Using Illusions to Understand Vision


MIT professor Edward Adelson uses remarkable visual illusions to help explain the workings of the human visual system. One such illusion is shown above. Believe it or not, the square marked A is the same shade of gray on your computer screen as the square marked B.


Here’s a “proof,” provided by Adelson. Two strips of constant grayness are aligned on top of the picture. You can see that the A square is the same shade as the strips near it and the B square is the same shade as the strips near it. Perhaps you still don’t believe that the strips are of constant grayness. In that case, put some paper up next to your computer screen to block off everything except for the strips; you’ll see it’s true.

Adelson explains the illusion here. The point is that our visual system is not meant to be used as a light meter; instead it is trying to solve the much more important problem (for our survival) of determining the true shade (that is, the color of the attached “paint”) of the objects it is looking at.

You can find more interesting illusions and demos from Adelson and other members of the perceptual science group at MIT, but don’t fail to also take a look at the illusions collected by the lab of Dale Purves at Duke. I particularly recommend the cube color contrast demo, where you can see that gray can be made to look yellow or blue.

Purves, together with R. Beau Lotto, wrote the book “Why We See What We Do: An Empirical Theory of Vision,” which collects these remarkable illusions and also expounds on a theory explaining them. The theory, to summarize it very briefly, says that what humans actually see is a “reflexive manifestation of the past rather than a logical analysis of the present.” I found myself quite uncomfortable with the theory for much the same reasons as given in Alan Gilchrist‘s review.

I also would prefer a more mathematical theory than Purves and Lotto give. It seems to me that we should in general try to explain illusions in terms of a Bayesian analysis of the most probable scene given the evidence provided by the light. My collaborators Bill Freeman and Yair Weiss (both former students of Adelson’s) have long worked along these lines; see for example Yair’s excellent Ph.D. thesis from 1998, explaining motion illusions.

In fact, I would like to go beyond a mathematical explanation of illusions to an algorithmic one. I would argue that a good computer vision system should “suffer” from the same illusions as a human, even though it has neither the same evolutionary history nor the same life history. To take an example of what I have in mind, the famous Necker cube illusion presumably arises naturally from the fact that the two interpretations are both local optima, with respect to probability, so a good artificial system should use an algorithm that settles into one interpretation, but then still be able to spontaneously switch to the other.


Tags: , , , , ,

5 Responses to “Using Illusions to Understand Vision”

  1. miro slodki Says:


    I love your site
    and have put you in my blog roll for others to find as well

    wondering if you could do a piece to follow up on Kaiku’s book re hyperspace. I think this is the kind of stuff your readers would thoroughly enjoy

    again kudo’s to you

    I might suggest that you leave a guest book for people to post these general OMG TGIFI (thank God I found It) type of messages

  2. Barak Says:

    To amplify Miro Slodki’s comment, I suggest you put a list of books that you recommend here on the side column. (If you link it to a bookseller, they will even give you a commission when people use your link to buy the book.)

  3. Jonathan Yedidia Says:

    Thank you Miro and Barak for your comments and suggestions! You may well see some features like you suggest in the near future. (EDIT: Both these features have now been implemented.)

  4. subcorpus Says:

    i still cant wrap my mind around this …
    hehe …
    but i saw this illusion on wikipedia few months back … couldn’t understand it then either … hehe …

  5. Charlie Dean’s Blog » Sunday Follow-up Says:

    […] you want to stare at the checkerboard some more, you can go to the Nerd Wisdom blog, which is the blog of an MIT professor. There are links to more optical illusions […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: