Do We See the World Realistically?
13
It is no different when you set out to solve a technical problem involving a
real-world situation: You don’t want a photograph of the objective situation,
but a diagram showing just the necessary information. An effective diagram
is both simple and suggestive: Its labeled nodes are tokens, or place-holders,
and don’t resemble what they represent. Lines are drawn to indicate the flow
of information, or the flow of interaction between parts. A diagram is a pragmatic tool designed to reveal working details and not to be realistic. Visual
representation is not a diagram, but the pragmatic intent is similar.
Hoffman and his associates Justin Mark and Brian Marion have carried out
experiments, using mathematical modeling, to test this hypothesis. They modeled an environment in which creatures with realistic vision competed against
creatures whose vision gave them non-realistic but practical details. They found
that faithful representation is driven to extinction by non-veridical strategies
based on utility rather than objective reality. This suggests that natural selection
must do the same: It is not likely to favor rigorously correct representation, but
instead, would want organisms to experience visual displays in a form which
is most effective for rapidly sizing up events and making instantaneous (and
appropriate) decisions.
Hoffman’s theory is called the interface theory of perception. This word
refers to computing devices having a user-friendly interface: Such a device
(e.g., your laptop) carries out all its computing in the background and presents
you only with the finished result. In the very same way, your brain computes
in the background and delivers to you a carefully designed version of your
surroundings, representing all the information that you need to know, and
withholding details unrelated to your present activity.
In the interface theory, it is important that the forms we experience seeing
are completely consistent with measurement and other ways of knowing. What
we see as a straight line may be confirmed with a ruler, and what looks round
to us has objective curvature in the mathematical sense. The appearance of
things is not physically distorted. However, forms should have a quality in
experience which is maximally informative to a creature in its moment-tomoment transactions with the world.
In order to see a display or scene—that is, to take it in mentally and understand it—it is necessary for the display to be clearly divided into separate
objects arrayed in space. The raw scene projected onto the retina of your eyes
is a kaleidoscopic montage of colored patches, and the first challenge of the
visual brain is to find the meaningful objects deeply embedded in this display.
This is an enormously difficult task, beyond the power of present-day computers, yet the brain seems to do it effortlessly. It is the way all creatures see
13
It is no different when you set out to solve a technical problem involving a
real-world situation: You don’t want a photograph of the objective situation,
but a diagram showing just the necessary information. An effective diagram
is both simple and suggestive: Its labeled nodes are tokens, or place-holders,
and don’t resemble what they represent. Lines are drawn to indicate the flow
of information, or the flow of interaction between parts. A diagram is a pragmatic tool designed to reveal working details and not to be realistic. Visual
representation is not a diagram, but the pragmatic intent is similar.
Hoffman and his associates Justin Mark and Brian Marion have carried out
experiments, using mathematical modeling, to test this hypothesis. They modeled an environment in which creatures with realistic vision competed against
creatures whose vision gave them non-realistic but practical details. They found
that faithful representation is driven to extinction by non-veridical strategies
based on utility rather than objective reality. This suggests that natural selection
must do the same: It is not likely to favor rigorously correct representation, but
instead, would want organisms to experience visual displays in a form which
is most effective for rapidly sizing up events and making instantaneous (and
appropriate) decisions.
Hoffman’s theory is called the interface theory of perception. This word
refers to computing devices having a user-friendly interface: Such a device
(e.g., your laptop) carries out all its computing in the background and presents
you only with the finished result. In the very same way, your brain computes
in the background and delivers to you a carefully designed version of your
surroundings, representing all the information that you need to know, and
withholding details unrelated to your present activity.
In the interface theory, it is important that the forms we experience seeing
are completely consistent with measurement and other ways of knowing. What
we see as a straight line may be confirmed with a ruler, and what looks round
to us has objective curvature in the mathematical sense. The appearance of
things is not physically distorted. However, forms should have a quality in
experience which is maximally informative to a creature in its moment-tomoment transactions with the world.
In order to see a display or scene—that is, to take it in mentally and understand it—it is necessary for the display to be clearly divided into separate
objects arrayed in space. The raw scene projected onto the retina of your eyes
is a kaleidoscopic montage of colored patches, and the first challenge of the
visual brain is to find the meaningful objects deeply embedded in this display.
This is an enormously difficult task, beyond the power of present-day computers, yet the brain seems to do it effortlessly. It is the way all creatures see