NTT Researcher Finds Key Mechanism to Human Vision
-Finding paves way for understanding how we "sees" things-

Whereas it is widely believed that the various mechanisms responsible for the processing of "shape," "motion," "position," and "color," etc., function independently within the human visual system, the finding reveals that the mechanism of "motion" perception, in fact, has so strong an influence on those of "position" and "shape" as to give rise to visual illusions. This finding clarifies the previously unexplained mechanism by which a moving object is seen at the correct position. The new perspectives opened up by the finding are also expected to lead to innovative engineering applications in the video communications field.

The results of this research were published in the scientific journal "Nature" on February 18, 1999.


Background

1. Significance of Research on Vision

Between the moment when an image of an object is projected on the retina of the eye and the moment when we become conscious of having "seen" the object, the neural networks in the retina and the visual cortex must process a great deal of complex information. The researchers' breakthrough into the mechanism of visual information processing sheds light on an important problem of neuroscience and is also expected to lead to engineering applications in areas such as image compression and machine recognition.

2. Separate Processing of Visual Attributes

The "visual" world is composed of a variety of image attributes, such as brightness, color, texture, shape, motion, and position. It is the combination of the mechanisms that handle these attributes that allows us to see things. Recent developments in neuroscience have shown that the mechanisms responsible for different attributes are located in separate areas of the cerebral cortex. For instance, it has been demonstrated that the processing of information on motion is closely related to an area known as V5, whereas the processing of information on color is related to the area known as V4.

3. Motion and Position Change

In the physical world, the motion of objects is accompanied by changes in their position and orientation. By measuring how much position shifts in a given time, it is possible to compute the speed of motion. In the "visual" world, however, the change in position and the speed of motion do not necessarily correspond. This is thought to be because the two visual attributes are processed in different parts of the brain.

4. The Paradox of Motion Aftereffect

The inconsistency of the perception of position and that of motion is clearly demonstrated in the visual illusion known as the "motion aftereffect" (MAE) or as the "waterfall illusion." This is a well-known illusion in which, after adaptation to motion in a particular direction, a static pattern appears to be moving in the opposite direction. As a result of adaptation of the mechanism for motion processing, the perception of a motion that does not, in fact, exist in the stimulus, is generated in the brain (Note 1). The observer experiences the paradox that the physically stationary object is perceptually moving.

5. Perception of Position Shift in the MAE

It is widely believed that, in the MAE, the position of an apparently moving pattern does not appear to change. This is thought to imply that separately processed motion and position information is perceived without affecting each other. In other words, the mechanisms that handle the various visual attributes are thought to be independent.

However, the finding suggests that this argument is incorrect.


The Finding

1. Basic Phenomenon (see Figure 1)

The subject was first shown a rotating windmill pattern. The subject was then shown a stationary windmill, which was to serve as the test pattern. Due to the MAE, the test windmill appeared to the subject to be rotated in the opposite direction. Then the subject was asked to judge the orientation of the impeller of a test windmill, which was physically vertical. If the MAE is not accompanied by a position shift, as is widely believed, then the orientation should appear vertical. In practice, however, the orientation was perceived as being inclined in the direction that it appeared to be rotating.

2. Results of the experiments (see Figure 2)

When the magnitude of apparent orientation shift was measured as a function of time, for the first few seconds after seeing the test pattern, the orientation shift grew larger, just as if the windmill were really moving. However, the speed of this increase was slow, less than 10% of the speed of the MAE actually perceived. In other words, apparent position does not change proportionately to the apparent speed of motion. The perceptions of motion and position do not conform.

The shift in orientation started to decline a few seconds after the test onset. This is because the MAE decays over time, but, interestingly, the perception of orientation shift persisted for some time after the disappearance of the MAE. Nishida and Johnston suggest this is because the brain, when estimating position shift from motion information, uses a kind of buffer that stores the influence of past motion and integrates it over time.

When the windmill is actually rotated in the opposite direction to the perceived MAE, it appears stationary although it is, in fact, rotating. Under these conditions, orientation shift was not perceived. This means that the motion signals within the brain give rise to the perceived orientation shift.


Significance of the Finding

1. Interaction of Motion and Position (see Figure 3)

The finding does not reject the notion that motion and position information are processed separately to some extent, but suggests that there exist, within the brain, pathways through which motion information affects the perception of position. Another important revelation of these experiments is that the position shift due to MAE generates orientation shift. It may be that orientation is more closely related to the processing of shape information than to position. The classical view is that shape and motion are representative attributes of two major pathways known as the temporal lobe and the parietal lobe systems. This finding suggests the possibility that the processing of visual information by these two pathways is even more closely related than was previously thought.

2. Seeing a Moving Object at the Correct Position (see Figure 4)

Why, then, should the brain contain pathways through which the judgment of position is affected by motion? "Seeing" an object requires a processing time of a fraction of a second. Consequently, when a ball comes flying towards us, the delay in processing should mean that when the ball appears to be at a certain position, the actual ball is no longer at that position. To resolve this problem and see a moving object in the correct position, it is necessary to extrapolate the future position of the ball from motion information and see it "where it will be." In order to predict the future, the brain must integrate the speed of motion over time, estimate the degree of position shift, and combine this with the appearance of the object as seen at the present time. Nishida and Johnston suggest that the realization of such a function is one of the purposes of this interaction between motion and position found in the course of their research.

3. Engineering Applications

Recent developments in multimedia have drawn attention to the need for efficient technology for the compression of moving images. In compression, the principal difficulty lies in choosing which kind of information to preserve and which to discard. If this choice is not based on the mechanism by which we see things, it will not be possible to develop communications technology that is comfortable to use. Nishida and Johnston's research reveals that motion information is used not only to see motion itself, but also in seeing position and shape. This clarification of new characteristics of motion information may lead to new developments in moving image compression technology.


Note

Note 1: The mechanism of the motion aftereffect (MAE)

The MAE is thought to arise by the following mechanism. The human brain contains groups of cells whose function is to detect motion in various directions. Let us label two of these cells, whose function is to detect motion to left and right, Cell L and Cell R. Normally, these two cells respond to a static stimulus to the same extent. The brain interprets the fact that the two cells have responded to the same extent as indicating that the stimulus is stationary. Next, if the subject is shown motion to the right, Cell R responds more strongly. If this situation continues for some time, the adaptation phenomenon causes the response of Cell R to weaken gradually. In this case, adaptation can be thought of as a process similar to fatigue. If the subject is then shown a static stimulus once again, because the response of Cell R, which has been responding, is significantly reduced, the response of Cell L will be stronger in relative terms. Because this pattern of cell responses is the same as that which occurs when the subject sees motion to the left, the brain's interpretation causes the stimulus to appear to be moving to the left, although it is, in fact, stationary.


Paper Published

Publication: Nature, vol. 397, pp. 610-612 (February 18, 1999 issue)
Title: Influence of motion signals on the perceived position of spatial pattern
Authors: Shin'ya Nishida (1) and Alan Johnston (2)
Affiliation: (1) Senior Researcher, Information Science Research Laboratory, NTT Basic Research Laboratories; (2) Reader, Department of Psychology and Institute of Cognitive Neuroscience, University College London



Attachment

Figure 1: Apparent inclination induced by the MAE
Figure 2: Temporal characteristics of orientation shift
Figure 3: Model
Figure 4: Mechanism to see a moving object at the correct position