May 26, 2004

A new visual mechanism indicated by electric signboards
- Involvement of motion processing in seeing shape -

Nippon Telegraph And Telephone Corp. (NTT; Head Office: Chiyoda-ku, Tokyo; President: Norio Wada) has found that the human brain employs visual motion information for clear perception of the spatial shape of moving objects. This discovery was made from psychophysical investigations of a visual illusion used for electric signboards. It argues against the prevailing hypothesis of separate processing of shape and motion in the human brain, and provides useful insight into visual telecommunications technology.

In neuroscience, it has been widely believed that the shape and motion of an object are separately processed by independent specialized neural mechanisms in the visual system (Fig. 1). Inconsistent with this view, the new finding indicates that the processing of shape and that of motion are tightly linked in an early stage of visual processing. Furthermore, the finding provides a useful guideline for designing effective display systems that use image motion to improve the clarity of an image. The results of this research were reported in the British science magazine, Current Biology (May 25, 2004).

1. Summary of the experiments
NTT Communication Science Laboratories have been studying the human brain mechanisms responsible for the perception in multi-slit viewing. In multi-slit viewing, shapes, such as letters, are presented through an array of narrow slits (Fig. 2). The slit separation is so wide that the presented shapes cannot be recognized as long as they are stationary, but the shapes can be clearly seen when scrolled behind the stationary slits. This illusion has been known since 1970's and is applied to electric signboards on streets. However, the principle underlying the perception of this illusion has not been fully understood.

One classical account of multi-slit viewing is that the moving shapes are "painted" on the retina as a result of an observer's tracking eye movement. To exclude this possibility, all experiments were conducted under a condition where observers could not track the moving shapes (letters), yet the stimulus movements did improve the shape recognition. This implies that the shapes seen in multi-slit viewing are reconstructed in the brain. An issue of long-standing controversy, however, has been whether the motion information is used for the shape reconstruction, as opposed to the general belief that the shape processing is independent of the motion processing. The present study revealed two lines of evidence indicating that the mechanisms for multi-slit viewing have "direction selectivity(*1)", a property widely regarded as critical to motion processing.

(1) Examination of direction selectivity from noise masking and adaptation
The masking effect of a moving noise pattern, presented in the inter-slit area, on letter identification in multi-slit viewing was evaluated by changing the intensity (contrast), direction and speed of the noise pattern. The results indicated that the masking effect depends on the direction of noise motion, with the maximum effect occurring when the noise moves in the same direction and at the same speed as the letter movement (Fig. 3). This indicates that the neural mechanism underlying multi-slit viewing is direction selective. Another experiment demonstrated a direction-selective adaptation effect in which the letter perception was worse after prolonged adaptation to a noise pattern moving in the same direction as the letter movement than after adaptation to the opposite direction of motion. Although masking and adaptation are the standard psychophysical techniques to examine the stimulus specificity of neural mechanisms, this is the first time that direction selectivity of masking and adaptation for spatial pattern perception was shown.

(2) Examination of direction selectivity from the quality of perceived pattern
When an image is viewed though slits, the image quality is degraded since fine spatial components are lost. Theoretically, without knowing the moving direction of the presented patterns, an observer cannot reconstruct fine spatial shape components higher than the spatial frequency whose cycle is twice the slit interval (Nyquist frequency(*2)) (Fig. 4). Therefore, if the observer can perceive spatial frequencies higher than the Nyquist frequency, it follows that she/he uses motion information. To evaluate the quality of the image perceived in multi-slit viewing, the perceived spatial frequency map was estimated by a reverse correlation(*3) method. The obtained map indicated that the observers did indeed perceive spatial frequencies higher than the Nyquist frequency.

2. Background and implication of the study
It has been suggested that the visual system consists of multiple parallel processing pathways running from the retinal circuit to higher cortical areas and that shapes and movements are separately processed in different pathways. To be more specific, two types of neurons in early visual cortical areas - one sensitive to fine shapes but insensitive to motion directions and the other sensitive to motion directions but insensitive to fine shapes - have been respectively regarded as initial shape analyzers and initial motion analyzers. Although several recent findings argue against the notion of independent processing of shape and motion, most of them can be interpreted as late interactions taking place after shape and motion are separately analyzed. The present finding, on the other hand, indicates that early visual neurons sensitive to motion directions are involved in shape perception, as well as in motion perception, thus casting doubt on the notion of independent processing per se.

Our eyes never stand still. Even when stationary objects are seen, the image on the retina is not stationary. Although the present discoveries were made in a special case (i.e., multi-slit viewing), the neural mechanisms revealed are always operating in our daily life, enabling us to perceive a clear dynamic visual world.

The multi-slit display is also of interest from the engineering point of view. Although the spatial resolution of this display is very low, it can provide high-definition images just by moving them. In this sense, the present study has revealed how high-definition images are reconstructed in the human brain. This could lead to future development of displays based on the similar principle, and also provides clues for solving problems in current display technology, such as color breaking (*4) in DLP displays.

3. Future research
Future study will reveal whether motion information is also used for the perception of color. Through examination of human cognitive functions, NTT will continue to clarify the basic principles underlying the representation and compression of information to provide a basis for future advanced telecommunications technologies.

*1 Direction selectivity: A neural mechanism responding to visual inputs is said to be direction selective when it responds to a given motion direction more strongly to the opposite direction.

*2 Nyquist frequency: It is known that when a signal (e.g., an image) is sampled at a constant interval, the original signal cannot be recovered from the sampled signal if the signal contains components higher than a given frequency. This limit is called Nyquist frequency, and its cycle is twice the sampling interval. Note however that this theorem assumes that the signal is stationary. When a moving signal is sampled, the Nyqusit frequency has a slightly different meaning, as shown in Fig. 3.

*3 Reverse correlation: The output of a system is measured while changing the input stimulus quasi-randomly. By taking the correlation of a given output with the input pattern, the stimulus structure that elicits the output can be estimated. This method has been used in electroencephalograph and electrophysiology and was recently introduced in psychophysics.

*4 Color breaking: For DLP displays, the observer sometimes perceives decomposition of the RGB colors that compose a full-color image. This is because a variation of color is made by temporally mixing the RGB colors, but the human visual mechanism is also related to the generation of this annoying effect.

- Fig 1: The relation ship between the known processing pathways and the new pathways suggested by the present study.
- Fig. 2: Multi slit viewing.
- Fig. 3: Direction-selective masking by noise.
- Fig. 4: Theoretical limitation of shape perception in multi-slit viewing.

For further information, contact:
Minako Sawaki and Hirofumi Motai
Planning Division
NTT Science and Core Technology Laboratory Group
Tel: 046-240-5152


Copyright (c) 2004 Nippon telegraph and telephone corporation