Representation of motion boundaries in retinotopic human visual cortical areas

Edges are important in the interpretation of the retinal image. Although luminance edges have been studied extensively, much less is known about how or where the primate visual system detects boundaries defined by differences in surface properties such as texture, motion or binocular disparity. Here we use functional magnetic resonance imaging (fMRI) to localize human visual cortical activity related to the processing of one such higher-order edge type: motion boundaries. We describe a robust fMRI signal that is selective for motion segmentation. This boundary-specific signal is present, and retinotopically organized, within early visual areas, beginning in the primary visual cortex (area V1). Surprisingly, it is largely absent from the motion-selective area MT/V5 and far extrastriate visual areas. Changes in the surface velocity defining the motion boundaries affect the strength of the fMRI signal. In parallel psychophysical experiments, the perceptual salience of the boundaries shows a similar dependence on surface velocity. These results demonstrate that information for segmenting scenes by relative motion is represented as early as V1.     

Processing of first- and second-order motion signals by neurons in area MT of the macaque monkey

Extrastriate cortical area MT is thought to process behaviorally important visual motion signals. Psychophysical studies suggest that visual motion signals may be analyzed by multiple mechanisms, a "first-order" one based on luminance, and a "second-order" one based upon higher level cues (e.g. contrast, flicker). Second-order motion is visible to human observers, but should be invisible to first-order motion sensors. To learn if area MT is involved in the analysis of second-order motion, we measured responses to first- and second-order gratings of single neurons in area MT (and in one experiment, in area V1) in anesthetized, paralyzed macaque monkeys. For each neuron, we measured directional and spatio-temporal tuning with conventional first-order gratings and with second-order gratings created by spatial modulation of the flicker rate of a random texture. A minority of MT and V1 neurons exhibited significant selectivity for direction or orientation of second-order gratings. In nearly all cells, response to second-order motion was weaker than response to first-order motion. MT cells with significant selectivity for second-order motion tended to be more responsive and more sensitive to luminance contrast, but were in other respects similar to the remaining MT neurons; they did not appear to represent a distinct subpopulation. For those cells selective for second-order motion, we found a correlation between the preferred directions of first- and second-order motion, and weak correlations in preferred spatial frequency. These cells preferred lower temporal frequencies for second-order motion than for first-order motion. A small proportion of MT cells seemed to remain selective and responsive for second-order motion. None of our small sample of V1 cells did. Cells in this small population, but not others, may perform "form-cue invariant" motion processing (Albright, 1992).     

Functional MRI reveals spatially specific attentional modulation in human primary visual cortex

Selective visual attention can strongly influence perceptual processing, even for apparently low-level visual stimuli. Although it is largely accepted that attention modulates neural activity in extrastriate visual cortex, the extent to which attention operates in the first cortical stage, striate visual cortex (area V1), remains controversial. Here, functional MRI was used at high field strength (3 T) to study humans during attentionally demanding visual discriminations. Similar, robust attentional modulations were observed in both striate and extrastriate cortical areas. Functional mapping of cortical retinotopy demonstrates that attentional modulations were spatially specific, enhancing responses to attended stimuli and suppressing responses when attention was directed elsewhere. The spatial pattern of modulation reveals a complex attentional window that is consistent with object-based attention but is inconsistent with a simple attentional spotlight. These data suggest that neural processing in V1 is not governed simply by sensory stimulation, but, like extrastriate regions, V1 can be strongly and specifically influenced by attention.     

Motion integration in a thalamic visual nucleus

Thalamic nuclei have long been regarded as passive relay stations for sensory information en route to higher level processing in the cerebral cortex. Recently, physiological and theoretical studies have reassessed the role of the thalamus and it has been proposed that thalamic nuclei may actively participate with cortical areas in processing specific information. In support of this idea, we now show that a subset of neurons in an extrageniculate visual nucleus, the lateral-posterior pulvinar complex, can signal the true direction of motion of a plaid pattern, indicating that thalamic cells can integrate different motion signals into a coherent moving percept. This is the first time that these computations have been found to occur outside the higher-order cortical areas. Our findings implicate extrageniculate cortico-thalamo-cortical loops in the dynamic processing of image motion, and, more generally, as basic computational modules involved in analysing specific features of complex visual scenes.     

Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys

We have previously shown that some neurons in extrastriate area MT are capable of signaling the global motion of complex patterns; neurons randomly sampled from V1, on the other hand, respond only to the motion of individual oriented components. Because only a small fraction of V1 neurons projects to MT, we wished to establish the processing hierarchy more precisely by studying the properties of those neurons projecting to MT, identified by antidromic responses to electrical stimulation of MT. The neurons that project from V1 to MT were directionally selective and, like other V1 neurons, responded only to the motion of the components of complex patterns. The projection neurons were predominantly "special complex," responsive to a broad range of spatial and temporal frequencies, and sensitive to very low stimulus contrasts. The projection neurons thus comprise a homogeneous and highly specialized subset of V1 neurons, consistent with the notion that V1 acts as clearing house of basic visual measurements, distributing information appropriately to higher cortical areas for specialized analysis.     

Working memory in young children: evidence for modality-specificity and implications for cerebral reorganization in early childhood

Digit span (DS) and visual-spatial memory span (VMS) tasks have been considered indices of auditory and visual spatial processing, respectively, often classified as "primary memory" or "attention". There has been limited evidence for their modality specificity, however. We present two children who showed visual spatial processing deficiencies (including VMS) and non-dominant manual inefficiency with normal visual-spatial perception, auditory-verbal processing and dominant fine manual skills. These children support a distinction between auditory and visual-spatial memory span. These findings are discussed with regard to a hypothesis that the unique expression of VMS is time-limited, that visual-spatial processing becomes more verbalized as children learn to read and that these behavioral changes produce a lateral shift in cortical processing of visual spatial information.     

Similar electrophysiological correlates of texture segregation induced by luminance, orientation, motion and stereo

Certain local features induce preattentive texture segregation. Recently, components in the visual evoked potential (VEP) associated with preattentive texture segregation (tsVEPs) have been demonstrated. To assess the similarity and dissimilarity of visual processing across visual dimensions, we compared VEPs and tsVEPs in texture segregation by luminance, orientation, motion and stereo disparity. We found tsVEPs across these four visual dimensions to be remarkably similar when compared to the "low-level" VEPs. The tsVEPs were always negative; their implicit time, peak latency and amplitude were (in msec/msec/microV): 91/234/-5.7, luminance; 84/257/-3.9, orientation; 80/295/-8.3, motion; and 95/310/-5.0 for stereo. The cross-correlation function, as a quantitative measure for similarity, on average was higher for the tsVEPs by a factor of 4.2 as compared to the low-level VEPs (P < 0.0001). The results suggest (1) that the tsVEPs represent activity of neural mechanisms that have generalised to some degree across visual dimensions; and (2) that these hypothetical generalisation mechanisms might exist already in the primary visual cortex.     

Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons

A single visual stimulus activates neurons in many different cortical areas. A major challenge in cortical physiology is to understand how the neural activity in these numerous active zones leads to a unified percept of the visual scene. The anatomical basis for these interactions is the dense network of connections that link the visual areas. Within this network, feedforward connections transmit signals from lower-order areas such as V1 or V2 to higher-order areas. In addition, there is a dense web of feedback connections which, despite their anatomical prominence, remain functionally mysterious. Here we show, using reversible inactivation of a higher-order area (monkey area V5/MT), that feedback connections serve to amplify and focus activity of neurons in lower-order areas, and that they are important in the differentiation of figure from ground, particularly in the case of stimuli of low visibility. More specifically, we show that feedback connections facilitate responses to objects moving within the classical receptive field; enhance suppression evoked by background stimuli in the surrounding region; and have the strongest effects for stimuli of low salience.     

Neuronal responses to edges defined by luminance vs. temporal texture in macaque area V1

We examined the responsivity, orientation selectivity, and direction selectivity of a sample of neurons in cortical area V1 of the macaque using visual stimuli consisting of drifting oriented contours defined by each of two very different figural cues: luminance contrast and temporal texture. Comparisons of orientation and direction tuning elicited by the different cues were made in order to test the hypothesis that the neuronal representations of these parameters are form-cue invariant. The majority of the sampled cells responded to both stimulus types, although responses to temporal texture stimuli were generally weaker than those elicited by luminance-defined stimuli. Of those units exhibiting orientation selectivity when tested with the luminance-defined stimuli, more than half were also selective for the orientation of the temporal texture stimuli. There was close correspondence between the preferred orientations and tuning bandwidths revealed with the two stimulus types. Of those units exhibiting directional selectivity when tested with the luminance-defined stimuli, about two-thirds were also selective for the direction of the temporal texture stimuli. There was close correspondence between the preferred directions revealed with the two stimulus types, although bidirectional responses were somewhat more common when temporal texture stimuli were used. These results indicate that many V1 neurons encode orientation and direction of motion of retinal image features in a manner that is largely independent of whether the feature is defined by luminance or temporal texture contrast. These neurons may contribute to perceptual phenomena in which figural cue identity is disregarded.     

Fine grain of the neural representation of human spatial vision

It is widely held that in human spatial vision the visual scene is initially processed through visual filters, each of which is responsive to narrow ranges of image spatial frequencies. The physiological basis of these filters are thought to be cortical neurons with receptive fields of different sizes. The grain of the neural representation of spatial vision is much finer than had been supposed. Using laser interferometry, which effectively bypasses the demodulation of the optics of the eye, we measured discrimination of, and adaptation to, high spatial frequency laser interference fringe patterns. Spatial frequency discrimination was good right up to the visual resolution limit (average Weber fractions of 0.13 at 50 c/deg). Both contrast and spatial frequency matches made after adapting to extremely fine interference fringes strongly suggested that there existed even finer, relatively unadapted, filters (mechanisms with small receptive fields). The smallest cortical receptive fields processing spatial information in human vision are so small that they can possess receptive field centers hardly wider than single cone photoreceptors.     

The retinotopy of visual spatial attention

We used high-field (3T) functional magnetic resonance imaging (fMRI) to label cortical activity due to visual spatial attention, relative to flattened cortical maps of the retinotopy and visual areas from the same human subjects. In the main task, the visual stimulus remained constant, but covert visual spatial attention was varied in both location and load. In each of the extrastriate retinotopic areas, we found MR increases at the representations of the attended target. Similar but smaller increases were found in V1. Decreased MR levels were found in the same cortical locations when attention was directed at retinotopically different locations. In and surrounding area MT+, MR increases were lateralized but not otherwise retinotopic. At the representation of eccentricities central to that of the attended targets, prominent MR decreases occurred during spatial attention.     

Neural dynamics of motion processing and speed discrimination

A neural network model of visual motion perception and speed discrimination is presented. The model shows how a distributed population code of speed tuning, that realizes a size-speed correlation, can be derived from the simplest mechanisms whereby activations of multiple spatially short-range filters of different size are transformed into speed-turned cell responses. These mechanisms use transient cell responses to moving stimuli, output thresholds that covary with filter size, and competition. These mechanisms are proposed to occur in the V1-->MT cortical processing stream. The model reproduces empirically derived speed discrimination curves and simulates data showing how visual speed perception and discrimination can be affected by stimulus contrast, duration, dot density and spatial frequency. Model motion mechanisms are analogous to mechanisms that have been used to model 3-D form and figure-ground perception. The model forms the front end of a larger motion processing system that has been used to simulate how global motion capture occurs, and how spatial attention is drawn to moving forms. It provides a computational foundation for an emerging neural theory of 3-D form and motion perception.     

Object-based attention in the primary visual cortex of the macaque monkey

Typical natural visual scenes contain many objects, which need to be segregated from each other and from the background. Present theories subdivide the processes responsible for this segregation into a pre-attentive and attentive system. The pre-attentive system segregates image regions that 'pop out' rapidly and in parallel across the visual field. In the primary visual cortex, responses to pre-attentively selected image regions are enhanced. When objects do not segregate automatically from the rest of the image, the time-consuming attentive system is recruited. Here we investigate whether attentive selection is also associated with a modulation of firing rates in area V1 of the brain in monkeys trained to perform a curve-tracing task. Neuronal responses to the various segments of a target curve were simultaneously enhanced relative to responses evoked by a distractor curve, even if the two curves crossed each other. This indicates that object-based attention is associated with a response enhancement at the earliest level of the visual cortical processing hierarchy.     

A model for the neuronal implementation of selective visual attention based on temporal correlation among neurons

We propose a model for the neuronal implementation of selective visual attention based on temporal correlation among groups of neurons. Neurons in primary visual cortex respond to visual stimuli with a Poisson distributed spike train with an appropriate, stimulus-dependent mean firing rate. The spike trains of neurons whose receptive fields do not overlap with the "focus of attention" are distributed according to homogeneous (time-independent) Poisson process with no correlation between action potentials of different neurons. In contrast, spike trains of neurons with receptive fields within the focus of attention are distributed according to non-homogeneous (time-dependent) Poisson processes. Since the short-term average spike rates of all neurons with receptive fields in the focus of attention covary, correlations between these spike trains are introduced which are detected by inhibitory interneurons in V4. These cells, modeled as modified integrate-and-fire neurons, function as coincidence detectors and suppress the response of V4 cells associated with non-attended visual stimuli. The model reproduces quantitatively experimental data obtained in cortical area V4 of monkey by Moran and Desimone (1985).     

Psychohysical hallucinations of orientation and spatial frequency

After inspection of vertical sinusoidal gratings at least three distinct types of subjective or "hallucinated" patterns can be seen on a uniform test field. One type, here called horizontal streaming (H), is already well-known from the work of MacKay. A second type (V) looks like aroughly sinusoidal grating about 1-5 octaves above the adapting spitial frequency. Under optimal conditions a second vertical component appears at about 2 octaves below the adapting frequency. The third category of aftereffect consists of diagonal lines (D) at two orientations (about +/-40 degrees from vertical). The spatial-frequency band at these two orientations appears to be fairly broad, but roughly similar to the adapting frequency. The duration and strength of D increased, while V declined, at higher adapting spatial frequencies. D and V were increasing functions of adapting contrast, while H appeared abruptly only after the highest adapting contrast. H, D, and V are thus all functionally distinct. A schematic model of cortical organization is proposed to account for these phenomena. Pattern channels selective for a given orientation are grouped together with movement channels selective for the orthogonal direction. Antagonism between channels within such "modules" accounts for the streaming effect (H). Inhibition between modules tuned to different orientations and spatial frequencies accounts for the D and V effects: after adaptation of channels in one module, neighbouring module(s) are released from inhibition to produce a spurious response which is seen as a grating-like object in the adapted part of the visual field. During flickering adaptation a "halluncinated" lattice can be seen superimposed on the adapting grating. It apparently consists of Fourier components more remote from the adapting pattern than D and V are. This disinhibitory effect is strong confirmation of the inhibitory model. The regular and highly organized matrix of channels implied by these experiments may constitute a cortical hypercolumn conducting a coarse, piecewise Fourier transformation of the retinal image.     

Visual processing levels revealed by response latencies to changes in different visual attributes

Visual latencies, and their variation with stimulus attributes, can provide information about the level in the visual system at which different attributes of the image are analysed, and decisions about them made. A change in the colour, structure or movement of a visual stimulus brings about a highly reproducible transient constriction of the pupil that probably depends on visual cortical mechanisms. We measured this transient response to changes in several attributes of visual stimuli, and also measured manual reaction times to the same stimulus changes. Through analysis of latencies, we hoped to establish whether changes in different stimulus attributes were processed by mechanisms at the same or different levels in the visual pathway. Pupil responses to a change in spatial structure or colour are almost identical, but both are ca. 40 ms slower than those to a change in light flux, which are thought to depend largely on subcortical pathways. Manual reaction times to a change in spatial structure or colour, or to the onset of coherent movement, differ reliably, and all are longer than the reaction time to a change in light flux. On average, observers take 184 ms to detect a change in light flux, 6 ms more to detect the onset of a grating, 30 ms more to detect a change in colour, and 37 ms more to detect the onset of coherent motion. The pattern of latency variation for pupil responses and reaction times suggests that the mechanisms that trigger the responses lie at different levels in cortex. Given our present knowledge of visual cortical organization, the long reaction time to the change in motion is surprising. The range of reaction times across different stimuli is consistent with decisions about the onset of a grating being made in V1 and decisions about the change in colour or change in motion being made in V4.     

Correlation analysis of corticotectal interactions in the cat visual system

We have studied the temporal relationship between visual responses in various visual cortical areas [17, 18, postero medial lateral suprasylvian (PMLS), postero lateral lateral suprasylvian (PLLS), 21a]) and the superficial layers of the cat superior colliculus (SC). To this end, simultaneous recordings were performed in one or several visual cortical areas and the SC of anesthetized paralyzed cats, and visually evoked multiunit responses were subjected to correlation analysis. Significant correlations occurred in 117 (24%) of 489 cortex-SC pairs and were found for all cortical areas recorded. About half of the significant correlograms showed an oscillatory modulation. In these cases, oscillation frequencies covered a broad range, the majority being in the alpha- and beta-band. On average, significant center peaks in cross-correlograms had a modulation amplitude of 0.34. Our analysis revealed a considerable intertrial variability of correlation patterns with respect to both correlation strength and oscillation frequency. Furthermore, cortical areas differed in their corticotectal correlation patterns. The percentage of cells involved a corticotectal correlation, as well as the percentage of significantly modulated correlograms in such cases, was low for areas 17 and PMLS but high for areas 18 and PLLS. Analysis of the cortical layers involved in these interactions showed that consistent temporal relationships between cortical and collicular responses were not restricted to layer V. Our data demonstrate a close relationship between corticotectal interactions and intracortical or intracollicular synchronization. Trial-by-trial analysis from these sites revealed a clear covariance of corticotectal correlations with intracortical synchronization. The probability of observing corticotectal interactions increased with enhanced local cortical and collicular synchronization and, in particular, with interareal cortical correlations. Corticotectal correlation patterns resemble in many ways those described among areas of the visual cortex. However, the correlations observed are weaker than those between nearby cortical sites, exhibit usually broader peaks and for some cortical areas show consistent phase-shifts. Corticotectal correlations represent population phenomena that reflect both the local and global temporal organization of activity in the cortical and collicular network and do not arise from purely monosynaptic interactions. Our findings show that both striate and extrastriate inputs affect the superficial SC in a cooperative manner and, thus, do not support the view that responses in the superficial SC depend exclusively on input from the primary visual areas as implied by the concept of "two corticotectal systems." We conclude that the corticotectal projections convey temporal activation patterns with high reliability, thus allowing the SC evaluation of information encoded in the temporal relations between responses of spatially disseminated cortical neurons. As a consequence, information distributed across multiple cortical areas can affect the SC neurons in a coherent way.     

Evolving concepts of spatial channels in vision: from independence to nonlinear interactions

By the 1960s it was evident from neuroanatomy that there were extensive recurrent interactions, both excitatory and inhibitory, among visual cortical neurons. Nevertheless, the psychophysical discovery of 'spatial-frequency channels' gave rise to a decade in which parallel, independent channels were thought to subserve early spatial vision. Recent work, however, has clearly demonstrated that early visual channels do not perform a Fourier or wavelet decomposition of the image. Instead, they interact through a variety of nonlinear pooling mechanisms. Such nonlinear interactions perform important computations in texture perception, stereopsis, and motion and form vision.     

Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device

The ability of organisms to categorize diverse and often novel stimuli depends on ongoing interactions with their environment. In a modality such as vision, categorization requires the generation of both selective and invariant responses of cortical neurons to complex visual stimuli. How does behavior contribute to shaping the responses of these neurons? Analysis of this question is made difficult by the complex multilevel interactions between many neural and behavioral variables. To mitigate this difficulty, we studied the development and ongoing plasticity of pattern-selective neuronal responses by means of synthetic neural modeling. For this purpose, we constructed Darwin V, which consists of a simulated neuronal model embedded in a real-world device that is capable of motion and autonomous behavior. The neuronal model consists of four major components: a visual system (containing cortical and subcortical networks); a taste system based on conductance; sets of motor neurons capable of triggering behavior; and a diffuse ascending (value) system. The modeled visual cortex consists of two areas: a topographic map responsive to elementary features connected to a higher-order map composed of initially non-selective neuronal units. During behavior over time in its environment, Darwin V encounters numerous objects consisting of black metal cubes displaying different patterns of white blobs and stripes. Initially, the lack of specific higher-order visual responses does not allow visual pattern discrimination, and appetitive and aversive behaviors are triggered by the 'taste' (surface conductivity of objects) alone. In the course of sensory experience, however, changes occur in visual and sensorimotor connection strengths, with two major consequences. First, units within the higher visual area acquire responses that are both pattern selective and translation invariant. Second, as a result of the operation of the value system, these responses become linked to appropriate behaviors. Analysis of Darwin V after such changes indicates that the continuity of self-generated movements is essential for the development of pattern-selective and translation-invariant responses. The concomitant development of a preference for foveal over parafoveal objects was found to be due to increased behavioral interactions with object cubes gripped by the centrally mounted effector (snout) of Darwin V. Finally, even after development of higher-order visual responses, visual responses to more frequently encountered objects continued to be enhanced, while other responses were diminished. Overall, the detailed study of Darwin V over multiple levels of organization provides a heuristically revealing example of the crucial role played by behavioral and environmental interactions in the development of complex responses by specialized neurons.     

Magnocellular and parvocellular contributions to the responses of neurons in macaque striate cortex

Anatomical and physiological studies of the primate visual system have suggested that the signals relayed by the magnocellular and parvocellular subdivisions of the LGN remain segregated in visual cortex. It has been suggested that this segregation may account for the known differences in visual function between the parietal and temporal cortical processing streams in extrastriate visual cortex. To test directly the hypothesis that the temporal stream of processing receives predominantly parvocellular signals, we recorded visual responses from the superficial layers of V1 (striate cortex), which give rise to the temporal stream, while selectively inactivating either the magnocellular or parvocellular subdivisions of the LGN. Inactivation of the parvocellular subdivision reduced neuronal responses in the superficial layers of V1, but the effects of magnocellular blocks were generally as pronounced or slightly stronger. Individual neurons were found to receive contributions from both pathways. We furthermore found no evidence that magnocellular contributions were restricted to either the cytochrome oxidase blobs or interblobs in V1. Instead, magnocellular signals made substantial contributions to responses throughout the superficial layers. Thus, the regions within V1 that constitute the early stages of the temporal processing stream do not appear to contain isolated parvocellular signals. These results argue against a direct mapping of the subcortical magnocellular and parvocellular pathways onto the parietal and temporal streams of processing in cortex.