Fixational instability and natural image statistics: implications for early visual representations

Under natural viewing conditions, small movements of the eye, head and body prevent the maintenance of a steady direction of gaze. It is known that stimuli tend to fade when they are stabilized on the retina for several seconds. However, it is unclear whether the physiological motion of the retinal image serves a visual purpose during the brief periods of natural visual fixation. This study examines the impact of fixational instability on the statistics of the visual input to the retina and on the structure of neural activity in the early visual system. We show that fixational instability introduces a component in the retinal input signals that, in the presence of natural images, lacks spatial correlations. This component strongly influences neural activity in a model of the LGN. It decorrelates cell responses even if the contrast sensitivity functions of simulated cells are not perfectly tuned to counter-balance the power-law spectrum of natural images. A decorrelation of neural activity at the early stages of the visual system has been proposed to be beneficial for discarding statistical redundancies in the input signals. The results of this study suggest that fixational instability might contribute to the establishment of efficient representations of natural stimuli.     

Is the early visual system optimised to be energy efficient?

This paper demonstrates that a representation which balances natural image encoding with metabolic energy efficiency shows many similarities to the neural organisation observed in the early visual system. A simple linear model was constructed that learned receptive fields by optimally balancing information coding with metabolic expense for an entire visual field in a 2-stage visual system. The input to the model consists of a space variant retinal array of photoreceptors. Natural images were then encoded through a bottleneck such as the retinal ganglion cells that form the optic nerve. The natural images represented by the activity of retinal ganglion cells were then encoded by many more 'cortical' cells in a divergent representation. Qualitatively, the system learnt by optimising information coding and energy expenditure and matched (1) the centre surround organisation of retinal ganglion cells; (2) the Gabor-like organisation of cortical simple cells; (3) higher densities of receptive fields in the fovea decreasing in the periphery; (4) smaller receptive fields in the fovea increasing in size in the periphery; (5) spacing ratios of retinal cells; and (6) aspect ratios of cortical receptive fields. Quantitatively, however, there are small but significant discrepancies between density slopes which may be accounted for by taking optic blur and fixation induced image statistics into account. In addition, the model cortical receptive fields are more broadly tuned than biological cortical neurons; this may be accounted for by the computational limitation of modelling a relatively low number of neurons. This paper shows that retinal receptive field properties can be understood in terms of balancing coding with synaptic energy expenditure and cortical receptive fields with firing rate energy expenditure, and provides a sound biological explanation of why 'sparse' distributions are beneficial.     

Do simple cells in primary visual cortex form a tight frame?

Sets of neuronal tuning curves, which describe the responses of neurons as functions of a stimulus, can serve as a basis for approximating other functions of stimulus parameters. In a function-approximating network, synaptic weights determined by a correlation-based Hebbian rule are closely related to the coefficients that result when a function is expanded in an orthogonal basis. Although neuronal tuning curves typically are not orthogonal functions, the relationship between function approximation and correlation-based synaptic weights can be retained if the tuning curves satisfy the conditions of a tight frame. We examine whether the spatial receptive fields of simple cells in cat and monkey primary visual cortex (V1) form a tight frame, allowing them to serve as a basis for constructing more complicated extrastriate receptive fields using correlation-based synaptic weights. Our calculations show that the set of V1 simple cell receptive fields is not tight enough to account for the acuity observed psychophysically.     

Self-organization and functional role of lateral connections and multisize receptive fields in the primary visual cortex

Cells in the visual cortex are selective not only to ocular dominance and orientation of the input, but also to its size and spatial frequency. The simulations reported in this paper show how size selectivity could develop through Hebbian self-organization, and how receptive fields of different sizes could organize into columns like those for orientation and ocular dominance. The lateral connections in the network self-organize cooperatively and simultaneously with the receptive field sizes, and produce patterns of lateral connectivity that closely follow the receptive field organization. Together with our previous work on ocular dominance and orientation selectivity, these results suggest that a single Hebbian self-organizing process can give rise to all the major receptive field properties in the visual cortex, and also to structured patterns of lateral interactions, some of which have been verified experimentally and others predicted by the model. The model also suggests a functional role for the self-organized structures: The afferent receptive fields develop a sparse coding of the visual input, and the recurrent lateral interactions eliminate redundancies in cortical activity patterns, allowing the cortex to efficiently process massive amounts of visual information.     

Analysis of a correlation-based model for the development of orientation-selective receptive fields in the visual cortex

We analyse a model for the development of orientation-selective receptive fields of simple cells in a locally connected network of cortical neurons. The Hebbian learning rule that underlies the development is described by a linear differential equation. The structure of the emerging cortical map can be predicted by deriving the eigenfunctions corresponding to the leading eigenvalues of the associated matrix. We show that the receptive fields have the typical form of a wavelet. Mathematically, receptive fields are given by a Hermitian polynomial with Gaussian cut-off and a phase factor. Both the phase of the wavelet and the orientation are changing periodically along the surface of the cortical map as suggested by previous simulation studies and as also found in experiments. In order to get orientation-selective receptive fields, the spatial correlation function of the inputs that drive the development must have a zero crossing.     

Sparse coding on the spot: spontaneous retinal waves suffice for orientation selectivity

Ohshiro, Hussain, and Weliky (2011) recently showed that ferrets reared with exposure to flickering spot stimuli, in the absence of oriented visual experience, develop oriented receptive fields. They interpreted this as refutation of efficient coding models, which require oriented input in order to develop oriented receptive fields. Here we show that these data are compatible with the efficient coding hypothesis if the influence of spontaneous retinal waves is considered. We demonstrate that independent component analysis learns predominantly oriented receptive fields when trained on a mixture of spot stimuli and spontaneous retinal waves. Further, we show that the efficient coding hypothesis provides a compelling explanation for the contrast between the lack of receptive field changes seen in animals reared with spot stimuli and the significant cortical reorganisation observed in stripe-reared animals.     

A scene registration method based on a dynamical receptive field model of biological vision

A scene registration method based on a model of dynamical receptive field organization of biological vision is presented. In this model, the receptive fields of simple cells have differently oriented Gabor-type receptive field functions. Collectively, the simple cells in a hypercolumn extract an HC-vector from local sensory input serving as the place token for the scene registration. By incorporating a data driven dynamical tuning mechanism of simple cells, the place tokens are invariant to distortions. The initial experiments show that this biologically motivated method is accurate and robust.     

Fixational eye movements, natural image statistics, and fine spatial vision

Perception and motor control are often regarded as two separate branches of neuroscience. Like most species, however, humans are not passively exposed to the incoming flow of sensory data, but actively seek useful information. By shaping input signals in ways that simplify perceptual tasks, behavior might play an important role in establishing efficient sensory representations in the brain. Under natural viewing conditions, the main source of motion of the stimulus on the retina is not the scene but our own behavior. The retinal image is never still, even during visual fixation, when small eye movements combine with movements of the head and body to continually perturb the location of gaze. This article examines the impact of the fixational motion of the retinal image on the statistics of visual input and the neural encoding of visual information. Building upon recent theoretical and experimental results, it is argued that an unstable fixation constitutes an efficient strategy for acquiring information from natural scenes. According to this theory, the fluctuations of luminance caused by the incessant motion of the eye equalize the power present at different spatial frequencies in the spatiotemporal stimulus on the retina. This phenomenon yields compact neural representations, emphasizes fine spatial detail, and might enable a temporal multiplexing of visual information from the retina to the cortex. This theory posits motor contributions to early visual representations and suggests that perception and behavior are more intimately tied than commonly thought.     

基于V1细胞特性的边缘检测

针对均衡边缘检测精度和抗噪性能难度大的问题,借鉴初级视皮层（V1）细胞的动静态感知特性,建立具有方位选择性的V1细胞模型应用于图像边缘检测。采用时空滤波器来模拟简单细胞的感受野,通过使用能量模型和归一化来整合简单细胞的响应得到V1细胞模型,从而利用V1细胞静态感知特性来检测自然图像边缘。仿真结果表明,所提V1细胞模型能够基本拟合生物数据,具有生物上的普适性;与传统的边缘检测算子相比,该模型的性能更优,鲁棒性更强。依据生物实验结论来构建生物视觉模型并用于图像处理,对生物视觉和计算机视觉的融合进行了有益的探索。     

一种脑启发式的边缘检测模型

边缘是物体的基础特征,传统边缘检测方法具有一定的局限性。鉴于人类视觉系统能高效准确地感知物体的边缘信息,根据大脑侧膝体（Lateral Geniculate Nucleus,LGN）和初级视皮层（primary visual cortex,V1）简单细胞的感受野特性,提出一种脑启发式的前馈LGN-V1（Feedforward LGN-V1,FLV）视觉感知模型。首先用高斯函数之差模拟单个LGN细胞的同心圆式感受野,再通过同类LGN细胞的联合构建细胞组,最后将两类细胞组分别共线排列并平行放置模拟得到特定偏好朝向V1简单细胞。通过多简单细胞响应的整合获取全体V1简单细胞的响应。实验结果表明,FLV模型能体现真实简单细胞的生物特性。较传统的边缘检测方法而言,所提模型效果更优,具有更好的鲁棒性。     

Active vision and receptive field development in evolutionary robots

In this paper, we describe the artificial evolution of adaptive neural controllers for an outdoor mobile robot equipped with a mobile camera. The robot can dynamically select the gazing direction by moving the body and/or the camera. The neural control system, which maps visual information to motor commands, is evolved online by means of a genetic algorithm, but the synaptic connections (receptive fields) from visual photoreceptors to internal neurons can also be modified by Hebbian plasticity while the robot moves in the environment. We show that robots evolved in physics-based simulations with Hebbian visual plasticity display more robust adaptive behavior when transferred to real outdoor environments as compared to robots evolved without visual plasticity. We also show that the formation of visual receptive fields is significantly and consistently affected by active vision as compared to the formation of receptive fields with grid sample images in the environment of the robot. Finally, we show that the interplay between active vision and receptive field formation amounts to the selection and exploitation of a small and constant subset of visual features available to the robot.     

Efficient encoding of natural optic flow

Statistically efficient processing schemes focus the resources of a signal processing system on the range of statistically probable signals. Relying on the statistical properties of retinal motion signals during ego-motion we propose a nonlinear processing scheme for retinal flow. It maximizes the mutual information between the visual input and its neural representation, and distributes the processing load uniformly over the neural resources. We derive predictions for the receptive fields of motion sensitive neurons in the velocity space. The properties of the receptive fields are tightly connected to their position in the visual field, and to their preferred retinal velocity. The velocity tuning properties show characteristics of properties of neurons in the motion processing pathway of the primate brain.     

Spatiochromatic receptive field properties derived from information-theoretic analyses of cone mosaic responses to natural scenes

Neurons in the early stages of processing in the primate visual system efficiently encode natural scenes. In previous studies of the chromatic properties of natural images, the inputs were sampled on a regular array, with complete color information at every location. However, in the retina cone photoreceptors with different spectral sensitivities are arranged in a mosaic. We used an unsupervised neural network model to analyze the statistical structure of retinal cone mosaic responses to calibrated color natural images. The second-order statistical dependencies derived from the covariance matrix of the sensory signals were removed in the first stage of processing. These decorrelating filters were similar to type I receptive fields in parvo- or konio-cellular LGN in both spatial and chromatic characteristics. In the subsequent stage, the decorrelated signals were linearly transformed to make the output as statistically independent as possible, using independent component analysis. The independent component filters showed luminance selectivity with simple-cell-like receptive fields, or had strong color selectivity with large, often double-opponent, receptive fields, both of which were found in the primary visual cortex (V1). These results show that the "form" and "color" channels of the early visual system can be derived from the statistics of sensory signals.     

Replicating receptive fields of simple and complex cells in primary visual cortex in a neuronal network model with temporal and population sparseness and reliability

We propose a new principle for replicating receptive field properties of neurons in the primary visual cortex. We derive a learning rule for a feedforward network, which maintains a low firing rate for the output neurons (resulting in temporal sparseness) and allows only a small subset of the neurons in the network to fire at any given time (resulting in population sparseness). Our learning rule also sets the firing rates of the output neurons at each time step to near-maximum or near-minimum levels, resulting in neuronal reliability. The learning rule is simple enough to be written in spatially and temporally local forms. After the learning stage is performed using input image patches of natural scenes, output neurons in the model network are found to exhibit simple-cell-like receptive field properties. When the output of these simple-cell-like neurons are input to another model layer using the same learning rule, the second-layer output neurons after learning become less sensitive to the phase of gratings than the simple-cell-like input neurons. In particular, some of the second-layer output neurons become completely phase invariant, owing to the convergence of the connections from first-layer neurons with similar orientation selectivity to second-layer neurons in the model network. We examine the parameter dependencies of the receptive field properties of the model neurons after learning and discuss their biological implications. We also show that the localized learning rule is consistent with experimental results concerning neuronal plasticity and can replicate the receptive fields of simple and complex cells.     

Self-organisation of symmetry networks: transformation invariance from the spontaneous symmetry-breaking mechanism

Symmetry networks use permutation symmetries among synaptic weights to achieve transformation-invariant response. This article proposes a generic mechanism by which such symmetries can develop during unsupervised adaptation: it is shown analytically that spontaneous symmetry breaking can result in the discovery of unknown invariances of the data's probability distribution. It is proposed that a role of sparse coding is to facilitate the discovery of statistical invariances by this mechanism. It is demonstrated that the statistical dependences that exist between simple-cell-like threshold feature detectors, when exposed to temporally uncorrelated natural image data, can drive the development of complex-cell-like invariances, via single-cell Hebbian adaptation. A single learning rule can generate both simple-cell-like and complex-cell-like receptive fields.     

基于主视通路结构分级响应模型的轮廓检测方法EI北大核心CSCD

基于视通路结构分级响应与动态传递的方式,本文提出了一种图像轮廓检测的新方法.针对视网膜感光细胞的暗视觉特性,建立亮度自适应的暗视野调节模型,利用多尺度经典感受野的方位选择性,构建高级轮廓与全局轮廓的检测路径;模拟外侧膝状体(Lateral geniculate nucleus,LGN)细胞特性对信息进行纹理稀疏编码,并结合非经典感受野的侧抑制作用抑制背景强纹理;另外在LGN区提出微动整合机制,减少纹理冗余信息,再经适应性突触实现信息关联传递;最后将初级轮廓响应跨视区前馈至V1区并经全局轮廓修正后,与高级轮廓响应实现快速融合.分别以RuG40、BSDS500图像库中的自然图像作为实验数据,检测结果与基准轮廓图的平均最优P指标分别为0.50、0.32,结果表明本方法能更有效地区分轮廓与纹理边缘,凸显主体轮廓.本文利用视神经细胞的内在机制以及神经信息的动态传递过程实现图像轮廓信息的编码与检测,也为研究后续高级视皮层的视觉感知提供了新思路.     

A new simple /spl infin/OH neuron model as a biologically plausible principal component analyzer

A new approach to unsupervised learning in a single-layer neural network is discussed. An algorithm for unsupervised learning based upon the Hebbian learning rule is presented. A simple neuron model is analyzed. A dynamic neural model, which contains both feed-forward and feedback connections between the input and the output, has been adopted. The, proposed learning algorithm could be more correctly named self-supervised rather than unsupervised. The solution proposed here is a modified Hebbian rule, in which the modification of the synaptic strength is proportional not to pre- and postsynaptic activity, but instead to the presynaptic and averaged value of postsynaptic activity. It is shown that the model neuron tends to extract the principal component from a stationary input vector sequence. Usually accepted additional decaying terms for the stabilization of the original Hebbian rule are avoided. Implementation of the basic Hebbian scheme would not lead to unrealistic growth of the synaptic strengths, thanks to the adopted network structure.     

Simple-cell-like receptive fields maximize temporal coherence in natural video

Recently, statistical models of natural images have shown the emergence of several properties of the visual cortex. Most models have considered the nongaussian properties of static image patches, leading to sparse coding or independent component analysis. Here we consider the basic time dependencies of image sequences instead of their nongaussianity. We show that simple-cell-type receptive fields emerge when temporal response strength correlation is maximized for natural image sequences. Thus, temporal response strength correlation, which is a nonlinear measure of temporal coherence, provides an alternative to sparseness in modeling simple-cell receptive field properties. Our results also suggest an interpretation of simple cells in terms of invariant coding principles, which have previously been used to explain complex-cell receptive fields.     

A Learning Rule to Model the Development of Orientation Selectivity in Visual Cortex

This paper presents a learning rule, CBA, to develop oriented receptive fields similar to those founded in cat striate cortex. The inherent complexity of the development of selectivity in visual cortex has led most authors to test their models by using a restricted input environment. Only recently, some learning rules (the PCA and the BCM rules) have been studied in a realistic visual environment. For these rules, which are based upon Hebbian learning, single neuron models have been proposed in order to get a better understanding of their properties and dynamics. These models suffered from unbounded growing of synaptic strength, which is remedied by a normalization process. However, normalization seems biologically implausible, given the non-local nature of this process. A detailed stability analysis of the proposed rule proves that the CBA attains a stable state without any need for normalization. Also, a comparison among the results achieved in different types of visual environments by the PCA, the BCM and the CBA rules is provided. The final results show that the CBA rule is appropriate for studying the biological process of receptive field formation and its application in image processing and artificial vision tasks.     

Hebbian-based neural networks for bottom-up visual attention and its applications to ship detection in SAR images

This paper proposes a bottom-up attention model based on pulsed Hebbian neural networks. The salience of the visual input can be generated through the networks using a simple normalization process, which can be calculated rapidly. Moreover, visual salience in this model can be represented as binary codes that mimic neuronal pulses in the human brain. Experimental results on psychophysical patterns and eye fixation prediction for natural images prove the effectiveness and efficiency of the model. In an arduous task of detecting ships in synthetic aperture radar (SAR) images, there are large amounts of data to be processed in real time. As a fast and effective technique for saliency detection, the proposed model is applied to ship detection in SAR images and its robustness against speckles is further proved.