Learning lateral interactions for feature binding and sensory segmentation from prototypic basis interactions

We present a hybrid learning method bridging the fields of recurrent neural networks, unsupervised Hebbian learning, vector quantization, and supervised learning to implement a sophisticated image and feature segmentation architecture. This architecture is based on the competitive layer model (CLM), a dynamic feature binding model, which is applicable on a wide range of perceptual grouping and segmentation problems. A predefined target segmentation can be achieved as attractor states of this linear threshold recurrent network, if the lateral weights are chosen by Hebbian learning. The weight matrix is given by the correlation matrix of special pattern vectors with a structure dependent on the target labeling. Generalization is achieved by applying vector quantization on pair-wise feature relations, like proximity and similarity, defined by external knowledge. We show the successful application of the method to a number of artificial test examples and a medical image segmentation problem of fluorescence microscope cell images.     

Integrating feature maps and competitive layer architectures for motion segmentation

We present a generic approach to integrate feature maps with a competitive layer architecture to enable segmentation by a competitive neural dynamics specified in terms of the latent space mappings constructed by the feature maps. We demonstrate the underlying ideas for the case of motion segmentation, using a system that employs Unsupervised Kernel Regression (UKR) for the creation of the feature maps, and the Competitive Layer Model (CLM) for the competitive layer architecture. The UKR feature maps hold learned representations of a set of candidate motions and the CLM dynamics, working on features defined in the UKR domain, implements the segmentation of observed trajectory data according to the competing candidates. We also demonstrate how the introduction of an additional layer can provide the system with a parametrizable rejection mechanism for previously unknown observations. The evaluation on trajectories describing four different letters yields improved classification results compared to our previous, pure manifold approach.     

Neural-network models for the blood glucose metabolism of adiabetic

We study the application of neural networks to modeling the blood glucose metabolism of a diabetic. In particular we consider recurrent neural networks and time series convolution neural networks which we compare to linear models and to nonlinear compartment models. We include a linear error model to take into account the uncertainty in the system and for handling missing blood glucose observations. Our results indicate that best performance can be achieved by the combination of the recurrent neural network and the linear error model.     

Dynamic recurrent neural networks: a dynamical analysis

In this paper, we explore the dynamical features of a neural network model which presents two types of adaptative parameters: the classical weights between the units and the time constants associated with each artificial neuron. The purpose of this study is to provide a strong theoretical basis for modeling and simulating dynamic recurrent neural networks. In order to achieve this, we study the effect of the statistical distribution of the weights and of the time constants on the network dynamics and we make a statistical analysis of the neural transformation. We examine the network power spectra (to draw some conclusions over the frequential behaviour of the network) and we compute the stability regions to explore the stability of the model. We show that the network is sensitive to the variations of the mean values of the weights and the time constants (because of the temporal aspects of the learned tasks). Nevertheless, our results highlight the improvements in the network dynamics due to the introduction of adaptative time constants and indicate that dynamic recurrent neural networks can bring new powerful features in the field of neural computing.     

Medical image segmentation using a contextual-constraint-based Hopfield neural cube

Neural-network-based image techniques such as the Hopfield neural networks have been proposed as an alternative approach for image segmentation and have demonstrated benefits over traditional algorithms. However, due to its architecture limitation, image segmentation using traditional Hopfield neural networks results in the same function as thresholding of image histograms. With this technique high-level contextual information cannot be incorporated into the segmentation procedure. As a result, although the traditional Hopfield neural network was capable of segmenting noiseless images, it lacks the capability of noise robustness. In this paper, an innovative Hopfield neural network, called contextual-constraint-based Hopfield neural cube (CCBHNC) is proposed for image segmentation. The CCBHNC uses a three-dimensional architecture with pixel classification implemented on its third dimension. With the three-dimensional architecture, the network is capable of taking into account each pixel's feature and its surrounding contextual information. Besides the network architecture, the CCBHNC also differs from the original Hopfield neural network in that a competitive winner-take-all mechanism is imposed in the evolution of the network. The winner-take-all mechanism adeptly precludes the necessity of determining the values for the weighting factors for the hard constraints in the energy function in maintaining feasible results. The proposed CCBHNC approach for image segmentation has been compared with two existing methods. The simulation results indicate that CCBHNC can produce more continuous, and smoother images in comparison with the other methods.     

Computational design and nonlinear dynamics of a recurrent network model of the primary visual cortex

Recurrent interactions in the primary visual cortex make its output a complex nonlinear transform of its input. This transform serves preattentive visual segmentation, that is, autonomously processing visual inputs to give outputs that selectively emphasize certain features for segmentation. An analytical understanding of the nonlinear dynamics of the recurrent neural circuit is essential to harness its computational power. We derive requirements on the neural architecture, components, and connection weights of a biologically plausible model of the cortex such that region segmentation, figure-ground segregation, and contour enhancement can be achieved simultaneously. In addition, we analyze the conditions governing neural oscillations, illusory contours, and the absence of visual hallucinations. Many of our analytical techniques can be applied to other recurrent networks with translation-invariant neural and connection structures.     

A new method based on the CLM of the LV RNN for brain MR image segmentation

A new method based on the competitive layer model (CLM) implemented by Lotka–Volterra recurrent neural networks (LV RNNs) is proposed for brain MR image segmentation. This method firstly divides an MR image into sub-images, and segments each sub-image by the CLM of the LV RNN to obtain a lot of 4-connected regions. Secondly, any two neighboring regions that are similar to each other are merged to form one region. Finally, all remaining regions are clustered by the RFCM into background, CSF, GM and WM. Compared with other three methods using numerical simulations, our method is shown to be more effective.     

Feature extraction and image segmentation using self-organizing networks

Feature extraction and image segmentation (FEIS) are two primary goals of almost all image-understanding systems. They are also the issues at which we look in this paper. We think of FEIS as a multilevel process of grouping and describing at each level. We emphasize the importance of grouping during this process because we believe that many features and events in real images are only perceived by combining weak evidence of several organized pixels or other low-level features. To realize FEIS based on this formulation, we must deal with such problems as how to discover grouping rules, how to develop grouping systems to integrate grouping rules, how to embed grouping processes into FEIS systems, and how to evaluate the quality of extracted features at various levels. We use self-organizing networks to develop grouping systems that take the organization of human visual perception into consideration. We demonstrate our approach by solving two concrete problems: extracting linear features in digital images and partitioning color images into regions. We present the results of experiments on real images.     

基于ACNNC模型的中文分词方法

目前,现有中文分词模型大多是基于循环神经网络的,其能够捕捉序列整体特征,但存在忽略了局部特征的问题。针对这种问题,该文综合了注意力机制、卷积神经网络和条件随机场,提出了注意力卷积神经网络条件随机场模型(Attention Convolutional Neural Network CRF, ACNNC)。其中,嵌入层训练词向量,自注意力层代替循环神经网络捕捉序列全局特征,卷积神经网络捕捉序列局部特征和位置特征,经融合层的特征输入条件随机场进行解码。实验证明该文提出的模型在BACKOFF 2005测试集上有更好的分词效果,并在PKU、MSR、CITYU和AS上取得了96.2%、96.4%、96.1%和95.8%的F₁值。     

Using competitive layer model implemented by Lotka–Volterra recurrent neural networks for detecting brain activated regions from fMRI data

The competitive layer model (CLM) implemented by the Lotka–Volterra recurrent neural networks (LV RNNs) is prominently characterized by its capability of binding neurons with similar feature into the same layer by competing among neurons at different layers in a column. This paper proposes to use the CLM of the LV RNN for detecting brain activated regions from the fMRI data. The correlated voxels from brain fMRI data can be obtained, and the clusters from fMRI time series can be uncovered. Experiments on synthetic and real fMRI data demonstrate the effectiveness of binding activated voxels into the ‘active’ layers of the CLM. The activated voxels can be detected more accurately than some existing methods by the proposed method.

     

Dynamics of Winner-Take-All Competition in Recurrent Neural Networks With Lateral Inhibition

This paper studies the behavior of recurrent neural networks with lateral inhibition. Such network architecture is important in biological neural systems. General conditions determining the existence, number, and stability of network equilibria are derived. The manner in which these features depend upon steepness of neuronal activation functions and the strength of lateral inhibition is demonstrated for a broad range of nondecreasing activation functions including the discontinuous threshold function which represents the infinite gain limit. For uniform lateral inhibitory networks, the lateral inhibition is shown to sharpen neuron output patterns by increasing separation of suprathreshold activity levels of competing neurons. This results in the tendency of one neuron's output to dominate those of the others which can afford a "winner-take-all" (WTA) mechanism. Importantly, multiple stable equilibria may exist and shifts in inputs levels may yield network state transitions that exhibit hysteresis. A limitation of using lateral inhibition to implement WTA is further demonstrated. The possible significance of these identified network dynamics to physiology and pathophysiology of the striatum (particularly in Parkinsonian rest tremor) is discussed     

General-purpose computation with neural networks: a survey of complexity theoretic results

We survey and summarize the literature on the computational aspects of neural network models by presenting a detailed taxonomy of the various models according to their complexity theoretic characteristics. The criteria of classification include the architecture of the network (feedforward versus recurrent), time model (discrete versus continuous), state type (binary versus analog), weight constraints (symmetric versus asymmetric), network size (finite nets versus infinite families), and computation type (deterministic versus probabilistic), among others. The underlying results concerning the computational power and complexity issues of perceptron, radial basis function, winner-take-all, and spiking neural networks are briefly surveyed, with pointers to the relevant literature. In our survey, we focus mainly on the digital computation whose inputs and outputs are binary in nature, although their values are quite often encoded as analog neuron states. We omit the important learning issues.     

A winner-take-all Lotka–Volterra recurrent neural network with only one winner in each row and each column

A winner-take-all Lotka–Volterra recurrent neural network with N × N neurons is proposed in this paper. Sufficient conditions for existence of winner-take-all stable equilibrium points in the network are obtained. These conditions guarantee that there is one and only one winner in each row and each column at any stable equilibrium point. In addition, rigorous convergence analysis is carried out. It is proven that the proposed network model is convergent. The conditions for the winner-take-all behavior obtained in this paper provide design guidelines for network implementation and fabrication. Simulations are also presented to illustrate the theoretical findings.     

残差密集空间金字塔网络的城市遥感图像分割

目的 遥感图像语义分割是根据土地覆盖类型对图像中每个像素进行分类,是遥感图像处理领域的一个重要研究方向。由于遥感图像包含的地物尺度差别大、地物边界复杂等原因,准确提取遥感图像特征具有一定难度,使得精确分割遥感图像比较困难。卷积神经网络因其自主分层提取图像特征的特点逐步成为图像处理领域的主流算法,本文将基于残差密集空间金字塔的卷积神经网络应用于城市地区遥感图像分割,以提升高分辨率城市地区遥感影像语义分割的精度。方法 模型将带孔卷积引入残差网络,代替网络中的下采样操作,在扩大特征图感受野的同时能够保持特征图尺寸不变;模型基于密集连接机制级联空间金字塔结构各分支,每个分支的输出都有更加密集的感受野信息;模型利用跳线连接跨层融合网络特征,结合网络中的高层语义特征和低层纹理特征恢复空间信息。结果 基于ISPRS （International Society for Photogrammetry and Remote Sensing） Vaihingen地区遥感数据集展开充分的实验研究,实验结果表明,本文模型在6种不同的地物分类上的平均交并比和平均F₁值分别达到69.88%和81.39%,性能在数学指标和视觉效果上均优于SegNet、pix2pix、Res-shuffling-Net以及SDFCN （symmetrical dense-shortcut fully convolutional network）算法。结论 将密集连接改进空间金字塔池化网络应用于高分辨率遥感图像语义分割,该模型利用了遥感图像不同尺度下的特征、高层语义信息和低层纹理信息,有效提升了城市地区遥感图像分割精度。     

A Proposed Hybrid Recurrent Neural Control System for Two Co-operating Robots

This paper discusses a model refernce adaptive (MRAC) position/force controller using proposed neural networks for two co-operating planar robots. The proposed neural network is a recurrent hybrid network. The recurrent networks have feedback connections and thus an inherent memory for dynamics, which makes them suitable for representing dynamic systems. A feature of the networks adopted is their hybrid hidden layer, which includes both linear and nonlinear neurons. On the other hand, the results of the case of a single robot under position control alone are presented for comparison. The results presented show the superior ability of the proposed neural network based model reference adaptive control scheme at adapting to changes in the dynamics parameters of robots.     

基于随机退火的神经网络学习算法及其应用

提出了一种基于随机退火机制的竞争层神经网络学习算法,并将其应用于解决图像特征绑定问题。该算法将竞争层神经网络的串行迭代模式改为随机优化模式,通过采用退火技术避免网络收敛到能量函数的局部极小点。通过理论分析证明了该算法与竞争层神经网络动力学方程的等价性。通过对比实验验证了算法能够在加快网络收敛速度的同时提高特征绑定结果的 合理性。     

An ANN based approach to recognize initial phonemes of spoken words of Assamese language

Initial phoneme is used in spoken word recognition models. These are used to activate words starting with that phoneme in spoken word recognition models. Such investigations are critical for classification of initial phoneme into a phonetic group. A work is described in this paper using an artificial neural network (ANN) based approach to recognize initial consonant phonemes of Assamese words. A self organizing map (SOM) based algorithm is developed to segment the initial phonemes from its word counterpart. Using a combination of three types of ANN structures, namely recurrent neural network (RNN), SOM and probabilistic neural network (PNN), the proposed algorithm proves its superiority over the conventional discrete wavelet transform (DWT) based phoneme segmentation. The algorithm is exclusively designed on the basis of Assamese phonemical structure which consists of certain unique features and are grouped into six distinct phoneme families. Before applying the segmentation approach using SOM, an RNN is used to take some localized decision to classify the words into six phoneme families. Next the SOM segmented phonemes are classified into individual phonemes. A two-class PNN classification is performed with clean Assamese phonemes, to recognize the segmented phonemes. The validation of recognized phonemes is checked by matching the first formant frequency of the phoneme. Formant frequency of Assamese phonemes, estimated using the pole or formant location determination from the linear prediction model of vocal tract, is used effectively as a priori knowledge in the proposed algorithm.     

混沌神经网络及其在最优化问题中的应用

首先评述了三种混沌神经网络模型,然后提出了一种新的混沌模拟退火算法。其次将四种方法分别应用于10个城市的施行推销商问题。文中给出了每一模型神经元输出和能量函数随时间演变过程曲线。根据仿真结果,讨论了四种方法的特性与有效。其结论为：提出的模拟退火神经网络比其它网络模型更能获得全局最小解。     

Weight adaptation and oscillatory correlation for imagesegmentation

We propose a method for image segmentation based on a neural oscillator network. Unlike previous methods, weight adaptation is adopted during segmentation to remove noise and preserve significant discontinuities in an image. Moreover, a logarithmic grouping rule is proposed to facilitate grouping of oscillators representing pixels with coherent properties. We show that weight adaptation plays the roles of noise removal and feature preservation. In particular, our weight adaptation scheme is insensitive to termination time and the resulting dynamic weights in a wide range of iterations lead to the same segmentation results. A computer algorithm derived from oscillatory dynamics is applied to synthetic and real images, and simulation results show that the algorithm yields favorable segmentation results in comparison with other recent algorithms. In addition, the weight adaptation scheme can be directly transformed to a novel feature-preserving smoothing procedure. We also demonstrate that our nonlinear smoothing algorithm achieves good results for various kinds of images.     

Modeling of inhibition/excitation firing in olfactory bulb through spiking neurons

Spiking neural systems are based on biologically inspired neural models of computation since they take into account the precise timing of spike events and therefore are suitable to analyze dynamical aspects of neuronal signal transmission. These systems gained increasing interest because they are more sophisticated than simple neuron models found in artificial neural systems; they are closer to biophysical models of neurons, synapses, and related elements and their synchronized firing of neuronal assemblies could serve the brain as a code for feature binding and pattern segmentation. The simulations are designed to exemplify certain properties of the olfactory bulb (OB) dynamics and are based on an extension of the integrate-and-fire (IF) neuron, and the idea of locally coupled excitation and inhibition cells. We introduce the background theory to making an appropriate choice of model parameters. The following two forms of connectivity offering certain computational and analytical advantages, either through symmetry or statistical properties in the study of OB dynamics have been used:

• all-to-all coupling,
• receptive field style coupling.

Our simulations showed that the inter-neuron transmission delay controls the size of spatial variations of the input and also smoothes the network response. Our IF extended model proves to be a useful basis from which we can study more sophisticated features as complex pattern formation, and global stability and chaos of OB dynamics.