Dynamical behavior of autoassociative memory performing noveltyfiltering for signal enhancement

This paper deals with the dynamical behavior, in probabilistic sense, of a simple perceptron network with sigmoidal output units performing autoassociation for novelty filtering. Networks of retinotopic topology having a one-to-one correspondence between input and output units can be readily trained using the delta learning rule, to perform autoassociative mappings. A novelty filter is obtained by subtracting the network output from the input vector. Then the presentation of a "familiar" pattern tends to evoke a null response; but any anomalous component is enhanced. Such a behavior exhibits a promising feature for enhancement of weak signals in additive noise. This paper shows that the probability density function of the weight converges to Gaussian when the input time series is statistically characterized by nonsymmetrical probability density functions. It is shown that the probability density function of the weight satisfies the Fokker-Planck equation. By solving the Fokker-Planck equation, it is found that the weight is Gaussian distributed with time dependent mean and variance.     

脉冲噪声环境下的韧性匹配滤波检测方法

针对传统局部最优匹配滤波检测性能在脉冲噪声环境下会明显退化的问题,提出一种韧性匹配滤波检测方法。利用噪声分布概率密度函数求出非线性变换公式,对观测信号作非线性变换和匹配滤波检测。应用于BPSK调制信号的匹配检测滤波信号仿真表明,该算法在抑制脉冲噪声方面对噪声特征指数具有良好韧性,在脉冲噪声环境下比传统局部最优匹配滤波方法具有更好的检测性能。     

Edge preservation of impulse noise filtered images by improved anisotropic diffusion

This paper provides a robust scheme for random valued impulsive noise reduction along with edge preservation by anisotropic diffusion with improved diffusivity. The defective impulse noisy pixels are detected by Laplacian based second order pixel difference operation where these defective pixels are replaced by appropriate values with regard of the gray level of their four directional neighbors. This de-noised image undergoes the diffusion operation where diffusion coefficient function is modified to make it adaptive by incorporating local gray level variance information. The proposed modified diffusion scheme effectively restore the edges and fine details destroyed during impulse noise reduction process. The effect of proposed diffusion scheme has been studied on various images and the results are compared with some existing diffusion methods which are independently used for impulse noise reduction and edge preservation. The results shows that the prior removal of impulsive noise before the application of diffusion process is advantageous over the direct application of diffusion for removing the impulsive noise. In addition, the results of the proposed diffusion scheme are compared with some of the median filter based methods which are effectively used for impulse noise reduction without caring of edge preservation. The proposed diffusion scheme sufficiently preserves the edges without boosting of impulsive noise components on images corrupted up to 50 % of the impulsive noise density.     

Noise in integrate-and-fire neurons: from stochastic input to escape rates

We analyze the effect of noise in integrate-and-fire neurons driven by time-dependent input and compare the diffusion approximation for the membrane potential to escape noise. It is shown that for time-dependent subthreshold input, diffusive noise can be replaced by escape noise with a hazard function that has a gaussian dependence on the distance between the (noise-free) membrane voltage and threshold. The approximation is improved if we add to the hazard function a probability current proportional to the derivative of the voltage. Stochastic resonance in response to periodic input occurs in both noise models and exhibits similar characteristics.     

Stochastic Resonance in Continuous and Spiking Neuron Models With Levy Noise

Levy noise can help neurons detect faint or subthreshold signals. Levy noise extends standard Brownian noise to many types of impulsive jump-noise processes found in real and model neurons as well as in models of finance and other random phenomena. Two new theorems and the ItÔ calculus show that white Levy noise will benefit subthreshold neuronal signal detection if the noise process's scaled drift velocity falls inside an interval that depends on the threshold values. These results generalize earlier “forbidden interval” theorems of neuronal “stochastic resonance” (SR) or noise-injection benefits. Global and local Lipschitz conditions imply that additive white Levy noise can increase the mutual information or bit count of several feedback neuron models that obey a general stochastic differential equation (SDE). Simulation results show that the same noise benefits still occur for some infinite-variance stable Levy noise processes even though the theorems themselves apply only to finite-variance Levy noise. The Appendix proves the two ItÔ-theoretic lemmas that underlie the new Levy noise-benefit theorems.      

极大并联阈值网络中噪声改善信号的相关性

讨论了极大并联阈值网络中噪声改善信号相关性问题。当输入噪声为单峰高斯噪声时，输入信号在阈下时噪声才能改善信号的相关性，即随机谐振现象存在。而当输入噪声为双峰高斯混合噪声时，不仅输入信号在阈下时随机谐振现象有时存在，而且输入信号在阈上时噪声往往也能改善信号的相关性，即阈上随机谐振现象存在。噪声改善信号相关性随着网络中单元数的调整而改善。这些结果进一步说明了随机谐振或阈上随机谐振对噪声分布的依赖性，同时也拓广了随机谐振或阈上随机谐振在数字信号处理方面的应用。     

Analytic expressions for rate and CV of a type I neuron driven by white gaussian noise

We study the one-dimensional normal form of a saddle-node system under the influence of additive gaussian white noise and a static "bias current" input parameter, a model that can be looked upon as the simplest version of a type I neuron with stochastic input. This is in contrast with the numerous studies devoted to the noise-driven leaky integrate-and-fire neuron. We focus on the firing rate and coefficient of variation (CV) of the interspike interval density, for which scaling relations with respect to the input parameter and noise intensity are derived. Quadrature formulas for rate and CV are numerically evaluated and compared to numerical simulations of the system and to various approximation formulas obtained in different limiting cases of the model. We also show that caution must be used to extend these results to the Theta neuron model with multiplicative gaussian white noise. The correspondence between the first passage time statistics for the saddle-node model and the Theta neuron model is obtained only in the Stratonovich interpretation of the stochastic Theta neuron model, while previous results have focused only on the Ito interpretation. The correct Stratonovich interpretation yields CVs that are still relatively high, although smaller than in the Ito interpretation; it also produces certain qualitative differences, especially at larger noise intensities. Our analysis provides useful relations for assessing the distance to threshold and the level of synaptic noise in real type I neurons from their firing statistics. We also briefly discuss the effect of finite boundaries (finite values of threshold and reset) on the firing statistics.     

Learning sensory maps with real-world stimuli in real time using abiophysically realistic learning rule

We present a real-time model of learning in the auditory cortex that is trained using real-world stimuli. The system consists of a peripheral and a central cortical network of spiking neurons. The synapses formed by peripheral neurons on the central ones are subject to synaptic plasticity. We implemented a biophysically realistic learning rule that depends on the precise temporal relation of pre- and postsynaptic action potentials. We demonstrate that this biologically realistic real-time neuronal system forms stable receptive fields that accurately reflect the spectral content of the input signals and that the size of these representations can be biased by global signals acting on the local learning mechanism. In addition, we show that this learning mechanism shows fast acquisition and is robust in the presence of large imbalances in the probability of occurrence of individual stimuli and noise.     

Histogram-based fuzzy filter for image restoration

In this paper, we present a novel approach to the restoration of noise-corrupted image, which is particularly effective at removing highly impulsive noise while preserving image details. This is accomplished through a fuzzy smoothing filter constructed from a set of fuzzy membership functions for which the initial parameters are derived in accordance with input histogram. A principle of conservation in histogram potential is incorporated with input statistics to adjust the initial parameters so as to minimize the discrepancy between a reference intensity and the output of defuzzification process. Similar to median filters (MF), the proposed filter has the benefits that it is simple and it assumes no a priori knowledge of specific input image, yet it shows superior performance over conventional filters (including MF) for the full range of impulsive noise probability. Unlike in many neuro-fuzzy or fuzzy-neuro filters where random strategy is employed to choose initial membership functions for subsequent lengthy training, the proposed filter can achieve satisfactory performance without any training.     

分数极点系统中非高斯稳定噪声的逆滤波方法

!-稳定分布可以更好地描述实际应用中所遇到的具有显著脉冲特性的随机信号和噪声。!稳定分布没有统一闭式的概率密度函数,其二阶及二阶以上统计量均不存在。针对分数极点系统中存在的独立S!S噪声,论文提出一种分数极点系统中稳定分布噪声的逆滤波方法,并分析了算法的长记忆、最小相位、收敛特性。计算机模拟实验结果表明,这种算法是一种在S!S分布噪声条件下具有良好韧性的逆滤波方法。     

一阶自回归模型中噪声改善信号的相关性

讨论一阶自回归模型中三种典型噪声改善信号的相关性问题。当输入信号在阈下或部分在阈下时,随着噪声强度的增加,输出信号与输入信号的相关系数先递增后递减,适量的噪声改善了信号的相关性,随机谐振现象存在。随着阈值的增加,随机谐振功效降低、最佳噪声值变大;随着噪声密度函数在零均值周边脉冲值变大和拖尾变厚,随机谐振功效也降低。存在一个噪声范围,其间输入信号与输出信号相关系数大于输入信号与噪声信号相关系数,一阶自回归模型中输出信号比噪声信号与输入信号更相关。这些结果说明在离散时间系统中噪声改善信号的相关性,随机谐振现象存在,且随机谐振对噪声具有鲁棒性。这些结果也拓广了随机谐振在数字信号处理中的应用。     

An adaptive conscientious competitive learning algorithm and its applications

An adaptive conscientious competitive learning (ACCL) algorithm is proposed in this paper. The ACCL algorithm can adjust the conscience parameter itself according to the feedback information about the practical winning situation of all neurons during the learning process. The a priori information about the distribution range of the input patterns which is required for the conventional conscientious competitive learning (CCL) algorithm, is no longer required in the ACCL algorithm. The “neurons get stuck” problem of the competitive learning (CL) algorithm and conscientious competitive learning (CCL) algorithm with small conscience parameter is overcome. At the same time, neurons will not be tangled together as in the case of the CCL algorithm with large conscience parameter. The ACCL algorithm is applied to vector quantization (VQ) and probability density function estimation (PDFE). It can generate better results than the conventional CL and CCL algorithms. Experimental results are also included to demonstrate its effectiveness.     

Spatiotemporal spike encoding of a continuous external signal

Interspike intervals of spikes emitted from an integrator neuron model of sensory neurons can encode input information represented as a continuous signal from a deterministic system. If a real brain uses spike timing as a means of information processing, other neurons receiving spatiotemporal spikes from such sensory neurons must also be capable of treating information included in deterministic interspike intervals. In this article, we examine functions of neurons modeling cortical neurons receiving spatiotemporal spikes from many sensory neurons. We show that such neuron models can encode stimulus information passed from the sensory model neurons in the form of interspike intervals. Each sensory neuron connected to the cortical neuron contributes equally to the information collection by the cortical neuron. Although the incident spike train to the cortical neuron is a superimposition of spike trains from many sensory neurons, it need not be decomposed into spike trains according to the input neurons. These results are also preserved for generalizations of sensory neurons such as a small amount of leak, noise, inhomogeneity in firing rates, or biases introduced in the phase distributions.     

Robust control of the output probability density functions formultivariable stochastic systems with guaranteed stability

Presents two robust solutions to the control of the output probability density function for general multi-input and multi-output stochastic systems. The control inputs of the system appear as a set of variables in the probability density functions of the system output, and the signal available to the controller is the measured probability density function of the system output. A type of dynamic probability density model is formulated by using a B-spline neural network with all its weights dynamically related to the control input. It has been shown that the so-formed robust control algorithms can control the shape of the output probability density function and can guaranteed the closed-loop stability when the system is subjected to a bounded unknown input. An illustrative example is included to demonstrate the use of the developed control algorithms, and desired results have been obtained     

突触噪声作用下的IF阈值神经元模型的随机共振

基于带阈值的积分放电模型研究了神经元在突触递质噪声和周期信号驱动下的随机共振现象.利用平均法得到系统输出幅值增益的精确表达式,考察了输出幅值增益与信号频率、噪声强度、相关时间及非对称度的关系.发现输出幅值增益随着这些参量的演化曲线在一定条件下呈非单调的,这些都表明在突触递质噪声和周期信号驱动下的神经发放确实存在随机共振现象.     

Delay-dependent fault detection and diagnosis using B-spline neural networks and nonlinear filters for time-delay stochastic systems

A new fault detection and diagnosis (FDD) scheme is studied in this paper for the continuous-time stochastic dynamic systems with time delays, where the available information for the FDD is the input and the measured output probability density functions (PDFs) of the system. The square-root B-spline neural networks is used to formulate the output PDFs with the dynamic weightings. As a result, the concerned FDD problem can be transformed into a robust FDD problem subjected to a continuous time uncertain nonlinear system with time delays. Delay-dependent criteria to detect and diagnose the system fault are provided by using linear matrix inequality (LMI) techniques. It is shown that this new criterion can provide higher sensitivity performance than the existing result. Simulations are given to demonstrate the efficiency of the proposed approach.     

Pipelined robust M-estimate adaptive second-order Volterra filter against impulsive noise

The main limits on adaptive Volterra filters are their computational complexity in practical implementation and significant performance degradation under the impulsive noise environment. In this paper, a low-complexity pipelined robust M-estimate second-order Volterra (PRMSOV) filter is proposed to reduce the computational burdens of the Volterra filter and enhance the robustness against impulsive noise. The PRMSOV filter consists of a number of extended second-order Volterra (SOV) modules without feedback input cascaded in a chained form. To apply to the modular architecture, the modified normalized least mean M-estimate (NLMM) algorithms are derived to suppress the effect of impulsive noise on the nonlinear and linear combiner subsections, respectively. Since the SOV-NLMM modules in the PRMSOV can operate simultaneously in a pipelined parallelism fashion, they can give a significant improvement of computational efficiency and robustness against impulsive noise. The stability and convergence on nonlinear and linear combiner subsections are also analyzed with the contaminated Gaussian (CG) noise model. Simulations on nonlinear system identification and speech prediction show the proposed PRMSOV filter has better performance than the conventional SOV filter and joint process pipelined SOV (JPPSOV) filter under impulsive noise environment. The initial convergence, steady-state error, robustness and computational complexity are also better than the SOV and JPPSOV filters.     

Stochastic resonance in parallel concatenated turbo code decoding

In an array of threshold devices, we examine the effect of noise in improving performance of turbo code decoding. Such a phenomenon of noise-enhanced effect is termed stochastic resonance (SR). When signal is subthreshold, SR is observed by using Gaussian noise during iterative decoding. That is, the minimal bit error ratio (BER) is achieved at some non-zero noise intensity level. Besides, the larger the number of threshold devices is, the more remarkable the SR effect becomes. Especially when noise intensity is nearly optimal, BER approximates to zero after a few decoding iterations. Moreover, when Gaussian mixture noise is utilized, suprathreshold stochastic resonance (SSR) occurs in turbo decoding. These results show the beneficial effect of noise in channel coding and decoding.     

基于改进遗传算法的图像小波阈值去噪研究

论文提出了一种基于改进遗传算法的图像小波阈值去噪方法。从理论上分析了小波阈值去噪的原理,并采用改进遗传算法来求小波变换各子带的最优阈值,计算时无需噪声方差等先验信息;通过综合交叉和随机变异,避免了人为确定交叉率和变异率,从而使算法更加稳健,在提高搜索效率的同时减少陷入局部最优的机会。实验结果表明,与普通的小波阈值去噪方法相比,该方法能较好地改善去噪后图像的视觉效果,提高峰值信噪比。     

Topographic map formation by maximizing unconditional entropy: aplausible strategy for “online” unsupervised competitivelearning and nonparametric density estimation

An unsupervised competitive learning rule, called the vectorial boundary adaptation rule (VBAR), is introduced for topographic map formation. Since VBAR is aimed at producing an equiprobable quantization of the input space, it yields a nonparametric model of the input probability density function. Furthermore, since equiprobable quantization is equivalent to unconditional entropy maximization, we argue that this is a plausible strategy for maximizing mutual information (Shannon information rate) in the case of "online" learning. We use mutual information as a tool for comparing the performance of our rule with Kohonen's self-organizing (feature) map algorithm.