Training neural networks with additive noise in the desired signal

A global optimization strategy for training adaptive systems such as neural networks and adaptive filters (finite or infinite impulse response) is proposed. Instead of adding random noise to the weights as proposed in the past, additive random noise is injected directly into the desired signal. Experimental results show that this procedure also speeds up greatly the backpropagation algorithm. The method is very easy to implement in practice, preserving the backpropagation algorithm and requiring a single random generator with a monotonically decreasing step size per output channel. Hence, this is an ideal strategy to speed up supervised learning, and avoid local minima entrapment when the noise variance is appropriately scheduled.     

Robust Blind Beamforming Algorithm Using Joint Multiple Matrix Diagonalization

The objective of the blind beamforming is to restore the unknown source signals simply based on the observations, without a priori knowledge of the source signals and the mixing matrix. In this paper, we propose a new joint multiple matrix diagonalization (JMMD) algorithm for the robust blind beamforming. This new JMMD algorithm is based on the iterative eigen decomposition of the fourth-order cumulant matrices. Therefore, it can avoid the problems of the stability and the misadjustment, which arise from the conventional steepest-descent approaches for the constant-modulus or cumulant optimization. Our Monte Carlo simulations show that our proposed algorithm significantly outperforms the ubiquitous joint approximate diagonalization of eigen-matrices algorithm, relying on the Givens rotations for the phase-shift keying source signals in terms of signal-to-interference-and-noise ratio for a wide variety of signal-to-noise ratios     

Analog Hardware Implementation of Continuous-Time Adaptive Filter Structures

We have implemented a four-tap adaptive filter in a continuous-time analog VLSI circuit. Since an ideal delay is impossible to implement in continuous-time hardware, we implemented the delay line as a cascade of low-pass filters (called the gamma filter). Since many years of research in our lab has shown that the gamma filter outperforms the ideal delay line for a wide range of applications, the gamma filter should not be considered merely a crude approximation of the ideal delay line. We show measured results from an analog chip that solves the problem of system identification–identifying an unknown linear circuit from its input/output relationship. Furthermore, we believe that a cascade of all-pass filters (called the Laguerre filter) will potentially outperform the gamma. We have built an adaptive Laguerre filter and show that its measured convergence rate is superior to that of the gamma. Finally, rather than perform gradient descent on a multimodal error function to determine a single optimal time constant, we propose multi-scale realizations of these delay line structures.     

Freeman olfactory cortex model: A multiplexed KII network implementation

This paper presents the results of a CMOS-VLSI implementation of a realistic computational model proposed by Walter Freeman for the olfactory system. This model, in later years, has been studied for engineering applications such as auto-association and classification. The analogue nature of the model motivates analogue VLSI implementations. However, the dimension and complexity of such system poses many obstacles to an analogue electronic implementation; one such is the massive interconnectivity which size increases with the square of the number of inputs (channels). We suggest a multiplexing procedure that puts the burden of interconnectivity over a digital system that is simpler to design and makes the analogue system more treatable. The procedure naturally samples the signals. To avoid smoothing filters, a discrete-time solution was also employed. Although with such approach the time resolution is reduced, the advantages overcome the detriments. Previous work has shown that the model can be efficiently discretized using DSP techniques, resulting on a system that is able to predict, on sample-by-sample basis, the behaviour of the VLSI circuit, allowing for a simple and flexible way to adjust the circuit parameters. We present the measured circuit results that are further confronted with the digital implementation.     

Fast RLS-Like Algorithm for Generalized Eigendecomposition and its Applications

Generalized eigendecomposition (GED) plays a vital role in many signal-processing applications. In this paper, we will propose a new method for computing the generalized eigenvectors, which is on-line and resembles the RLS algorithm for Wiener filtering. We further present a proof to show convergence to the exact solution and simulations have shown that the algorithm is faster than most of the traditional methods. This algorithm belongs to the class of fixed-point algorithms and hence does not require any external step-size parameters like the gradient-based methods. Simulations are performed on synthetic data and compared with other algorithms found in literature. Finally we will demonstrate the application of GED in the design of a CDMA receiver for direct-sequence spread spectrum signals.     

Adaptive blind deconvolution of linear channels using Renyi's entropy with Parzen window estimation

Blind deconvolution of linear channels is a fundamental signal processing problem that has immediate extensions to multiple-channel applications. In this paper, we investigate the suitability of a class of Parzen-window-based entropy estimates, namely Renyi's entropy, as a criterion for blind deconvolution of linear channels. Comparisons between maximum and minimum entropy approaches, as well as the effect of entropy order, equalizer length, sample size, and measurement noise on performance, will be investigated through Monte Carlo simulations. The results indicate that this nonparametric entropy estimation approach outperforms the standard Bell-Sejnowski and normalized kurtosis algorithms in blind deconvolution. In addition, the solutions using Shannon's entropy were not optimal either for super- or sub-Gaussian source densities.     

Correntropy: Properties and Applications in Non-Gaussian Signal Processing

The optimality of second-order statistics depends heavily on the assumption of Gaussianity. In this paper, we elucidate further the probabilistic and geometric meaning of the recently defined correntropy function as a localized similarity measure. A close relationship between correntropy and M-estimation is established. Connections and differences between correntropy and kernel methods are presented. As such correntropy has vastly different properties compared with second-order statistics that can be very useful in non-Gaussian signal processing, especially in the impulsive noise environment. Examples are presented to illustrate the technique.     

Mutagenicity of chemicals of industrial and agricultural relevance in Salmonella,streptomyces and Aspergillus

Five chemicals of industrial and agricultural relevance—ethylene dibromide, ethylene dichloride, propylene dichloride, allyl alcohol and sulphallate—were tested for their ability to induce reverse mutations in Salmonella typhimurium and forward mutations in Streptomyces coelicolor and Aspergillus nidulans. Ethylene dibromide was positive in all the genetic systems employed; sulphallate gave a positive response, to different degrees, in all the microorganisms; ethylene dichloride was weakly active in S. typhimurium following microsomal activation; propylene dichloride was detected as a direct acting mutagen in S. typhimurium and A. nidulans but not in S. coelicolor; allyl alcohol was completely negative in all test systems.     

The Kernel Least-Mean-Square Algorithm

The combination of the famed kernel trick and the least-mean-square (LMS) algorithm provides an interesting sample-by-sample update for an adaptive filter in reproducing kernel Hilbert spaces (RKHS), which is named in this paper the KLMS. Unlike the accepted view in kernel methods, this paper shows that in the finite training data case, the KLMS algorithm is well posed in RKHS without the addition of an extra regularization term to penalize solution norms as was suggested by Kivinen [Kivinen, Smola and Williamson, ldquoOnline Learning With Kernels,rdquo IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. 2165-2176, Aug. 2004] and Smale [Smale and Yao, ldquoOnline Learning Algorithms,rdquo Foundations in Computational Mathematics, vol. 6, no. 2, pp. 145-176, 2006]. This result is the main contribution of the paper and enhances the present understanding of the LMS algorithm with a machine learning perspective. The effect of the KLMS step size is also studied from the viewpoint of regularization. Two experiments are presented to support our conclusion that with finite data the KLMS algorithm can be readily used in high dimensional spaces and particularly in RKHS to derive nonlinear, stable algorithms with comparable performance to batch, regularized solutions.     

Modeling and inverse controller design for an unmanned aerial vehicle based on the self-organizing map

The next generation of aircraft will have dynamics that vary considerably over the operating regime. A single controller will have difficulty to meet the design specifications. In this paper, a self-organizing map (SOM)-based local linear modeling scheme of an unmanned aerial vehicle (UAV) is developed to design a set of inverse controllers. The SOM selects the operating regime depending only on the embedded output space information and avoids normalization of the input data. Each local linear model is associated with a linear controller, which is easy to design. Switching of the controllers is done synchronously with the active local linear model that tracks the different operating conditions. The proposed multiple modeling and control strategy has been successfully tested in a simulator that models the LoFLYTE UAV.