A posteriori real-time recurrent learning schemes for a recurrentneural network based nonlinear predictor

Recurrent neural networks (RNNs) are well established for the nonlinear and nonstationary signal prediction paradigm. Appropriate learning algorithms, such as the real-time recurrent learning (RTRL) algorithm, have been developed for that purpose. However, little is known about the RNN time-management policy. Here, insight is provided into the time-management of the RNN, and an a posteriori approach to the RNN based nonlinear signal prediction paradigm is offered. Based upon the chosen time-management policy, algorithms are developed, from the a priori learning-a priori error strategy through to the a posteriori learning-a posteriori error strategy. Compared with the a priori algorithms, the a posteriori algorithms offered are shown to provide a better prediction performance with little further expense in terms of computational complexity. Simulations undertaken on speech using the newly introduced algorithms confirm the theoretical results     

Towards the Optimal Learning Rate for Backpropagation

A backpropagation learning algorithm for feedforward neural networks withan adaptive learning rate is derived. The algorithm is based uponminimising the instantaneous output error and does not include anysimplifications encountered in the corresponding Least Mean Square (LMS)algorithms for linear adaptive filters. The backpropagation algorithmwith an adaptive learning rate, which is derived based upon the Taylorseries expansion of the instantaneous output error, is shown to exhibitbehaviour similar to that of the Normalised LMS (NLMS) algorithm. Indeed,the derived optimal adaptive learning rate of a neural network trainedby backpropagation degenerates to the learning rate of the NLMS for a linear activation function of a neuron. By continuity, the optimal adaptive learning rate for neural networks imposes additional stabilisationeffects to the traditional backpropagation learning algorithm.     

Study of the dead layer in germanium for the CDMS detectors

We present new measurements on a Cryogenic Dark Matter Search (CDMS) detector with electron, neutron, and gamma sources. The measurements have been performed to investigate the dead layer of one of the CDMS Z-dependent Ionization Phonon germanium detectors. The dead layer has been studied at both charge electrodes and at different electric field intensities. We also present a method to remove the dependence of athermal phonon measurements on event position.     

Analysis and online realization of the CCA approach for blind source separation.

A critical analysis of the canonical correlation analysis (CCA) approach in blind source separation (BSS) is provided. It is proved that by maximizing the autocorrelation functions of the recovered signals we can separate the source signals successfully. It is further shown that the CCA approach represents the same class of generalized eigenvalue decomposition (GEVD) problems as the matrix pencil method. Finally, online realizations of the CCA approach are discussed with a linear-predictor-based algorithm studied as an example.     

Influence of background and surround on image color matching

In this article, the corresponding‐color data for complex images reproduced on different media were obtained by simultaneous matching using an adjustment method. In our experiment printed color images and images displayed on a monitor were compared in different viewing conditions. The viewing condition varied in surround relative luminance and background. The experimental data show that surround relative luminance has little influence on color matching between printed and monitor images while changes in background modify color appearance. These results were used to evaluate different chromatic adaptation transforms (CAT). We found that for the same viewing conditions the SHARP transform shows the best agreement between the experimental and predicted data. SHARP transform can not predict accurately corresponding colors for blue and black regions. Therefore, we proposed new CAT that shows better characteristics than other transforms for cyan, green, and black colors and similar characteristics for other colors. © 2007 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 17, 244–251, 2007     

Characterisation of Signal Modality: Exploiting Signal Nonlinearity in Machine Learning and Signal Processing

A novel method for online tracking of the changes in the nonlinearity within both real-domain and complex–valued signals is introduced. This is achieved by a collaborative adaptive signal processing approach based on a hybrid filter. By tracking the dynamics of the adaptive mixing parameter within the employed hybrid filtering architecture, we show that it is possible to quantify the degree of nonlinearity within both real- and complex-valued data. Implementations for tracking nonlinearity in general and then more specifically sparsity are illustrated on both benchmark and real world data. It is also shown that by combining the information obtained from hybrid filters of different natures it is possible to use this method to gain a more complete understanding of the nature of the nonlinearity within a signal. This also paves the way for building multidimensional feature spaces and their application in data/information fusion.     

Indoor Power Harvesting Using Photovoltaic Cells for Low-Power Applications

Utilization of low-power indoor devices such as remote sensors, supervisory and alarm systems, distributed controls, and data transfer system is on steady rise. Due to remote and distributed nature of these systems, it is attractive to avoid using electrical wiring to supply power to them. Primary batteries have been used for this application for many years, but they require regular maintenance at usually hard to access places. This paper provides a complete analysis of a photovoltaic (PV) harvesting system for indoor low-power applications. The characteristics of a target load, PV cell, and power conditioning circuit are discussed. Different choices of energy storage are also explained. Implementation and test results of the system are presented, which highlights the practical issues and limitations of the system.     

Sequential Data Fusion via Vector Spaces: Fusion of Heterogeneous Data in the Complex Domain

A sequential data fusion approach via higher dimensional vector spaces is introduced. This is achieved by making use of the representation of directional signals within the field of complex numbers $C$ . The concept of data fusion is next introduced and the place of the proposed approach within that framework is identified. The benefits of such an approach are illustrated and a range of possible applications is shown. The concept introduced is supported by a real world case study which focuses on simultaneous forecasting of wind speed and direction. The architectures and learning algorithms which support this concept are introduced and their distributed sequential fusion nature is highlighted.     

Toward an optimal PRNN-based nonlinear predictor

We present an approach for selecting optimal parameters for the pipelined recurrent neural network (PRNN) in the paradigm of nonlinear and nonstationary signal prediction. We consider the role of nesting, which is inherent to the PRNN architecture. The corresponding number of nested modules needed for a certain prediction task, and their contribution toward the final prediction gain give a thorough insight into the way the PRNN performs, and offers solutions for optimization of its parameters. In particular, nesting allows the forgetting factor in the cost function of the PRNN to exceed unity, hence it becomes an emphasis factor. This compensates for the small contribution of the distant modules to the prediction process, due to nesting, and helps to circumvent the problem of vanishing gradient, experienced in RNNs for prediction. The PRNN is shown to outperform the linear least mean square and recursive least squares predictors, as well as previously proposed PRNN schemes, at no expense of additional computational complexity.     

A posteriori error learning in nonlinear adaptive filters

The authors provide relationships between the a priori and a posteriori errors of adaptation algorithms for real-time output-error nonlinear adaptive filters realised as feedforward or recurrent neural networks. The analysis is undertaken for a general nonlinear activation function of a neuron, and for gradient-based learning algorithms, for both a feedforward (FF) and recurrent neural network (RNN). Moreover, the analysis considers both contractive and expansive forms of the nonlinear activation functions within the networks. The relationships so obtained provide the upper and lower error bounds for general gradient based a posteriori learning in neural networks