Competitive principal component analysis for locally stationarytime series

A new unsupervised algorithm is proposed that performs competitive principal component analysis (PCA) of a time series. A set of expert PCA networks compete, through the mixture of experts (MOE) formalism, on the basis of their ability to reconstruct the original signal. The resulting network finds an optimal projection of the input onto a reduced dimensional space as a function of the input and, hence, of time. As a byproduct, the time series is both segmented and identified according to stationary regions. Examples showing the performance of the algorithm are included     

Making sense of a complex world [chaotic events modeling]

Addresses the identification of nonlinear systems from output time series, which we have called dynamic modeling. We start by providing the mathematical basis for dynamic modeling and show that it is equivalent to a multivariate nonlinear prediction problem in the reconstructed space. We address the importance of dynamic reconstruction for dynamic modeling. Recognizing that dynamic reconstruction is an ill-defined inverse problem, we describe a regularized radial basis function network for solving the dynamic reconstruction problem. Prior knowledge in the form of smoothness of the mapping is imposed on the solution via regularization. We also show that, in time-series analysis, some form of regularization can be accomplished by using the structure of the time series instead of imposing a smoothness constraint on the cost function. We develop a methodology based on iterated prediction to train the network weights with an error derived through trajectory learning. This method provides a robust performance because during learning the weights are constrained to follow a trajectory. The dynamic invariants estimated from the generated time series are similar to the ones estimated from the original time series, which means that the properties of the attractor have been captured by the neural network. We finally raise the question that generalized delay operators may have advantages in dynamic reconstruction, primarily in cases where the time series is corrupted by noise. We show how to set the recursive parameter of the gamma operator to attenuate noise and preserve the dynamics     

Spatial approximation of the radiation transport equation using a subgrid-scale finite element method

In this paper we present stabilized finite element methods to discretize in space the monochromatic radiation transport equation. These methods are based on the decomposition of the unknowns into resolvable and subgrid scales, with an approximation for the latter that yields a problem to be solved for the former. This approach allows us to design the algorithmic parameters on which the method depends, which we do here when the discrete ordinates method is used for the directional approximation. We concentrate on two stabilized methods, namely, the classical SUPG technique and the orthogonal subscale stabilization. A numerical analysis of the spatial approximation for both formulations is performed, which shows that they have a similar behavior: they are both stable and optimally convergent in the same mesh-dependent norm. A comparison with the behavior of the Galerkin method, for which a non-standard numerical analysis is done, is also presented.     

Adaptive blind deconvolution of linear channels using Renyi's entropy with Parzen window estimation

Blind deconvolution of linear channels is a fundamental signal processing problem that has immediate extensions to multiple-channel applications. In this paper, we investigate the suitability of a class of Parzen-window-based entropy estimates, namely Renyi's entropy, as a criterion for blind deconvolution of linear channels. Comparisons between maximum and minimum entropy approaches, as well as the effect of entropy order, equalizer length, sample size, and measurement noise on performance, will be investigated through Monte Carlo simulations. The results indicate that this nonparametric entropy estimation approach outperforms the standard Bell-Sejnowski and normalized kurtosis algorithms in blind deconvolution. In addition, the solutions using Shannon's entropy were not optimal either for super- or sub-Gaussian source densities.     

The Kernel Least-Mean-Square Algorithm

The combination of the famed kernel trick and the least-mean-square (LMS) algorithm provides an interesting sample-by-sample update for an adaptive filter in reproducing kernel Hilbert spaces (RKHS), which is named in this paper the KLMS. Unlike the accepted view in kernel methods, this paper shows that in the finite training data case, the KLMS algorithm is well posed in RKHS without the addition of an extra regularization term to penalize solution norms as was suggested by Kivinen [Kivinen, Smola and Williamson, ldquoOnline Learning With Kernels,rdquo IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. 2165-2176, Aug. 2004] and Smale [Smale and Yao, ldquoOnline Learning Algorithms,rdquo Foundations in Computational Mathematics, vol. 6, no. 2, pp. 145-176, 2006]. This result is the main contribution of the paper and enhances the present understanding of the LMS algorithm with a machine learning perspective. The effect of the KLMS step size is also studied from the viewpoint of regularization. Two experiments are presented to support our conclusion that with finite data the KLMS algorithm can be readily used in high dimensional spaces and particularly in RKHS to derive nonlinear, stable algorithms with comparable performance to batch, regularized solutions.     

The past, present, and future of neural networks for signalprocessing

The article provides a review of the fundamental of neural networks and reports recent progress. Topics covered include dynamic modeling, model-based neural networks, statistical learning, eigenstructure-based processing, active learning, and generalization capability. Current and potential applications of neural networks are also described in detail. Those applications include optical character recognition, speech recognition and synthesis, automobile and aircraft control, image analysis and neural vision, and several medical applications. Essentially, neural networks have become a very effective tool in signal processing, particularly in various recognition tasks     

Correntropy: Properties and Applications in Non-Gaussian Signal Processing

The optimality of second-order statistics depends heavily on the assumption of Gaussianity. In this paper, we elucidate further the probabilistic and geometric meaning of the recently defined correntropy function as a localized similarity measure. A close relationship between correntropy and M-estimation is established. Connections and differences between correntropy and kernel methods are presented. As such correntropy has vastly different properties compared with second-order statistics that can be very useful in non-Gaussian signal processing, especially in the impulsive noise environment. Examples are presented to illustrate the technique.     

Fast RLS-Like Algorithm for Generalized Eigendecomposition and its Applications

Generalized eigendecomposition (GED) plays a vital role in many signal-processing applications. In this paper, we will propose a new method for computing the generalized eigenvectors, which is on-line and resembles the RLS algorithm for Wiener filtering. We further present a proof to show convergence to the exact solution and simulations have shown that the algorithm is faster than most of the traditional methods. This algorithm belongs to the class of fixed-point algorithms and hence does not require any external step-size parameters like the gradient-based methods. Simulations are performed on synthetic data and compared with other algorithms found in literature. Finally we will demonstrate the application of GED in the design of a CDMA receiver for direct-sequence spread spectrum signals.     

Time-based compression and classification of heartbeats

Heart function measured by electrocardiograms (ECG) is crucial for patient care. ECG generated waveforms are used to find patterns of irregularities in cardiac cycles in patients. In many cases, irregularities evolve over an extended period of time that requires continuous monitoring. However, this requires wireless ECG recording devices. These devices consist of an enclosed system that includes electrodes, processing circuitry, and a wireless communication block imposing constraints on area, power, bandwidth, and resolution. In order to provide continuous monitoring of cardiac functions for real-time diagnostics, we propose a methodology that combines compression and analysis of heartbeats. The signal encoding scheme is the time-based integrate and fire sampler. The diagnostics can be performed directly on the samples avoiding reconstruction required by the competing finite rate of innovation and compressed sensing. As an added benefit, our scheme provides an efficient hardware implementation and a compressed representation for the ECG recordings, while still preserving discriminative features. We demonstrate the performance of our approach through a heartbeat classification application consisting of normal and irregular heartbeats known as arrhythmia. Our approach that uses simple features extracted from ECG signals is comparable to results in the published literature.     

Feature selection in MLPs and SVMs based on maximum output information

This paper presents feature selection algorithms for multilayer perceptrons (MLPs) and multiclass support vector machines (SVMs), using mutual information between class labels and classifier outputs, as an objective function. This objective function involves inexpensive computation of information measures only on discrete variables; provides immunity to prior class probabilities; and brackets the probability of error of the classifier. The maximum output information (MOI) algorithms employ this function for feature subset selection by greedy elimination and directed search. The output of the MOI algorithms is a feature subset of user-defined size and an associated trained classifier (MLP/SVM). These algorithms compare favorably with a number of other methods in terms of performance on various artificial and real-world data sets.