On a discrete-time stochastic learning control algorithm

In an earlier paper by the author (2001), the learning gain for a D-type learning algorithm, is derived based on minimizing the trace of the input error covariance matrix for linear time-varying systems. It is shown that, if the product of the input/output coupling matrices is full-column rank, then the input error covariance matrix converges uniformly to zero in the presence of uncorrelated random disturbances, whereas, the state error covariance matrix converges uniformly to zero in the presence of measurement noise. However, in general, the proposed algorithm requires knowledge of the state matrix. In this note, it is shown that equivalent results can be achieved without the knowledge of the state matrix. Furthermore, the convergence rate of the input error covariance matrix is shown to be inversely proportional to the number of learning iterations     

A discrete-time stochastic learning control algorithm

In this paper, the learning gain, for a selected learning algorithm, is derived based on minimizing the trace of the input error covariance matrix for linear time-varying systems. It is shown that, if the product of the input/output coupling matrices is a full-column rank, then the input error covariance matrix converges uniformly to zero in the presence of uncorrelated random disturbances. However, the state error covariance matrix converges uniformly to zero in presence of measurement noise. Moreover, it is shown that, if a certain condition is met, then the knowledge of the state coupling matrix is not needed to apply the proposed stochastic algorithm. The proposed algorithm is shown to suppress a class of nonlinear and repetitive state disturbance. The application of this algorithm to a class of nonlinear systems is also considered. A numerical example is included to illustrate the performance of the algorithm     

An enhanced adaptive Kalman filtering for linear systems with inaccurate noise statistics

This paper investigates the simultaneous state and noise covariance estimation for linear systems with inaccurate noise statistics. An enhanced adaptive Kalman filtering (EAKF) based on dynamic recursive nominal covariance estimation (DNRCE) and modified variational Bayesian (VB) inference is presented. The EAKF realizes the concurrently estimation of state and noise covariance matrices by introducing a nominal parameter in the traditional recursive covariance estimation and designing a new adaptive forgotten factor for the dynamic model of the estimated information propagation. The simulation of a target tracking example shows that, compared with the existing filters, the proposed filter has good adaptive performance for inaccurate and time-varying noise covariance matrices.     

Single‐look classification accuracy for polarimetric SAR

Polarimetric Synthetic Aperture Radar (SAR) systems such as ALOS‐PALSAR and Radarsat‐2 can operate in many different modes. The use of additional polarizations may require additional time and operating power and it is important to justify this by increased classification accuracy. A fully polarimetric, dual frequency AIRSAR scene from a rice‐growing area in Japan was classified by a maximum likelihood method based on the Wishart distribution. It is shown how the measured covariance matrices determine the separation accuracy between two classes. Closed form expressions are then given for the expected single‐look accuracy of the maximum likelihood classifier as a function of the class covariance matrices. This can be used to quickly compare the high spatial resolution classification performance of different polarimetric systems to decide upon a particular operating mode.     

On the error behavior in linear minimum variance estimation problems

For linear systems the error covariance matrix for the unbiased, minimum variance estimate of the state does not depend upon any specific realization of the measurement sequence. Thus it can be examined to determine the expected behavior of the error in the estimate before actually using the filter in practice. In this paper, the general linear system that contains both plant and measurement noise is shown to exhibit a decomposition property that permits the derivation of upper and lower bounds upon the error covariance matrix. This decomposition allows systems containing either plant or measurement noise, but not both, to be considered separately. Some general characteristics of these simpler systems are discussed and conditions for the positive definiteness and vanishing of the error covariance matrix are established. It is seen that the presence of plant noise, in general, prevents the error from vanishing. Alternatively, the condition ofq-stage observability is seen to be sufficient to insure that the error covariance matrix asymptotically approaches the zero matrix for systems with noise-free plants. These results are used to establish very specific lower bounds. Through the application of the duality principle, they can be applied directly to the analysis of the linear regulator problem.     

On the stability of fuzzy systems

Studies the global asymptotic stability of a class of fuzzy systems. It demonstrates the equivalence of stability properties of fuzzy systems and linear time invariant (LTI) switching systems. A necessary and sufficient condition for the stability of such systems are given, and it is shown that under the sufficient condition, a common Lyapunov function exists for the LTI subsystems. A particular case when the system matrices can be simultaneously transformed to normal matrices is shown to correspond to the existence of a common quadratic Lyapunov function. A constructive procedure to check the possibility of simultaneous transformation to normal matrices is provided     

Stochastic P-type/D-type iterative learning control algorithms

This paper presents stochastic algorithms that compute optimal and sub-optimal learning gains for a P-type iterative learning control algorithm (ILC) for a class of discrete-time-varying linear systems. The optimal algorithm is based on minimizing the trace of the input error covariance matrix. The state disturbance, reinitialization errors and measurement errors are considered to be zero-mean white processes. It is shown that if the product of the input-output coupling matrices C ( t + 1 ) B ( t ) is full column rank, then the input error covariance matrix converges to zero in presence of uncorrelated disturbances. Another sub-optimal P-type algorithm, which does not require the knowledge of the state matrix, is also presented. It is shown that the convergence of the input error covariance matrices corresponding to the optimal and sub-optimal P-type and D-type algorithms are equivalent, and all converge to zero at a rate inversely proportional to the number of learning iterations. A transient-response performance comparison, in the domain of learning iterations, for the optimal and sub-optimal P- and D-type algorithms is investigated. A numerical example is added to illustrate the results.     

Generalized characterization of tracking systems and linear multivariable plants

In order to design tracking systems incorporating linear multivariable plants with more controlled outputs than manipulated inputs, it is shown that a more'general tracking concept than set-point tracking is necessary. The inclusion of inequalities in tracking conditions facilitates the characterization of tracking systems and linear multivariable plants. It is shown that the possibility of undertracking (i.e. tracking with non-negative errors) is characterized by the separation theorem of convex analysis, that linear multivariable plants can be classified into Class I and Class II plants based upon their steady-state transfer-function matrices, and that, in the case of Class I plants, undertracking is possible for any set-point commands. Furthermore, the necessary and/or sufficient conditions for Class I plants are given. Finally, it is shown that, in the case of Class 1 plants, undertracking is also possible for any set-point commands and any constant disturbances.     

Minimax linear observers and regulators for stochastic systems with uncertain second-order statistics

The problem of minimax design of linear observers and regulators for linear time-varying multivariable stochastic systems with uncertain models of their second-order statistics is treated in this paper. General classes of allowable covariance matrices and means of the process and observation noises and of the random initial condition are considered. A game formulation of the problem is adopted and it is shown that the optimal filter for the least favorable set of covariances is minimax robust for each of the filtering situations analyzed. Conditions satisfied by the saddle-point solutions are given, and their utility for finding the worst case covariances is illustrated by way of several examples of uncertainty classes of practical interest.     

Distributed fusion cubature Kalman filters for nonlinear systems

This paper is concerned with the distributed fusion estimation problem for multisensor nonlinear systems. Based on the Kalman filtering framework and the spherical cubature rule, a general method for calculating the cross‐covariance matrices between any two local estimators is presented for multisensor nonlinear systems. In the linear unbiased minimum variance sense, based on the cross‐covariance matrices, a distributed fusion cubature Kalman filter weighted by matrices (MW‐CKF) is presented. The proposed MW‐CKF has better accuracy and robustness. An example verifies the effectiveness of the proposed algorithms.     

Evolutionary discriminant analysis

An evolutionary approach to the supervised reduction of dimensions is introduced in this paper. Traditionally, such reduction has been accomplished by maximizing one or another measure of class separation. Quite often, the rank deficiency of the involved covariance matrices precludes the application of this classical approach to real situations. Besides, the number of projections cannot be chosen freely, but it is bounded to be equal to the number of classes minus one. By contrast, our evolution strategy reduces dimensions by the direct minimization of the number of misclassified patterns. No matrices are involved whatsoever and the number of projections can be chosen without restrictions. This allows to obtain two-dimensional renderings of data sets with more than three classes such as the 19 class UCI soybean problem. A nonlinear generalization of this procedure based on the hierarchical composition of linear projections is shown to solve the UCI thyroid problem with state of the art recognition rates.     

Discriminant analysis using certain normed exponential densities with emphasis on remote sensing application

The Bayes discriminant analysis based upon the normality assumption for population models does not lead to an exact evaluation of probabilities of correct classification and of misclassification unless it is restricted to a simplest possible situation. In order to overcome this and other computational difficulties that one faces in a complex situation such as the remote sensing, certain alternative densities are posed as models for the observations. It is shown that for a Bayes discriminant analysis these densities lead to piecewise linear discriminant functions even when the covariance matrices are unequal (a property not enjoyed in the normal case) and provide a theoretical solution for evaluating probabilities of correct classification and of misclassification. Also, some computational advantages are achieved.     

Optimal non-linear estimation for distributed-parameter systems via the partition theorem

This paper considers the estimation problem for non-linear distributed-parameter systems via the ‘Partition Theorem’. First, the a posterioriprobability for the state is derived for the estimation of non-linear distributed-parameter systems. Secondly, linear systems excited by a white gaussian noise and with non-gaussian initial state are considered as a special class of the problem. The a posterioriprobability for the state, the optimal estimates and corresponding error covariance matrices are obtained by using the properties of the fundamental solution for the differential operator. Finally, it is shown that on approximate expression for the solution of the problem is also derived by applying a gaussian sum approximation technique.     

Fractional-Order Embedding Supervised Canonical Correlations Analysis with Applications to Feature Extraction and Recognition

Due to the noise disturbance and limited number of training samples, within-set and between-set sample covariance matrices in canonical correlations analysis (CCA) based methods usually deviate from the true ones. In this paper, we re-estimate the covariance matrices by embedding fractional order and incorporate the class label information. First, we illustrate the effectiveness of the fractional-order embedding model through theory analysis and experiments. Then, we quote fractional-order within-set and between-set scatter matrices, which can significantly reduce the deviation of sample covariance matrices. Finally, we incorporate the supervised information, novel generalized CCA and discriminative CCA are presented for multi-view dimensionality reduction and recognition, called fractional-order embedding generalized canonical correlations analysis and fractional-order embedding discriminative canonical correlations analysis. Extensive experimental results on various handwritten numeral, face and object recognition problems show that the proposed methods are very effective and obviously outperform the existing methods in terms of classification accuracy.     

Analysis of conditional covariance structure models

This paper presents a statistical development for analyzing covariance structure models with certain variables held constant. The theory enables one to assess partial correlations among variables. Models are defined based on the conditional covariance matrices, and they are studied by the maximum likelihood approach with the appropriate conditional sample covariance matrices. It is shown that existing computer package programs and statistical methods can be easily applied to analyze the new models. Examples from two real data sets are reported to illustrate the theory.     

Minimum phone error training of precision matrix models

Gaussian mixture models (GMMs) are commonly used as the output density function for large-vocabulary continuous speech recognition (LVCSR) systems. A standard problem when using multivariate GMMs to classify data is how to accurately represent the correlations in the feature vector. Full covariance matrices yield a good model, but dramatically increase the number of model parameters. Hence, diagonal covariance matrices are commonly used. Structured precision matrix approximations provide an alternative, flexible, and compact representation. Schemes in this category include the extended maximum likelihood linear transform and subspace for precision and mean models. This paper examines how these precision matrix models can be discriminatively trained and used on state-of-the-art speech recognition tasks. In particular, the use of the minimum phone error criterion is investigated. Implementation issues associated with building LVCSR systems are also addressed. These models are evaluated and compared using large vocabulary continuous telephone speech and broadcast news English tasks.     

实值空间的宽带信号测向新方法

为了降低宽带信号TCT方位估计算法的运算量和分辩门限,针对中心对称阵列,将实值处理过程和子空间投影MUSIC的思想引入宽带信号方位估计,提出一种宽带信号方位估计新方法。该方法首先各个频率子带的数据协方差矩阵去噪且双向平滑,然后使用左实变换矩阵将它们变换为实数矩阵,从而大大降低了后面矩阵特征分解时运算量,接着使用双边相关变换（TCT）方法选取聚焦矩阵对各个频率子带的实协方差矩阵进行变换得到同一参考频率点的数据协方差阵,最后利用基于子空间投影的MUSIC方法进行一系列一维搜索算法来求得各个目标的方位角。仿真实验结果表明,此方法比常规宽带TCT方法运算量小,且具有更低的分辨门限和更小的估计偏差。     

Noise covariance identification for time-varying and nonlinear systems

Kalman-based state estimators assume a priori knowledge of the covariance matrices of the process and observation noise. However, in most practical situations, noise statistics and initial conditions are often unknown and need to be estimated from measurement data. This paper presents an auto-covariance least-squares-based algorithm for noise and initial state error covariance estimation of large-scale linear time-varying (LTV) and nonlinear systems. Compared to existing auto-covariance least-squares based-algorithms, our method does not involve any approximations for LTV systems, has fewer parameters to determine and is more memory/computationally efficient for large-scale systems. For nonlinear systems, our algorithm uses full information estimation/moving horizon estimation instead of the extended Kalman filter, so that the stability and accuracy of noise covariance estimation for nonlinear systems can be guaranteed or improved, respectively.     

Performance loss and design method of Kalman filters for discrete-time linear systems with uncertainties†

This paper is divided into two parts. The first part is concerned with the performance loss of the discrete-time Kalman filter designed on the basis of the model with errors in both dynamical and observation systems. The difference equation which describes the evolution of the covariance matrix of actual estimation error is derived. Some numerical results are shown as the illustration of the technique.

The second half is devoted to the development of the method of designing the unbiased minimum variance linear filter for the random system whose elements of both the transition and observation matrices are Gaussian white noises. For this purpose the result of the first part is utilized.     

Low-complexity approximate iterative LMMSE detection for large-scale MIMO systems

This paper deals with iterative detection for uplink large-scale MIMO systems. The well-known iterative linear minimum mean squared error (LMMSE) detector requires quadratic complexity (per symbol per iteration) with the number of antennas, which may be a concern in large-scale MIMO. In this work, we develop approximate iterative LMMSE detectors based on transformed system models where the transformation matrices are obtained through channel matrix decompositions. It is shown that, with quasi-linear complexity (per symbol per iteration), the proposed detectors can achieve almost the same performance as the conventional LMMSE detector. It is worth mentioning that the linear transformations are also useful to reduce the complexity of downlink precoding, so the relevant computational complexity can be shared by both uplink and downlink.