Polynomial fixed-point smoothing of uncertainly observed signals based on covariances

In this article, the least-squares νth-order polynomial fixed-point smoothing problem of uncertainly observed signals is considered, when only some information about the moments of the processes involved is available. For this purpose, a suitable augmented observation equation is defined such that the optimal polynomial estimator of the original signal is obtained from the optimal linear estimator of the augmented signal based on the augmented observations and, hence, a recursive algorithm for this linear estimator is deduced. The proposed estimator does not require the knowledge of the state-space model of the signal, but only the moments (up to the 2νth one) of the signal and observation noise, as well as the probability that the signal exists in the observations.     

Nonparametric Stein-type shrinkage covariance matrix estimators in high-dimensional settings

Estimating a covariance matrix is an important task in applications where the number of variables is larger than the number of observations. Shrinkage approaches for estimating a high-dimensional covariance matrix are often employed to circumvent the limitations of the sample covariance matrix. A new family of nonparametric Stein-type shrinkage covariance estimators is proposed whose members are written as a convex linear combination of the sample covariance matrix and of a predefined invertible target matrix. Under the Frobenius norm criterion, the optimal shrinkage intensity that defines the best convex linear combination depends on the unobserved covariance matrix and it must be estimated from the data. A simple but effective estimation process that produces nonparametric and consistent estimators of the optimal shrinkage intensity for three popular target matrices is introduced. In simulations, the proposed Stein-type shrinkage covariance matrix estimator based on a scaled identity matrix appeared to be up to 80% more efficient than existing ones in extreme high-dimensional settings. A colon cancer dataset was analyzed to demonstrate the utility of the proposed estimators. A rule of thumb for adhoc selection among the three commonly used target matrices is recommended.     

Improved Stein-type shrinkage estimators for the high-dimensional multivariate normal covariance matrix

Many applications require an estimate for the covariance matrix that is non-singular and well-conditioned. As the dimensionality increases, the sample covariance matrix becomes ill-conditioned or even singular. A common approach to estimating the covariance matrix when the dimensionality is large is that of Stein-type shrinkage estimation. A convex combination of the sample covariance matrix and a well-conditioned target matrix is used to estimate the covariance matrix. Recent work in the literature has shown that an optimal combination exists under mean-squared loss, however it must be estimated from the data. In this paper, we introduce a new set of estimators for the optimal convex combination for three commonly used target matrices. A simulation study shows an improvement over those in the literature in cases of extreme high-dimensionality of the data. A data analysis shows the estimators are effective in a discriminant and classification analysis.     

自校正解耦信息融合Wiener状态估值器

对含未知噪声方差阵的多传感器系统,用现代时间序列分析方法.基于滑动平均(MA)新息模型的在线辨识和求解相关函数矩阵方程组,可得到估计噪声方差阵估值器,进而在按分量标量加权线性最小方差最优信息融合则下,提出了自校正解耦信息融合Wiener状态估值器.它的精度比每个局部自校正Wiener状态估值器精度高.它实现了状态分量的解耦局部Wiener估值器和解耦融合Wiener估值器.证明了它的收敛性,即若MA新息模型参数估计是一致的,则它将收敛于噪声统计已知时的最优解耦信息融合Wiener状态估值器,因而它具有渐近最优性.一个带3传感器的目标跟踪系统的仿真例子说明了其有效性.     

不完全量测下估计误差协方差与传感器精度上–下确界的关系

在不完全量测下估计系统状态时,状态的稳态误差协方差与各个传感器精度指标有关.今提出一种新算法.可以根据估计误差协方差确定出传感器精度的上-下确界.算法根据稳态卡尔曼滤波的估计误差协方差表达式,推出传感器探测概率以及量测噪声方差指标的容差,并结合线性矩阵不等式求出传感器量测噪声方差的上-下界.根据这些结果,可以对给定的估计误差协方差,采用传感器精度指标的下界,从而在满足其他工程要求的前提下,放宽采样频率,降低传感器成本.     

Estimation of time series noise covariance using correlation technology

Covariance of clean signal and observed noise is necessary for extracting clean signal from a time series.This is transferred to calculate the covariance of observed noise and clean signal’s MA process,when the clean signal is described by an autoregressive moving average (ARMA) model.Using the correlations of the innovations data from observed time series to form a least-squares problem,a concisely autocovariance least-square (CALS) method has been proposed to estimate the covariance.We also extended our work to the case of unknown MA process coefficients.Comparisons between Odelson’s autocovariance least-square (ALS) estimation algorithm and the proposed CALS method show that the CALS method could get a much more exact and compact estimation of the covariance than ALS and its extended form.     

Multi-sensor information fusion estimators for stochastic uncertain systems with correlated noises

The information fusion estimation problems are investigated for multi-sensor stochastic uncertain systems with correlated noises. The stochastic uncertainties caused by correlated multiplicative noises exist in the state and observation matrices. The process noise and the observation noises are one-step auto-correlated and two-step cross-correlated, respectively. While the observation noises of different sensors are one-step cross-correlated. The optimal centralized fusion filter, predictor and smoother are proposed in the linear minimum variance sense via an innovative analysis approach. To enhance the robustness and flexibility, a distributed fusion filter is put forward, which requires the calculation of filtering error cross-covariance matrices between any two local filters. To avoid the calculation of cross-covariance matrices, another distributed fusion filter is also presented by using the covariance intersection (CI) fusion algorithm, which can reduce the computational cost. A simulation example is given to show the effectiveness of the proposed algorithms.     

Blind system identification using precise and quantized observations

This paper studies the blind identification of multi-channel FIR systems using precise and quantized observations. First, a new deterministic blind identification (DBI) algorithm is presented for multi-channel FIR systems using precise observations, in which the system parameters can be consistently estimated and the common source signal can be stably recovered. When the observed samples are quantized by a static finite-level quantizer, an iterative deterministic blind identification (IDBI) method is then provided. The asymptotic characters of the proposed IDBI method are discussed and the quantization effect on the identification performance is analyzed. Numerical simulations are given to support the developed DBI method and IDBI method.     

New design of linear discrete-time predictor using covariance information

A discrete one-stage predictor algorithm using covariance information in linear systems is derived. The algorithm is obtained for white Gaussian observation noise. The signal is a nonstationary or stationary stochastic process. The auto-covariance function of the signal is expressed using a semi-degenerate kernel of discrete-time systems. The semi-degenerate kernel can represent general covariance functions of random processes by a finite sum of nonrandom functions.     

Modeling energy dependent nuclear reaction probabilities from pointwise measurements including full covariance information

For the design of large and highly complex facilities – like, the international project ITER (http://www.iter.org/) – reliable basic data with well established covariance information (uncertainties, correlations) are indispensable. Procedures to attain the requirements by means of a Bayesian generalized least squares method and recent approaches using theoretical nuclear reaction codes in combination with Monte Carlo techniques will be discussed and results compared. Particular emphasis is given to the procedures of constructing covariance matrices for the experimental data sets as this determines the quality of the result.     

带不确定协方差线性相关白噪声系统改进的鲁棒协方差交叉融合稳态Kalman 估值器

对于带有不确定协方差线性相关白噪声的多传感器系统, 利用Lyapunov 方程提出设计协方差交叉(CI) 融合极大极小鲁棒Kalman 估值器(预报器、滤波器、平滑器) 的一种统一方法. 利用保守的局部估值误差互协方差, 提出改进的CI 融合鲁棒稳态Kalman 估值器及其实际估值误差方差最小上界, 克服了用原始CI 融合方法给出的上界具有较大保守性的缺点, 改善了原始CI 融合器鲁棒精度. 跟踪系统的仿真例子验证了所提出方法的正确性和有效性.

     

Design of state estimators for uncertain linear systems using quadratic boundedness

The notion of quadratic boundedness, which allows one to address the stability of a dynamic system in the presence of bounded disturbances, is applied to the design of state estimators for discrete-time linear systems with polytopic uncertainties. Necessary and sufficient stability conditions are stated and upper bounding sequences on the estimation error are derived. For the purpose of design, such conditions can be expressed in terms of linear matrix inequalities (LMIs), thus guaranteeing the numerical tractability. Simulation results are reported to show the effectiveness of the approach.     

Identification of process and measurement noise covariance for state and parameter estimation using extended Kalman filter

The performance of Bayesian state estimators, such as the extended Kalman filter (EKF), is dependent on the accurate characterisation of the uncertainties in the state dynamics and in the measurements. The parameters of the noise densities associated with these uncertainties are, however, often treated as ‘tuning parameters’ and adjusted in an ad hoc manner while carrying out state and parameter estimation. In this work, two approaches are developed for constructing the maximum likelihood estimates (MLE) of the state and measurement noise covariance matrices from operating input-output data when the states and/or parameters are estimated using the EKF. The unmeasured disturbances affecting the process are either modelled as unstructured noise affecting all the states or as structured noise entering the process predominantly through known, but unmeasured inputs. The first approach is based on direct optimisation of the ML objective function constructed by using the innovation sequence generated from the EKF. The second approach - the extended EM algorithm - is a derivative-free method, that uses the joint likelihood function of the complete data, i.e. states and measurements, to compute the next iterate of the decision variables for the optimisation problem. The efficacy of the proposed approaches is demonstrated on a benchmark continuous fermenter system. The simulation results reveal that both the proposed approaches generate fairly accurate estimates of the noise covariances. Experimental studies on a benchmark laboratory scale heater-mixer setup demonstrate a marked improvement in the predictions of the EKF that uses the covariance estimates obtained from the proposed approaches.     

Recursive smoothing algorithms for the estimation of signals from uncertain observations via mixture approximations

Considering discrete-time systems with uncertain observations when the signal model is unknown, but only covariance information is available, and the signal and the observation additive noise are correlated and jointly Gaussian, we present recursive algorithms for suboptimal fixed-point and fixed-interval smoothing estimators. To derive the algorithms, we employ a technique consisting in approximating the conditional distributions of the signal given the observations by Gaussian distributions, taking successive approximations of the mixtures of normal distributions. The expectation of these approximations provides us with the suboptimal estimators. In a numerical simulation example, the performance of the proposed estimators is compared with that of linear ones, via the sample mean square values of the corresponding estimation errors.     

Relation between filter using covariance information and Kalman filter

This paper presents sequential algorithms for the optimal impulse function, Kalman gain and the error variance in linear least squares filtering problems, when the autocovariance function of the signal is given in the form of a semi-degenerate kernel, and the additive observation noise in white Gaussian. A digital simulation result indicates that the algorithms presented in this paper are feasible, and that the values of Kalman gain and the error variance calculated by these algorithms approach to those obtained by the Kalman filter theory, for time sufficiently large.     

Estimation of the disturbance structure from data using semidefinite programming and optimal weighting

Designing a state estimator for a linear state-space model requires knowledge of the characteristics of the disturbances entering the states and the measurements. In [Odelson, B. J., Rajamani, M. R., & Rawlings, J. B. (2006). A new autocovariance least squares method for estimating noise covariances. Automatica, 42(2), 303-308], the correlations between the innovations data were used to form a least-squares problem to determine the covariances for the disturbances. In this paper we present new and simpler necessary and sufficient conditions for the uniqueness of the covariance estimates. We also formulate the optimal weighting to be used in the least-squares objective in the covariance estimation problem to ensure minimum variance in the estimates. A modification to the above technique is then presented to estimate the number of independent stochastic disturbances affecting the states. This minimum number of disturbances is usually unknown and must be determined from data. A semidefinite optimization problem is solved to estimate the number of independent disturbances entering the system and their covariances.     

Centralized filtering and smoothing algorithms from outputs with random parameter matrices transmitted through uncertain communication channels

The least-squares linear centralized estimation problem is addressed for discrete-time signals from measured outputs whose disturbances are modeled by random parameter matrices and correlated noises. These measurements, coming from different sensors, are sent to a processing center to obtain the estimators and, due to random transmission failures, some of the data packet processed for the estimation may either contain only noise (uncertain observations), be delayed (sensor delays) or even be definitely lost (packet dropouts). Different sequences of Bernoulli random variables with known probabilities are employed to describe the multiple random transmission uncertainties of the different sensors. Using the last observation that successfully arrived when a packet is lost, the optimal linear centralized fusion estimators, including filter, multi-step predictors and fixed-point smoothers, are obtained via an innovation approach; this approach is a general and useful tool to find easily implementable recursive algorithms for the optimal linear estimators under the least-squares optimality criterion. The proposed algorithms are obtained without requiring the evolution model of the signal process, but using only the first and second-order moments of the processes involved in the measurement model.     

Learning with privileged information using Bayesian networks

For many supervised learning applications, additional information, besides the labels, is often available during training, but not available during testing. Such additional information, referred to the privileged information, can be exploited during training to construct a better classifier. In this paper, we propose a Bayesian network (BN) approach for learning with privileged information. We propose to incorporate the privileged information through a three-node BN. We further mathematically evaluate different topologies of the three-node BN and identify those structures, through which the privileged information can benefit the classification. Experimental results on handwritten digit recognition, spontaneous versus posed expression recognition, and gender recognition demonstrate the effectiveness of our approach.     

引入信息增益的层次聚类算法

层次聚类分析是模式识别和数据挖掘领域中一个非常重要的研究课题,具有广泛的应用前景。受决策树学习中选择最佳分类属性的启发,提出一种引入信息增益的层次聚类方法,该方法利用信息增益指导层次聚类中的属性加权,从而提高聚类结果质量。在UCI数据集上的实验结果表明,该算法性能明显优于原层次聚类算法。     

New results using an integrated model and recursive algorithm for image motion estimation

A linear equation in the affine parameters used to model image motion may be derived by Taylor series expansion and truncation, and windowed spatial integration. Two methods for reducing errors in the Taylor approximation are discussed and results are presented.