Iterative SNR estimation using a priori information

A class iterative signal-to-noise ratio (SNR) estimation algorithm is proposed in this paper. The data samples are governed by a given distribution with a priori. The expectation maximization (EM) algorithm is applied to iteratively maximize the likelihood function so as to realize the SNR estimation. Cramer–Rao bounds (CRB) with different a priori are compared for binary phase shift keying and orthogonal phase shift keying systems, which show the potential of the SNR estimator in turbo-like systems. In high-order modulations, simulation results show that the reduced-complexity iterative method with equal a priori has better performance in middle or high SNR region than the foregone ones. Moreover, the new method with feedback information is the best when its iteration number is 4 and extrinsic information larger than 0.4. These methods are applied in the bit-interleaved coded modulation with iterative decode (BICM-ID) system to validate the effect of the proposed methods.     

Regularization parameter estimation for feedforward neural networks

Under the framework of the Kullback-Leibler (KL) distance, we show that a particular case of Gaussian probability function for feedforward neural networks (NNs) reduces into the first-order Tikhonov regularizer. The smooth parameter in kernel density estimation plays the role of regularization parameter. Under some approximations, an estimation formula is derived for estimating regularization parameters based on training data sets. The similarity and difference of the obtained results are compared with other work. Experimental results show that the estimation formula works well in sparse and small training sample cases.     

A variant of finite-dimensional Tikhonov regularization with a-posteriori parameter choice

In this paper we consider a particular variant of finite-dimensional Tikhonov regularization for ill-posed operator equations. Convergence rates are established and an a-posteriori parameter choice method is derived that leads to optimal convergence rates with respect to data errors and with respect to the finite-dimensional subspace, without using any information about the exact solution. Finally, using linear splines we present several numerical examples that confirm the theoretical results.     

TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization

In this paper an improvement of the optimally pruned extreme learning machine (OP-ELM) in the form of a L₂ regularization penalty applied within the OP-ELM is proposed. The OP-ELM originally proposes a wrapper methodology around the extreme learning machine (ELM) meant to reduce the sensitivity of the ELM to irrelevant variables and obtain more parsimonious models thanks to neuron pruning. The proposed modification of the OP-ELM uses a cascade of two regularization penalties: first a L₁ penalty to rank the neurons of the hidden layer, followed by a L₂ penalty on the regression weights (regression between hidden layer and output layer) for numerical stability and efficient pruning of the neurons. The new methodology is tested against state of the art methods such as support vector machines or Gaussian processes and the original ELM and OP-ELM, on 11 different data sets; it systematically outperforms the OP-ELM (average of 27% better mean square error) and provides more reliable results - in terms of standard deviation of the results - while remaining always less than one order of magnitude slower than the OP-ELM.     

Echo state networks with Tikhonov regularization: optimization using integral gain

This study proposes feed-forward echo state networks (ESN) as an estimator, and couples it with second-order proportional-integral-derivative (PID) feedback extension to compensate for dead time in feedback systems. The system is tested for two-dimensional space motion patterns recognition and prediction using simulations, which allows control of noise input. Tikhonov regularization is employed for training readouts and second-order PID feedback minimizes prediction bias. Evaluation is done using mean squared error and the coupled system performs well compared to any of its standalone versions. The results suggest it is feasible to (1) ‘compress’ the memory capacity of the system, and (2) reduce the number optimization parameters, while maintaining the estimation performance and following the excitation property of the estimator. It is feasible to optimize the ESN using feedback gain although it plays a significant role in the proposed system because the improvement by bias correction is far greater than that of optimization; thus, simplifying the estimation to a feedback problem which is easily tuned using the Ziegler–Nichols method.     

Multidirectional Subspace Expansion for One-Parameter and Multiparameter Tikhonov Regularization

Tikhonov regularization is a popular method to approximate solutions of linear discrete ill-posed problems when the observed or measured data is contaminated by noise. Multiparameter Tikhonov regularization may improve the quality of the computed approximate solutions. We propose a new iterative method for large-scale multiparameter Tikhonov regularization with general regularization operators based on a multidirectional subspace expansion. The multidirectional subspace expansion may be combined with subspace truncation to avoid excessive growth of the search space. Furthermore, we introduce a simple and effective parameter selection strategy based on the discrepancy principle and related to perturbation results.     

Tikhonov正则化广义最小化求解振动速度

高精度生产及检测设备对环境微振动有较高要求,基于振动速度幅值计算的微振动等级可对环境振动情况进行定量评估.由于振动速度传感器难以实现对低频微振动的直接测量,因此需要使用低频高灵敏度的振动加速度传感器测量加速度信号,通过积分算法间接实现对振动速度的测量.基于Tik-honov正则化的广义最小化求解振动速度方法采用向量乘法运算、多线程并行运算实现对振动速度的快速测量;通过调节广义化阶数和正则化因子实现对不同环境的测量,克服了现有积分算法针对不同测量环境的适应性和计算速度等方面的不足.经实验验证,在有效抑制低频噪声的情况下,能够最大化地保留低频信息,较好地再现振动信号的时域瞬时特性.     

Iterative solvers for Tikhonov regularization of dense inverse problems

According to the special demands arising from the development of science and technology, in the last decades appeared a special class of problems that are inverse to the classical direct ones. Such an inverse problem is concerned with the opposite way, usually followed by a direct one: finding the cause of a given effect or finding the law of evolution given the cause and effect. Very frequently, such inverse problems are modelled by Fredholm first-kind integral equations that give rise after discretization to (very) ill-conditioned linear systems, in classical or least squares formulation. Then, an efficient numerical solution can be obtained by using the Tikhonov regularization technique. In this respect, in the present paper, we propose three Kovarik-like algorithms for numerical solution of the regularized problem. We prove convergence for all three methods and present numerical experiments on a mathematical model of an inverse problem concerned with the determination of charge distribution generating a given electric field.     

运用MMSE先验信噪比估计进行语音增强

针对DD（Decision-Directed）先验信噪比估计方法在处理语音时产生延迟以及非因果先验信噪比估计算法不具实时性的缺点,提出一种MMSE（Minimum Mean Square Error）先验信噪比估计方法。它在高斯语音模型假设的基础上,运用最小均方误差准则直接从带噪信号中估计先验信噪比。通过对增强语音信噪比、Itakura-Saito失真测度以及信号时域图和语谱图仿真,结果表明,该算法比DD算法能更好地抑制“音乐噪声”和防止语音畸变,且相对于非因果先验信噪比估计算法具有更强实时性。     

Two-stage procedure for parameter estimation in large-scale interconnected linear zero-memory systems

The problem of parameter estimation in interconnected complex systems composed of linear zero-memory elements is considered. A two-stage scheme for estimating system parameters is proposed, and the convergence in probability and with probability one, of the parameter estimates to the true values of system parameters, is shown. Some computational aspects of the algorithm are discussed and its recursive version is provided. The rate of convergence is also studied.     

Specific structures for large-scale state estimation algorithms having information exchange

This paper considers the design and evaluation of large-scale state estimation algorithms having specific structures which allow the subsystems to exchange information over noisy channels. The specific structures which are presented are first motivated by considering the relative performance between the surely locally unbiased filter and a global dynamics filter. The role of the surely locally unbiased filter in evaluating the tradeoffs between the cost of information transfer and filter performance is examined and a theorem is presented which forms the basis for an algorithm for calculating channel noise crossover levels. The theoretical results are illustrated via an application to a power system model.     

Maximum a posteriori parameter estimation in large-scale systems

In this paper we examine the use of the maximum a posteriori (MAP) approach for parameter estimation in large-scale interconnected dynamical systems. We examine both the suboptimal approach of Sage and the optimal approaches which arise on solving within a multi-level structure the resulting nonlinear two point boundary value problem. In particular we consider two optimal methods i.e. the costate prediction method and the state estimation approach of Chen and Perlis as applied to parameter estimation. We illustrate the approaches on a simple example which is used as a bench mark problem for purposes of comparison of the numerical efficiency of the various techniques.     

Cumulant-based parameter estimation using structured networks

A two-level three-layer structured network is developed to estimate the moving-average model parameters based on second-order and third-order cumulant matching. The structured network is a multilayer feedforward network composed of linear summers in which the weights of these summers have a clear physical meaning. The first level is composed of random access memory units, which are used to control the connectivities of the second-level summers. The second level is composed of three layers of linear summers in which the weight of any summer represents the moving-average parameter to be estimated. The connectivities among these summers are controlled by the first-level memory units in such a way that the outputs of the second-level structured network equal the desired second-order or third-order statistics if the summer weights equal their corresponding true moving-average parameter values. Each second-order and third-order cumulant is viewed as a pattern which the structured network needs to learn, and a steepest-descent algorithm is proposed for training the structured network. The author also presents extensions to particular sorts of estimation, and results of simulations.     

Using a priori information in regression analysis

The paper considers the methods to evaluate regression parameters under indefinite a priori information of two types: fuzzy and stochastic. Fuzzy a priori information is assumed to be formulated on the basis of fuzzy notions of the model designer. Stochastic a priori information is systems of equations, which are linear in regression parameters and whose right-hand sides are random variables. Regression parameters may both be constant and vary in time. A classification of the evaluation methods using indefinite a priori information is proposed and used to generalize well-known methods. An evaluation method is developed, which combines the fuzzy and stochastic a priori information about regression parameters.     

Likelihood parameter estimation for calibrating a soil moisture model using radar bakscatter

Land surface model parameter estimation can be performed using soil moisture information provided by synthetic aperture radar imagery. The presence of speckle necessitates aggregating backscatter measurements over large (> 100 m × 100 m) land areas in order to derive reliable soil moisture information from imagery, and a model calibrated to such aggregated information can only provide estimates of soil moisture at spatial resolutions required for reliable speckle accounting. A method utilizing the likelihood formulation of a probabilistic speckle model as the calibration objective function is proposed which will allow for calibrating land surface models directly to radar backscatter intensity measurements in a way which simultaneously accounts for model parameter- and speckle-induced uncertainty. The method is demonstrated using the NOAH land surface model and Advanced Integral Equation Method (AIEM) backscatter model calibrated to SAR imagery of an area in the Southwestern United States, and validated against in situ soil moisture measurements. At spatial resolutions finer than 100 m × 100 m NOAH and AIEM calibrated using the proposed radar intensity likelihood parameter estimation algorithm predict surface level soil moisture to within 4% volumetric water content 95% of the time, which is an improvement over a 95% prediction confidence of 10% volumetric water content by the same models calibrated directly to soil moisture information derived from synthetic aperture radar imagery at the same scales. Results suggest that much of this improvement is due to increased ability to simultaneously estimate NOAH parameters and AIEM surface roughness parameters.     

基于动态参数的Tikhonov方法在电容成像图像重建中的应用

该文针对电容成像(ECT)图像重建问题的病态性,采用Tikhonov正则化方法进行图像重建,并选用三种方法动态选择正则化参数。仿真结果表明对某些流型分布,用文中所述方法得到的重建结果优于目前普遍使用的线性反投影(LBP)算法。该方法为提高ECT图像重建质量提供了新的途径。     

The use of Morozov's discrepancy principle for Tikhonov regularization for solving nonlinear ill-posed problems

In this paper we investigate Morozov's Discrepancy Principle for choosing the regularization parameter in Tikhonov regularization for solving nonlinear ill-posed problems. Convergence rates and a saturation property of the regularized solutions, where the regularization parameter is chosen by the discrepancy principle, are investigated. Numerical results are presented to verify the theoretical results.     

Regularization of problem of processing measurement data under conditions of a priori uncertainty

Using regularization procedures, the problem of identification of the parameters of nonlinear objects with minimal a priori information on the probabilistic characteristics of the measurement noise and disturbandes is solved on the basis of the conception of the inverse problems of dynamics. The algorithm which is obtained remains stable in the presence of errors in the input data and is oriented towards the contemporary capabilities of computer engineering. Results of a numerical experiment are presented.