Non-linear system identification using the Hammerstein model

An algorithm for the identification of non-linear systems which can be described by a Hammerstein model consisting of a single-valued non-linearity followed by a linear system is presented. Cross-correlation techniques are employed to decouple the identification of the linear dynamics from the characterization of the non-linear element. These results are extended to include the identification of the component subsystems of a feedforward process consisting of a Hammerstein model in parallel with another linear system.     

改进的自适应权值核范数最小化去噪算法

为解决传统权值核范数最小化(WNNM)算法在最优参数选取过程中过度依赖经验值的问题,提出一种改进的自适应参数选取WNNM算法,其最大特点是在WNNM算法基础上增加了噪声评估模型。通过提取均值减损对比归一化系数和邻域系数的分布特征参数构成图像特征矢量,与其对应的噪声浓度共同组成样本集;利用支持向量回归对样本集进行训练得到噪声评估模型,快速有效地为算法提供最优参数。实验结果表明,相比传统WNNM算法,该算法在进行图像去噪时,效率更高,效果更好,具有良好的鲁棒性和泛化性。     

Continuous-time Hammerstein system identification

A continuous-time Hammerstein system, i.e., a system consisting of a nonlinear memoryless subsystem followed by a linear dynamic one, is identified. The system is driven and disturbed by white random signals. The a priori information about both subsystems is nonparametric, which means that functional forms of both the nonlinear characteristic and the impulse response of the dynamic subsystem are unknown. An algorithm to estimate the nonlinearity is presented and its pointwise convergence to the true characteristic is shown. The impulse response of the dynamic part is recovered with a correlation method. The algorithms are computationally independent. Results of a simulation example are given     

Nonlocal image denoising via adaptive tensor nuclear norm minimization

Nonlocal self-similarity shows great potential in image denoising. Therefore, the denoising performance can be attained by accurately exploiting the nonlocal prior. In this paper, we model nonlocal similar patches through the multi-linear approach and then propose two tensor-based methods for image denoising. Our methods are based on the study of low-rank tensor estimation (LRTE). By exploiting low-rank prior in the tensor presentation of similar patches, we devise two new adaptive tensor nuclear norms (i.e., ATNN-1 and ATNN-2) for the LRTE problem. Among them, ATNN-1 relaxes the general tensor N-rank in a weighting scheme, while ATNN-2 is defined based on a novel tensor singular-value decomposition (t-SVD). Both ATNN-1 and ATNN-2 construct the stronger spatial relationship between patches than the matrix nuclear norm. Regularized by ATNN-1 and ATNN-2 respectively, the derived two LRTE algorithms are implemented through the adaptive singular-value thresholding with global optimal guarantee. Then, we embed the two algorithms into a residual-based iterative framework to perform nonlocal image denoising. Experiments validate the rationality of our tensor low-rank assumption, and the denoising results demonstrate that our proposed two methods are exceeding the state-of-the-art methods, both visually and quantitatively.

     

基于带权核范数最小化和混合高斯模型的去噪模型

非局部自相似性（NSS）先验在图像恢复中发挥重要作用,如何充分利用这一先验提高图像恢复性能仍值得深入研究,提出一种基于带权核范数最小化和混合高斯模型的去噪模型。首先,采用混合高斯模型（GMM）对无噪声的自然图像非局部自相似图像块进行训练,再用训练好的混合高斯模型指导退化的图像产生非局部自相似图像块组;然后,结合带权的核范数最小化技术实现图像的去噪,并对模型的保真项进行一般性扩展,给出收敛的求解算法。仿真实验表明,所提方法与基于3D滤波的块匹配（BM3D）算法、同时稀疏编码学习（LSSC）算法和带权的核范数最小化（WNNM）模型相比,峰值信噪比（PSNR）提高0.11~0.49 dB。     

Adaptive surface-related multiple subtraction using sparse norm minimization method

Multiple attenuation, which is difficult to solve, is an important problem in the seismic data processing especially in the marine case. A strategy for multiple removal consists of estimating a model of the multiples and then adaptively subtracting this model from the data by estimating shaping filters. A classical approach of this strategy is the surface-related multiple elimination (SRME) method. In the SRME process, the subtraction stage plays an important role, because there are amplitude, phase, and frequency distortions in the predicted multiple model. Typically, in this stage the primaries are assumed to have minimum energy (l₂-norm) and the solving method is the least-square errors. Methods using this norm are robust in the presence of noise, but can produce bad results when primaries and multiples interfere. Replacing the l₂-norm, a sparseness constraint is used in our new approach. The sparseness constraint should give better results because the correct subtraction of the predicted multiples should lead to a primary estimation with a minimum number of events. We also develop a fast gradient method for solving the sparse norm minimization problem. The effective results of the new method are illustrated with the synthetic data in the one-dimensional and two-dimensional cases. It is shown that the sparse norm minimization problem with fast gradient solution methods leads to much improved attenuation of the multiples while the minimum energy assumption is violated. The multiples being subtracted fit the multiples well in the data while preserving the energy of primaries.     

Hammerstein system identification by non-parametric regression estimation

A discrete-time, multiple-input non-linear Hammerstein system is identified. The dynamical subsystem is recovered using the standard correlation method. The main results concern estimation of the non-linear memoryless subsystem. No conditions concerning the functional form of the transform characteristic of the subsystem are made and an algorithm for estimation of the characteristic is given. The algorithm is simply a non-parametric kernel estimate of the regression function calculated from the dependent data. It is shown that the algorithm converges to the characteristic of the subsystem in the pointwise as well as the global sense. For sufficiently smooth characteristics, the rate of convergence is o(n^-1/(2+d in probability, where d is the dimension of the input variable.     

Continuous-time Hammerstein system identification from sampled data

A continuous-time Hammerstein system driven by a random signal is identified from observations sampled in time. The sampling may be uniform or not. The a-priori information about the system is nonparametric, functional forms of both the nonlinear characteristic and the impulse response are completely unknown. Three kernel algorithms, one offline and two semirecursive are presented. Their convergence to the true characteristic of the nonlinear subsystem is shown. The distance between consecutive sampling times must not decrease too fast for the algorithms to converge.     

Hammerstein system identification by non-parametric instrumental variables

A mixed, parametric–non-parametric routine for Hammerstein system identification is presented. Parameters of a non-linear characteristic and of ARMA linear dynamical part of Hammerstein system are estimated by least squares and instrumental variables assuming poor a priori knowledge about the random input and random noise. Both subsystems are identified separately, thanks to the fact that the unmeasurable interaction inputs and suitable instrumental variables are estimated in a preliminary step by the use of a non-parametric regression function estimation method. A wide class of non-linear characteristics including functions which are not linear in the parameters is admitted. It is shown that the resulting estimates of system parameters are consistent for both white and coloured noise. The problem of generating optimal instruments is discussed and proper non-parametric method of computing the best instrumental variables is proposed. The analytical findings are validated using numerical simulation results.     

截断核范数和全变差正则化高光谱图像复原

目的 高光谱图像距具有较高的光谱分辨率,从而具备区分诊断性光谱特征地物的能力,但高光谱数据经常会受到如环境、设备等各种因素的干扰,导致数据污染,严重影响高光谱数据在应用中的精度和可信度。方法 根据高光谱图像光谱维度特征值大小与所包含信息的关系,利用截断核范数最小化方法表示光谱低秩先验,从而有效抑制稀疏噪声;再利用高光谱图像的空间稀疏先验建立正则化模型,达到去除高密度噪声的目的;最终,结合上述两种模型的优势,构建截断核范数全变差正则化模型去除高斯噪声、稀疏噪声及其他混合噪声等。结果 将本文与其他三种近期发表的主流去噪方法进行对比,模型平均峰信噪比提高3.20 dB,平均结构相似数值指标提高0.22,并可以应用到包含各种噪声、不同尺寸的图像,其模型平均峰信噪比提高1.33 dB。结论 本文方法在光谱低秩中更加准确地表示了观测数据的先验特征,利用高光谱遥感数据的空间和低秩先验信息,能够对含有高密度噪声以及稀疏异常值的图像进行复原。     

A noniterative method for identification using Hammerstein model

A noniterative method developed by Hsia for the particular case where the transfer function in the Hammerstein model has no zeros is extended to the general case where the transfer function may have zeros. Numerical examples show that the computation time by this method is considerably less than by the iteratire procedure proposed by Narendra and Gallman while the accuracy of the estimates is comparable.     

Convergence of the iterative Hammerstein system identification algorithm

The convergence of the iterative identification algorithm for the Hammerstein system has been an open problem for a long time. In this paper, a detailed study is carried out and various convergence properties of the iterative algorithm are derived. It is shown that the iterative algorithm with normalization is convergent in general. Moreover, it is shown that convergence takes place in one step (two least squares iterations) for finite-impulse response Hammerstein models with i.i.d. inputs.     

Making parametric Hammerstein system identification a linear problem

In this paper, we study the identification of parametric Hammerstein systems with FIR linear parts. By a proper normalization and a clever characterization, it is shown that the average squared error cost function for identification can be expressed in terms of the inner product between the true but unknown parameter vector and its estimate. Further, the cost function is concave in the inner product and linear in the inner product square. Therefore, the identification of parametric Hammerstein systems with FIR linear parts is a globally convergent problem and has one and only one (local and global) minimum. This implies that the identification of such systems is a linear problem in terms of the inner product square and any local search based identification algorithm converges globally.     

Non-linear system identification using Hammerstein and non-linear feedback models with piecewise linear static maps

We consider the identification of Hammerstein/non-linear feedback models by approximating internal non-linearities using piecewise linear static maps. The resulting method utilizes a point-slope parameterization that leads to a computationally tractable optimization problem. The computational appeal of this technique is derived from the fact that the method only requires a matrix inverse and singular value decomposition. Furthermore, the identification method simultaneously identifies the linear dynamic and static non-linear blocks without requiring prior assumptions on the form of the static non-linearity.     

一种多输入单输出Hammerstein系统的集成辨识方法

针对多输入单输出(MISO)Hammerstein系统提出了一种稳态与动态辨识相结合的集成辨识方法.该方法利用稳态信息获取稳态模型的强一致性估计,并通过稳态模型以神经网络获得其非线性逼近函数,再利用动态信息辨识获取多输入单输出(MISO)Hammerstein系统的线性子系统未知参数的一致性估计.仿真结果表明了该方法的有效性和实用性.     

Graph cuts via l1 norm minimization

Graph cuts have become an increasingly important tool for solving a number of energy minimization problems in computer vision and other fields. In this paper, the graph cut problem is reformulated as an unconstrained $\ell_1$ norm minimization which can be solved effectively using interior point methods. This reformulation exposes connections between the graph cuts and other related continuous optimization problems. Eventually the problem is reduced to solving a sequence of sparse linear systems involving the Laplacian of the underlying graph. The proposed procedure exploits the structure of these linear systems in a manner that is easily amenable to parallel implementations. Experimental results obtained by applying the procedure to graphs derived from image processing problems are provided.     

Impulse response constrained LS-SVM modelling for MIMO Hammerstein system identification

In this paper, a new methodology for identifying multiple inputs multiple outputs Hammerstein systems is presented. The proposed method aims at incorporating the impulse response of the system into a least-squares support vector machine (LS-SVM) formulation and therefore the regularisation capabilities of LS-SVM are applied to the system as a whole. One of the main advantages of this method comes from the fact that it is flexible concerning the class of problems it can model and that no previous knowledge about the underlying non-linearities is required except for very mild assumptions. Also, it naturally adapts to handle different numbers of inputs/outputs and performs well in the presence of white Gaussian noise. Finally, the method incorporates information about the structure of the system but still the solution of the model follows from a linear system of equations. The performance of the proposed methodology is shown through three simulation examples and compared with other methods in the literature.     

Hammerstein system identification through best linear approximation inversion and regularisation

Hammerstein systems are composed by the cascading of a static nonlinearity and a linear system. In this paper, a methodology for identifying such systems using a combination of least squares support vector machines (LS-SVM) and best linear approximation (BLA) techniques is proposed. To do this, a novel method for estimating the intermediate variable is presented allowing a clear separation of the identification steps. First, an approximation to the linear block is obtained through the BLA of the system. Then, an approximation to the intermediate variable is obtained using the inversion of the estimated linear block and the known output. Afterwards, a nonlinear model is calculated through LS-SVM using the estimated intermediate variable and the known input. To do this, the regularisation capabilities of LS-SVM play a crucial role. Finally, a parametric re-estimation of the linear block is made. The method was tested in three examples, two of them with hard nonlinearities, and was compared with four other methods showing very good performance in all cases. The obtained results demonstrate that also in the presence of noise, the method can effectively identify Hammerstein systems. The relevance of these findings lies in the fact that it is shown how the regularisation allows to bypass the usual problems associated with the noise backpropagation when the inversion of the estimated linear block is used to compute the intermediate variable.     

Detection of asymmetric control valve stiction from oscillatory data using an extended Hammerstein system identification method

The study in this paper is motivated by the detection of control valves with asymmetric stiction resulting in oscillations in feedback control loops. The joint characterization of the control valve and the controlled process is formulated as the identification of a class of extended Hammerstein systems. The input nonlinearity is described by a point-slope-based hysteretic model with two possibly asymmetric ascent and descent paths. An iterative identification method is proposed, based on the idea of separating the ascent and descent paths subject to the oscillatory input and output. The structure of the formulated extended Hammerstein system is shown to be identifiable, and the oscillatory signals in feedback control loops are proved to be informative by exploiting the cyclo-stationarity of these oscillatory signals. Numerical, experimental and industrial examples are provided to illustrate the effectiveness of the proposed identification method.     

Transductive learning with nonlinear flexibility using trace norm minimization and fixed point continuation

In this paper, the affine Schatten p-norm minimisation problem with one-side inequality constraints is formulated. Through the optimisation process, many trace norm minimisation methods just achieve some degree of denoising. Contrarily, flexible nonlinearity relationship is established with this relaxed formulation, and thus existing traditional theories can be successfully applied for many real data-sets in machine learning, particularly for those violating the linearity assumption. We show that our new objective function is convex, and its global minimum can be obtained by a more general form of the Fixed-Point Continuation framework with almost the same computational cost. Experiments show that our algorithm is generally competitive among the state-of-the-art on widely used data-sets.