Echo state networks with Tikhonov regularization: optimization using integral gain

This study proposes feed-forward echo state networks (ESN) as an estimator, and couples it with second-order proportional-integral-derivative (PID) feedback extension to compensate for dead time in feedback systems. The system is tested for two-dimensional space motion patterns recognition and prediction using simulations, which allows control of noise input. Tikhonov regularization is employed for training readouts and second-order PID feedback minimizes prediction bias. Evaluation is done using mean squared error and the coupled system performs well compared to any of its standalone versions. The results suggest it is feasible to (1) ‘compress’ the memory capacity of the system, and (2) reduce the number optimization parameters, while maintaining the estimation performance and following the excitation property of the estimator. It is feasible to optimize the ESN using feedback gain although it plays a significant role in the proposed system because the improvement by bias correction is far greater than that of optimization; thus, simplifying the estimation to a feedback problem which is easily tuned using the Ziegler–Nichols method.     

多帧图像的Tikhonov正则化重建算法研究*

介绍了Tikhonov正则化超分辨率重建算法的基本原理和特点,在原有正则化空域图像复原方法的基础上,根据多帧序列图像之间的互补信息,提出一种改进的正则化空域图像复原的新方法,该算法直接将正则化函数作用于图像超分辨率重建算法的条件概率项内,提高了正则化项的校正效率,并用共轭梯度运算来改善算法的收敛性,节省了图像重建所需的时间。实验和仿真结果表明,与传统方法相比,该算法不仅减轻了图像边缘纹理的模糊性,提高了图像的清晰度,而且收敛速度快。     

Iterative solvers for Tikhonov regularization of dense inverse problems

According to the special demands arising from the development of science and technology, in the last decades appeared a special class of problems that are inverse to the classical direct ones. Such an inverse problem is concerned with the opposite way, usually followed by a direct one: finding the cause of a given effect or finding the law of evolution given the cause and effect. Very frequently, such inverse problems are modelled by Fredholm first-kind integral equations that give rise after discretization to (very) ill-conditioned linear systems, in classical or least squares formulation. Then, an efficient numerical solution can be obtained by using the Tikhonov regularization technique. In this respect, in the present paper, we propose three Kovarik-like algorithms for numerical solution of the regularized problem. We prove convergence for all three methods and present numerical experiments on a mathematical model of an inverse problem concerned with the determination of charge distribution generating a given electric field.     

Tikhonov正则化广义最小化求解振动速度

高精度生产及检测设备对环境微振动有较高要求,基于振动速度幅值计算的微振动等级可对环境振动情况进行定量评估.由于振动速度传感器难以实现对低频微振动的直接测量,因此需要使用低频高灵敏度的振动加速度传感器测量加速度信号,通过积分算法间接实现对振动速度的测量.基于Tik-honov正则化的广义最小化求解振动速度方法采用向量乘法运算、多线程并行运算实现对振动速度的快速测量;通过调节广义化阶数和正则化因子实现对不同环境的测量,克服了现有积分算法针对不同测量环境的适应性和计算速度等方面的不足.经实验验证,在有效抑制低频噪声的情况下,能够最大化地保留低频信息,较好地再现振动信号的时域瞬时特性.     

A method for regularization of evolutionary polynomial regression

While many applications require models that have no acceptable linear approximation, the simpler nonlinear models are defined by polynomials. The use of genetic algorithms to find polynomial models from data is known as evolutionary polynomial regression (EPR). This paper introduces evolutionary polynomial regression with regularization, an algorithm extending EPR with a regularization term to control polynomial complexity. The article also describes a set of experiences to compare both flavors of EPR against other methods including linear regression, regression trees and support vector regression. These experiments show that evolutionary polynomial regression with regularization is able to achieve better fitting and needs less computation time than plain EPR.     

近似最优正则化参数方法在电容成像图像重建中的应用

将基于近似最优正则化参数的Tikhonov方法应用于电容成像(Electrical capacitance tomography，ECT)图像重建以解决其中存在的病态性问题，并利用几种典型分布对该方法进行仿真测试。数值结果表明，在先验知识满足的条件下，近似最优参数法所找到的正则化参数是对最优正则化参数的较合理近似。在重建结果方面，基于近似最优参数的Tikhonov方法在不同的介电常数分布下与目前普遍采用的线性反投影算法(Linear back projection，LBP)各有优势。结果表明，该方法尚不能完全取代LBP算法，但能在一定程度上弥补LBP算法的不足。     

Multi-step virtual metrology for semiconductor manufacturing: A multilevel and regularization methods-based approach

In semiconductor manufacturing, wafer quality control strongly relies on product monitoring and physical metrology. However, the involved metrology operations, generally performed by means of scanning electron microscopes, are particularly cost-intensive and time-consuming. For this reason, in common practice a small subset only of a productive lot is measured at the metrology stations and it is devoted to represent the entire lot. Virtual Metrology (VM) methodologies are used to obtain reliable predictions of metrology results at process time, without actually performing physical measurements. This goal is usually achieved by means of statistical models and by linking process data and context information to target measurements. Since semiconductor manufacturing processes involve a high number of sequential operations, it is reasonable to assume that the quality features of a given wafer (such as layer thickness and critical dimensions) depend on the whole processing and not on the last step before measurement only. In this paper, we investigate the possibilities to enhance VM prediction accuracy by exploiting the knowledge collected in the previous process steps. We present two different schemes of multi-step VM, along with dataset preparation indications. Special emphasis is placed on regression techniques capable of handling high-dimensional input spaces. The proposed multi-step approaches are tested on industrial production data.     

A general learning framework using local and global regularization

In this paper, we propose a general learning framework based on local and global regularization. In the local regularization part, our algorithm constructs a regularized classifier for each data point using its neighborhood, while the global regularization part adopts a Laplacian regularizer to smooth the data labels predicted by those local classifiers. We show that such a learning framework can easily be incorporated into either unsupervised learning, semi-supervised learning, and supervised learning paradigm. Moreover, many existing learning algorithms can be derived from our framework. Finally we present some experimental results to show the effectiveness of our method.     

热传导方程反问题的Tikhonov正则化法

为了分析热传导方程反问题所涉及的初始条件.论文把这一类问题转化成第一类Fredholm积分方程,运用Tikhonov正则化的反演法和牛顿法获取正则化参数,得到这一问题的数值解.通过数值实验,验证了这一算法在实际应用中的有效性.     

Segmentation of ARX-models using sum-of-norms regularization

Segmentation of time-varying systems and signals into models whose parameters are piecewise constant in time is an important and well studied problem. Here it is formulated as a least-squares problem with sum-of-norms regularization over the state parameter jumps, a generalization of ?₁-regularization. A nice property of the suggested formulation is that it only has one tuning parameter, the regularization constant which is used to trade-off fit and the number of segments.     

An Algorithm for Image Denoising with Automatic Noise Estimate

In this paper, we propose a general algorithm for image denoising when no a priori information on the noise is available. The image denoising problem is formulated as an inequality constrained minimization problem where the objective is a general convex regularization functional and the right-hand side of the constraint depends on the noise norm and is not known. The proposed method is an iterative procedure which, at each iteration, automatically computes both an approximation of the noise norm and an approximate solution of the minimization problem. Experimental results demonstrate the effectiveness of the proposed automatic denoising procedure.

基于动态参数的Tikhonov方法在电容成像图像重建中的应用

该文针对电容成像(ECT)图像重建问题的病态性,采用Tikhonov正则化方法进行图像重建,并选用三种方法动态选择正则化参数。仿真结果表明对某些流型分布,用文中所述方法得到的重建结果优于目前普遍使用的线性反投影(LBP)算法。该方法为提高ECT图像重建质量提供了新的途径。     

基于替代函数及贝叶斯框架的1范数ELM算法

针对极端学习机 (Extreme learning machine, ELM)算法的不适定问题和模型规模控制问题,本文提出基于1范数正则项的改进型ELM算法. 通过在二次损失函数基础上引入1范数正则项以控制模型规模,改善ELM的泛化能力.此外,为简化1范数 正则化方法的求解过程,利用边际优化方法,构建适当的替代函数,以便于采用贝叶斯方法代替计算复杂的 交叉检验方法,并实现正则化参数的自适应估计.仿真结果表明,本文所提算法能够有效简化模型结构,并 保持较高的预测精度.     

Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization

Some regularization methods, including the group lasso and the adaptive group lasso, have been developed for the automatic selection of grouped variables (factors) in conditional mean regression. In many practical situations, such a problem arises naturally when a set of dummy variables is used to represent a categorical factor and/or when a set of basis functions of a continuous variable is included in the predictor set. Complementary to these earlier works, the simultaneous and automatic factor selection is examined in quantile regression. To incorporate the factor information into regularized model fitting, the adaptive sup-norm regularized quantile regression is proposed, which penalizes the empirical check loss function by the sum of factor-wise adaptive sup-norm penalties. It is shown that the proposed method possesses the oracle property. A simulation study demonstrates that the proposed method is a more appropriate tool for factor selection than the adaptive lasso regularized quantile regression.     

全局和局部结构内容自适应正则化的单幅图像超分辨模型

目的 基于正则化的重建是单幅图像超分辨的重要方法之一.其中,如何构造合适的图像先验,增强超分辨重建过程中的边缘和纹理保持能力是该类方法的关键.提出一个全局和局部结构内容自适应正则化的单幅图像超分辨模型.方法 该模型综合了图像梯度的全局非高斯性和局部结构方向自适应回归特性.首先,利用广义高斯分布拟合图像梯度模的重尾特性,由最大后验概率框架构造了图像全局内容感知的lα(0＜α＜1)范数稀疏性度量;然后,利用图像局部内容的各向异性相关性,给出基于Geman-McClure(GM)权函数加权的局部结构方向自适应回归先验;最后利用半二次惩罚和变量分裂法,设计了该优化模型快速求解的超分辨算法.结果 实验结果表明:在客观评价上,本文方法在峰值信噪比与结构相似度两方面优于现有的一些超分辨方法,在主观视觉效果上,能够很好的恢复图像的纹理细节和边缘信息.结论 基于全局和局部结构内容自适应正则化的单幅图像超分辨方法在保持图像边缘和恢复图像纹理细节方面取得较好的重建性能.     

结合栈式监督AE与可变加权ELM的回归预测模型

在现代工业生产过程中,许多关键变量与产品质量或生产效率密切相关,关键变量的实时监测是实现利润最大化及节能降耗的有效途径。针对回归预测任务中目标特征提取不全面、预测精度较低等问题,提出一种基于栈式监督自编码器与可变加权极限学习机的回归预测模型。通过堆叠多层自编码器并在每层自编码器中添加回归网络,同时以有监督方式对栈式自编码器（SAE）进行逐层预训练,得到与输出变量相关的特征表示。利用反向传播算法对网络参数进行微调,优化自编码器模型参数。在分析提取特征与输出变量的相关性基础上,对极限学习机（ELM）的输入权值和偏置进行加权得到预测结果。实验结果表明,与基于ELM和SAE-ELM的回归预测模型相比,该模型在多晶硅铸锭的G6产品数据集上的均方根误差降低0.056 7和0.011 2、决定系数提高0.489 3和0.290 3,具有更高的回归预测准确性及更强的鲁棒性与泛化性能。     

应用谱回归和图正则最小二乘回归的数据降维

数据降维对于提高高维数据处理的效率具有重要意义,稀疏编码是目前受到广泛关注的主流降维方法。针对该方法在降维过程中不能保持样本空间几何结构信息的不足,提出一种基于谱回归和图正则最小二乘回归的改进方案,以2个图像数据集和2个基因表达数据集为样本的实验表明该方法优于未加改进的稀疏编码降维法。     

A new investigation into regularization techniques for the method of fundamental solutions

This study examines different regularization approaches to investigate the solution stability of the method of fundamental solutions (MFS). We compare three regularization methods in conjunction with two different regularization parameters to find the optimal stable MFS scheme. Meanwhile, we have investigated the relationship among the condition number, the effective condition number, and the MFS solution accuracy. Numerical results show that the damped singular value decomposition under the parameter choice of the generalized cross-validation performs the best in terms of the MFS stability analysis. We also find that the condition number is a superior criterion to the effective condition number.     

A comprehensive survey on regularization strategies in machine learning

In machine learning, the model is not as complicated as possible. Good generalization ability means that the model not only performs well on the training data set, but also can make good prediction on new data. Regularization imposes a penalty on model’s complexity or smoothness, allowing for good generalization to unseen data even when training on a finite training set or with an inadequate iteration. Deep learning has developed rapidly in recent years. Then the regularization has a broader definition: regularization is a technology aimed at improving the generalization ability of a model. This paper gave a comprehensive study and a state-of-the-art review of the regularization strategies in machine learning. Then the characteristics and comparisons of regularizations were presented. In addition, it discussed how to choose a regularization for the specific task. For specific tasks, it is necessary for regularization technology to have good mathematical characteristics. Meanwhile, new regularization techniques can be constructed by extending and combining existing regularization techniques. Finally, it concluded current opportunities and challenges of regularization technologies, as well as many open concerns and research trends.     

基于低频唤醒和极限学习机的无线定位系统

设计了一种基于低频(LF)唤醒技术和极限学习机(ELM)分类算法的无线定位系统.实现了低频唤醒、射频应答的电路结构和通信回路,实现了有源应答器的超低待机监听.通过比较各类型的多种射频定位算法,选用基于ELM分类的定位算法,对低频唤醒接收信号强度指示(RSSI)数据进行分类并实现定位,有效降低了定位算法在线阶段的计算量,在单片机系统中实现了实时定位计算.测试结果表明:定位系统在有效范围内定位精度可达15 cm,定位正确率可达95％以上,在定位精度和稳定性方面明显优于超高频(UHF)频段射频定位系统.