一种基于L1范数正则化的回声状态网络

针对回声状态网络存在的病态解以及模型规模控制问题,本文提出一种基于L1范数正则化的改进回声状态网络.该方法通过在目标函数中添加L1范数惩罚项,提高模型求解的数值稳定性,同时借助于L1范数正则化的特征选择能力,控制网络的复杂程度,防止出现过拟合.对于L1范数正则化的求解,采用最小角回归算法计算正则化路径,通过贝叶斯信息准则进行模型选择,避免估计正则化参数.将模型应用于人造数据和实际数据的时间序列预测中,仿真结果证明了本文方法的有效性和实用性.     

L1+L2正则化逻辑斯蒂模型分类算法

提出一种L1+L2范数正则化逻辑斯蒂模型分类算法。该算法引入L2范数正则化,解决L1正则化逻辑斯蒂算法迭代过程奇异问题,通过引入样本向量的扩展和新的权值向量完成L1范数非平滑问题,最终使用共轭梯度方法求解经过转化的最优化问题。在各种实际数据集上的实验结果表明,该算法优于L2范数、L1范数和Lp范数正则化逻辑斯蒂模型,具有较好的特征选择和分类性能。     

组合范数正则化稀疏编码和自适应加权残差的鲁棒跟踪

针对基于稀疏表示的目标跟踪中编码系数采用L_0或L_1范数正则,易造成NP难优化或预估偏差增大等问题,提出一种基于贝叶斯框架下的组合范数正则化稀疏编码和自适应加权残差的鲁棒跟踪算法.首先提出组合范数正则化稀疏编码,对目标函数编码系数同时进行L_0和L_1正则,根据其贡献程度赋予不同的权值,以增强目标外观模型的鲁棒性;其次在目标函数中引入残差项,赋予其自适应权重来缓解噪声、腐蚀和光照等离群子干扰;最后求解目标函数最小化所涉及的非凸病态问题,在加速近邻梯度算法框架下提出一种广义阈值法来迭代求解最优值.采用大量具有挑战性的序列进行实验的结果表明,与现阶段其他主流算法相比,该算法具有更好的鲁棒性.     

Ll-norm Regularized SVM Clustering Algorithm

提出L1范数正则化支持向量机(SVM)聚类算法.该算法能够同时实现聚类和特征选择功能.给出LI范数正则化SVM聚类原问题和对偶问题形式,采用类似迭代坐标下降的方法求解困难的混合整数规划问题.在多组数据集上的实验结果表明,L1范数正则化SVM聚类算法聚类准确率与L2范数正则化SVM聚类算法相近,而且能够实现特征选择.     

紧框架域混合正则化模型在图像恢复中的应用

目的 有界变差函数容易造成恢复图像纹理信息丢失,并产生虚假边缘,为克服此缺点,在紧框架域,提出一种保护图像纹理信息,抑制虚假边缘产生的混合正则化模型,并推导出交替方向迭代乘子算法。方法 首先,在紧框架域,对系统和泊松噪声模糊的图像,用Kullback-Leibler函数作为拟合项,用有界变差函数半范数和L₁范数组成混合正则项,二者加权组成能量泛函正则化模型。其次,分析混合正则化模型解的存在性和唯一性。再次,通过引入辅助变量,利用交替方向迭代乘子算法,将混合正则化模型最小化问题分解为4个容易处理的子问题。最后,子问题交替迭代形成有效的优化算法。结果 紧框架域混合正则化模型有效地克服有界变差函数容易导致纹理信息丢失、产生虚假边缘的不足。相对经典算法,本文算法提高峰值信噪比大约0.10.7 dB。结论 与其他图像恢复正则化模型相比,本文算法有利于保护图像的纹理,抑制虚假边缘,取得较高的峰值信噪比和结构相似测度,适用于恢复系统和泊松噪声模糊的图像。     

基于Bregman迭代的CT图像重建算法

针对大规模集成电路领域CT重建图像的特点,提出TV约束条件下采用l1范数作正则项的重建模型,并给出了基于Bregman迭代的模型求解算法.算法分为两步: 1)采用Bregman迭代求解图像的l1范数作为正则项,误差的加权l2范数作为保真项的约束极值问题;2) 采用TV约束对1)中得到的重建图像进行修正.算法对TV约束条件下采用l1作正则项的重建模型分开求解,降低了算法的复杂度,加快了收敛速度.算法在稀疏投影数据下可以快速重建CT图像且质量较好.本文采用经典的Shepp-Logan图像进行仿真实验并对实际得到的电路板投影数据进行重建,结果表明该算法可满足重建质量要求且重建速度有较大提升.     

对偶算法在紧框架域TV-L1去模糊模型中的应用

目的 建立准确的数学模型并获得有效的求解算法是图像恢复面临的“两难”问题,非光滑型能量泛函有利于准确描述图像的特征,但很难获得有效的求解算法。提出一种拟合项和正则项都是非光滑型能量泛函正则化模型,并推导出有效的交替迭代算法。方法 首先,对系统和椒盐噪声模糊的图像,在紧框架域,用L₁范数描述拟合项,用加权有界变差函数半范数描述正则项。其次,通过引入辅助变量,将图像恢复正则化模型转化为增广拉格朗日模型。再次,利用变量分裂技术,将转化模型分解为两个子问题。最后,利用Fenchel变换和不动点迭代原理,将子问题分别转化为对偶迭代子问题和松弛迭代子问题,并证明迭代子问题的收敛性。结果 针对图像恢复模型的非光滑性,提出一种交替迭代算法。仿真实验表明,相对传统算法,本文算法能有效地恢复系统和椒盐噪声模糊的图像,提高峰值信噪比大约0.51分贝。结论 该正则化模型能有效地恢复图像的边缘,取得较高的峰值信噪比和结构相似测度,具有较快的收敛速度,适用于恢复椒盐噪声模糊的图像。     

p范数正则化支持向量机分类算法

L2范数罚支持向量机(Support vector machine,SVM)是目前使用最广泛的分类器算法之一,同时实现特征选择和分类器构造的L1范数和L0范数罚SVM算法也已经提出.但是,这两个方法中,正则化阶次都是事先给定,预设p=2或p=1.而我们的实验研究显示,对于不同的数据,使用不同的正则化阶次,可以改进分类算法的预测准确率.本文提出p范数正则化SVM分类器算法设计新模式,正则化范数的阶次p可取范围为02范数罚SVM,L1范数罚SVM和L0范数罚SVM.     

基于AdaBoost的弹性网型正则化多核学习算法

在正则化多核学习中,稀疏的核函数权值会导致有用信息丢失和泛化性能退化,而通过非稀疏模型选取所有核函数则会产生较多的冗余信息并对噪声敏感。针对上述问题,基于AdaBoost框架提出一种弹性网型正则化多核学习算法。在迭代选取基本分类器时对核函数的权值进行弹性网型正则化约束,即混合L_1范数和L_p范数约束,构造基于多个基本核最优凸组合的基本分类器,并将其集成到最终的强分类器中。实验结果表明,该算法在保留集成算法优势的同时,能够实现核函数权值稀疏性和非稀疏性的平衡,与L_1-MKL和L_p-MKL算法相比,能够以较少的迭代次数获得分类精度较高的分类器。     

L1范数正则化SVM聚类算法

提出L1范数正则化支持向量机(SVM)聚类算法。该算法能够同时实现聚类和特征选择功能。给出L1范数正则化SVM聚类原问题和对偶问题形式,采用类似迭代坐标下降的方法求解困难的混合整数规划问题。在多组数据集上的实验结果表明,L1范数正则化SVM聚类算法聚类准确率与L2范数正则化SVM聚类算法相近,而且能够实现特征选择。     

L_(2,1)范数正则化的不相关判别分析及其在人脸识别中的应用

对高维数据降维并选取有效特征对分类起着关键作用。针对人脸识别中存在的高维和小样本问题,从特征选取和子空间学习入手,提出了一种L_(2,1)范数正则化的不相关判别分析算法。该算法首先对训练样本矩阵进行奇异值分解;然后通过一系列变换,将原非线性的Fisher鉴别准则函数转化为线性模型;最后加入L_(2,1)范数惩罚项进行求解,得到一组最佳鉴别矢量。将训练样本和测试样本投影到该低维子空间中,利用最近欧氏距离分类器进行分类。由于加入了L_(2,1)范数惩罚项,该算法能使特征选取和子空间学习同时进行,有效改善识别性能。在ORL、YaleB及PIE人脸库上的实验结果表明,算法在有效降维的同时能进一步提高鉴别能力。     

基于Radon变换的运动模糊图像恢复

提出了一种快速自然运动模糊图像恢复算法,采用一种新的基于Radon变换算法来确定模糊核函数;在确定模糊核函数后,对于模糊图像的恢复采用了一种改进的基于l1范数和l2范数混合保真项的变分图像恢复算法。实验结果表明,与Fergus的算法和Levinss的算法比较,所提算法对于一类线性运动占主要因素的强噪声模糊图像的恢复具有更快的速度和良好的恢复效果。     

改进的弹性网模型在深度神经网络中的应用

由于具有较高的模型复杂度，深层神经网络容易产生过拟合问题，为了减少该问题对网络性能的不利影响，提出一种基于改进的弹性网模型的深度学习优化方法。首先，考虑到变量之间的相关性，对弹性网模型中的L1范数的不同变量进行自适应加权，从而得到L2范数与自适应加权的L1范数的线性组合。其次，将改进的弹性网络模型与深度学习的优化模型相结合，给出在这种新正则项约束下求解神经网络参数的过程。然后，推导出改进的弹性网模型在神经网络优化中具有群组选择能力和Oracle性质，进而从理论上保证该模型是一种更加鲁棒的正则化方法。最后，在多个回归问题和分类问题的实验中，相对于L1、L2和弹性网正则项，该方法的回归测试误差可分别平均降低87.09、88.54和47.02，分类测试准确度可分别平均提高3.98、2.92和3.58个百分点。由此，在理论和实验两方面验证了改进的弹性网模型可以有效地增强深层神经网络的泛化能力，提升优化算法的性能，解决深度学习的过拟合问题。     

基于图像局部梯度L0范数正规化的图像分解算法

针对基于梯度L0范数正规化的变分泛函最优化分解图像时误判噪声梯度为边缘梯度的问题,提出一种基于图像局部梯度的L0范数正规化的图像分解算法。该算法构造了一个由保真函数和正则项构成的适用于图像分解的变分泛函,其中正则项用图像的局部梯度的L0范数进行估计,进而通过求解泛函的最小值,以分解出图像的结构信息(即图像的边缘)。与直接基于图像一阶梯度的L0范数的分解算法相比,该算法可以去除噪声梯度的干扰,从而使分解出的图像边缘中不含有噪声。实验结果表明,该算法在分解图像结构和纹理时,既能很好地把边缘保留在图像结构层中,也可把噪声分解到图像结构层外。     

Unsupervised change detection in SAR images using curvelet and L1-norm based soft segmentation

In this article, we propose a novel unsupervised change detection method for synthetic aperture radar (SAR) images. First, we generate a difference image as a weighted average of a log-ratio image and a mean-ratio image, which has the advantage of enhancing the information of changed regions and restraining the information of unchanged background regions simultaneously. Second, we propose a variational soft segmentation model based on non-differentiable curvelet regularization and L1-norm fidelity. Numerically, by using the split Bregman technique for curvelet regularization term and reformulating the L1-norm fidelity as weighted L2-norm fidelity, we get an effective algorithm in which each sub-problem has a closed-form solution. The numerical experiments and comparisons with several existing methods show that the proposed method is promising, with not only high robustness to non-Gaussian noise or outliers but also high change detection accuracy. Moreover, the proposed method is good at detecting fine-structured change areas. Especially, it outperforms other methods in preserving edge continuity and detecting curve-shaped changed areas.     

Self-representation graph feature selection method for classification

This paper proposed a novel feature selection method that includes a self-representation loss function, a graph regularization term and an \({l_{2,1}}\)-norm regularization term. Different from traditional least square loss function which focuses on achieving the minimal regression error between the class labels and their corresponding predictions, the proposed self-representation loss function pushes to represent each feature with a linear combination of its relevant features, aim at effectively selecting representative features and ensuring the robustness to outliers. The graph regularization terms include two kinds of inherent information, i.e., the relationship between samples (the sample–sample relation for short) and the relationship between features (the feature–feature relation for short). The feature–feature relation reflects the similarity between two features and preserves the relation into the coefficient matrix, while the sample–sample relation reflects the similarity between two samples and preserves the relation into the coefficient matrix. The \({l_{2,1}}\)-norm regularization term is used to conduct feature selection, aim at selecting the features, which satisfies the characteristics mentioned above. Furthermore, we put forward a new optimization method to solve our objective function. Finally, we feed reduced data into support vector machine (SVM) to conduct classification on real datasets. The experimental results showed that the proposed method has a better performance comparing with state-of-the-art methods, such as k nearest neighbor, ridge regression, SVM and so on.     

Sparse regularization for semi-supervised classification

Manifold regularization (MR) is a promising regularization framework for semi-supervised learning, which introduces an additional penalty term to regularize the smoothness of functions on data manifolds and has been shown very effective in exploiting the underlying geometric structure of data for classification. It has been shown that the performance of the MR algorithms depends highly on the design of the additional penalty term on manifolds. In this paper, we propose a new approach to define the penalty term on manifolds by the sparse representations instead of the adjacency graphs of data. The process to build this novel penalty term has two steps. First, the best sparse linear reconstruction coefficients for each data point are computed by the l¹-norm minimization. Secondly, the learner is subject to a cost function which aims to preserve the sparse coefficients. The cost function is utilized as the new penalty term for regularization algorithms. Compared with previous semi-supervised learning algorithms, the new penalty term needs less input parameters and has strong discriminative power for classification. The least square classifier using our novel penalty term is proposed in this paper, which is called the Sparse Regularized Least Square Classification (S-RLSC) algorithm. Experiments on real-world data sets show that our algorithm is very effective.     

基于洛伦兹范数的医学图像超分辨率重建研究

在超分辨率图像重建(SR)模型中,为了达到良好的重建效果,选择一个合适的代价函数是研究的重点。采用SR重建模型中的差错项选择了洛伦兹范数,正则化项选择了吉洪诺夫正则化,重建过程采用了迭代方法。提出的算法可以有效地解决医学图像SR重建过程中的去异值点和图像边缘保持的两大关键问题,达到良好的重建效果。为了验证上述算法的有效性,就一系列添加了运动模糊和不同噪声的低分辨率MRI医学图像进行了SR重建,并且与基于L2范数的重建算法的重建效果进行了比较分析。实验结果显示,所提算法具有良好的实用性和有效性。     

On extreme learning machine for ε-insensitive regression in the primal by Newton method

In this paper, extreme learning machine (ELM) for ε-insensitive error loss function-based regression problem formulated in 2-norm as an unconstrained optimization problem in primal variables is proposed. Since the objective function of this unconstrained optimization problem is not twice differentiable, the popular generalized Hessian matrix and smoothing approaches are considered which lead to optimization problems whose solutions are determined using fast Newton–Armijo algorithm. The main advantage of the algorithm is that at each iteration, a system of linear equations is solved. By performing numerical experiments on a number of interesting synthetic and real-world datasets, the results of the proposed method are compared with that of ELM using additive and radial basis function hidden nodes and of support vector regression (SVR) using Gaussian kernel. Similar or better generalization performance of the proposed method on the test data in comparable computational time over ELM and SVR clearly illustrates its efficiency and applicability.