A Variational Approach to Remove Outliers and Impulse Noise

We consider signal and image restoration using convex cost-functions composed of a non-smooth data-fidelity term and a smooth regularization term. We provide a convergent method to minimize such cost-functions. In order to restore data corrupted with outliers and impulsive noise, we focus on cost-functions composed of an ?₁ data-fidelity term and an edge-preserving regularization term. The analysis of the minimizers of these cost-functions provides a natural justification of the method. It is shown that, because of the ?₁ data-fidelity, these minimizers involve an implicit detection of outliers. Uncorrupted (regular) data entries are fitted exactly while outliers are replaced by estimates determined by the regularization term, independently of the exact value of the outliers. The resultant method is accurate and stable, as demonstrated by the experiments. A crucial advantage over alternative filtering methods is the possibility to convey adequate priors about the restored signals and images, such as the presence of edges. Our variational method furnishes a new framework for the processing of data corrupted with outliers and different kinds of impulse noise.     

基于非负邻域嵌入和非局部正则化的单帧图像超分辨率重建算法

单帧图像超分辨率重建是指利用一幅低分辨率图像,通过相应的算法来获取一幅高分辨率图像的技术。提出了一种基于 非负邻域嵌入和 非局部正则化 的单帧图像超分辨率重建算法,以弥补传统邻域嵌入算法的不足。在训练阶段,首先对低分辨率图像预放大2倍,以保证在放大倍数较大时,高、低分辨率图像块之间的邻域关系也能得到较好的保持;在重建阶段,使用非负邻域嵌入来有效地解决近邻数的选取问题;最后利用图像块的非局部相似性构造非局部正则项对重建结果进行修正。实验结果表明,相对于传统算法,本方法的重建结果纹理丰富、边缘清晰。     

Noise-robust semi-supervised learning via fast sparse coding

This paper presents a novel noise-robust graph-based semi-supervised learning algorithm to deal with the challenging problem of semi-supervised learning with noisy initial labels. Inspired by the successful use of sparse coding for noise reduction, we choose to give new L₁-norm formulation of Laplacian regularization for graph-based semi-supervised learning. Since our L₁-norm Laplacian regularization is explicitly defined over the eigenvectors of the normalized Laplacian matrix, we formulate graph-based semi-supervised learning as an L₁-norm linear reconstruction problem which can be efficiently solved by sparse coding. Furthermore, by working with only a small subset of eigenvectors, we develop a fast sparse coding algorithm for our L₁-norm semi-supervised learning. Finally, we evaluate the proposed algorithm in noise-robust image classification. The experimental results on several benchmark datasets demonstrate the promising performance of the proposed algorithm.     

Feature-aware regularization for sparse online learning

Learning a compact predictive model in an online setting has recently gained a great deal of attention.The combination of online learning with sparsity-inducing regularization enables faster learning with a smaller memory space than the previous learning frameworks.Many optimization methods and learning algorithms have been developed on the basis of online learning with L1-regularization.L1-regularization tends to truncate some types of parameters,such as those that rarely occur or have a small range of values,unless they are emphasized in advance.However,the inclusion of a pre-processing step would make it very difficult to preserve the advantages of online learning.We propose a new regularization framework for sparse online learning.We focus on regularization terms,and we enhance the state-of-the-art regularization approach by integrating information on all previous subgradients of the loss function into a regularization term.The resulting algorithms enable online learning to adjust the intensity of each feature’s truncations without pre-processing and eventually eliminate the bias of L1-regularization.We show theoretical properties of our framework,the computational complexity and upper bound of regret.Experiments demonstrated that our algorithms outperformed previous methods in many classification tasks.     

基于洛伦兹范数的医学图像超分辨率重建研究

在超分辨率图像重建(SR)模型中,为了达到良好的重建效果,选择一个合适的代价函数是研究的重点。采用SR重建模型中的差错项选择了洛伦兹范数,正则化项选择了吉洪诺夫正则化,重建过程采用了迭代方法。提出的算法可以有效地解决医学图像SR重建过程中的去异值点和图像边缘保持的两大关键问题,达到良好的重建效果。为了验证上述算法的有效性,就一系列添加了运动模糊和不同噪声的低分辨率MRI医学图像进行了SR重建,并且与基于L2范数的重建算法的重建效果进行了比较分析。实验结果显示,所提算法具有良好的实用性和有效性。     

Multiplicative Denoising Based on Linearized Alternating Direction Method Using Discrepancy Function Constraint

The multiplicative noise (speckle) in coherent imaging systems such as synthetic aperture radar makes it difficult to interpret observed images. Recently, the total variation (TV) models have received much interest in removing the speckle due to the strong edge preserving ability and low computational cost of the TV regularizer. However, the classical methods have difficulties in two aspects: one is how to efficiently compute the solution of the models with special data-fidelity terms, the other is how to choose the regularization parameter since the variational models are rather sensitive to the parameter. In this paper, we propose a new linearized alternating direction method, which is able to handle the data-fidelity term efficiently, and meanwhile estimate the optimal value of the regularization parameter exactly based on a discrepancy function constraint. We further establish the global convergence of the proposed algorithm. Numerical experiments demonstrate that our methods overall outperform the current state-of-the-art methods for multiplicative noise removal.     

Statistical Multiresolution Estimation for Variational Imaging: With an Application in Poisson-Biophotonics

In this paper we present a spatially-adaptive method for image reconstruction that is based on the concept of statistical multiresolution estimation as introduced in Frick et al. (Electron. J. Stat. 6:231–268, 2012). It constitutes a variational regularization technique that uses an ? _∞-type distance measure as data-fidelity combined with a convex cost functional. The resulting convex optimization problem is approached by a combination of an inexact alternating direction method of multipliers and Dykstra’s projection algorithm. We describe a novel method for balancing data-fit and regularity that is fully automatic and allows for a sound statistical interpretation. The performance of our estimation approach is studied for various problems in imaging. Among others, this includes deconvolution problems that arise in Poisson nanoscale fluorescence microscopy.     

A pruning algorithm with L1/2regularizer for extreme learning machine简

Compared with traditional learning methods such as the back propagation （BP） method, extreme learning machine provides much faster learning speed and needs less human intervention, and thus has been widely used. In this paper we combine the L1/2 regularization method with extreme learning machine to prune extreme learning machine. A variable learning coefficient is employed to prevent too large a learning increment. A numerical experiment demonstrates that a network pruned by L1/2 regularization has fewer hidden nodes but provides better performance than both the original network and the network pruned by L2 regularization.     

A local structure adaptive super-resolution reconstruction method based on BTV regularization

Super-resolution (SR) image reconstruction has been one of the hottest research fields in recent years. The main idea of SR is to utilize complementary information from a set of low resolution (LR) images of the same scene to reconstruct a high-resolution image with more details. Under the framework of the regularization based SR, this paper presents a local structure adaptive BTV regularization based super-resolution reconstruction method to overcome the shortcoming of the Bilateral Total Variation (BTV) super resolution reconstruction model. The proposed method adaptively chooses prior model and regularization parameter according to the local structures. Experimental results show that the proposed method can get better reconstruction results and significantly reduces the manual workload of the regularization parameter selection.     

A novel L1/2 regularization shooting method for Cox’s proportional hazards model

Nowadays, a series of methods are based on a L ₁ penalty to solve the variable selection problem for a Cox’s proportional hazards model. In 2010, Xu et al. have proposed a L _1/2 regularization and proved that the L _1/2 penalty is sparser than the L ₁ penalty in linear regression models. In this paper, we propose a novel shooting method for the L _1/2 regularization and apply it on the Cox model for variable selection. The experimental results based on comprehensive simulation studies, real Primary Biliary Cirrhosis and diffuse large B cell lymphoma datasets show that the L _1/2 regularization shooting method performs competitively.     

基于结构张量的视频超分辨率算法

针对传统正则化超分辨率（SR）重建模型中,正则化参数选择过大会使重建结果模糊,导致边缘和纹理等细节丢失,选择过小模型去噪能力又不足的问题,提出一种基于结构张量的双正则化参数的视频超分辨率重建算法。首先,利用局部结构张量对图像进行平滑区域和边缘的检测;然后,利用差异曲率对全变分（TV）进行先验信息加权;最后,对平滑区域和边缘采用不同的正则化参数进行超分辨率重建。实验数据显示提出的算法将峰值信噪比（PSNR）提高了0.033～0.11 dB,具有较好的重建效果。实验结果表明：该算法能够有效地提升低分辨率（LR）视频帧重建效果,可应用于低分辨率视频增强、车牌识别和视频监控中感兴趣目标增强等方面。     

Video super-resolution network using detail component extraction and optical flow enhancement algorithm

The video super-resolution (SR) task refers to the use of corresponding low-resolution (LR) frames and multiple neighboring frames to generate high-resolution (HR) frames. Existing deep learning-based approaches usually utilize LR optical flow for video SR tasks. However, the accuracy of LR optical flow is not enough to recover the fine detail part. In this paper, we propose a video SR network that uses optical flow SR and optical flow enhancement algorithms to provide accurate temporal dependency. And extract the detail component of LR adjacent frames as supplementary information for accurate feature extraction. Firstly, the network infers HR optical flow from LR optical flow, and uses the optical flow enhancement algorithm to enhance HR optical flow. Then the processed HR optical flows are used as the input of the motion compensation network. Secondly, we extract detail component to reduce the error caused by motion compensation based on optical flow. Finally, the SR results are generated through the SR network. We perform comprehensive comparative experiments on two datasets: Vid4 and DAVIS. The results show that, compared with other state-of-the-art methods, the proposed video SR method achieves the better performance.

     

对偶算法在紧框架域TV-L1去模糊模型中的应用

目的 建立准确的数学模型并获得有效的求解算法是图像恢复面临的“两难”问题,非光滑型能量泛函有利于准确描述图像的特征,但很难获得有效的求解算法。提出一种拟合项和正则项都是非光滑型能量泛函正则化模型,并推导出有效的交替迭代算法。方法 首先,对系统和椒盐噪声模糊的图像,在紧框架域,用L₁范数描述拟合项,用加权有界变差函数半范数描述正则项。其次,通过引入辅助变量,将图像恢复正则化模型转化为增广拉格朗日模型。再次,利用变量分裂技术,将转化模型分解为两个子问题。最后,利用Fenchel变换和不动点迭代原理,将子问题分别转化为对偶迭代子问题和松弛迭代子问题,并证明迭代子问题的收敛性。结果 针对图像恢复模型的非光滑性,提出一种交替迭代算法。仿真实验表明,相对传统算法,本文算法能有效地恢复系统和椒盐噪声模糊的图像,提高峰值信噪比大约0.51分贝。结论 该正则化模型能有效地恢复图像的边缘,取得较高的峰值信噪比和结构相似测度,具有较快的收敛速度,适用于恢复椒盐噪声模糊的图像。     

Mesh Denoising using Extended ROF Model with L1 Fidelity

This paper presents a variational algorithm for feature‐preserved mesh denoising. At the heart of the algorithm is a novel variational model composed of three components: fidelity, regularization and fairness, which are specifically designed to have their intuitive roles. In particular, the fidelity is formulated as an L₁ data term, which makes the regularization process be less dependent on the exact value of outliers and noise. The regularization is formulated as the total absolute edge‐lengthed supplementary angle of the dihedral angle, making the model capable of reconstructing meshes with sharp features. In addition, an augmented Lagrange method is provided to efficiently solve the proposed variational model. Compared to the prior art, the new algorithm has crucial advantages in handling large scale noise, noise along random directions, and different kinds of noise, including random impulsive noise, even in the presence of sharp features. Both visual and quantitative evaluation demonstrates the superiority of the new algorithm.     

Regularization techniques for ill-posed inverse problems in data assimilation

Optimal state estimation from given observations of a dynamical system by data assimilation is generally an ill-posed inverse problem. In order to solve the problem, a standard Tikhonov, or L₂, regularization is used, based on certain statistical assumptions on the errors in the data. The regularization term constrains the estimate of the state to remain close to a prior estimate. In the presence of model error, this approach does not capture the initial state of the system accurately, as the initial state estimate is derived by minimizing the average error between the model predictions and the observations over a time window. Here we examine an alternative L₁ regularization technique that has proved valuable in image processing. We show that for examples of flow with sharp fronts and shocks, the L₁ regularization technique performs more accurately than standard L₂ regularization.     

Dual Norm Based Iterative Methods for Image Restoration

A convergent iterative regularization procedure based on the square of a dual norm is introduced for image restoration models with general (quadratic or non-quadratic) convex fidelity terms. Iterative regularization methods have been previously employed for image deblurring or denoising in the presence of Gaussian noise, which use L ² (Tadmor et?al. in Multiscale Model. Simul. 2:554?C579, 2004; Osher et?al. in Multiscale Model. Simul. 4:460?C489, 2005; Tadmor et?al. in Commun. Math. Sci. 6(2):281?C307, 2008), and L ¹ (He et?al. in J.?Math. Imaging Vis. 26:167?C184, 2005) data fidelity terms, with rigorous convergence results. Recently, Iusem and Resmerita (Set-Valued Var. Anal. 18(1):109?C120, 2010) proposed a proximal point method using inexact Bregman distance for minimizing a convex function defined on a non-reflexive Banach space (e.g. BV(??)), which is the dual of a separable Banach space. Based on this method, we investigate several approaches for image restoration such as image deblurring in the presence of noise or image deblurring via (cartoon+texture) decomposition. We show that the resulting proximal point algorithms approximate stably a true image. For image denoising-deblurring we consider Gaussian, Laplace, and Poisson noise models with the corresponding convex fidelity terms as in the Bayesian approach. We test the behavior of proposed algorithms on synthetic and real images in several numerical experiments and compare the results with other state-of-the-art iterative procedures based on the total variation penalization as well as the corresponding existing one-step gradient descent implementations. The numerical experiments indicate that the iterative procedure yields high quality reconstructions and superior results to those obtained by one-step standard gradient descent, with faster computational time.     

正则化和交叉验证在组合预测模型中的应用

组合预测模型的权重确定方式对于提高模型精度至关重要,为研究正则化与交叉验证是否能改善组合预测模型的预测效果,提出将正则化和交叉验证应用于基于最小二乘法的组合预测模型.通过在组合模型的最优化求解中分别加入L₁、L₂范数正则化项,并对数据集进行留一交叉验证后发现：L₁、L₂范数正则化都对组合模型的预测精度具有改善效果,且L₁范数正则化比L₂范数正则化对组合预测模型的改善效果更好,并且参与组合预测的单项预测模型越多,正则化的改善效果越好,交叉验证对组合预测模型的改善效果则与给定实验数据量呈现正相关.     

TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization

In this paper an improvement of the optimally pruned extreme learning machine (OP-ELM) in the form of a L₂ regularization penalty applied within the OP-ELM is proposed. The OP-ELM originally proposes a wrapper methodology around the extreme learning machine (ELM) meant to reduce the sensitivity of the ELM to irrelevant variables and obtain more parsimonious models thanks to neuron pruning. The proposed modification of the OP-ELM uses a cascade of two regularization penalties: first a L₁ penalty to rank the neurons of the hidden layer, followed by a L₂ penalty on the regression weights (regression between hidden layer and output layer) for numerical stability and efficient pruning of the neurons. The new methodology is tested against state of the art methods such as support vector machines or Gaussian processes and the original ELM and OP-ELM, on 11 different data sets; it systematically outperforms the OP-ELM (average of 27% better mean square error) and provides more reliable results - in terms of standard deviation of the results - while remaining always less than one order of magnitude slower than the OP-ELM.     

压缩图像空时自适应正则化超分辨率重建

所谓超分辨率(SR)技术就是由低分辨率(LR)图像序列来重建高分辨率(HR)图像的技术,而基于压缩图像的SR技术正成为当前研究的热点。为了提高压缩图像的重建质量,在正则化理论的基础上,通过利用比特流中的信息,提出了一种新颖的空时自适应超分辨率重建算法,该算法先利用正则化代价函数控制时域数据和空域先验信息之间的平衡,使正则化参数在SR重建过程中得到自适应地调整,然后利用迭代梯度下降法进行超分辨率重建。仿真实验表明,该自适应算法比采用传统算法重建的图像的主、客观质量有一定的提高,适合压缩图像的应用。     

The L1/2 regularization method for variable selection in the Cox model

In this paper, we investigate to use the L_1/2 regularization method for variable selection based on the Cox's proportional hazards model. The L_1/2 regularization can be taken as a representative of L_q (0 < q < 1) regularizations and has been demonstrated many attractive properties. To solve the L_1/2 penalized Cox model, we propose a coordinate descent algorithm with a new univariate half thresholding operator which is applicable to high-dimensional biological data. Simulation results based on standard artificial data show that the L_1/2 regularization method can be more accurate for variable selection than Lasso and SCAD methods. The results from real DNA microarray datasets indicate the L_1/2 regularization method performs competitively.