An efficient deconvolution algorithm for estimating oxygen consumption during muscle activities

The reconstruction of an unknown input function from noisy measurements in a biological system is an ill-posed inverse problem. Any computational algorithm for its solution must use some kind of regularization technique to neutralize the disastrous effects of amplified noise components on the computed solution. In this paper, following a hierarchical Bayesian statistical inversion approach, we seek estimates for the input function and regularization parameter (hyperparameter) that maximize the posterior probability density function. We solve the maximization problem simultaneously for all unknowns, hyperparameter included, by a suitably chosen quasi-Newton method. The optimization approach is compared to the sampling-based Bayesian approach. We demonstrate the efficiency and robustness of the deconvolution algorithm by applying it to reconstructing the time courses of mitochondrial oxygen consumption during muscle state transitions (e.g., from resting state to contraction and recovery), from the simulated noisy output of oxygen concentration dynamics on the muscle surface. The model of oxygen transport and metabolism in skeletal muscle assumes an in vitro cylindrical structure of the muscle in which the oxygen from the surrounding oxygenated solution diffuses into the muscle and is then consumed by the muscle mitochondria. The algorithm can be applied to other deconvolution problems by suitably replacing the forward model of the system.     

The Bayesian evidence scheme for regularizing probability-density estimating neural networks

Training probability-density estimating neural networks with the expectation-maximization (EM) algorithm aims to maximize the likelihood of the training set and therefore leads to overfitting for sparse data. In this article, a regularization method for mixture models with generalized linear kernel centers is proposed, which adopts the Bayesian evidence approach and optimizes the hyperparameters of the prior by type II maximum likelihood. This includes a marginalization over the parameters, which is done by Laplace approximation and requires the derivation of the Hessian of the log-likelihood function. The incorporation of this approach into the standard training scheme leads to a modified form of the EM algorithm, which includes a regularization term and adapts the hyperparameters on-line after each EM cycle. The article presents applications of this scheme to classification problems, the prediction of stochastic time series, and latent space models.     

Density-driven generalized regression neural networks (DD-GRNN) for function approximation.

This paper proposes a new nonparametric regression method, based on the combination of generalized regression neural networks (GRNNs), density-dependent multiple kernel bandwidths, and regularization. The presented model is generic and substitutes the very large number of bandwidths with a much smaller number of trainable weights that control the regression model. It depends on sets of extracted data density features which reflect the density properties and distribution irregularities of the training data sets. We provide an efficient initialization scheme and a second-order algorithm to train the model, as well as an overfitting control mechanism based on Bayesian regularization. Numerical results show that the proposed network manages to reduce significantly the computational demands of having individual bandwidths, while at the same time, provides competitive function approximation accuracy in relation to existing methods.     

基于替代函数及贝叶斯框架的1范数ELM算法

针对极端学习机 (Extreme learning machine, ELM)算法的不适定问题和模型规模控制问题,本文提出基于1范数正则项的改进型ELM算法. 通过在二次损失函数基础上引入1范数正则项以控制模型规模,改善ELM的泛化能力.此外,为简化1范数 正则化方法的求解过程,利用边际优化方法,构建适当的替代函数,以便于采用贝叶斯方法代替计算复杂的 交叉检验方法,并实现正则化参数的自适应估计.仿真结果表明,本文所提算法能够有效简化模型结构,并 保持较高的预测精度.     

Variational Bayesian sparse additive matrix factorization

Principal component analysis (PCA) approximates a data matrix with a low-rank one by imposing sparsity on its singular values. Its robust variant can cope with spiky noise by introducing an element-wise sparse term. In this paper, we extend such sparse matrix learning methods, and propose a novel framework called sparse additive matrix factorization (SAMF). SAMF systematically induces various types of sparsity by a Bayesian regularization effect, called model-induced regularization. Although group LASSO also allows us to design arbitrary types of sparsity on a matrix, SAMF, which is based on the Bayesian framework, provides inference without any requirement for manual parameter tuning. We propose an efficient iterative algorithm called the mean update (MU) for the variational Bayesian approximation to SAMF, which gives the global optimal solution for a large subset of parameters in each step. We demonstrate the usefulness of our method on benchmark datasets and a foreground/background video separation problem.     

正则化参数求解方法研究

针对BP神经网络算法训练过程中出现的过拟合问题,提出了利用一阶原点矩,二阶原点矩,方差和极大似然估计概念的推广来计算L2正则化中正则化参数λ值的方法。该方法通过对算法数据集[X,Y]中的X矩阵进行运算得到的四个λ值,BP神经网络算法训练时通常采用的是贝叶斯正则化方法,贝叶斯正则化方法存在着对先验分布和数据分布依赖等问题,而利用上述概念的推广计算的参数代入L2正则化的方法简便没有应用条件限制;在BP神经网络手写数字识别的实验中,将该方法与贝叶斯正则化方法应用到实验中后的算法识别结果进行比较,正确率提高了1.14-1.50个百分点;因而计算得到的λ值应用到L2正则化方法与贝叶斯正则化方法相比更能使得BP神经网络算法的泛化能力强,证明了该算法的有效性。     

A New Solution Path Algorithm in Support Vector Regression

In this paper, regularization path algorithms were proposed as a novel approach to the model selection problem by exploring the path of possibly all solutions with respect to some regularization hyperparameter in an efficient way. This approach was later extended to a support vector regression (SVR) model called epsiv -SVR. However, the method requires that the error parameter epsiv be set a priori. This is only possible if the desired accuracy of the approximation can be specified in advance. In this paper, we analyze the solution space for epsiv-SVR and propose a new solution path algorithm, called epsiv-path algorithm, which traces the solution path with respect to the hyperparameter epsiv rather than lambda. Although both two solution path algorithms possess the desirable piecewise linearity property, our epsiv-path algorithm overcomes some limitations of the original lambda-path algorithm and has more advantages. It is thus more appealing for practical use.     

求解大规模问题的多核学习正则化路径算法

多核学习在处理异构、不规则和分布不平坦的样本数据时表现出良好的灵活性和可解释性.针对精确正则化路径算法难以处理大规模样本数据的问题,文中提出正则化路径近似算法.根据采样分布函数进行抽样,在原始核矩阵的基础上生成近似矩阵,同时在拉格朗日乘子向量中抽取对应行,实现矩阵乘积的近似计算,提高多核学习正则化路径的求解效率.最后分析多核学习正则化路径近似算法的近似误差界和计算复杂性.在标准数据集上的实验验证文中算法的合理性和计算效率.     

最小平方误差算法的正则化核形式

最小平方误差算法是最常用的一种经典模式识别和回归分析方法,其目标是使线性函 数输出与期望输出的误差平方和为最小.该文应用满足Meteer条件的核函数和正则化技术,改 造经典的最小平方误差算法,提出了基于核函数和正则化技术的非线性最小平方误差算法,即 最小平方误差算法的正则化核形式,其目标函数包含基于核的非线性函数的输出与期望输出的 误差平方和,及一个适当的正则项.正则化技术可以处理病态问题,同时可以减小解空间和控制 解的推广性,文中采用了三种平方型的正则项,并且根据正则项的概率解释,详细比较了三种正 则项之间的差别.最后,用仿真资料和实际资料进一步分析算法的性能.     

A sparse Bayesian approach for joint feature selection and classifier learning

In this paper we present a new method for Joint Feature Selection and Classifier Learning using a sparse Bayesian approach. These tasks are performed by optimizing a global loss function that includes a term associated with the empirical loss and another one representing a feature selection and regularization constraint on the parameters. To minimize this function we use a recently proposed technique, the Boosted Lasso algorithm, that follows the regularization path of the empirical risk associated with our loss function. We develop the algorithm for a well known non-parametrical classification method, the relevance vector machine, and perform experiments using a synthetic data set and three databases from the UCI Machine Learning Repository. The results show that our method is able to select the relevant features, increasing in some cases the classification accuracy when feature selection is performed.     

基于Moreau-包络的近似平滑迭代磁共振图像重建算法

针对基于压缩感知（CS）的磁共振成像（MRI）稀疏重建中存在的两个非平滑正则项问题,提出了一种基于Moreau包络的近似平滑迭代算法（PSIA）。基于CS的经典MRI稀疏重建是求解一个由最小二乘保真项、小波变换稀疏正则项和总变分（TV）正则项线性组合成的目标函数最小化问题。首先,对目标函数中的小波变换正则项作平滑近似;然后,将数据保真项与平滑近似后的小波正则项的线性组合看成一个新的可以连续求导的凸函数;最后,采用PSIA对新的优化问题进行求解。该算法不仅可以同时处理优化问题中的两个正则约束项,还避免了固定权重带来的算法鲁棒性问题。仿真得到的体模图像及真实磁共振图像的实验结果表明,所提算法与四种经典的稀疏重建算法：共轭梯度（CG）下降算法、TV1范数压缩MRI（TVCMRI）算法、部分k空间重建算法（RecPF）和快速复合分离算法（FCSA）相比,在图像信噪比、相对误差和结构相似性指数上具有更好的重建结果,且在算法复杂度上与现有最快重建算法即FCSA相当。     

一种改进的正则图象

把正则图象的误差分成两部分：一部分为逼近误差，它是和原图象相关的确定型误差，另一部分为噪声传递误差，它是和噪声相关的随机型误差．通过正则图象代替原图象，求得逼近误差的近似值，然后将逼近误差近似值补给正则解，得到改进型正则解．对正则解以及逼近误差的计算，提出了快速算法，并有严密的理论推导．实验结果表明改进型正则图象的恢复效果优于传统正则解．在一敷的正则化方法中，可以借鉴本文提到的方法改进正则解．     

一类多项式光滑函数的逼近精度

针对一类支持向量机的多项式光滑函数,采用二分法求解它们尚未解决的逼近精度问题。为克服二分法可能会漏根的缺点,首先把多项式光滑函数的逼近精度问题表示为一个求逼近函数的最大值问题,把这个逼近函数分成4 段,分别求出每段的最大值,然后得到逼近函数在整个x轴上的最大值。并以1阶和2阶多项式光滑函数为例,用二分法解决了它们的逼近精度问题。研究表明,二分法是求解这类多项式光滑函数逼近精度的有效方法。     

Bayesian regression filters and the issue of priors

We propose a Bayesian framework for regression problems, which covers areas usually dealt with by function approximation. An online learning algorithm is derived which solves regression problems with a Kalman filter. Its solution always improves with increasing model complexity, without the risk of over-fitting. In the infinite dimension limit it approaches the true Bayesian posterior. The issues of prior selection and over-fitting are also discussed, showing that some of the commonly held beliefs are misleading. The practical implementation is summarised. Simulations using 13 popular publicly available data sets are used to demonstrate the method and highlight important issues concerning the choice of priors.     

一种基于L1范数正则化的回声状态网络

针对回声状态网络存在的病态解以及模型规模控制问题,本文提出一种基于L1范数正则化的改进回声状态网络.该方法通过在目标函数中添加L1范数惩罚项,提高模型求解的数值稳定性,同时借助于L1范数正则化的特征选择能力,控制网络的复杂程度,防止出现过拟合.对于L1范数正则化的求解,采用最小角回归算法计算正则化路径,通过贝叶斯信息准则进行模型选择,避免估计正则化参数.将模型应用于人造数据和实际数据的时间序列预测中,仿真结果证明了本文方法的有效性和实用性.     

正则化路径上的支持向量机模型组合方法

模型组合是提高支持向量机泛化性的重要方法,但存在计算效率较低的问题。提出一种基于正则化路径上贝叶斯模型平均的支持向量机模型组合方法,在提高支持向量机泛化性的同时,具有较高的计算效率。基于正则化路径算法建立初始模型集,引入对支持向量机的概率解释。模型的先验可看做是一个高斯过程,模型的后验概率通过贝叶斯公式求得,使用贝叶斯模型平均对模型进行组合。在标准数据集上,实验比较了所提出的模型组合方法与交叉验证及广义近似交叉验证（GACV）方法的性能,验证所提出的模型组合方法的有效性。     

贝叶斯网络分类器近似学习算法

贝叶斯网络在很多领域应用广泛,作为分类器更是一种有效的常用分类方法,它有着很高复杂度,这使得贝叶斯网络分类器在应用中受到诸多限制。通过对贝叶斯网络分类器算法的近似处理,可以有效减少计算量,并且得到令人满意的分类准确率。通过分析一种将判别式算法变为产生式算法的近似方法,介绍了这种算法的近似过程,并将其应用在了贝叶斯网分类算法中。接着对该算法进行分析,利用该算法的稳定性特点,提出Bagging-aCLL 集成分类算法,它进一步提高了该近似算法的分类精度。最后通过实验确定了该算法在分类准确率上确有不错的表现。     

紧框架域混合正则化模型在图像恢复中的应用

目的 有界变差函数容易造成恢复图像纹理信息丢失,并产生虚假边缘,为克服此缺点,在紧框架域,提出一种保护图像纹理信息,抑制虚假边缘产生的混合正则化模型,并推导出交替方向迭代乘子算法。方法 首先,在紧框架域,对系统和泊松噪声模糊的图像,用Kullback-Leibler函数作为拟合项,用有界变差函数半范数和L₁范数组成混合正则项,二者加权组成能量泛函正则化模型。其次,分析混合正则化模型解的存在性和唯一性。再次,通过引入辅助变量,利用交替方向迭代乘子算法,将混合正则化模型最小化问题分解为4个容易处理的子问题。最后,子问题交替迭代形成有效的优化算法。结果 紧框架域混合正则化模型有效地克服有界变差函数容易导致纹理信息丢失、产生虚假边缘的不足。相对经典算法,本文算法提高峰值信噪比大约0.10.7 dB。结论 与其他图像恢复正则化模型相比,本文算法有利于保护图像的纹理,抑制虚假边缘,取得较高的峰值信噪比和结构相似测度,适用于恢复系统和泊松噪声模糊的图像。     

改进的正则化模型在图像恢复中的应用

目的 由拟合项与正则项组成的海森矩阵,如果不具有特殊结构,其逆矩阵计算比较困难,为克服此缺点,提出一种海森矩阵可分块对角化的牛顿投影迭代算法。方法 首先,用L₂范数描述拟合项,用自变量是有界变差函数的复合函数刻画正则项,建立能量泛函正则化模型。其次,引入势函数,将正则化模型转化为增广能量泛函。再次,构造预条件矩阵,使得海森矩阵可分块对角化。最后,为防止牛顿投影迭代算法收敛到局部最优解,采用回溯线性搜索算法和改进的Barzilai-Borwein步长更新准则使得算法全局收敛。结果 针对图像去模糊正则化模型容易使边缘平滑和产生阶梯效应“两难”问题,提出一种新的正则化模型和牛顿投影迭代算法。仿真结果表明,“两难”问题通过本文算法得到了很好的解决。结论 与其他正则化图像去模糊模型相比,本文算法明显改善图像的质量,如有效地保护图像的边缘,抑制阶梯效应,相对偏差和误差较小,较高的峰值信噪比和结构相似测度。     

变分贝叶斯概率数据关联算法

针对杂波环境下的目标跟踪问题,提出了一种基于变分贝叶斯的概率数据关联算法(Variational Bayesian based probabilistic data association algorithm, VB-PDA).该算法首先将关联事件视为一个随机变量并利用多项分布对其进行建模,随后基于数据集、目标状态、关联事件的联合概率密度函数求取关联事件的后验概率密度函数,最后将关联事件的后验概率密度函数引入变分贝叶斯框架中以获取状态近似后验概率密度函数.相比于概率数据关联算法, VB-PDA算法在提高算法实时性的同时在权重Kullback-Leibler (KL)平均准则下获取了近似程度更高的状态后验概率密度函数.相关仿真实验对提出算法的有效性进行了验证.