一种具有最优收敛速度的正则化境面下降算法

Pegasos算法是求解大规模支持向量机问题的有效方法,在随机梯度下降过程中植入多阶段循环步骤,能使该算法得到最优的收敛速度O(1/T)。COMID算法是由镜面下降算法推广得到的正则化随机形式,可保证正则化项的结构,但对于强凸的优化问题,该算法的收敛速度仅为O(logT/T)。为此,在COMID算法中引入多阶段循环步骤,提出一种求解L1+L2混合正则化项问题的最优正则化镜面下降算法,证明其具有最优的收敛速度O(1/T),以及与COMID算法相同的稀疏性。在大规模数据库上的实验结果验证了理论分析的正确性和所提算法的有效性。     

一种求解截断L1正则化项问题的坐标下降算法

L1正则化在稀疏学习的研究中起关键作用,使用截断L1正则化项往往可以获得更好的准确率,但却导致了非凸优化问题.目前,主要采用多阶段凸松弛(multi-stage convex relaxation,MSCR)算法进行求解,由于每一阶段都需要求解一个凸优化问题,计算代价较大.为了弥补上述不足,提出了一种求解截断L1正则化项非凸学习问题的坐标下降算法(Non-convex CD).该算法只需在多阶段凸松弛算法的每一阶段执行单步的坐标下降算法,有效降低了计算复杂性.理论分析表明所提出的算法是收敛的.针对Lasso问题,在大规模真实数据库作了实验,实验结果表明,Non-convex CD在取得和MSCR几乎相同准确率的基础上,求解的CPU时间甚至优于求解凸问题的坐标下降方法.为了进一步说明所提算法的性能,进一步研究了Non-convex CD在图像去模糊化中的应用问题.     

基于次梯度的L1正则化Hinge损失问题求解研究

Hinge损失函数是支持向量机(support vector machines,SVM)成功的关键,L1正则化在稀疏学习的研究中起关键作用.鉴于两者均是不可导函数,高阶梯度信息无法使用.利用随机次梯度方法系统研究L1正则化项的Hinge损失大规模数据问题求解.首先描述了直接次梯度方法和投影次梯度方法的随机算法形式,并对算法的收敛性和收敛速度进行了理论分析.大规模真实数据集上的实验表明,投影次梯度方法对于处理大规模稀疏数据具有更快的收敛速度和更好的稀疏性.实验进一步阐明了投影阈值对算法稀疏度的影响.     

弹性网络核极限学习机的多标记学习算法

将正则化极限学习机或者核极限学习机理论应用到多标记分类中,一定程度上提高了算法的稳定性。但目前这些算法关于损失函数添加的正则项都基于L2正则,导致模型缺乏稀疏性表达。同时,弹性网络正则化既保证模型鲁棒性且兼具模型稀疏化学习,但结合弹性网络的极限学习机如何解决多标记问题鲜有研究。基于此,本文提出一种对核极限学习机添加弹性网络正则化的多标记学习算法。首先,对多标记数据特征空间使用径向基核函数映射;随后,对核极限学习机损失函数施加弹性网络正则项;最后,采用坐标下降法迭代求解输出权值以得到最终预测标记。通过对比试验和统计分析表明,提出的算法具有更好的性能表现。     

基于L1/2正则化的超分辨率图像重建算法

为提高图像重建质量,研究超分辨率图像重建技术与稀疏表示理论,提出一种基于L1/2正则化的超分辨率图像重建算法.将L1/2正则化理论运用到字典学习中,利用学习得到的字典重建高分辨率图像.实验结果表明,该算法的图像重建效果优于基于L1正则化的超分辨率图像重建算法.     

一种具有O(1/T)收敛速率的稀疏随机算法

随机梯度下降(stochastic gradient descent,SGD)是一种求解大规模优化问题的简单高效方法,近期的研究表明,在求解强凸优化问题时其收敛速率可通过α-suffix平均技巧得到有效的提升.但SGD属于黑箱方法,难以得到正则化优化问题所期望的实际结构效果.另一方面,COMID(composite objective mirror descent)是一种能保证L1正则化结构的稀疏随机算法,但对于强凸优化问题其收敛速率仅为O(logT?T).主要考虑"L1+Hinge"优化问题,首先引入L2强凸项将其转化为强凸优化问题,进而将COMID算法和α-suffix平均技巧结合得到L1MD-α算法.证明了L1MD-α具有O(1?T)的收敛速率,并且获得了比COMID更好的稀疏性.大规模数据库上的实验验证了理论分析的正确性和所提算法的有效性.     

结合L1和L2正则化约束的隐语义预测模型研究

在大数据领域中预测高维稀疏矩阵中的缺失数据，通常采用随机梯度下降算法构造隐语义模型来对缺失数据进行预测。在随机梯度下降算法来求解模型的过程中经常加入正则化项来提高模型的性能，由于[L1]正则化项不可导，目前在隐语义模型中主要通过加入[L2]正则化项来构建隐语义模型（SGD_LF）。但因为[L1]正则化项能提高模型的稀疏性增强模型求解能力，因此提出一种基于[L1]和[L2]正则化约束的隐语义（SPGD_LF）模型。在通过构建目标函数时，同时引入[L1]和[L2]正则化项。由于目标函数满足利普希茨条件，并通过二阶的泰勒展开对目标函数进行逼近，构造出随机梯度下降的求解器，在随机梯度下降求解隐语义模型的过程中通过软阈值来处理[L1]正则化项所对应的边界优化问题。通过此优化方案，可以更好地表达目标矩阵中的已知数据在隐语义空间中的特征和对应的所属社区关系，提高了模型的泛化能力。通过在大型工业数据集上的实验表明，SPGD_LF模型的预测精度、稀疏性和收敛速度等性能都有显著提高。     

一种求解强凸优化问题的最优随机算法

随机梯度下降(SGD)算法是处理大规模数据的有效方法之一.黑箱方法SGD在强凸条件下能达到最优的O(1/T)收敛速率,但对于求解L1+L2正则化学习问题的结构优化算法,如COMID(composite objective mirror descent)仅具有O(lnT/T)的收敛速率.提出一种能够保证稀疏性基于COMID的加权算法,证明了其不仅具有O(1/T)的收敛速率,还具有on-the-fly计算的优点,从而减少了计算代价.实验结果表明了理论分析的正确性和所提算法的有效性.     

稀疏Tikhonov正则化多分类支持向量机

Tikhonov正则化多分类支持向量机是一种将多分类问题简化为单个优化问题的新型支持向量机.由于Tikhonov正则化多分类支持向量机利用全部类别数据样本构建核函数矩阵,因此不适合大规模数据集的模式分类问题,鉴于该原因,一种稀疏Tikhonov正则化多分类支持量机被建立,其训练算法首先构建样本重要性评价标准,在标准下通过迭代学习获取约简集,最后利用约简集构建核函数矩阵并训练支持向量机.仿真实验结果表明稀疏Tikhonov正则化多分类支持向量机在训练速度和稀疏性方面具有很大的优越性.     

图像复原的Bregman迭代双正则化方法

正则化图像复原最终会导致一个大规模优化问题,提出了一种基于Bregman迭代双正则化的图像复原方法。该方法中目标函数同时考虑总变分正则化和小波域稀疏正则化,在Bregman框架下解决图像复原问题,并且给出了用于解该问题的分裂Bregman迭代算法。该算法将复杂的优化问题转化为几十次简单的迭代加以解决,每次迭代只需几次快速傅里叶变换和收缩操作即可。实验结果表明,提出的复原算法不论从客观改善信噪比还是主观视觉,都能取得很好的效果。同时与目前的复原算法相比,该算法有更快的收敛速度。     

Learning dynamic algorithm portfolios

Algorithm selection can be performed using a model of runtime distribution, learned during a preliminary training phase. There is a trade-off between the performance of model-based algorithm selection, and the cost of learning the model. In this paper, we treat this trade-off in the context of bandit problems. We propose a fully dynamic and online algorithm selection technique, with no separate training phase: all candidate algorithms are run in parallel, while a model incrementally learns their runtime distributions. A redundant set of time allocators uses the partially trained model to propose machine time shares for the algorithms. A bandit problem solver mixes the model-based shares with a uniform share, gradually increasing the impact of the best time allocators as the model improves. We present experiments with a set of SAT solvers on a mixed SAT-UNSAT benchmark; and with a set of solvers for the Auction Winner Determination problem. This work was supported by SNF grant 200020-107590/1.     

回归最小二乘支持向量机的增量和在线式学习算法

首先给出回归最小二乘支持向量机的数学模型，并分析了它的性质，然后在此基础上根据分块矩阵计算公式和核函数矩阵本身的特点设计了支持向量机的增量式学习算法和在线学习算法．该算法能充分利用历史的训练结果，减少存储空间和计算时间．仿真实验表明了这两种学习方法的有效性．     

老年人跌倒检测算法的研究现状

近些年,老年人的健康问题越来越受到重视,跌倒作为影响老年人健康安全问题的主要原因之一,其研究热度一直居高不下,高质量的跌倒检测算法层出不穷。总结了跌倒检测的研究意义和现有的热门研究方法,分别从单一算法和混合算法的角度概述基于阈值、机器学习与深度学习三个方面的跌倒检测算法,介绍各算法的检测方式、判定方式、总体性能和各类单一算法的优缺点,并且从时间、空间和时空三重维度重点阐述了卷积神经网络在跌倒领域发挥的显著作用及应用;同时介绍了跌倒检测算法所使用的数据集及其特点,便于研究者了解跌倒检测在阈值、机器学习与深度学习方面的最新研究进展。最后,对跌倒检测算法所面临的挑战及未来发展进行了展望。     

Optimization of metal-forming process via a hybrid intelligent optimization technique

In recent years, finite element simulation has been increasingly combined with optimization techniques and applied to optimization of various metal-forming processes. The robustness and efficiency of process optimization are critical factors to obtain ideal results, especially for those complicated metal-forming processes. Gradient-based optimization algorithms are subject to mathematical restrictions of discontinuous searching space, while nongradient optimization algorithms often lead to excessive computation time. This paper presents a novel intelligent optimization approach that integrates machine learning and optimization techniques. An intelligent gradient-based optimization scheme and an intelligent response surface methodology are proposed, respectively. By machine learning based on the rough set algorithm, initial total design space can be reduced to self-continuous hypercubes as effective searching spaces. Then optimization algorithms can be implemented more effectively to find optimal design results. An extrusion forging process and a U channel roll forming process are studied as application samples and the effectiveness of the proposed approach is verified.     

机器学习算法在中医诊疗中的研究综述

机器学习算法包括传统机器学习算法和深度学习算法。传统机器学习算法在中医诊疗领域中的应用研究较多,为探究中医辩证规律提供了参考,也为中医诊疗过程的客观化提供了依据。与此同时,随着其在多个领域不断取得成功,深度学习算法在中医诊疗中的价值越来越多地得到业界的重视。通过对中医诊疗领域中使用到的传统机器学习算法与深度学习算法进行述评,总结了两类算法在中医领域中的研究与应用现状,分析了两类算法的特点以及对中医的应用价值,以期为机器学习算法在中医诊疗领域的进一步研究提供参考。     

基于Spark的分布式大数据机器学习算法

对于大数据而言,机器学习技术是不可或缺的;对于机器学习而言,大规模的数据可以提升模型的精准度。然而复杂的机器学习算法从时间和性能上都急需分布式内存计算这种关键技术。Spark分布式内存计算可以实现算法的并行操作,有利于机器学习算法处理大数据集。因此本文提出在Spark分布式内存环境下实现非线性机器学习算法,其中包括多层可变神经网络、BPPGD SVM、K-means,并在实现的基础上进行数据压缩、数据偏向抽样或者数据加载等方面的优化。为了实现充分配置资源批量运行脚本,本文也实现SparkML调度框架来调度以上优化算法。实验结果表明,优化后的3种算法平均误差降低了40%,平均时间缩短了90%。     

Real-time bidirectional search: coordinated problem solving inuncertain situations

This paper investigates real-time bidirectional search (RTBS) algorithms, where two problem solvers, starting from the initial and goal states, physically move toward each other. To evaluate the RTBS performance, two kinds of algorithms are proposed and are compared to real-time unidirectional search. One is called centralized RTBS where a supervisor always selects the best action from all possible moves of the two problem solvers. The other is called decoupled RTBS where no supervisor exists and the two problem solvers independently select their next moves. Experiments on mazes and n-puzzles show that: 1) in clear situations decoupled RTBS performs better, while in uncertain situations, centralized RTBS becomes more efficient; and 2) RTBS is more efficient than real-time unidirectional search for 15-and 24-puzzles but not for randomly generated mazes. It is shown that the selection of the problem solving organization is the selection of the problem space, which determines the baseline of the organizational efficiency; once a difficult problem space is selected, the local coordination among problem solvers hardly overcome the deficit     

Learning from positive and unlabeled examples

In many machine learning settings, labeled examples are difficult to collect while unlabeled data are abundant. Also, for some binary classification problems, positive examples which are elements of the target concept are available. Can these additional data be used to improve accuracy of supervised learning algorithms? We investigate in this paper the design of learning algorithms from positive and unlabeled data only. Many machine learning and data mining algorithms, such as decision tree induction algorithms and naive Bayes algorithms, use examples only to evaluate statistical queries (SQ-like algorithms). Kearns designed the statistical query learning model in order to describe these algorithms. Here, we design an algorithm scheme which transforms any SQ-like algorithm into an algorithm based on positive statistical queries (estimate for probabilities over the set of positive instances) and instance statistical queries (estimate for probabilities over the instance space). We prove that any class learnable in the statistical query learning model is learnable from positive statistical queries and instance statistical queries only if a lower bound on the weight of any target concept f can be estimated in polynomial time. Then, we design a decision tree induction algorithm POSC4.5, based on C4.5, that uses only positive and unlabeled examples and we give experimental results for this algorithm. In the case of imbalanced classes in the sense that one of the two classes (say the positive class) is heavily underrepresented compared to the other class, the learning problem remains open. This problem is challenging because it is encountered in many real-world applications.     

Efficient computation for Whittaker-Henderson smoothing

Efficient algorithms that compute both the estimates and the generalized cross-validation score for the problem of Whittaker-Henderson smoothing are presented. Algorithm efficiency results from carefully exploiting the problem's rich structure to reduce execution time and memory use. The algorithms are much faster than existing ones, and use significantly less memory. MATLAB M-files are included.     

回归核极限学习机的多标记学习算法

基于极限学习机(ELM)的多标记学习算法多使用ELM分类模式,忽略标记之间存在的相关性.为此,文中提出结合关联规则与回归核极限学习机的多标记学习算法(ML-ASRKELM).首先通过关联规则分析标记空间,提取标记之间的规则向量.然后通过提出的多标记回归核极限学习机(ML-RKELM)得出预测结果.若规则向量不为空,将规则向量与预测结果运算得出最终预测结果,否则最终结果即为ML-RKELM的预测结果.对比实验表明ML-ASRKELM与ML-RKELM性能较优,统计假设检验进一步说明文中算法的有效性.