基于PSO算法的probit模型参数估计

针对二值probit回归模型中的参数估计问题,提出一种基于粒子群优化（PSO）的参数估计算法。该算法采用以最大似然准则作为PSO的适应度函数,建立二值probit回归模型中的参数估计计算模型。数值仿真分析表明,该算法性能较好,回归结果具有较高的拟合优度。     

A Fast Tracking Algorithm for Generalized LARS/LASSO

This letter gives an efficient algorithm for tracking the solution curve of sparse logistic regression with respect to the regularization parameter. The algorithm is based on approximating the logistic regression loss by a piecewise quadratic function, using Rosset and Zhu's path tracking algorithm on the approximate problem, and then applying a correction to get to the true path. Application of the algorithm to text classification and sparse kernel logistic regression shows that the algorithm is efficient.     

精确增量式在线v型支持向量回归机学习算法

为了解决v型支持向量回归机(v-supportvector regression,v-SVR)对偶问题的目标函数中增加的额外线性项从而导致无法产生有效初始解的问题和在绝缘增量调整过程中可能存在的解路径不可行更新问题,提出了精确增量式在线v-SVR学习算法.首先基于v-SVR的等价形式,利用提前调整,宽松的绝缘增量调整和精确的恢复调整有效地解决了v-SVR对偶问题存在的上述问题.然后分别对算法的可行性和有限收敛性进行了理论分析.最后在四个基准测试数据集上的仿真结果进一步验证了该算法的每一步调整都是可靠的,经过有限次数调整最终收敛到最小化问题的最优解,而且与批处理学习算法相比,随着训练样本的增加,算法在缩短学习时间上的优势显著.     

Fitting Sugeno integral for learning fuzzy measures using PAVA isotone regression

The discrete Sugeno integral is an aggregation function particularly suited to aggregation of ordinal inputs. It accounts for inputs interactions, such as redundancy and complementarity, and its learning from empirical data is a challenging optimisation problem. The methods of ordinal regression involve an expensive objective function, whose complexity is quadratic in the number of data. We formulate ordinal regression using a much less expensive objective computed in linear time by the pool-adjacent-violators algorithm. We investigate the learning problem numerically and show the superiority of the new algorithm.     

一种基于结构化学习的排序算法

传统排序算法将排序问题转换成分类或回归问题来求解,这样得到的模型不够精确。对此提出一种新的排序算法,该算法把排序问题看成一个结构化学习过程,即通过训练集来学习一个排序结构。算法首先定义了一个查询级的目标函数,针对算法约束条件太多,难以直接优化,提出使用割平面算法进行求解。对于算法中的“寻找最违约排列”子问题,将其变换成为一个简单的降序排列问题。基于基准数据集的实验表明,相比起传统的排序算法,所提算法更为有效。     

多尺度最小二乘小波支持向量机的回归建模

普通最小二乘支持向量机算法用于多尺度回归建模时精度较低。针对该问题,选取墨西哥草帽小波函数作为最小二乘支持向量机的核函数,设计一种基于小波核的多尺度最小二乘小波支持向量机。在此基础上,通过解二次优化问题求出多尺度回归建模问题的全局最优解,最终得出的多尺度回归模型能够有效地逼近多尺度信号。仿真结果表明,该算法具有较高的精度。     

On extreme learning machine for ε-insensitive regression in the primal by Newton method

In this paper, extreme learning machine (ELM) for ε-insensitive error loss function-based regression problem formulated in 2-norm as an unconstrained optimization problem in primal variables is proposed. Since the objective function of this unconstrained optimization problem is not twice differentiable, the popular generalized Hessian matrix and smoothing approaches are considered which lead to optimization problems whose solutions are determined using fast Newton–Armijo algorithm. The main advantage of the algorithm is that at each iteration, a system of linear equations is solved. By performing numerical experiments on a number of interesting synthetic and real-world datasets, the results of the proposed method are compared with that of ELM using additive and radial basis function hidden nodes and of support vector regression (SVR) using Gaussian kernel. Similar or better generalization performance of the proposed method on the test data in comparable computational time over ELM and SVR clearly illustrates its efficiency and applicability.     

基于最小二乘支持向量回归的水质预测

水质系统是一个开放的、复杂的、非线性动力学系统,具有时变复杂性,针对水质预测方法的研究虽然已经取得了一些成果,但也存在预测精度与计算复杂度等难题。为此,本文提出一种基于最小二乘支持向量回归的水质预测算法。支持向量机是机器学习中一种常用的分类模型,通过核函数将非线性数据从低维映射到高维空间,在高维空间实现线性分类和回归,最小二乘支持向量回归（LS-SVR）利用所有的样本参与回归拟合,使得回归的损失函数不再只与小部分支持向量样本有关,而是由所有样本参与学习修正误差,提高预测精度;同时该算法将标准SVR求解问题由不等式的约束条件及凸二次规划问题转化成线性方程组来求解,提高了运算速度,解决了非线性复杂特性的水质预测问题。     

基于稀疏张量判别分析的人体行为识别

在模式识别中,如何在提取关键特征的同时对样本进行降维与识别是研究的热点之一。在局部Fisher判别分析（LFDA）的基础上,结合张量表示和稀疏分析,本文提出一种基于稀疏张量的特征提取方法：稀疏张量局部Fisher判别分析（STLFDA）。该方法把张量局部Fisher判别分析（TLFDA）算法中特征分解问题转化为线性回归问题,并用弹性网络解决线性回归中的特征选择问题,既满足了张量局部Fisher判别分析的目标,又保证了得到的投影矩阵的稀疏性。通过在Weizmann人体行为数据库上的实验,表明了稀疏张量局部Fisher判别分析（STLFDA）算法的有效性。     

Dynamic parameter inverse analysis of concrete dams based on Jaya algorithm with Gaussian processes surrogate model

A dynamic parameter inverse analysis process for concrete dams based on Gaussian process regression and Jaya algorithm is presented. Gaussian process regression is used to establish a response surface representing the relationship between dynamic elastic modulus and modal parameters (natural frequency and mode shape). The Jaya algorithm is applied for dynamic parameter identification by minimizing the objective function. To verify the performance of the proposed method, we consider a concrete single buttress dam and a hyperbolic concrete arch dam as numerical examples. Numerical results show that Gaussian process regression can significantly improve the parameter identification efficiency without compromising on accuracy. Furthermore, the Jaya algorithm is compared with particle swarm optimization algorithm and genetic algorithm; the results show that the Jaya algorithm is promising in parameter recognition.     

一种基于ExtraTrees的差分隐私保护算法

为在同等隐私保护级别下提高模型的预测准确率并降低误差,提出一种基于ExtraTrees的差分隐私保护算法DiffPETs。在决策树生成过程中,根据不同的准则计算出各特征的结果值,利用指数机制选择得分最高的特征,通过拉普拉斯机制在叶子节点上进行加噪,使算法能够提供ε-差分隐私保护。将DiffPETs算法应用于决策树分类和回归分析中,对于分类树,选择基尼指数作为指数机制的可用性函数并给出基尼指数的敏感度,在回归树上,将方差作为指数机制的可用性函数并给出方差的敏感度。实验结果表明,与决策树差分隐私分类和回归算法相比,DiffPETs算法能有效降低预测误差。     

基于径向基函数的多目标回归特征构建算法

多目标回归（MTR）是一种针对单个样本同时具有多个连续型输出的回归问题。现有的多目标回归算法都基于同一个特征空间学习回归模型,而忽略了各输出目标本身的特殊性质。针对这一问题,提出基于径向基函数的多目标回归特征构建算法。首先,将各目标的输出作为额外的特征对各输出目标进行聚类,根据聚类中心在原始特征空间构成了目标特定特征空间的基;然后,通过径向基函数将原始特征空间映射到目标特定特征空间,构造目标特定的特征,并基于这些目标特定特征构建各输出目标的基回归模型;最后,用基回归模型的输出组成隐藏空间,采用低秩学习算法在其中发掘和利用输出目标之间的关联。在18个多目标回归数据集上进行实验,并把所提算法与层叠单目标回归（SST）、回归器链集成（ERC）、多层、多目标回归（MMR）等经典的多目标回归算法进行对比,结果表明所提算法在14个数据集上都取得了最好的性能,并且在18个数据集上的平均性能排序居第一位。可见所提算法构建的目标特定特征能够提高各输出目标的预测准确性,并结合低秩学习得到输出目标间的关联性以从整体上提升多目标回归的预测性能。     

基于粒子群算法的Logistic回归模型参数估计

针对Logistic回归模型中的参数估计计算复杂难题,提出一种基于粒子群优化算法（PSO）的估计方法。以最大似然准则作为粒子群优化算法的适应度函数,建立了Logistic回归模型中的参数估算模型。数值仿真分析表明,粒子群优化算法可以更精确地计算出相关参数。     

岭参数优化的ELM岭回归学习算法

针对ELM(extreme learning machine,极限学习机)学习算法可能存在的解的奇异问题,提出了岭参数优化的ELM岭回归学习算法(ELMRR).该算法利用岭回归方法代替原有的线性回归算法,以均方根误差为性能指标,采用粒子群优化算法确定最佳岭参数.为了验证该方法的有效性,对函数回归和分类问题进行仿真实验分析,结果表明该方法改善了ELM的预测性能且克服了传统岭回归算法岭参数难以确定的缺点.     

LS-Ensem:一种用于回归的集成算法

借鉴Friedman提出的基于函数空间的梯度下降搜索的思想,提出了一种新的集成学习算法——LS-Ensem算法.该算法只要求个体函数满足一个很宽松的条件,从而避免了每轮迭代中寻找最优个体函数所需的大量计算,显著地降低了算法的计算复杂性.理论分析表明该算法具有指数级收敛速度以及良好的泛化性能,文中还给出了泛化误差的界.仿真结果验证了理论分析的结论,并且还显示出LS-Ensem算法能够有效地抑制过拟合发生.     

Sparse Regression Ensembles in Infinite and Finite Hypothesis Spaces

We examine methods for constructing regression ensembles based on a linear program (LP). The ensemble regression function consists of linear combinations of base hypotheses generated by some boosting-type base learning algorithm. Unlike the classification case, for regression the set of possible hypotheses producible by the base learning algorithm may be infinite. We explicitly tackle the issue of how to define and solve ensemble regression when the hypothesis space is infinite. Our approach is based on a semi-infinite linear program that has an infinite number of constraints and a finite number of variables. We show that the regression problem is well posed for infinite hypothesis spaces in both the primal and dual spaces. Most importantly, we prove there exists an optimal solution to the infinite hypothesis space problem consisting of a finite number of hypothesis. We propose two algorithms for solving the infinite and finite hypothesis problems. One uses a column generation simplex-type algorithm and the other adopts an exponential barrier approach. Furthermore, we give sufficient conditions for the base learning algorithm and the hypothesis set to be used for infinite regression ensembles. Computational results show that these methods are extremely promising.     

基于多层递阶回归分析的轧钢煤气用量预测

以某钢铁企业为背景,基于煤气用户的历史数据,通过多层递阶回归分析建立相应的消耗预报模型,从而对煤气用量进行预测。首先把统计样本中的各个高相关因子作为回归变量进行线性回归处理,然后以回归系数与预报因子的乘积作为对修正量来进行多层递阶预报。这种多层递阶与回归分析方法,既能较好地体现高相关因子在预报模型中的重要作用,又具有较强的适应性,可提高预报精度。实际工业应用证明了方法的有效性。     

一种基于马氏距离的线性判别分析分类算法

对于一个特定的模式识别问题,表达和识别模式的特征具有不同的形式,它们在物理意义上是完全不同的,而且在数量级具有很大差别。该文提出了一种基于马氏距离的线性判别分析分类算法,选取判别函数为马氏距离,可以适用于具有不同类型特征值的分类问题。将该算法应用于UCI中Credit-A、Credit-G、Iris和Vehicle四个数据库的分类,并采用K次交叉验证方法进行实验。从实验结果中可知,与ENTROPY算法和C4．5（8）算法分类效果相比较,该文所提出的线性判别分析算法计算简单,识别率较高,是一种实际可行的分类算法。     

一种用于监控系统的监控指标趋势分析方法

提出一种用于监控系统的监控指标的趋势分析方法,包括两种趋势分析,分别为预测分析和周期性分析,前者通过线性回归算法和指数回归算法预测指标的未来值;后者通过三角函数回归算法计算指标的周期性参数。根据趋势分析结果和事先设定的报警阈值确定是否预先报警,绘制该指标变化的周期性曲线。该方法能够实现网络监控系统在监控指标发生违例前预先报警,自动生成指标变化周期性曲线,为监控系统增加了趋势分析功能。     

Calculation of simplicial depth estimators for polynomial regression with applications

A fast algorithm for calculating the simplicial depth of a single parameter vector of a polynomial regression model is derived. Additionally, an algorithm for calculating the parameter vectors with maximum simplicial depth within an affine subspace of the parameter space or a polyhedron is presented. Since the maximum simplicial depth estimator is not unique, l₁ and l₂ methods are used to make the estimator unique. This estimator is compared with other estimators in examples of linear and quadratic regression. Furthermore, it is shown how the maximum simplicial depth can be used to derive distribution-free asymptotic α-level tests for testing hypotheses in polynomial regression models. The tests are applied on a problem of shape analysis where it is tested how the relative head length of the fish species Lepomis gibbosus depends on the size of these fishes. It is also tested whether the dependency can be described by the same polynomial regression function within different populations.