A multiple-kernel LSSVR method for separable nonlinear system identification

In some nonlinear dynamic systems, the state variables function usually can be separated from the control variables function, which brings much trouble to the identification of such systems. To well solve this problem, an improved least squares support vector regression （LSSVR） model with multiple-kernel is proposed and the model is applied to the nonlinear separable system identification. This method utilizes the excellent nonlinear mapping ability of Morlet wavelet kernel function and combines the state and control variables information into a kernel matrix. Using the composite wavelet kernel, the LSSVR includes two nonlinear functions, whose variables are the state variables and the control ones respectively, in this way, the regression function can gain better nonlinear mapping ability, and it can simulate almost any curve in quadratic continuous integral space. Then, they are used to identify the two functions in the separable nonlinear dynamic system. Simulation results show that the multiple-kernel LSSVR method can greatly improve the identification accuracy than the single kernel method, and the Morlet wavelet kernel is more efficient than the other kernels.     

Parameter selection of support vector regression based on hybrid optimization algorithm and its application

Choosing optimal parameters for support vector regression （SVR） is an important step in SVR. design, which strongly affects the pefformance of SVR. In this paper, based on the analysis of influence of SVR parameters on generalization error, a new approach with two steps is proposed for selecting SVR parameters, First the kernel function and SVM parameters are optimized roughly through genetic algorithm, then the kernel parameter is finely adjusted by local linear search, This approach has been successfully applied to the prediction model of the sulfur content in hot metal. The experiment results show that the proposed approach can yield better generalization performance of SVR than other methods,     

基于凸优化算法的无人水下航行器协同定位

In this paper, a cooperative localization algorithm for autonomous underwater vehicles (AUVs) is proposed. A ``parallel" model is adopted to describe the cooperative localization problem instead of the traditional ``leader-follower" model, and a linear programming associated with convex optimization method is used to deal with the problem. After an unknown-but-bounded model for sensor noise is assumed, bearing and range measurements can be modeled as linear constraints on the configuration space of the AUVs. Merging these constraints induces a convex polyhedron representing the set of all configurations consistent with the sensor measurements. Estimates for the uncertainty in the position of a single AUV or the relative positions of two or more nodes can then be obtained by projecting this polyhedron onto appropriate subspaces of the configuration space. Two different optimization algorithms are given to recover the uncertainty region according to the number of the AUVs. Simulation results are presented for a typical localization example of the AUV formation. The results show that our positioning method offers a good localization accuracy, although a small number of low-cost sensors are needed for each vehicle, and this validates that it is an economical and practical positioning approach compared with the traditional approach.     

Diversity of Pareto front: A multiobjective genetic algorithm based on dominating information

In this paper, the diversity information included by dominating number is analyzed, and the probabilistic relationship between dominating number and diversity in the space of objective function is proved. A ranking method based on dominating number is proposed to build the Pareto front. Without increasing basic Pareto method’s computation complexity and introducing new parameters, a new multiobjective genetic algorithm based on proposed ranking method (MOGA-DN) is presented. Simulation results on function optimization and parameters optimization of control system verify the efficiency of MOGA-DN.     

Dm-KDE： dynamical kernel density estimation by sequences of KDE estimators with fixed number of components over data streams

In many data stream mining applications, traditional density estimation methods such as kemel density estimation, reduced set density estimation can not be applied to the density estimation of data streams because of their high computational burden, processing time and intensive memory allocation requirement. In order to reduce the time and space complexity, a novel density estimation method Dm-KDE over data streams based on the proposed algorithm m-KDE which can be used to design a KDE estimator with the fixed number of kernel components for a dataset is proposed. In this method, Dm-KDE sequence entries are created by algorithm m-KDE instead of all kemels obtained from other density estimation methods. In order to further reduce the storage space, Dm-KDE sequence entries can be merged by calculating their KL divergences. Finally, the probability density functions over arbitrary time or entire time can be estimated through the obtained estimation model. In contrast to the state-of-the-art algorithm SOMKE, the distinctive advantage of the proposed algorithm Dm-KDE exists in that it can achieve the same accuracy with much less fixed number of kernel components such that it is suitable for the scenarios where higher on-line computation about the kernel density estimation over data streams is required. We compare Dm-KDE with SOMKE and M-kernel in terms of density estimation accuracy and running time for various stationary datasets. We also apply Dm-KDE to evolving data streams. Experimental results illustrate the effectiveness of the pro- posed method.     

Nonlinear optimized adaptive trajectory control of helicopter

This paper attempts to develop an optimized adaptive trajectory control system for helicopters based on the dynamic inversion method. This control algorithm is implemented by three time-scale separation architectures. Pseudo control hedging (PCH) is used to protect the adaptive element from actuator saturation nonlinearities and also from the inner-outer-loop interaction. In addition, to augment the attitude control system, two online adaptive architectures that employ a neural network are used. By tuning the neural network based on the system model, a better and faster learning will be achieved, but this is a frustrating and time consuming process. Due to complexity in accurate tuning of neural network, this paper introduces a non-dominated sorting genetic algorithm II (NSGA-II) for off-line optimization of the neural network. Thus, in the proposed method, the neural network can compensate model inversion error caused by the deficiency of full knowledge of helicopter dynamics more accurately. The effectiveness of proposed method is demonstrated by numerical simulations.     

Semi-supervised classification based on Markov Random Field and Robust Error Function

We focus on semi-supervised classifier, where a decision rule is to be learned from labeled and unlabeled data. A model to semi-supervised classification is proposed to overcome the problem induced by mislabeled samples. A new energy function based on robust error function is used in Markov Random Field. Also two algorithms based on iterative condition mode and markov chain monte carlo respectively are designed to infer the label of both labeled and unlabeled samples. Our experiments demonstrate that the proposed method is efficient for real-world dataset.     

Optimizing radial basis function neural network based on rough sets and affinity propagation clustering algorithm

A novel method based on rough sets (RS) and the affinity propagation (AP) clustering algorithm is developed to optimize a radial basis function neural network (RBFNN). First, attribute reduction (AR) based on RS theory, as a preprocessor of RBFNN, is presented to eliminate noise and redundant attributes of datasets while determining the number of neurons in the input layer of RBFNN. Second, an AP clustering algorithm is proposed to search for the centers and their widths without a priori knowledge about the number of clusters. These parameters are transferred to the RBF units of RBFNN as the centers and widths of the RBF function. Then the weights connecting the hidden layer and output layer are evaluated and adjusted using the least square method (LSM) according to the output of the RBF units and desired output. Experimental results show that the proposed method has a more powerful generalization capability than conventional methods for an RBFNN.     

Automatic mass segmentation on mammograms combining random walks and active contour

Accurate mass segmentation on mammograms is a critical step in computer-aided diagnosis (CAD) systems. It is also a challenging task since some of the mass lesions are embedded in normal tissues and possess poor contrast or ambiguous margins. Besides, the shapes and densities of masses in mammograms are various. In this paper, a hybrid method combining a random walks algorithm and Chan-Vese (CV) active contour is proposed for automatic mass segmentation on mammograms. The data set used in this study consists of 1095 mass regions of interest (ROIs). First, the original ROI is preprocessed to suppress noise and surrounding tissues. Based on the preprocessed ROI, a set of seed points is generated for initial random walks segmentation. Afterward, an initial contour of mass and two probability matrices are produced by the initial random walks segmentation. These two probability matrices are used to modify the energy function of the CV model for prevention of contour leaking. Lastly, the final segmentation result is derived by the modified CV model, during which the probability matrices are updated by inserting several rounds of random walks. The proposed method is tested and compared with other four methods. The segmentation results are evaluated based on four evaluation metrics. Experimental results indicate that the proposed method produces more accurate mass segmentation results than the other four methods.     

Preservation of local linearity by neighborhood subspace scaling for solving the pre-image problem

An important issue involved in kernel methods is the pre-image problem. However, it is an ill-posed problem, as the solution is usually nonexistent or not unique. In contrast to direct methods aimed at minimizing the distance in feature space, indirect methods aimed at constructing approximate equivalent models have shown outstanding performance. In this paper, an indirect method for solving the pre-image problem is proposed. In the proposed algorithm, an inverse mapping process is constructed based on a novel framework that preserves local linearity. In this framework, a local nonlinear transformation is implicitly conducted by neighborhood subspace scaling transformation to preserve the local linearity between feature space and input space. By extending the inverse mapping process to test samples, we can obtain pre-images in input space. The proposed method is non-iterative, and can be used for any kernel functions. Experimental results based on image denoising using kernel principal component analysis （PCA） show that the proposed method outperforms the state-of-the-art methods for solving the pre-image problem.     

组合核支持向量机的模式分析新方法

基于支持向量机核函数的条件,将Sobolev Hilbert空间的再生核函数和多项式核函数进行有效的线性组合,给出一种新的支持向量机的组合核函数,提出一种基于再生核的组合核函数支持向量机的模式分析方法,该方法兼具了全局核函数与局部核函数的优点,且算法的复杂度被降低。仿真实验结果表明：支持向量机的核函数采用基于再生核的组合核函数是可行的,且此核函数不仅具有核函数的非线性映射特征,而且也继承了核函数对非线性逐级精细逼近的特征,模式分析的效果比单核函数可以更加细腻。     

多输出数据依赖核支持向量回归快速学习算法

针对基于递推下降法的多输出支持向量回归算法在模型参数拟合过程中收敛速度慢、预测精度低的情况,使用一种基于秩2校正规则且具有二阶收敛速度的修正拟牛顿算法（BFGS）进行多输出支持向量回归算法的模型参数拟合,同时为了保证模型迭代过程中的下降量和全局收敛性,应用非精确线性搜索技术确定步长因子。通过分析支持向量机（SVM）中核函数的几何结构,构造数据依赖核函数替代传统核函数,生成多输出数据依赖核支持向量回归模型。将模型与基于梯度下降法、修正牛顿法拟合的多输出支持向量回归模型进行对比。实验结果表明,在200个样本下该算法的迭代时间为72.98 s,修正牛顿法的迭代时间为116.34 s,递推下降法的迭代时间为2065.22 s。所提算法能够减少模型迭代时间,具有更快的收敛速度。     

Multi-layer radial basis function neural network based on multi-scale kernel learning

The conversion functions in the hidden layer of radial basis function neural networks (RBFNN) are Gaussian functions. The Gaussian functions are local to the kernel centers. In most of the existing research, the spatial local response of the sample is inaccurately calculated because the kernels have the same shape as a hypersphere, and the kernel parameters in the network are determined by experience. The influence of the fine structure in the local space is not considered during feature extraction. In addition, it is difficult to obtain a better feature extraction ability with less computational complexity. Therefore, this paper develops a multi-scale RBF kernel learning algorithm and proposes a new multi-layer RBF neural network model. For the samples of each class, the expectation maximization (EM) algorithm is used to obtain multi-layer nested sub-distribution models with different local response ranges, which are called multi-scale kernels in the network. The prior information of each sub-distribution is used as the connection weight between the multi-scale kernels. Finally, feature extraction is implemented using multi-layer kernel subspace embedding. The multi-scale kernel learning model can efficiently and accurately describe the fine structure of the samples and is fault tolerant to setting the number of kernels to a certain extent. Considering the prior probability of each kernel as the weight makes the feature extraction process satisfy the Bayes rule, which can enhance the interpretability of feature extraction in the network. This paper also theoretically proves that the proposed neural network is a generalized version of the original RBFNN. The experimental results show that the proposed method has better performance compared with some state-of-the-art algorithms.     

基于最小二乘支持向量回归的水质预测

水质系统是一个开放的、复杂的、非线性动力学系统,具有时变复杂性,针对水质预测方法的研究虽然已经取得了一些成果,但也存在预测精度与计算复杂度等难题。为此,本文提出一种基于最小二乘支持向量回归的水质预测算法。支持向量机是机器学习中一种常用的分类模型,通过核函数将非线性数据从低维映射到高维空间,在高维空间实现线性分类和回归,最小二乘支持向量回归（LS-SVR）利用所有的样本参与回归拟合,使得回归的损失函数不再只与小部分支持向量样本有关,而是由所有样本参与学习修正误差,提高预测精度;同时该算法将标准SVR求解问题由不等式的约束条件及凸二次规划问题转化成线性方程组来求解,提高了运算速度,解决了非线性复杂特性的水质预测问题。     

一种改进的再生核支持向量机回归模型

基于支持向量机核函数的条件,将Sobolev Hilbert空间的再生核函数进行改进,给出一种新的支持向量机核函数,并提出一种改进的最小二乘再生核支持向量机的回归模型,该回归模型的参数被减少,且仿真实验结果表明：最小二乘支持向量机的核函数采用改进的再生核函数是可行的,改进后的再生核函数不仅具有核函数的非线性映射特征,而且也继承了该再生核函数对非线性逐级精细逼近的特征,回归的效果比一般的核函数更为细腻。     

改进的核回归图像恢复

Steering核回归是一种自适应的、有效的图像恢复方法,在图像去噪、放大和去模糊中都得到了广泛应用。但此模型以高斯函数为核函数,故得到的恢复图像边缘,尤其是细小边缘常常会因过分平滑而模糊。提出基于鲁棒统计的各向异性核回归图像恢复模型,该模型在Steering核回归模型基础上,结合各向异性距离,以鲁棒统计权函数代替高斯核函数。大量图像恢复实验结果显示,与Steering核回归方法相比较,所提出方法得到的恢复图像质量显著提高,尤其是在细小边缘保持方面更具有明显优势。     

一种基于Cholesky分解的动态无偏LS-SVM学习算法

针对最小二乘支持向量机用于在线建模时存在的计算复杂性问题,提出一种动态无偏最小二乘支持向量回归模型.该模型通过改进标准最小二乘支持向量机结构风险的形式消除了偏置项.得到了无偏的最小二乘支持向量机,简化了回归系数的求解.根据模型动态变化过程中核函数矩阵的特点,设计了基于Cholesky分解的在线学习算法.该算法能充分利用历史训练结果,减少计算复杂性.仿真实验表明了所提出模型的有效性.     

Kernel sparse representation for hyperspectral unmixing based on high mutual coherence spectral library

ABSTRACT

Sparse regression is now a popular method for hyperspectral unmixing relying on a prior spectral library. However, it is limited by the high mutual coherence spectral library which contains high similarity atoms. In order to improve the accuracy of sparse unmixing with a high mutual coherence spectral library, a new algorithm based on kernel sparse representation unmixing model with total variation constraint is proposed in this paper. By constructing an appropriate kernel function to expand similarity measure scale, library atoms and hyperspectral data are mapped to kernel space where sparse regression algorithms are then applied. Experiments conducted with both simulated and real hyperspectral data sets indicate that the proposed algorithm effectively improves the unmixing performance when using a high mutual coherence spectral library because of its ability to precisely extract endmembers in hyperspectral images. Compared with other state-of-the-art algorithms, the proposed algorithm obtains low reconstruction errors in pixels with different mixed degree.     

基于测度学习支持向量机的钢琴乐谱难度等级识别

现有钢琴乐谱难度分类主要由人工方式完成,效率不高,而自动识别乐谱难度等级的算法对类别的拟合度较低。因此,与传统将乐谱难度等级识别归结为回归问题不同,本文直接将其建模为基于支持向量机的分类问题。并结合钢琴乐谱分类主观性强、特征之间普遍存在相关性等特点,利用测度学习理论有难度等级标签乐谱的先验知识,依据特征对难度区分的贡献度,改进高斯径向基核函数,从而提出一种测度学习支持向量机分类算法——ML-SVM算法。在9类和4类难度两个乐谱数据集上,我们将ML-SVM算法与逻辑回归,基于线性核函数、多项式核函数、高斯径向基核函数的支持向量机算法以及结合主成分分析的各个支持向量机算法进行了对比,实验结果表明我们提出算法的识别正确率优于现有算法,分别为68.74%和84.67%。所提算法有效提高了基于高斯径向基核函数支持向量机算法在本应用问题中的分类性能。     

Ridgelet kernel regression

A ridgelet kernel regression method is presented in this paper to approximate multi-dimensional functions, especially those with certain kinds of spatial inhomogeneities. This method is based on ridgelet theory, kernel and regularization techniques from which we can deduce a regularized kernel regression form. By representing this form with quadratic programming and taking the obtained solution to define a fitness function, we use particle swarm optimization to optimize the directions of ridgelets. The properties of ridgelet can guarantee the stability of this method in approximating multi-dimensional functions, as well as its superiority for functions with linear singularities. Additionally, the regularized technique employed in this model leads to smaller generalization error. Experiments in the tasks of regression and classification show its effectiveness.