Leave-one-out bounds for support vector ordinal regression machine

The success of support vector machine depends upon its parameters. The leave-one-out (LOO) method provides a quantitative criterion for selecting those parameters. However, one shortcoming of the LOO method is that it is highly time consuming. An effective approach is to approximate the LOO error by an upper bound. This paper is concerned with the support vector ordinal regression machine (SVORM). Two bounds of the LOO error for SVORM are presented. The first bound is based on the geometrical concept of a span. The second one is based on the concept of support vector. Preliminary numerical experiments show the validity of the bounds.     

Optimizing resources in model selection for support vector machine

Tuning support vector machine (SVM) hyperparameters is an important step in achieving a high-performance learning machine. It is usually done by minimizing an estimate of generalization error based on the bounds of the leave-one-out (LOO) such as radius-margin bound and on the performance measures such as generalized approximate cross-validation (GACV), empirical error, etc. These usual automatic methods used to tune the hyperparameters require an inversion of the Gram-Schmidt matrix or a resolution of an extra-quadratic programming problem. In the case of a large data set these methods require the addition of huge amounts of memory and a long CPU time to the already significant resources used in SVM training. In this paper, we propose a fast method based on an approximation of the gradient of the empirical error, along with incremental learning, which reduces the resources required both in terms of processing time and of storage space. We tested our method on several benchmarks, which produced promising results confirming our approach. Furthermore, it is worth noting that the gain time increases when the data set is large.     

多输入多输出非线性系统的最小二乘支持向量机广义逆控制

针对神经网络逆控制存在的不足, 对一类模型未知且某些状态量较难测得的多输入多输出(MIMO)非线性系统, 在状态软测量函数存在的前提下, 提出一种最小二乘支持向量机(LSSVM)广义逆辨识控制策略. 通过广义逆将原被控系统转化为伪线性复合系统, 并可使其极点任意配置, 采用LSSVM代替神经网络拟合广义逆系统中的静态非线性映射. 将系统的状态量辨识与LSSVM逆模型辨识结合, 通过LSSVM训练拟合同时实现软测量功能. 最后以双电机变频调速系统为对象, 采用该控制策略进行仿真研究, 结果验证了本文算法的有效性.     

Evolutionary model selection in a wavelet-based support vector machine for automated seizure detection

Support vector machines (SVM) have in recent years been gainfully used in various pattern recognition applications. Based on statistical learning theory, this paradigm promises strong robustness to noise and generalization to unseen data. As in any classification technique, appropriate choice of the kernels and input features play an important role in SVM performance. In this study, an evolutionary scheme searches for optimal kernel types and parameters for automated seizure detection. We consider the Lyapunov exponent, fractal dimension and wavelet entropy for possible feature extraction. The classification accuracy of this approach is examined by applying the MIT (Massachusetts Institute of Technology) dataset and comparing results with the SVM. The MIT-BIH dataset has the electrocardiographic (ECG) changes in patients with partial epilepsy which two types ECG beats (partial epilepsy and normal). A comparison of results shows that performance of the evolutionary scheme outweighs that of support vector machine. In the best condition, the accuracy rate of the proposed approaches reaches 100% for specificity and 96.29% for sensitivity.     

支持向量机的一种特征选取算法

支持向量机（Support Vector Machine,SVM）是一种有效的分类方法,其学习本质是通过对偶问题求解原问题,但是它不能直接获得特征重要性。提出一种新的特征选取算法,实验表明,该特征选取算法与一般特征选取算法（如F-Score算法）相比,对同一测试数据集计算的结果具有相同的降序排列结果,而且有更好的特征刻画量化指标,分界线更明显,表明新的特征选取算法具有更佳的合理性。     

802.11n与ZigBee共存:子载波置零多输入多输出物理层模型

针对802.11n与ZigBee共享ISM频段造成的WiFi与ZigBee信道重叠,进而导致网络间相互干扰使得网络性能下降,以及当前载波侦听多路访问/冲突避免(CSMA/CA）可能导致的频谱资源利用率较低的问题,提出一个采用子载波置零技术的2×2非相干多输入多输出(MIMO）物理层模型。该模型中,为了避免共信道干扰,WiFi发送端在发送数据前首先对其当前使用的信道中可能存在的ZigBee信号进行检测,若检测到ZigBee信号则对已被占用的频谱对应的子载波置零,使用余下频谱不重叠子载波进行通信。接收端对发送端使用的子载波进行识别,并完成后续工作。通过使WiFi与ZigBee信号频谱分离来消除信号间干扰,解决两者共存问题,实现WiFi与ZigBee数据并行传输。在由GNURadio/USRP软件无线电设备和ZigBee节点搭建的实验床上进行的实验结果表明,采用子载波置零技术的2×2非相干MIMO可以获得全带宽发送状态下50%~70%的吞吐量,同时在数据并行传输过程中ZigBee的正确收包百分比达到90%以上。     

一种多输入多输出系统模型辨识方法及其应用

目前的辨识方法一般需要在系统输入端加入激励信号，而且多输入多输出系统的在线辨识仍很困难。本文提出一种基于牛顿迭代法的多输入、多输出对象模型迭代辨识方法，模型参数更新的依据是使模型预测输出与全部采样时刻的对象实际输出之间的均方差递减，直到收敛。这种基于全局数据迭代的辨识方法可进行闭环辨识，无需外加激励信号，适用于多输入多输出对象的在线辨识。对一个两输入、两输出对象模型的仿真研究和某电厂300MW机组负荷被控对象的计算结果表明，辨识效果令人满意。     

Feature selection in the Laplacian support vector machine

Traditional classifiers including support vector machines use only labeled data in training. However, labeled instances are often difficult, costly, or time consuming to obtain while unlabeled instances are relatively easy to collect. The goal of semi-supervised learning is to improve the classification accuracy by using unlabeled data together with a few labeled data in training classifiers. Recently, the Laplacian support vector machine has been proposed as an extension of the support vector machine to semi-supervised learning. The Laplacian support vector machine has drawbacks in its interpretability as the support vector machine has. Also it performs poorly when there are many non-informative features in the training data because the final classifier is expressed as a linear combination of informative as well as non-informative features. We introduce a variant of the Laplacian support vector machine that is capable of feature selection based on functional analysis of variance decomposition. Through synthetic and benchmark data analysis, we illustrate that our method can be a useful tool in semi-supervised learning.     

A novel neural internal model control for multi-input multi-output nonlinear discrete-time processes

An internal model-based neural network control is proposed for unknown non-affine discrete-time multi-input multi-output (MIMO) processes in nonlinear state space form under model mismatch and disturbances. Based on the neural state-space model built for an unknown nonlinear MIMO state space process, an approximate internal model and approximate decoupling controllers are derived simultaneously. Thus, the learning of the inverse process dynamics is not required. A neural network model-based extended Kalman observer is used to estimate the states of a nonlinear process as not all states are accessible. The proposed neural internal model control can work for open-loop unstable processes with its closed-loop stability derived analytically. The application to a distributed thermal process shows the effectiveness of the proposed approach for suppressing nonlinear coupling and external disturbances and its feasibility for the control of unknown non-affine nonlinear discrete-time MIMO state space processes.     

Multi-innovation stochastic gradient algorithms for multi-input multi-output systems

This paper develops a multi-innovation stochastic gradient (MISG) algorithm for multi-input multi-output systems by expanding the innovation vector to an innovation matrix. The convergence analysis shows that the parameter estimates by the MISG algorithm consistently converge to the true parameters under the persistent excitation condition. The MISG algorithm uses not only the current innovation but also the past innovation at each iteration and repeatedly utilizes the available input–output data, thus the parameter estimation accuracy can be improved. The simulation example confirms the theoretical results.     

支持向量机核函数选择研究与仿真

支持向量机是一种基于核的学习方法,核函数选取对支持向量机性能有着重要的影响,如何有效地进行核函数选择是支持向量机研究领域的一个重要问题。目前大多数核选择方法不考虑数据的分布特征,没有充分利用隐含在数据中的先验信息。为此,引入能量熵概念,借助超球体描述和核函数蕴藏的度量特征,提出一种基于样本分布能量熵的支持向量机核函数选择方法,以提高SVM学习能力和泛化能力。数值实例仿真验证表明了该方法的可行性和有效性。     

基于支持向量机的隐含语意特征选择方法

隐含语意索引(LSI)是一个能有效捕获文档中词的隐含语意特征的方法。然而,用该方法选择的特征空间对文本分类来说可能不是最适合的,因为这种方法按照词的变化排序特征,而没有考虑到分类能力。支持向量机(SVM)高度的泛化能力使它特别适用于高维数据例如文档的分类。为此提出基于支持向量机的特征提取方法用于选择适于分类的LSI特征。该方法利用SVM高度泛化的分类能力, 通过使用在每一个规则下训练的分类器的参数对第k个特征对反向平方分解面的贡献w2k的值进行估计。实验表明当需要比LSI更少的训练和测试时间时,该方法能够以更为紧凑的表示方式提高分类性能。     

基于网格模式搜索的支持向量机模型选择

支持向量机的模型选择问题就是对于一个给定的核函数,调节核参数和惩罚因子C。分析了网格搜索算法和模式搜索算法,通过结合上述两种算法的优点提出了网格模式搜索算法。其核心原理是先用网格算法在全局范围内进行快速搜索,找到最优解的最小区间,再在这个最小区间内用模式搜索算法找到最优解。实验证明,网格模式搜索具有学习精度高和速度快的优点。     

Set-point-related indirect iterative learning control for multi-input multi-output systems

A form of iterative learning control (ILC) is used to update the set-point for the local controller. It is referred to as set-point-related (SPR) indirect ILC. SPR indirect ILC has shown excellent performance: as a supervision module for the local controller, ILC can improve the tracking performance of the closed-loop system along the batch direction. In this study, an ILC-based P-type controller is proposed for multi-input multi-output (MIMO) linear batch processes, where a P-type controller is used to design the control signal directly and an ILC module is used to update the set-point for the P-type controller. Under the proposed ILC-based P-type controller, the closed-loop system can be transformed to a 2-dimensional (2D) Roesser s system. Based on the 2D system framework, a sufficient condition for asymptotic stability of the closed-loop system is derived in this paper. In terms of the average tracking error (ATE), the closed-loop control performance under the proposed algorithm can be improved from batch to batch, even though there are repetitive disturbances. A numerical example is used to validate the proposed results.     

Adaptive sliding mode control for discrete-time multi-input multi-output systems

In this paper, robust adaptive sliding mode tracking control for discrete-time multi-input multi-output systems with unknown parameters and disturbance is considered. The robust tracking controller is comprised of adaptive control and sliding mode control design. Bounded motion of the system around the sliding surface and stability of the global system in the sense that all signals remain bounded are guaranteed. If the disturbance and the reference signal are slowly varying with respect to the sampling frequency, the proposed sliding mode controller can reject the disturbance and output tracking can be approximately achieved. Simulation results are presented to illustrate the proposed approach.     

Pole assignment for multi-input multi-output systems using output feedback

In this paper the problem of pole assignment using constant gain output feedback is studied for MIMO system with system order n > m + l − 1, where m and l are the number of inputs and outputs, respectively. A new procedure is presented to design a constant gain output feedback matrix which assigns (m + l − 2) poles exactly to the desired locations and shifts all the unassigned poles to suitable locations using root locus techniques.     

Minimal unknown-input functional observers for multi-input multi-output LTI systems

Designing minimum possible order (minimal) disturbance-decoupled proper functional observers for multi-input multi-output (MIMO) linear time-invariant (LTI) systems is studied. It is not necessary that a minimum-order unknown-input functional observer (UIFO) exists in our proposed design procedure. If the minimum-order observer cannot be attained, the observer's order is increased sequentially through a recursive algorithm, so that the minimal order UIFO can be obtained. To the best of our knowledge, this is the first time that this specific problem is addressed. It is assumed that the system is unknown-input functional detectable, which is the least requirement for the existence of a stable UIFO. This condition also is a certificate for the convergence of our observer's order-increase algorithm. Two methodologies are demonstrated to solve the observer design equations. The second presented scheme, is a new design method that based on our observations has a better numerical performance than the first conventional one. Numerical examples and simulation results in the MATLAB/Simulink environment describe the overall observer design procedure, and highlight the efficacy of our new methodology to solve the observer equations in comparison to the conventional one.     

多输入/多输出系统动态矩阵控制鲁棒稳定性

研究了基于脉冲响应模型的动态矩阵预测控制(DMC)算法,针对多输入、多输出(MIMO)系统脉冲响应模型的特点,利用脉冲响应系数误差矩阵范数平方和定义预测模型的模型误差,以线性矩阵不等式(LMI)的形式提出了DMC闭环鲁棒稳定充要条件,将DMC算法闭环稳定问题转换为一类线性矩阵不等式的可解问题.并且研究了模型误差与闭环系统稳定性之间的关系,给出了保证系统稳定条件下模型误差界的求取方法,通过求解一个线性矩阵不等式约束的凸优化问题得到保证闭环系统稳定的误差界.最后,利用算例对本文方法的有效性进行了验证.     

An enhanced support vector machine model for urban expansion prediction

Given the dynamic and convoluted nature of urban expansion process and the necessity of handling continuous and categorical variables, non-normal distributed data, and non-linear relationships, urban expansion modeling is challenging. To handle these issues effectively and enhance the quality of urban expansion prediction, the capabilities of support vector machine (SVM) technique are explored in this study. A binary SVM model is developed using three different data sampling methods and nineteen predictor variables, four of which are first introduced in this study. The model is configured by regulating the penalty parameter, selecting the most appropriate kernel function, and setting the best value for the kernel function's parameter. A novel combination of goodness-of-fit metrics is used to more realistically evaluate the model accuracy to predict built and unbuilt land cells as well as changed and unchanged land cells in the whole study area. The implementation of the developed model in Guilford County, NC, over the period of 2001–2011, as a case study, demonstrated highly accurate and reliable results. The best performance of the model with the training accuracy of 98% and the testing accuracy of 85% was achieved using a balanced sampling method, fourteen predictor variables, the penalty parameter equal to 1, the radial basis function (RBF) kernel, and the value of 2 for the kernel's parameter. The urban expansion model based on SVM method can substantially improve the prediction accuracy and would be helpful for making appropriate plans and policies to mitigate the adverse impacts of urban expansion.     

Feature clustering based support vector machine recursive feature elimination for gene selection

In a DNA microarray dataset, gene expression data often has a huge number of features(which are referred to as genes) versus a small size of samples. With the development of DNA microarray technology, the number of dimensions increases even faster than before, which could lead to the problem of the curse of dimensionality. To get good classification performance, it is necessary to preprocess the gene expression data. Support vector machine recursive feature elimination (SVM-RFE) is a classical method for gene selection. However, SVM-RFE suffers from high computational complexity. To remedy it, this paper enhances SVM-RFE for gene selection by incorporating feature clustering, called feature clustering SVM-RFE (FCSVM-RFE). The proposed method first performs gene selection roughly and then ranks the selected genes. First, a clustering algorithm is used to cluster genes into gene groups, in each which genes have similar expression profile. Then, a representative gene is found to represent a gene group. By doing so, we can obtain a representative gene set. Then, SVM-RFE is applied to rank these representative genes. FCSVM-RFE can reduce the computational complexity and the redundancy among genes. Experiments on seven public gene expression datasets show that FCSVM-RFE can achieve a better classification performance and lower computational complexity when compared with the state-the-art-of methods, such as SVM-RFE.