A multiple-kernel support vector regression approach for stock market price forecasting

Support vector regression has been applied to stock market forecasting problems. However, it is usually needed to tune manually the hyperparameters of the kernel functions. Multiple-kernel learning was developed to deal with this problem, by which the kernel matrix weights and Lagrange multipliers can be simultaneously derived through semidefinite programming. However, the amount of time and space required is very demanding. We develop a two-stage multiple-kernel learning algorithm by incorporating sequential minimal optimization and the gradient projection method. By this algorithm, advantages from different hyperparameter settings can be combined and overall system performance can be improved. Besides, the user need not specify the hyperparameter settings in advance, and trial-and-error for determining appropriate hyperparameter settings can then be avoided. Experimental results, obtained by running on datasets taken from Taiwan Capitalization Weighted Stock Index, show that our method performs better than other methods.     

A hybrid stock selection model using genetic algorithms and support vector regression

In the areas of investment research and applications, feasible quantitative models include methodologies stemming from soft computing for prediction of financial time series, multi-objective optimization of investment return and risk reduction, as well as selection of investment instruments for portfolio management based on asset ranking using a variety of input variables and historical data, etc. Among all these, stock selection has long been identified as a challenging and important task. This line of research is highly contingent upon reliable stock ranking for successful portfolio construction. Recent advances in machine learning and data mining are leading to significant opportunities to solve these problems more effectively. In this study, we aim at developing a methodology for effective stock selection using support vector regression (SVR) as well as genetic algorithms (GAs). We first employ the SVR method to generate surrogates for actual stock returns that in turn serve to provide reliable rankings of stocks. Top-ranked stocks can thus be selected to form a portfolio. On top of this model, the GA is employed for the optimization of model parameters, and feature selection to acquire optimal subsets of input variables to the SVR model. We will show that the investment returns provided by our proposed methodology significantly outperform the benchmark. Based upon these promising results, we expect this hybrid GA-SVR methodology to advance the research in soft computing for finance and provide an effective solution to stock selection in practice.     

Fuzzy nile-based support vector regression system

In this paper, we design a fuzzy rule-based support vector regression system. The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set. Based on the first-order hnear Tagaki-Sugeno （TS） model, the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method. Our model is applied to the real world regression task. The simulation results gives promising performances in terms of a set of fuzzy hales, which can be easily interpreted by humans.     

Efficient music note recognition based on a self-organizing map tree and linear vector quantization

Using classical signal processing and filtering techniques for music note recognition faces various kinds of difficulties. This paper proposes a new scheme based on neural networks for music note recognition. The proposed scheme uses three types of neural networks: time delay neural networks, self-organizing maps, and linear vector quantization. Experimental results demonstrate that the proposed scheme achieves 100% recognition rate in moderate noise environments. The basic design of two potential applications of the proposed scheme is briefly demonstrated.     

Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR)

A parallel randomized support vector machine (PRSVM) and a parallel randomized support vector regression (PRSVR) algorithm based on a randomized sampling technique are proposed in this paper. The proposed PRSVM and PRSVR have four major advantages over previous methods. (1) We prove that the proposed algorithms achieve an average convergence rate that is so far the fastest bounded convergence rate, among all SVM decomposition training algorithms to the best of our knowledge. The fast average convergence bound is achieved by a unique priority based sampling mechanism. (2) Unlike previous work (Provably fast training algorithm for support vector machines, 2001) the proposed algorithms work for general linear-nonseparable SVM and general non-linear SVR problems. This improvement is achieved by modeling new LP-type problems based on Karush–Kuhn–Tucker optimality conditions. (3) The proposed algorithms are the first parallel version of randomized sampling algorithms for SVM and SVR. Both the analytical convergence bound and the numerical results in a real application show that the proposed algorithm has good scalability. (4) We present demonstrations of the algorithms based on both synthetic data and data obtained from a real word application. Performance comparisons with SVM^light show that the proposed algorithms may be efficiently implemented.     

A hybrid procedure for stock price prediction by integrating self-organizing map and genetic programming

Stock price prediction is a very important financial topic, and is considered a challenging task and worthy of the considerable attention received from both researchers and practitioners. Stock price series have properties of high volatility, complexity, dynamics and turbulence, thus the implicit relationship between the stock price and predictors is quite dynamic. Hence, it is difficult to tackle the stock price prediction problems effectively by using only single soft computing technique. This study hybridizes a self-organizing map (SOM) neural network and genetic programming (GP) to develop an integrated procedure, namely, the SOM-GP procedure, in order to resolve problems inherent in stock price predictions. The SOM neural network is utilized to divide the sample data into several clusters, in such a manner that the objects within each cluster possess similar properties to each other, but differ from the objects in other clusters. The GP technique is applied to construct a mathematical prediction model that describes the functional relationship between technical indicators and the closing price of each cluster formed in the SOM neural network. The feasibility and effectiveness of the proposed hybrid SOM-GP prediction procedure are demonstrated through experiments aimed at predicting the finance and insurance sub-index of TAIEX (Taiwan stock exchange capitalization weighted stock index). Experimental results show that the proposed SOM-GP prediction procedure can be considered a feasible and effective tool for stock price predictions, as based on the overall prediction performance indices. Furthermore, it is found that the frequent and alternating rise and fall, as well as the range of daily closing prices during the period, significantly increase the difficulties of predicting.     

Short term forecasting with support vector machines and application to stock price prediction

Forecasting a stock price movement is one of the most difficult problems in finance. The reason is that financial time series are complex, non stationary. Furthermore, it is also very difficult to predict this movement with parametric models. Instead of parametric models, we propose two techniques, which are data driven and non parametric. Based on the idea that excess returns would be possible with publicly available information, we developed two models in order to forecast the short term price movements by using technical indicators. Our assumption is that the future value of a stock price depends on the financial indicators although there is no parametric model to explain this relationship. This relationship comes from the technical analysis. Comparison shows that support vector regression (SVR) out performs the multi layer perceptron (MLP) networks for a short term prediction in terms of the mean square error. If the risk premium is used as a comparison criterion, then the SVR technique is as good as the MLP method or better.     

Cycle-breaking acceleration for support vector regression

Support vector regression (SVR) is a powerful tool in modeling and prediction tasks with widespread application in many areas. The most representative algorithms to train SVR models are Shevade et al.'s Modification 2 and Lin's WSS1 and WSS2 methods in the LIBSVM library. Both are variants of standard SMO in which the updating pairs selected are those that most violate the Karush-Kuhn-Tucker optimality conditions, to which LIBSVM adds a heuristic to improve the decrease in the objective function. In this paper, and after presenting a simple derivation of the updating procedure based on a greedy maximization of the gain in the objective function, we show how cycle-breaking techniques that accelerate the convergence of support vector machines (SVM) in classification can also be applied under this framework, resulting in significantly improved training times for SVR.     

结构可调的支持向量回归估计

针对定义域各分区间内样本数据的噪声强度不同,以及在局部范围内数据变化急剧等复杂情况,提出了结构可调的支持向量回归估计(AS-SVR)方法,包括采用不同的损失函数,对各样本点自适应地选用不同的参数等。推导了求解公式,给出了调整算法。实例测试表明,AS-SVR方法的楚模效果优于常规方法。     

Predicting corporate bankruptcy using a self-organizing map: An empirical study to improve the forecasting horizon of a financial failure model

The aim of this study is to show how a Kohonen map can be used to increase the forecasting horizon of a financial failure model. Indeed, most prediction models fail to forecast accurately the occurrence of failure beyond 1 year, and their accuracy tends to fall as the prediction horizon recedes. So we propose a new way of using a Kohonen map to improve model reliability. Our results demonstrate that the generalization error achieved with a Kohonen map remains stable over the period studied, unlike that of other methods, such as discriminant analysis, logistic regression, neural networks and survival analysis, traditionally used for this kind of task.     

A hybrid genetic-neural architecture for stock indexes forecasting

In this paper, a new approach for time series forecasting is presented. The forecasting activity results from the interaction of a population of experts, each integrating genetic and neural technologies. An expert of this kind embodies a genetic classifier designed to control the activation of a feedforward artificial neural network for performing a locally scoped forecasting activity. Genetic and neural components are supplied with different information: The former deal with inputs encoding information retrieved from technical analysis, whereas the latter process other relevant inputs, in particular past stock prices. To investigate the performance of the proposed approach in response to real data, a stock market forecasting system has been implemented and tested on two stock market indexes, allowing for account realistic trading commissions. The results pointed to the good forecasting capability of the approach, which repeatedly outperformed the “Buy and Hold” strategy.     

Emotion recognition from geometric facial features using self-organizing map

This paper presents a novel emotion recognition model using the system identification approach. A comprehensive data driven model using an extended Kohonen self-organizing map (KSOM) has been developed whose input is a 26 dimensional facial geometric feature vector comprising eye, lip and eyebrow feature points. The analytical face model using this 26 dimensional geometric feature vector has been effectively used to describe the facial changes due to different expressions. This paper thus includes an automated generation scheme of this geometric facial feature vector. The proposed non-heuristic model has been developed using training data from MMI facial expression database. The emotion recognition accuracy of the proposed scheme has been compared with radial basis function network, multi-layered perceptron model and support vector machine based recognition schemes. The experimental results show that the proposed model is very efficient in recognizing six basic emotions while ensuring significant increase in average classification accuracy over radial basis function and multi-layered perceptron. It also shows that the average recognition rate of the proposed method is comparatively better than multi-class support vector machine.     

Classified self-organizing map with adaptive subcodebook for edge preserving vector quantization

This paper presents a novel classified self-organizing map method for edge preserving quantization of images using an adaptive subcodebook and weighted learning rate. The subcodebook sizes of two classes are automatically adjusted in training iterations based on modified partial distortions that can be estimated incrementally. The proposed weighted learning rate updates the neuron efficiently no matter of how large the weighting factor is. Experimental results show that the new method achieves better quality of reconstructed edge blocks and more spread out codebook and incurs a significantly less computational cost as compared to the competing methods.     

Error tolerance based support vector machine for regression

Most existing online algorithms in support vector machines (SVM) can only grow support vectors. This paper proposes an online error tolerance based support vector machine (ET-SVM) which not only grows but also prunes support vectors. Similar to least square support vector machines (LS-SVM), ET-SVM converts the original quadratic program (QP) in standard SVM into a group of easily solved linear equations. Different from LS-SVM, ET-SVM remains support vectors sparse and realizes a compact structure. Thus, ET-SVM can significantly reduce computational time while ensuring satisfactory learning accuracy. Simulation results verify the effectiveness of the newly proposed algorithm.     

Real-time prediction of order flowtimes using support vector regression

In a make-to-order production system, a due date must be assigned to new orders that arrive dynamically, which requires predicting the order flowtime in real-time. This study develops a support vector regression model for real-time flowtime prediction in multi-resource, multi-product systems. Several combinations of kernel and loss functions are examined, and results indicate that the linear kernel and the ε$\epsilon$-insensitive loss function yield the best generalization performance. The prediction error of the support vector regression model for three different multi-resource systems of varying complexity is compared to that of classic time series models (exponential smoothing and moving average) and to a feedforward artificial neural network. Results show that the support vector regression model has lower flowtime prediction error and is more robust. More accurately predicting flowtime using support vector regression will improve due-date performance and reduce expenses in make-to-order production environments.     

回归型支持向量机的调节熵函数法

基于最优化理论中的ＫＫＴ 互补条件建立支持向量回归机的无约束不可微优化模型,并给出了一种有效的光滑近似解法———调节熵函数方法．该方法不需参数取值很大便可逼近问题的最优解,从而避免了一般熵函数法为了逼近精确解,参数取得过大而导致数值的溢出现象,为求解支持向量回归机提供了一条新途径．数值实验结果表明,回归型支持向量机的调节熵函数法改善了支持向量机的回归性能和效率．     

A teaching method using a self-organizing map for reinforcement learning

We described a new preteaching method for re-inforcement learning using a self-organizing map (SOM). The purpose is to increase the learning rate using a small amount of teaching data generated by a human expert. In our proposed method, the SOM is used to generate the initial teaching data for the reinforcement learning agent from a small amount of teaching data. The reinforcement learning function of the agent is initialized by using the teaching data generated by the SOM in order to increase the probability of selecting the optimal actions it estimates. Because the agent can get high rewards from the start of reinforcement learning, it is expected that the learning rate will increase. The results of a mobile robot simulation showed that the learning rate had increased even though the human expert had showed only a small amount of teaching data. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

A novel model by evolving partially connected neural network for stock price trend forecasting

This paper proposes a novel model by evolving partially connected neural networks (EPCNNs) to predict the stock price trend using technical indicators as inputs. The proposed architecture has provided some new features different from the features of artificial neural networks: (1) connection between neurons is random; (2) there can be more than one hidden layer; (3) evolutionary algorithm is employed to improve the learning algorithm and training weights. In order to improve the expressive ability of neural networks, EPCNN utilizes random connection between neurons and more hidden layers to learn the knowledge stored within the historic time series data. The genetically evolved weights mitigate the well-known limitations of gradient descent algorithm. In addition, the activation function is defined using sin(x) function instead of sigmoid function. Three experiments were conducted which are explained as follows. In the first experiment, we compared the predicted value of the trained EPCNN model with the actual value to evaluate the prediction accuracy of the model. Second experiment studied the over fitting problem which occurred in neural network training by taking different number of neurons and layers. The third experiment compared the performance of the proposed EPCNN model with other models like BPN, TSK fuzzy system, multiple regression analysis and showed that EPCNN can provide a very accurate prediction of the stock price index for most of the data. Therefore, it is a very promising tool in forecasting of the financial time series data.     

Parameter-insensitive kernel in extreme learning for non-linear support vector regression

Support vector regression (SVR) is a state-of-the-art method for regression which uses the ε‐sensitive loss and produces sparse models. However, non-linear SVRs are difficult to tune because of the additional kernel parameter. In this paper, a new parameter-insensitive kernel inspired from extreme learning is used for non-linear SVR. Hence, the practitioner has only two meta-parameters to optimise. The proposed approach reduces significantly the computational complexity yet experiments show that it yields performances that are very close from the state-of-the-art. Unlike previous works which rely on Monte-Carlo approximation to estimate the kernel, this work also shows that the proposed kernel has an analytic form which is computationally easier to evaluate.