Surveying stock market forecasting techniques – Part II: Soft computing methods

The key to successful stock market forecasting is achieving best results with minimum required input data. Given stock market model uncertainty, soft computing techniques are viable candidates to capture stock market nonlinear relations returning significant forecasting results with not necessarily prior knowledge of input data statistical distributions. This paper surveys more than 100 related published articles that focus on neural and neuro-fuzzy techniques derived and applied to forecast stock markets. Classifications are made in terms of input data, forecasting methodology, performance evaluation and performance measures used. Through the surveyed papers, it is shown that soft computing techniques are widely accepted to studying and evaluating stock market behavior.     

Trend following deep Q-Learning strategy for stock trading

Computers and algorithms are widely used to help in stock market decision making. A few questions with regards to the profitability of algorithms for stock trading are can computers be trained to beat the markets? Can an algorithm take decisions for optimal profits? And so forth. In this research work, our objective is to answer some of these questions. We propose an algorithm using deep Q-Reinforcement Learning techniques to make trading decisions. Trading in stock markets involves potential risk because the price is affected by various uncertain events ranging from political influences to economic constraints. Models that trade using predictions may not always be profitable mainly due to the influence of various unknown factors in predicting the future stock price. Trend Following is a trading idea in which, trading decisions, like buying and selling, are taken purely according to the observed market trend. A stock trend can be up, down, or sideways. Trend Following does not predict the stock price but follows the reversals in the trend direction. A trend reversal can be used to trigger a buy or a sell of a certain stock. In this research paper, we describe a deep Q-Reinforcement Learning agent able to learn the Trend Following trading by getting rewarded for its trading decisions. Our results are based on experiments performed on the actual stock market data of the American and the Indian stock markets. The results indicate that the proposed model outperforms forecasting-based methods in terms of profitability. We also limit risk by confirming trading actions with the trend before actual trading.     

Expert system for predicting stock market timing using a candlestick chart

It has been one of the greatest challenges to predict the stock market. Since stock prices vary dramatically, it is important to determine when to buy and sell stocks in order to get high returns from stock investment. In this study, we have developed a candlestick chart analysis expert system, or a chart interpreter, for predicting the best stock market timing. The expert system has patterns and rules which can predict future stock price movements. Defined patterns are classified into five groups with respect to their meanings: falling, rising, neutral, trend-continuation and trend-reversal patterns. The experimental results revealed that the developed knowledge base could provide excellent indicators with an average hit ratio of 72% to help investors get high returns from their stock investment. Through experiments from January 1992 to June 1997, it was proven that the developed knowledge base was time- and field-independent.     

A stock market portfolio recommender system based on association rule mining

We propose a stock market portfolio recommender system based on association rule mining (ARM) that analyzes stock data and suggests a ranked basket of stocks. The objective of this recommender system is to support stock market traders, individual investors and fund managers in their decisions by suggesting investment in a group of equity stocks when strong evidence of possible profit from these transactions is available.Our system is different compared to existing systems because it finds the correlation between stocks and recommends a portfolio. Existing techniques recommend buying or selling a single stock and do not recommend a portfolio.We have used the support confidence framework for generating association rules. The use of traditional ARM is infeasible because the number of association rules is exponential and finding relevant rules from this set is difficult. Therefore ARM techniques have been augmented with domain specific techniques like formation of thematical sectors, use of cross-sector and intra-sector rules to overcome the disadvantages of traditional ARM.We have implemented novel methods like using fuzzy logic and the concept of time lags to generate datasets from actual data of stock prices.Thorough experimentation has been performed on a variety of datasets like the BSE-30 sensitive Index, the S&P CNX Nifty or NSE-50, S&P CNX-100 and DOW-30 Industrial Average. We have compared the returns of our recommender system with the returns obtained from the top-5 mutual funds in India. The results of our system have surpassed the results from the mutual funds for all the datasets.Our approach demonstrates the application of soft computing techniques like ARM and fuzzy classification in the design of recommender systems.     

A hybrid genetic-neural architecture for stock indexes forecasting

In this paper, a new approach for time series forecasting is presented. The forecasting activity results from the interaction of a population of experts, each integrating genetic and neural technologies. An expert of this kind embodies a genetic classifier designed to control the activation of a feedforward artificial neural network for performing a locally scoped forecasting activity. Genetic and neural components are supplied with different information: The former deal with inputs encoding information retrieved from technical analysis, whereas the latter process other relevant inputs, in particular past stock prices. To investigate the performance of the proposed approach in response to real data, a stock market forecasting system has been implemented and tested on two stock market indexes, allowing for account realistic trading commissions. The results pointed to the good forecasting capability of the approach, which repeatedly outperformed the “Buy and Hold” strategy.     

Planning stock portfolios by means of weighted frequent itemsets

Planning stock portfolios is a challenging task, because investors have to forecast stock market trends. To limit losses due to wrong forecasts a common strategy is diversification, which consists in buying stocks belonging to different sectors/markets to spread bets across different assets. Since the amount of stock market data is continuously growing, an appealing research strategy is to first apply data mining algorithms to discover significant patterns from potentially large stock datasets and then exploit them to support investor decision-making.This article presents an itemset-based approach to supporting buy-and-hold investors in technical analyses by automatically identifying promising sets of high-yield yet diversified stocks to buy. Specifically, it investigates the use of itemsets to generate stock portfolios from historical stock data and recommend them for buy-and-hold investments. To achieve this goal, it analyzes stock market datasets, which contain for each stock the closing prices on different trading days. Datasets are enriched with (analyst-provided) taxonomies, which are used to classify stocks as the corresponding sectors. Unlike previous approaches, it generates a model composed of a subset of potentially interesting itemsets, which are then used to support investors in decision-making. The selected itemsets represent promptly usable stock portfolios satisfying expert’s requirements on minimal average return and minimal level of diversification across sectors.The experiments performed on real stock datasets acquired under different market conditions demonstrate the effectiveness of the proposed approach compared to real stock funds.     

Mining stock price using fuzzy rough set system

In this study of mining stock price data, we attempt to predict the stronger rules of stock prices. To address this problem, we proposed an effective method, a fuzzy rough set system to predict a stock price at any given time. Our system has two agents: one is a visual display agent that helps stock dealers monitor the current price of a stock and the other is a mining agent that helps stock dealers make decisions about when to buy or sell stocks. To demonstrate that our system is effective, we used it to predict the stronger rules of stock price and achieved at least 93% accuracy after 180 trials.     

基于理性指标的马尔可夫链股市态势预测方法

由于股票市场存在人为扰动性,使得基于情绪的股市预测算法效果不佳。针对股市的诱多诱空问题,提出一种基于理性指标的马尔可夫链股市态势预测算法（RI_MCA）。提取股市的主要理性特征,并对这些理性特征进行量化;通过主成分分析将这些理性特征融合成理性指标,并利用理性指标获取股市的买卖点;将买卖点所对应的股市状态引入到马尔可夫链中,实现股市态势预测。在理性指标和股市状态相背离情况下会降低买卖点的可靠性,因而通过将特征背离引入到RI_MCA算法中提出了RICD_MCA算法,RICD_MCA算法根据特征背离程度对RI_MCA算法的结果进行调整优化。在上证指数上的实验比较与分析结果表明,RICD_MCA算法具有更高的预测精度。     

The Tech Buzz Game [stock market prediction]

Prediction markets, also known as information or decision markets, are designed to forecast future events or trends. Internet-based prediction markets can easily aggregate the insights of an unlimited number of potentially knowledgeable people asynchronously. The Tech Buzz Game - a joint venture between Yahoo! Research Labs and O'Reilly Media - is a fantasy prediction market launched in March 2005 at the O'Reilly Emerging Technology (ETech) Conference. The game consists of multiple sub-markets that pit a handful of rival technologies, each represented by a stock, against one another. The game's object is to anticipate future search buzz and buy and sell stocks accordingly. Thus, a player who believes BitTorrent stock is undervalued might buy shares, while a player who thinks BitTorrent is overpriced might sell the stock or instead purchase shares in a competing peer-to-peer technology. The Tech Buzz Game serves two key research-oriented goals. One is to evaluate the power of prediction markets to forecast high-tech trends. The other goal of the Tech Buzz Game is to field test the dynamic pari-mutuel market, a Yahoo! Research Labs trading mechanism designed to price and allocate shares.     

意愿计算的股市突变点预测方法

由于传统分段线性表示方法没有考虑股市数据分布变化导致分段不合理,同时股市突变点相关特征的局部性导致突变点难以有效预测,所以在分段线性表示方法的基础上提出一种意愿计算的股市突变点预测方法(WC-WSVM)。首先,给出一种波动率分布变化的分段线性表示(V-PLR)方法,通过波动率分布变化自适应地优化PLR分段阈值;然后,提取与主力买卖股票意愿相关的股市特征并进行量化,利用逻辑回归(LR)对于所提取的特征进行融合得到意愿计算结果;最后,将意愿计算结果与PLR-WSVM算法输入特征共同代入到WSVM中,进行突变点预测。在真实数据上的实验结果表明,算法具有强适应性,预测精度得到有效提升。     

An integrated portfolio optimisation procedure based on data envelopment analysis,artificial bee colony algorithm and genetic programming

Portfolio optimisation is an important issue in the field of investment/financial decision-making and has received considerable attention from both researchers and practitioners. However, besides portfolio optimisation, a complete investment procedure should also include the selection of profitable investment targets and determine the optimal timing for buying/selling the investment targets. In this study, an integrated procedure using data envelopment analysis (DEA), artificial bee colony (ABC) and genetic programming (GP) is proposed to resolve a portfolio optimisation problem. The proposed procedure is evaluated through a case study on investing in stocks in the semiconductor sub-section of the Taiwan stock market for 4 years. The potential average 6-month return on investment of 9.31% from 1 November 2007 to 31 October 2011 indicates that the proposed procedure can be considered a feasible and effective tool for making outstanding investment plans, and thus making profits in the Taiwan stock market. Moreover, it is a strategy that can help investors to make profits even when the overall stock market suffers a loss.     

模糊认知图在股票市场预测中的应用研究

复杂系统中存在着大量的过程依赖、自组织,并且一直是进化的,用传统的方法对其建模十分困难。模糊认知图作为一种模糊逻辑和神经网络相结合的产物,为复杂系统建模提供了一种有效工具。文中根据模糊认知图的特点,提出了用遗传学习算法建立系统的模糊认知图方法,为复杂系统分析及预测提供了一种解决方案。最后,以股票市场的数据为例进行了分析和预测模拟,结果表明,该方法是有效的。     

Forecasting stock market short-term trends using a neuro-fuzzy based methodology

A neuro-fuzzy system composed of an Adaptive Neuro Fuzzy Inference System (ANFIS) controller used to control the stock market process model, also identified using an adaptive neuro-fuzzy technique, is derived and evaluated for a variety of stocks. Obtained results challenge the weak form of the Efficient Market Hypothesis (EMH) by demonstrating much improved and better predictions, compared to other approaches, of short-term stock market trends, and in particular the next day’s trend of chosen stocks. The ANFIS controller and the stock market process model inputs are chosen based on a comparative study of fifteen different combinations of past stock prices performed to determine the stock market process model inputs that return the best stock trend prediction for the next day in terms of the minimum Root Mean Square Error (RMSE). Gaussian-2 shaped membership functions are chosen over bell shaped Gaussian and triangular ones to fuzzify the system inputs due to the lowest RMSE. Real case studies using data from emerging and well developed stock markets – the Athens and the New York Stock Exchange (NYSE) – to train and evaluate the proposed system illustrate that compared to the “buy and hold” strategy and several other reported methods, the proposed approach and the forecasting trade accuracy are by far superior.     

Financial time series forecasting using rough sets with time-weighted rule voting

In the paper we investigate experimentally the feasibility of rough sets in building profitable trend prediction models for financial time series. In order to improve the decision process for long time series, a novel time-weighted rule voting method, which accounts for information aging, is proposed. The experiments have been performed using market data of multiple stock market indices. The classification efficiency and financial performance of the proposed rough sets models was verified and compared with that of support vector machines models and reference financial indices. The results showed that the rough sets approach with time weighted rule voting outperforms the classical rough sets and support vector machines decision systems and is profitable as compared to the buy and hold strategy. In addition, with the use of variable precision rough sets, the effectiveness of generated trading signals was further improved.     

A feature weighted support vector machine and K-nearest neighbor algorithm for stock market indices prediction

This study investigates stock market indices prediction that is an interesting and important research in the areas of investment and applications, as it can get more profits and returns at lower risk rate with effective exchange strategies. To realize accurate prediction, various methods have been tried, among which the machine learning methods have drawn attention and been developed. In this paper, we propose a basic hybridized framework of the feature weighted support vector machine as well as feature weighted K-nearest neighbor to effectively predict stock market indices. We first establish a detailed theory of feature weighted SVM for the data classification assigning different weights for different features with respect to the classification importance. Then, to get the weights, we estimate the importance of each feature by computing the information gain. Lastly, we use feature weighted K-nearest neighbor to predict future stock market indices by computing k weighted nearest neighbors from the historical dataset. Experiment results on two well known Chinese stock market indices like Shanghai and Shenzhen stock exchange indices are finally presented to test the performance of our established model. With our proposed model, it can achieve a better prediction capability to Shanghai Stock Exchange Composite Index and Shenzhen Stock Exchange Component Index in the short, medium and long term respectively. The proposed algorithm can also be adapted to other stock market indices prediction.     

A fuzzy GARCH model applied to stock market scenario using a genetic algorithm

In this paper, we derive a new application of fuzzy systems designed for a generalized autoregression conditional heteroscedasticity (GARCH) model. In general, stock market performance is time-varying and nonlinear, and exhibits properties of clustering. The latter means simply that certain large changes tend to follow other large changes, and in general small changes tend to follow other small changes. This paper shows results from using the method of functional fuzzy systems to analyze the clustering in the case of a GARCH model.The optimal parameters of the fuzzy membership functions and GARCH model are extracted using a genetic algorithm (GA). The GA method aims to achieve a global optimal solution with a fast convergence rate for this fuzzy GARCH model estimation problem. From the simulation results, we have determined that the performance is significantly improved if the leverage effect of clustering is considered in the GARCH model. The simulations use stock market data from the Taiwan weighted index (Taiwan) and the NASDAQ composite index (NASDAQ) to illustrate the performance of the proposed method.     

An intelligent short term stock trading fuzzy system for assisting investors in portfolio management

Financial markets are complex systems influenced by many interrelated economic, political and psychological factors and characterised by inherent nonlinearities. Recently, there have been many efforts towards stock market prediction, applying various fuzzy logic techniques and using technical analysis methods.This paper presents a short term trading fuzzy system using a novel trading strategy and an “amalgam” between altered commonly used technical indicators and rarely used ones, in order to assist investors in their portfolio management. The sample consists of daily data from the general index of the Athens Stock Exchange over a period of more than 15 years (15/11/1996 to 5/6/2012), which was also divided into distinctive groups of bull and bear market periods.The results suggest that, with or without taking into consideration transaction costs, the return of the proposed fuzzy model is superior to the returns of the buy and hold strategy. Τhe proposed system can be characterised as conservative, since it produces smaller losses during bear market periods and smaller gains during bull market periods compared with the buy and hold strategy.     

A novel stock recommendation system using Guba sentiment analysis

Investment recommendation has been one of the hottest topics in the finance area which can help investors to get more profits and to avoid loss. Existing recommendation systems mostly depend on analysis of trading data and company profit prediction. Though many works show that there is a positive correlation between investors’ sentiment and the finance market trends, few recommendation theories have been built based on sentiment. The primary reason is the difficulty to measure investors’ sentiment. In this work, a novel stock recommendation system is developed based on a proposed theory concerning the correlation between Guba-based sentiment of the retail investors and the stock market trends in China. To verify four hypotheses of the theory, a novel method is proposed to measure the investors’ sentiment by exploiting the large volumes of emotion enriched texts posted in Guba, which is online social platform for individual investors to share news and opinions concerning their favorite stocks. Results show the correctness of the proposed theory: (1) there is a positive correlation between Guba-based sentiment and the stock market trends; 2) the higher the post volumes and agreement, more proficiency the bullishness would be; and (3) a long-lasting negative Guba-based sentiment indicates the arrival of the bear market. The proposed recommendation system consists of three criteria accordingly to ensure the portfolio to meet requirements of the theory. Finally, experiments are implemented using the real data of Chinese stock market from March 2009 to March 2016 and the results show the effectiveness of the proposed system in recommending lucrative stocks and the theoretical cumulate profit is about eight times of the CSI300 in the period.     

A stock trading expert system based on the rule-base evidential reasoning using Level 2 Quotes

Generally, stock trading expert systems (STES) called also “mechanical trading systems” are based on the technical analysis, i.e., on methods for evaluating securities by analyzing statistics generated by the market activity, such as past prices and volumes (number of transactions during a unit of a timeframe). In other words, such STES are based on the Level 1 information. Nevertheless, currently the Level 2 information is available for the most of traders and can be successfully used to develop trading strategies especially for the day trading when a significant amount of transactions are made during one trading session. The Level 2 tools show in-depth information on a particular stock. Traders can see not only the “best” bid (buying) and ask (selling) orders, but the whole spectrum of buy and sell orders at different volumes and different prices. In this paper, we propose some new technical analysis indices bases on the Level 2 and Level 1 information which are used to develop a stock trading expert system. For this purpose we adapt a new method for the rule-base evidential reasoning which was presented and used in our recent paper for building the stock trading expert system based the Level 1 information. The advantages of the proposed approach are demonstrated using the developed expert system optimized and tested on the real data from the Warsaw Stock Exchange.     

A gradient boosting decision tree approach for insider trading identification: An empirical model evaluation of China stock market

Insider trading is a kind of criminal behavior in stock market by using nonpublic information. In recent years, it has become the major illegal activity in China’s stock market. In this study, a combination approach of GBDT (Gradient Boosting Decision Tree) and DE (Differential Evolution) is proposed to identify insider trading activities by using data of relevant indicators. First, insider trading samples occurred from year 2007 to 2017 and corresponding non-insider trading samples are collected. Next, the proposed method is trained by the GBDT, and initial parameters of the GBDT are optimized by the DE. Finally, out-of-samples are classified by the trained GBDT–DE model and its performances are evaluated. The experiment results show that our proposed method performed the best for insider trading identification under time window length of ninety days, indicating the relevant indicators under 90-days time window length are relatively more useful. Additionally, under all three time window lengths, relative importance result shows that several indicators are consistently crucial for insider trading identification. Furthermore, the proposed approach significantly outperforms other benchmark methods, demonstrating that it could be applied as an intelligent system to improve identification accuracy and efficiency for insider trading regulation in China stock market.