Adaptive stock trading with dynamic asset allocation using reinforcement learning

Stock trading is an important decision-making problem that involves both stock selection and asset management. Though many promising results have been reported for predicting prices, selecting stocks, and managing assets using machine-learning techniques, considering all of them is challenging because of their complexity. In this paper, we present a new stock trading method that incorporates dynamic asset allocation in a reinforcement-learning framework. The proposed asset allocation strategy, called meta policy (MP), is designed to utilize the temporal information from both stock recommendations and the ratio of the stock fund over the asset. Local traders are constructed with pattern-based multiple predictors, and used to decide the purchase money per recommendation. Formulating the MP in the reinforcement learning framework is achieved by a compact design of the environment and the learning agent. Experimental results using the Korean stock market show that the proposed MP method outperforms other fixed asset-allocation strategies, and reduces the risks inherent in local traders.     

An intelligent stock trading system using comprehensive features

The aim of this study is to predict automatic trading decisions in stock markets. Comprehensive features (CF) for predicting future trend are very difficult to generate in a complex environment, especially in stock markets. According to related work, the relevant stock information can help investors formulate objects that may result in better profits. With this in mind, we present a framework of an intelligent stock trading system using comprehensive features (ISTSCF) to predict future stock trading decisions. The ISTSCF consists of stock information extraction, prediction model learning and stock trading decision. We apply three different methods to generate comprehensive features, including sentiment analysis (SA) that provides sensitive market events from stock news articles for sentiment indices (SI), technical analysis (TA) that yields effective trading rules based on trading information on the stock exchange for technical indices (TI), as well as the trend-based segmentation method (TBSM) that raises trading decisions from stock price for trading signals (TS). Experiments on the Taiwan stock market show that the results of employing comprehensive features are significantly better than traditional methods using numeric features alone (without textual sentiment features).     

Trend following deep Q-Learning strategy for stock trading

Computers and algorithms are widely used to help in stock market decision making. A few questions with regards to the profitability of algorithms for stock trading are can computers be trained to beat the markets? Can an algorithm take decisions for optimal profits? And so forth. In this research work, our objective is to answer some of these questions. We propose an algorithm using deep Q-Reinforcement Learning techniques to make trading decisions. Trading in stock markets involves potential risk because the price is affected by various uncertain events ranging from political influences to economic constraints. Models that trade using predictions may not always be profitable mainly due to the influence of various unknown factors in predicting the future stock price. Trend Following is a trading idea in which, trading decisions, like buying and selling, are taken purely according to the observed market trend. A stock trend can be up, down, or sideways. Trend Following does not predict the stock price but follows the reversals in the trend direction. A trend reversal can be used to trigger a buy or a sell of a certain stock. In this research paper, we describe a deep Q-Reinforcement Learning agent able to learn the Trend Following trading by getting rewarded for its trading decisions. Our results are based on experiments performed on the actual stock market data of the American and the Indian stock markets. The results indicate that the proposed model outperforms forecasting-based methods in terms of profitability. We also limit risk by confirming trading actions with the trend before actual trading.     

人工神经网络在证券价格预测中的应用

证券市场中成功的交易模式是可以模仿及学习的.证券价格走势实质是一种复杂时序函数.人工神经网络是在模仿人脑处理问题过程中发展起来的新型智能信息处理系统,人工神经网络可以通过调节连接权值以任意精度逼近任何连续函数,因此也可以逼近证券价格随时间变换这种函数.文中采用基于BP模型的神经网络,用BP算法和遗传算法来训练网络权值,同时也采用了动量法和学习率自适应调整相结合的策略,对证券市场的价格进行建模和预测,结果表明,此模型具有较好的学习、泛化能力,对股票市场或其他类似的非线性经济系统的走势预测决策具有较好的效果.     

Stock Trading Using RSPOP: A Novel Rough Set-Based Neuro-Fuzzy Approach

This paper investigates the method of forecasting stock price difference on artificially generated price series data using neuro-fuzzy systems and neural networks. As trading profits is more important to an investor than statistical performance, this paper proposes a novel rough set-based neuro-fuzzy stock trading decision model called stock trading using rough set-based pseudo outer-product (RSPOP) which synergizes the price difference forecast method with a forecast bottleneck free trading decision model. The proposed stock trading with forecast model uses the pseudo outer-product based fuzzy neural network using the compositional rule of inference [POPFNN-CRI(S)] with fuzzy rules identified using the RSPOP algorithm as the underlying predictor model and simple moving average trading rules in the stock trading decision model. Experimental results using the proposed stock trading with RSPOP forecast model on real world stock market data are presented. Trading profits in terms of portfolio end values obtained are benchmarked against stock trading with dynamic evolving neural-fuzzy inference system (DENFIS) forecast model, the stock trading without forecast model and the stock trading with ideal forecast model. Experimental results showed that the proposed model identified rules with greater interpretability and yielded significantly higher profits than the stock trading with DENFIS forecast model and the stock trading without forecast model.     

Stock trading with cycles: A financial application of ANFIS and reinforcement learning

Based on the principles of technical analysis, this paper proposes an artificial intelligence model, which employs the Adaptive Network Fuzzy Inference System (ANFIS) supplemented by the use of reinforcement learning (RL) as a non-arbitrage algorithmic trading system. The novel intelligent trading system is capable of identifying a change in a primary trend for trading and investment decisions. It dynamically determines the periods for momentum and moving averages using the RL paradigm and also appropriately shifting the cycle using ANFIS-RL to address the delay in the predicted cycle. This is used as a proxy to determine the best point in time to go LONG and visa versa for SHORT. When this is coupled with a group of stocks, we derive a simple form of “riding the cycles – waves”. These are the derived features of the underlying stock movement. It provides a learning framework to trade on cycles. Initial experimental results are encouraging. Firstly, the proposed framework is able to outperform DENFIS and RSPOP in terms of true error and correlation. Secondly, based on the test trading with five US stocks, the proposed trading system is able to beat the market by about 50 percentage points over a period of 13 years.     

大数据分析的应用案例——投资模型的稳健性

交易模型的稳健性,指的是该模型的利润率曲线的波动性较小,没有大起大落。针对一个基于支持向量回归（SVR）技术的算法交易模型的稳健性问题,提出了使用若干导出指标训练统一的交易模型的策略,以及投资组合多样化的方法。首先,介绍基于支持向量回归技术的算法交易模型;然后,基于常用指标,构造了若干导出指标,用于股票价格的短期预测。这些指标,刻画了近期价格运动的典型模式、超买/超卖市场状态,以及背离市场状态。对这些指标进行了规范化,用于训练交易模型,使得模型可以泛化到不同的股票;最后,设计了投资组合多样化方法。在投资组合里,各个股票之间的相关性,有时会导致较大的投资损失;因为具有较强相关关系的股票,其价格朝相同方向变化。如果交易模型预测的价格走势不正确,引起止损操作,那么这些具有较强相关关系的股票,将引发雪崩式的止损,于是导致损失加剧。把股票根据相似性聚类到不同类别,通过从不同聚类类别中选择若干股票来构成多样化的投资组合,其中,股票的相似性,通过交易模型在不同股票上近期的利润曲线的相似度进行计算。在900只股票10年的价格大数据上进行了实验,实验结果显示,交易模型能够获得超过定期存款的超额利润率,年化利润率为8.06%。交易模型的最大回撤由13.23%降为5.32%,夏普指数由81.23%提高到88.79%,交易模型的利润率曲线波动性降低,说明交易模型的稳健性获得了提高。     

A multiple fuzzy inference systems framework for daily stock trading with application to NASDAQ stock exchange

The aim of this study is to develop an expert system for predicting daily trading decisions in a typical financial market environment. The developed system thus employs a Multiple FISs framework consisting of three dedicated FISs for stock trading decisions, Buy, Hold and Sell respectively. As input to the Multiple FISs framework, the system takes the fundamental information of the respective companies and the historical prices of the stocks which are processed to give the technical information. The framework suggests the investor to Buy, Sell or Hold on a daily basis for a portfolio of stock taken into consideration. Experimenting the framework on selected stocks of NASDAQ stock exchange shows that including the fundamental data of the stocks as input along with the technical data significantly improves the profit return than that of the system taking only technical information as input data. Characterised as a stock market indicator, the framework performs better than some of the most popularly used technical indicators such as Moving Average Convergence/Divergence (MACD), Relative Strength Index (RSI), Stochastic Oscillator (SO) and Chaikin Oscillator (CO). The developed framework also gives better profit return compared to an existing model with similar objective.     

Portfolio management system in equity market neutral using reinforcement learning

Portfolio management involves position sizing and resource allocation. Traditional and generic portfolio strategies require forecasting of future stock prices as model inputs, which is not a trivial task since those values are difficult to obtain in the real-world applications. To overcome the above limitations and provide a better solution for portfolio management, we developed a Portfolio Management System (PMS) using reinforcement learning with two neural networks (CNN and RNN). A novel reward function involving Sharpe ratios is also proposed to evaluate the performance of the developed systems. Experimental results indicate that the PMS with the Sharpe ratio reward function exhibits outstanding performance, increasing return by 39.0% and decreasing drawdown by 13.7% on average compared to the reward function of trading return. In addition, the proposed PMS_CNN model is more suitable for the construction of a reinforcement learning portfolio, but has 1.98 times more drawdown risk than the PMS_RNN. Among the conducted datasets, the PMS outperforms the benchmark strategies in TW50 and traditional stocks, but is inferior to a benchmark strategy in the financial dataset. The PMS is profitable, effective, and offers lower investment risk among almost all datasets. The novel reward function involving the Sharpe ratio enhances performance, and well supports resource-allocation for empirical stock trading.

     

Ensemble learning of rule-based evolutionary algorithm using multi-layer perceptron for supporting decisions in stock trading problems

Classification is a major research field in pattern recognition and many methods have been proposed to enhance the generalization ability of classification. Ensemble learning is one of the methods which enhance the classification ability by creating several classifiers and making decisions by combining their classification results. On the other hand, when we consider stock trading problems, trends of the markets are very important to decide to buy and sell stocks. In this case, the combinations of trading rules that can adapt to various kinds of trends are effective to judge the good timing of buying and selling. Therefore, in this paper, to enhance the performance of the stock trading system, ensemble learning mechanism of rule-based evolutionary algorithm using multi-layer perceptron (MLP) is proposed, where several rule pools for stock trading are created by rule-based evolutionary algorithm, and effective rule pools are adaptively selected by MLP and the selected rule pools cooperatively make decisions of stock trading. In the simulations, it is clarified that the proposed method shows higher profits or lower losses than the method without ensemble learning and buy&hold.     

An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown

Dynamic control theory has long been used in solving optimal asset allocation problems, and a number of trading decision systems based on reinforcement learning methods have been applied in asset allocation and portfolio rebalancing. In this paper, we extend the existing work in recurrent reinforcement learning (RRL) and build an optimal variable weight portfolio allocation under a coherent downside risk measure, the expected maximum drawdown, E(MDD). In particular, we propose a recurrent reinforcement learning method, with a coherent risk adjusted performance objective function, the Calmar ratio, to obtain both buy and sell signals and asset allocation weights. Using a portfolio consisting of the most frequently traded exchange-traded funds, we show that the expected maximum drawdown risk based objective function yields superior return performance compared to previously proposed RRL objective functions (i.e. the Sharpe ratio and the Sterling ratio), and that variable weight RRL long/short portfolios outperform equal weight RRL long/short portfolios under different transaction cost scenarios. We further propose an adaptive E(MDD) risk based RRL portfolio rebalancing decision system with a transaction cost and market condition stop-loss retraining mechanism, and we show that the proposed portfolio trading system responds to transaction cost effects better and outperforms hedge fund benchmarks consistently.     

Empirical analysis: stock market prediction via extreme learning machine

How to predict stock price movements based on quantitative market data modeling is an attractive topic. In front of the market news and stock prices that are commonly believed as two important market data sources, how to extract and exploit the hidden information within the raw data and make both accurate and fast predictions simultaneously becomes a challenging problem. In this paper, we present the design and architecture of our trading signal mining platform that employs extreme learning machine (ELM) to make stock price prediction based on those two data sources concurrently. Comprehensive experimental comparisons between ELM and the state-of-the-art learning algorithms, including support vector machine (SVM) and back-propagation neural network (BP-NN), have been undertaken on the intra-day tick-by-tick data of the H-share market and contemporaneous news archives. The results have shown that (1) both RBF ELM and RBF SVM achieve higher prediction accuracy and faster prediction speed than BP-NN; (2) the RBF ELM achieves similar accuracy with the RBF SVM and (3) the RBF ELM has faster prediction speed than the RBF SVM. Simulations of a preliminary trading strategy with the signals are conducted. Results show that strategy with more accurate signals will make more profits with less risk.     

Intelligent stock trading system with price trend prediction and reversal recognition using dual-module neural networks

This article presents an intelligent stock trading system that can generate timely stock trading suggestions according to the prediction of short-term trends of price movement using dual-module neural networks(dual net). Retrospective technical indicators extracted from raw price and volume time series data gathered from the market are used as independent variables for neural modeling. Both neural network modules of thedual net learn the correlation between the trends of price movement and the retrospective technical indicators by use of a modified back-propagation learning algorithm. Reinforcing the temporary correlation between the neural weights and the training patterns, dual modules of neural networks are respectively trained on a short-term and a long-term moving-window of training patterns. An adaptive reversal recognition mechanism that can self-tune thresholds for identification of the timing for buying or selling stocks has also been developed in our system. It is shown that the proposeddual net architecture generalizes better than one single-module neural network. According to the features of acceptable rate of returns and consistent quality of trading suggestions shown in the performance evaluation, an intelligent stock trading system with price trend prediction and reversal recognition can be realized using the proposed dual-module neural networks.     

Using genetic algorithm to support clustering-based portfolio optimization by investor information

A clustering-based portfolio optimization scheme that employs a genetic algorithm (GA) based on investor information for active portfolio management is presented. Whereas numerous studies have investigated trading behaviors, investor performance, and portfolio investment strategies, few works have developed investment strategies based on investor information. This study is conducted in two phases. First, a basket of portfolio (i.e., a collection of stocks held in individual portfolios) is developed through a cluster analysis of investor information. A GA is then employed to optimize the weights of the selected stocks. And the optimized portfolio is rebalanced to get excess return. It is concluded that the proposed multistage portfolio optimization scheme for active portfolio management generates superior results than previously proposed methods for the Korean stock market.     

Optimal portfolio trading subject to stochastic dominance constraints under second‐order autoregressive price dynamics

This paper studies the optimal portfolio trading problem under the generalized second‐order autoregressive execution price model. The problem of minimizing expected execution cost under the proposed price model is formulated as a quadratic programming (QP) problem. For a risk‐averse trader, problem formulation under the second‐order stochastic dominance constraints results in a quadratically constrained QP problem. Under some conditions on the execution price model, it is proved that the portfolio trading problems for risk‐neutral and risk‐averse traders become convex programming problems, which have many theoretical and computational advantages over the general class of optimization problems. Extensive numerical illustrations are provided, which render the practical significance of the proposed execution price model and the portfolio trading problems.     

Combining KADS with ZEUS to Develop a Multi-Agent E-Commerce Application

A KADS based requirement analysis for the management of stock trading portfolios is presented. This provides a theoretical foundation for a stock trading system. This system is designed around portfolio management tasks that include eliciting user profiles, collecting information on the user's portfolio position, monitoring the environment on behalf of the user, and making decision suggestions to meet the user's investment goals. The requirement analysis defines a framework for a Multi-Agent System for Stock Trading (MASST). Experiments in task decomposition and agent interaction using a partially implemented system are described.     

Trading With a Stock Chart Heuristic

The efficient market hypothesis (EMH) is a cornerstone of financial economics. The EMH asserts that security prices fully reflect all available information and that the stock market prices securities at their fair values. Therefore, investors cannot consistently ldquobeat the marketrdquo because stocks reside in perpetual equilibrium, making research efforts futile. This flies in the face of the conventional nonacademic wisdom that astute analysts can beat the market using technical or fundamental stock analysis. The purpose of this research is to partially assess whether technical analysts, who predict future stock prices by analyzing past stock prices, can consistently achieve a trading return that outperforms the stock market average return. This is tested using knowlege engineering experimentation with one price history pattern - the ldquobull flag stock chartrdquo - which signals technical analysts of a future stock market price increase. A recognizer for the stock chart pattern is built using a template-matching technique from pattern recognition. The recognizer and associated trading rules are then tested by simulating trading on over 35 years of daily closing price data for the New York stock exchange composite index. The experiment is then replicated using the horizontal rotation or mirror image pattern of the ldquobull flagrdquo (or ldquobear flagrdquo stock chart) that signals a future stock market decrease. Results are systematic, statistically significant, and fail to confirm the null hypothesis based on a corollary to the EMH: that profit realized from trading determined by this heuristic method is no better than what would be realized from trading decisions based on random choice.     

Constructing a dynamic stock portfolio decision-making assistance model: using the Taiwan 50 Index constituents as an example

There are several decisions in investment management process. Security selection is the most time-consuming stage. Tatical allocation is in order to take advantage of market opportunities based on short-term prediction (Amenc and Le Sourd in Portfolio theory and performance analysis. Wiley, 2003). Although it is difficult to keep track of the fluctuations of volatile financial markets, the capacity of artificial intelligence to perform spatial search and obtain feasible solutions has led to its recent widespread adoption in the resolution of financial problems. Classifier systems possess a dynamic learning mechanism, they can be used to constantly explore environmental conditions, and immediately provide appropriate decisions via self-aware learning. This study consequently employs a classifier system in conjunction with real number encoding to investigate how to obtain optimal stock portfolio based on investor adjustment cycle. We examine the constituents of the TSEC Taiwan 50 Index taking moving average (MA), stochastic indicators (KD), moving average convergence divergence (MACD), relative strength index (RSI) and Williams %R (WMS %R) as input factors, adopting investor-determined adjustment cycle to allocate capital, and then constructing stock portfolio. We have conducted empirical testing using weekly and monthly adjustment cycle; the results revealed that this study’s decision-making assistance model yields average annual interest rate of 49.35%, which is significantly better than the −6.59% of a random purchase model. This research indicates that a classifier system can effectively monitor market fluctuations and help investors obtain relatively optimal returns. The assistance model proposed in this study thus can provide really helpful decision-making information to investors.     

基于深度迁移学习的多尺度股票预测

股票市场不仅是上市公司的重要融资渠道,也是重要的投资市场,股票预测一直受到人们的关注。为了充分利用来自不同股票价格的信息,提高股票的预测效果,提出一种多尺度股票价格预测模型TL-EMD-LSTM-MA（TELM）。TELM模型通过经验模态分解将收盘价分解为多个时间尺度分量,不同时间尺度分量震荡频率不同,反映了不同的周期性信息;根据分量的震荡频率选择不同方法进行预测,高频分量利用深度迁移学习的方法训练堆叠LSTM,低频分量利用移动平均法进行预测;将所有分量的预测值相加作为收盘价的最终预测输出。通过深度迁移学习训练的堆叠LSTM,包含来自不同股票的信息,具备更多行业或市场的知识,能有效降低预测误差。利用移动平均法预测低频分量,更有效捕获股票的总体趋势。对中国A股市场内500支股票以及上证指数、深证成指等指数进行预测,结果表明,与其他模型相比,TELM预测误差最低,拟合优度最高。根据TELM预测的股票收盘价模拟股票交易过程,结果表明TELM投资风险低、收益高。     

A portfolio optimization model using Genetic Network Programming with control nodes

Many evolutionary computation methods applied to the financial field have been reported. A new evolutionary method named “Genetic Network Programming” (GNP) has been developed and applied to the stock market recently. The efficient trading rules created by GNP has been confirmed in our previous research. In this paper a multi-brands portfolio optimization model based on Genetic Network Programming with control nodes is presented. This method makes use of the information from technical indices and candlestick chart. The proposed optimization model, consisting of technical analysis rules, are trained to generate trading advice. The experimental results on the Japanese stock market show that the proposed optimization system using GNP with control nodes method outperforms other traditional models in terms of both accuracy and efficiency. We also compared the experimental results of the proposed model with the conventional GNP based methods, GA and Buy&Hold method to confirm its effectiveness, and it is clarified that the proposed trading model can obtain much higher profits than these methods.