Robustness Analysis of Deep Reinforcement Learning for Online Portfolio Selection
Keywords:
Online Portfolio Selection, Deep Reinforcement Learning, Resource Allocation, Robustness Analysis, Market RepresentationSynopsis
This is a Chapter in:
Book:
Intelligent Computing and Consumer Support Applications
Series:
Chronicle of Computing
Chapter Abstract:
Online Portfolio Selection (OLPS) requires a careful mix of assets to minimize risk and maximize rewards over a trading episode. The stochastic, non-stationary aspect of the market makes decision-making very complex. Heuristic methods relying on historical returns were traditionally used to select assets that found a balance of risk and reward. However, improvements in modeling time series from Neural Networks led to new solutions. Deep Reinforcement Learning (DRL) has become a popular approach to solve this problem, but its methods rarely reach a consensus among publications. In other fields, solutions using non-Markovian state representations are frequent. Crafting rewards to improve agent learning is common but has effects on the resulting behaviors. The resulting processes are rarely compared to other recent State-of-the-Art solutions but to heuristic algorithms. The proliferation of approaches motivated us to benchmark them using traditional financial metrics, and evaluate their robustness over time and across market conditions. We aim to evaluate the contributions to measured performance from each method in market representation, policy learning and value estimation.
Keywords:
Online Portfolio Selection, Deep Reinforcement Learning, Resource Allocation, Robustness Analysis
Cite this paper as:
Velay M., Doan B.-L., Rimmel A., Popineau F., Daniel F. (2023) Robustness Analysis of Deep Reinforcement Learning for Online Portfolio Selection. In: Tiako P.F. (ed) Intelligent Computing and Consumer Support Applications. Chronicle of Computing. OkIP. https://doi.org/10.55432/978-1-6692-0003-1_2
Presented at:
The 2022 OkIP International Conference on Automated and Intelligent Systems (CAIS) in Oklahoma City, Oklahoma, USA, and Online, on October 3-6, 2022
Contact:
Marc Velay
marc.velay@centralesupelec.fr
References
Eric Benhamou et al. (2020). Bridging the Gap Between Markowitz Planning and Deep Reinforcement Learning. en. SSRN Scholarly Paper ID 3702112. Rochester, NY: Social Science Research Network. DOI: 10. 2139/ssrn.3702112.
Ricard Durall. (2022) Asset Allocation: From Markowitz to Deep Reinforcement Learning.
Xinyi Li et al. (2019a). Optimistic Bull or Pessimistic Bear: Adaptive Deep Reinforcement Learning for Stock Portfolio Allocation. In: arXiv:1907.01503.
Yang Li, Wanshan Zheng, and Zibin Zheng. (2019b). Deep Robust Reinforcement Learning for Practical Algorithmic Trading. en. In: IEEE Access 7, pp. 108014– 108022. ISSN: 2169-3536. DOI: 10 . 1109 / ACCESS . 2019.2932789.
Zhipeng Liang et al. (2018) Adversarial Deep Reinforcement Learning in Portfolio Management. In: arXiv:1808.09940.
Harry Markowitz. (1952). Portfolio Selection. In: The Journal of Finance 7.1, pp. 77–91. ISSN: 0022-1082. DOI: 10/bxrq3f.
Janosch Moos et al. (2022). Robust Reinforcement Learning: A Review of Foundations and Recent Advances. en. In: Machine Learning and Knowledge Extraction 4.1, pp. 276–315. ISSN: 2504-4990. DOI: 10. 3390/make4010013.
Uta Pigorsch and Sebastian Schafer. (2021) High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning. In: arXiv:2112.04755.
Yunan Ye et al. (2020). Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States. In: arXiv:2002.05780.
Yifan Zhang et al. (2020). Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning. In: IEEE Transactions on Knowledge and Data Engineering PP, pp. 1–1. DOI: 10/gj6rzg.
