A novel deep reinforcement learning framework with task-incremental continual forecasting for trading systems
Аннотация
The increasingly complex and non-stationary nature of financial markets has created a growing need for automated trading systems capable of adapting to evolving market conditions. Although Deep Reinforcement Learning (DRL) has shown promising performance in financial trading, its effectiveness may decline when market patterns change over time. To address this limitation, we propose the Continual Forecasting Fusion Deep Reinforcement Learning (CFFDRL) framework, which incorporates Streaming Continual Learning (SCL) through a Continuous Piggyback GRU forecasting module to adaptively generate future price predictions for a DRL-based trading agent. cPB is an SCL method that adapts to newly generated data by learning task-specific masks over a frozen pre-trained backbone without modifying the original weights. Specifically, in CFFDRL, the cPB concept is incorporated into a GRU-based forecasting module, called cPB-GRU, which incrementally predicts future prices from historical OHLC data. The module is continuously updated during both training and testing, enabling adaptation to new market conditions. The generated forecasts are concatenated with the OHLC data and used as the observation space for the DRL agent. Experimental results on six datasets show that CFFDRL achieves an average cumulative return of 50.09%, compared with 33.28% for DRL-PPO and 19.48% for PPO-Static-GRU. It also achieves the highest average Sharpe ratio of 0.10 and the lowest average maximum drawdown of 19.46% among the evaluated PPO variants. These results show that continual forecasting improves both profitability and risk-adjusted performance.