Neural Computing and Applications Программное обеспечение

Benchmark of non-stationary bandit algorithms

Puchakayala, Hema Chandra, Wallett, Tyler, Jafari, Amir
Springer Nature · 2026-11-14 · DOI 10.1007/s00521-026-12583-1

Аннотация

Recommendation systems deployed in dynamic environments must adapt to users whose preferences drift over time, yet many bandit algorithms still assume stationary reward distributions. This work presents a reproducible simulation framework that models user drift as a latent preference vector over content categories evolving in a non-stationary multi-armed bandit setting. Within this framework we implement four adaptive algorithms: Non-Stationary $$\varepsilon $$ -Greedy, Sliding-Window UCB, CUSUM-UCB, and EXP3.S and evaluate them on synthetically generated drifting-user data using average and cumulative reward with confidence intervals over 50 runs. The results show that Sliding-Window UCB consistently achieves the highest long-run reward, followed by CUSUM-UCB and Non-Stationary $$\varepsilon $$ -Greedy, while the adversarially robust EXP3.S underperforms in this stochastic drift regime. These findings suggest that methods which explicitly discount or reset outdated observations are well suited to gradual user drift, and that adversarial bandit algorithms can be overly conservative when non-stationarity arises from user behavior rather than malicious manipulation.

Читать у издателя →