This paper by Kuleshov and Doina gives some practical advice and empirical evidence for multi-armed bandit (MAB) algorithms. They conduct several experiments in the standard MAB setting with $\epsilon$-greedy, upper confidence bound, learning, and Boltzmann algorithms. Their time horizon is only 1000 pulls, so, in general, the $\epsilon$-greedy and Boltzmann algorithms did better than the logarithmic regret algorithms.

Related Posts via Categories

• Link/Book(pdf): “Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems”
• Link: AlphaGo Wins against one of the best Go Players on the Planet
• Ted Dunning on Bandits
• “Stochastic Superoptimization” and “Programming by Optimization”
• “Generalized Thompson sampling for sequential decision-making and causal inference”
• “Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained”
• “An Empirical Evaluation of Thompson Sampling”
• “Linear Bandits in High Dimension and Recommendation Systems”
• “A Survey of Monte Carlo Tree Search Methods”
• Schwarzenegger Bandit Success Formula