Statistical Forward Planning (SFP) is a group of robust and general AI techniques that use a simulation model to adaptively search for effective sequences of actions in various games and other problems characterised as Markov Decision Processes or Extensive Form Games (and their partially observable versions, POMDPs and Factored Observation Stochastic Games).

SFP methods can operate without the need for prior training and can handle complex and dynamic environments. This tutorial will provide a tour of SFP from basics to finer points, complete with pointers to Python code (e.g. in OpenSpiel and other repositories). We will cover a number of powerful SFP algorithms including Monte Carlo Tree Search (MCTS), Rolling Horizon Evolutionary Algorithm (RHEA) and Monte Carlo Graph Search (MCGS), as well as handling partial observability with Information Set MCTS. We’ll also cover:

the relationship between SFP algorithms and Counterfactual Regret Minimisation (CFR)
incorporating policy and value functions (similar to AlphaZero)
efficient exploration functions for flat reward landscapes
handling combinatorial action spaces

Demonstrations will show these algorithms can play a variety of video games surprisingly well and provide insights into their working principles and behaviours. The tutorial will be suitable for those with no experience of SFP, but we also expect MCTS veterans to gain some fresh insights. We will conclude with a discussion of some of the most exciting challenges in the area.