Most, if not all, optimisation algorithms have exposed parameters and various components, while the choice of which affects the performance of the algorithm. Finding the right values of these parameters that maximise the performance is a challenging task itself. Even more so, when this is done in a manual iterative manner, which leads to a tedious and suboptimal workflow. Fortunately, there exist several configuration frameworks that automatically search for a good configuration of parameters for the task at hand, optimising specific performance measures. These automated algorithm configurators have demonstrated to be a powerful tool in the design and configuration of algorithms in the last two decades. Although the concept of automated algorithm configuration (AAC) is generally known to algorithm designers and practitioners, the adoption of these methods in empirical research studies stays behind. One reason for this limited adoption is that most AAC frameworks are presented for the general use case of finding an optimally configured algorithm for a specific problem domain and do not explicitly showcase their usability in a broader setting. In this tutorial we go beyond the default AAC scenario to demonstrate the potential and capabilities of several AAC techniques in an actual research context. We especially focus on scenarios with multiple objectives involved, both at an algorithm level (configuring multi-objective optimisers) as at the configuration level (generating optimal performance trade-offs).

The tutorial is structured into multiple parts, each describing a different use case for AAC. In each use case we show the usability of AAC in a practical application setting, followed by an actual demonstration of how to set up the experimental pipeline using various AAC frameworks, such as irace, (MO-)ParamILS or (MO-)SMAC. The use cases will cover four scenarios: 1) fairly comparing (multi-objective) optimisation algorithms with AAC, 2) configuring anytime behaviour of algorithms, 3) configuring algorithms to optimise multiple performance objectives simultaneously, and 4) improving algorithm understanding using AAC with additional analytical tools. The tutorial will end with an open discussion on how to integrate AAC in one’s own research.