Experimental data-efficient reinforcement learning with an ensemble of surrogate models.

Abstract

Model-based reinforcement learning methods enhance sample efficiency by generating synthetic data during training. However, modeling errors can undermine training, leading to failures when applied to the actual environment, especially due to discrepancies in the learned dynamics. In this paper, we propose a new ensemble of double surrogate models created using symbolic regression to uncover the fundamental physical principles governing system behavior, thereby enabling more data-efficient real-world applications. Symbolic regression enhances model interpretability by producing straightforward models that generalize well with limited data. These models interact with reinforcement learning algorithms, with interactions occurring solely within the synthetic models, significantly reducing the need for real experimental data. The structure of the double surrogate models mitigates model bias, preventing agents from exploiting inaccuracies in the environment that could lead to poor performance. Our approach demonstrates comparable training performance when validated in real environments, requiring less than 1 % of the experimental data typically needed for conventional reinforcement learning algorithms.