A physical theory of reinforcement learning

摘要: Recently, non-classical statistical patterns on outputs of current large language models, including Bell inequality violations in word distributions, have been reported. In this paper, we present a physical formulation of reinforcement learning (RL), involving the operator-theoretic framework of open/generalized quantum systems. Precisely, we construct physical models that realise RL as an open/generalized quantum evolution process beyond the mathematical model of classical Markov decision processes for RL, with classical and quantum evolutions recovered as two special cases. As an illustration, we apply the physical theory to the search problem in unstructured large datasets, and propose an associated scheme for diagnosing the effect of non-classical correlations to current AI systems. Our models clarify the physical origin of RL, and offer a perspective for understanding current AI systems through their underlying physical processes.