Research on multi-parameter self-tuning of digital low-level RF system based on reinforcement learning

摘要: To ensure the efficient operation of the upgraded digital LLRF system of the CSNS-II linear accelerator under higher precision and larger scale conditions, this study employs reinforcement learning (RL) algorithm to train an agent for autonomous optimization of LLRF control parameters. First, two RL algorithms, proximal policy optimization (PPO) and soft actor-critic (SAC), were trained respectively within the established LLRF simulation model to compare their efficiency and accuracy in LLRF parameter optimization. The study shows that, benefiting from its off-policy nature, the SAC algorithm outperforms PPO in terms of data utilization efficiency and convergence speed, and is thus selected as the baseline algorithm for agent training. Subsequently, validation experiments based on the SAC algorithm were conducted on the CSNS buncher cavity, where the trained agent successfully achieved autonomous closed-loop control parameter optimization, meeting the expected control objectives.