PWR loading pattern optimization with reinforcement learning

Abstract

The core loading pattern optimization problem belongs to the class of combinatorial optimization problem and has been studied since the dawn of commercial nuclear energy industry. It is characterized by multiple objectives and constraints, with a very high number of candidate patterns, which makes it impossible to solve explicitly. Stochastic optimization methodologies including Genetic Algorithms and Simulated Annealing are used by different nuclear utilities and vendors to perform fuel cycle reload design. Nevertheless, hand-designed solutions continue to be the prevalent method in the industry. To improve the state-of-the-art core reload patterns, we aim to create a method as scalable as possible, that agrees with the designer's goal of performance and safety. To help in this task Deep Reinforcement Learning (DRL), in particular Proximal Policy Optimization is leveraged. DRL has recently experienced a strong impetus from its successes applied to games, sometimes even reaching 'super-human' performances. This paper lays out the foundation of this method and proposes to study the behavior of several hyper-parameters that influence the DRL algorithm. The algorithm is highly dependent on multiple factors such as an exploration/exploitation trade-off that manifests through different parameters such as the number of loading patterns seen and the number of samples collected before a policy update, but also the shape of the objective function derived for the core design. Experimental results also demonstrate the effectiveness of the method in finding high-quality solutions from scratch within a reasonable amount of time. (authors)