Search NASA⌕ Search

SEARCH · Search NASA

Results for “PID Control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e

We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab). Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale. We treat the Mu2e accelerator system as a Markov Decision Process suitable for Reinforcement Learning (RL), utilizing PPO to reduce bias and enhance training stability. A key innovation in our approach is the integration of a neuralized Proportional-Integral-Derivative (PID) controller into the policy function, resulting in a significant improvement in the Spill Duty Factor (SDF) by 13.6%, surpassing the performance of the current PID controller baseline by an additional 1.6%. This paper presents the preliminary offline results based on a differentiable simulator of the Mu2e accelerator. It paves the groundwork for real-time implementations and applications, representing a crucial step towards automated proton beam intensity control for the Mu2e experiment.

43 PARTICLE ACCELERATORS↗

A Reinforcement Learning Approach to Augment Conventional PID Control in Nuclear Power Plant Transient Operation

The ability of nuclear reactors to operate their power conversion cycles more flexibly will enhance their value to energy grids with variable pricing. Current nuclear control systems are typically classical controllers that are often based on proportional-integral-derivative (PID) control. This paper presents a method of augmenting the existing PID control for difficult transient operations in nuclear power plants using a reinforcement learning–derived feedforward signal applied in real time. The agents, which are trained on a test thermal load-following problem, are designed to improve steam generator outlet temperature control for a range of fast load-following scenarios covering ramp rates from 9%/min to 15%/min. Several reinforcement learning algorithms were initially investigated for the training of the feedforward agents with deep Q-learning (DQN) and proximal policy optimization (PPO) networks, which were found to be the most promising. The DQN controllers utilize discrete actions, giving them a better disturbance rejection at steady state but inconsistent response to initial temperature deviations. In contrast, PPO-trained agents, which take continuous actions except for a dead zone around zero, were shown to have the best combination of high disturbance rejection at steady state and good tracking of the desired temperature value. The ability of the PPO agent was also examined, with the average time of decision making found to be on the order of 1 ms. The fault properties of the controller under the loss of the reinforcement learning agent feedforward signal were also examined. The controller showed strong performance in situations of “no-signal” faults. but was less good at handling “stuck-at” faults, where the feedforward signal remains at a set value. In both cases, however, the PID was able to successfully maintain stability, eventually returning the system to a steady state. It is hoped that this work will allow for the proposed control architecture to be examined for more difficult control problems such that it may eventually be used to adapt existing nuclear plants for more aggressive load-following on grids of the future.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Signal-Compensation-Based Adaptive PID Control for Fused Magnesia Smelting Processes

For fused magnesia smelting processes, the proportional-integral-derivative (PID) controller is difficult to regulate the smelting currents within desired ranges due to unexpected variations of the dynamics caused by material feeding operations. To solve this problem, we first proposed its dynamic model composed of a linear model and an uncertain nonlinear dynamic system that describes the unknown time-varying dynamics of nonlinearity and strong couplings among controlled variables. Then, using the accurately calculated value of the uncertain part at the previous sampling time instant and the tracking error that reflects the influence of its change rate, we developed a feedforward and a one-step optimal control to obtain two signal compensators to eliminate the impact of frequent variations of the uncertainties. For the compensators, we proposed a weight selection method to minimize the sum of the squared tracking error with ensured closed-loop stability. Based on the abovementioned modeling and control structure, we finally developed a novel adaptive PID controller augmented with signal compensators to control the whole process. Industrial applications have shown that the proposed method can always keep the tracking error and the control input within their targeted ranges, leading to a significantly reduced energy consumption per ton and increased fused magnesia production.

Wang, Weizhou↗

Robust PID Control Against Nonlinear Uncertainties in Modern Power Systems with Microgrids

This paper investigates the problem of robust PID control for nonlinear power systems with uncertainties. In this paper, we analyze the challenges posed by nonlinearities and uncertainties in power systems, as well as the limitations of traditional linearization methods. On this basis, a PID control design method is proposed, which utilizes only the Lipschitz constant information of the nonlinear uncertainties to achieve global stability control for the power systems. The paper provides a rigorous mathematical foundation to demonstrate the effectiveness and robustness of the controller. Finally, simulation studies are conducted to validate and evaluate the performance of the proposed model, algorithm, and controller design.

Yuan, Shuo↗

Integrate FARM with PID controllers: IES Simulation Ecosystem Control System Development

The FARM (Feasible Actuator Range Modifier) software module is a component of the RAVEN-based FORCE framework for analysis of Integrated Energy Systems (IES). FARM was designed to support the HERON software module in the solution of the optimal dispatch problem for IES units. As the result of HERON-FARM dispatch simulation, the set-point trajectories are optimized to meet constraints on both the production variables (i.e., the variables to be optimized such as the electrical power, the hydrogen production rate, etc.) and the process variables tied to the service life of equipment (e.g., steam flowrate, vessel pressure, turbine firing temperature, etc.) at a coarse time resolution (every 10 or 100 seconds) over long time horizons (several days or weeks). In case the operational constraints need to be met at finer time resolution, the computational burden of HERON-FARM would linearly increase with the sampling rate, and sub-optimal solutions might be obtained. System responses characterized by overshoots and damped oscillations temporarily violating the imposed constraints might occur during abrupt power transients. In this report, a hierarchical control system architecture for the operation of the Dynamic Energy Technology and Integration Laboratory (DETAIL) facility constructed at INL was proposed. First, the preliminary studies on the proposed control strategy for operating the facility and the designed PI controllers were reviewed. In particular, the current approach for generating the set-point trajectories was studied, and its limits were identified. To this aim, the inclusion of a Supervisory Control layer embedding a modified version of the FARM algorithm for preserving the system safe operation over both long and real-time horizons was proposed. In this way, FARM would be applied twice, i.e., the original version (“FARM Validator”) aiding the solution of the power dispatch problem, and the modified version (“FARM Supervisory” coordinating the PI controllers to address the real-time control tasks. Despite the kernel of the two modules is the same algorithm, their roles, tasks, and capabilities are quite different. A detailed description of the role of FARM at addressing low-level control tasks is provided, along with tentative operational procedures for training the models embedded into the algorithm by using the collected experimental data.

42 ENGINEERING↗

Hybrid Temperature-Reactivity PID Controllers for Nuclear Thermal Propulsion Startup

In this work, a novel temperature-reactivity controller is introduced to rotate control drums based on a chamber temperature signal. It consists of two components: (1) a reactivity-driven proportional controller converting the temperature demand into a reactivity signal, and (2) a temperature-driven proportional-derivative controller which relies on a derivative term to anticipate sudden changes in demand and limit temperature overshoot. A weighting function is proposed to continuously transition from one component to the other.

33 ADVANCED PROPULSION SYSTEMS↗

Hybrid Temperature-Reactivity PID Controllers for Nuclear Thermal Propulsion Startup

In this work, a novel temperature-reactivity controller is introduced to rotate control drums based on a chamber temperature signal. It consists of two components: (1) a reactivity-driven proportional controller converting the temperature demand into a reactivity signal, and (2) a temperature-driven proportional-derivative controller which relies on a derivative term to anticipate sudden changes in demand and limit temperature overshoot. A weighting function is proposed to continuously transition from one component to the other.

33 ADVANCED PROPULSION SYSTEMS↗

PID Temperature Controller Autotuning for Skipper-CCDs

Proportional-integral-derivative (PID) temperature control of Skipper-CCDs is important for dark matter direct-detection experiments, e.g., SENSEI@MINOS. The project goal is to create an autotuned PID controller in Python to replace expensive Lake Shore temperature controllers.

Jones, Evan↗

Design of a Combined Proportional Integral Derivative Controller to Regulate the Temperature Inside a High-Temperature Tubular Solar Reactor

Solar fuels are proven to be promising candidates for thermochemical energy storage. However, the transient nature of solar radiation is an obstacle to maintaining a stable operational temperature inside a solar reactor. To overcome this challenge, the temperature of a solar reactor can be regulated by controlling the incoming solar radiation or the feedstock flowrate inside the reactor. In this work, a combined proportional integral derivative (PID) controller is implemented to regulate the temperature inside a high-temperature tubular solar reactor with counter-current flowing gas/particles. The control model incorporates two control systems to regulate incoming solar radiation and gas flow simultaneously. The design of the controller is based on a reduced-order numerical model of a high-temperature tubular solar reactor that is vertically oriented with an upward gas flow and downward particle flow. The reactor receives heat circumferentially through its wall over a finite segment of its length. Formulation of the heat transfer model is presented by applying the energy balance for the reactor tube and considering heat and mass transfer inside. A set of governing differential equations are solved numerically by using the finite volume method to obtain reactor wall, particles, and gas temperatures along the reactor length with various boundary conditions. Simulation results are used to tune the PID controller parameters by utilizing the Ziegler–Nichols tuning method. Both the simulation results and the controller performance are visualized on the LABVIEW platform. The controller is challenged to track different temperature setpoints with different scenarios of transient solar radiation. Here, the performance of the PID controller was compared to experimental results obtained from an industrial PID controller embedded in a 7 kW electric furnace. Results show that the combined PID controller is successful in maintaining a stable temperature inside the reactor by regulating the incoming solar radiation and the flowrate via small steady-state error and reasonable settling time and overshoot.

14 SOLAR ENERGY↗

Simulating a Reactor with PID Temperature Control

A Simulation of a Reactor, Built Using CrowPis, which are simple and accessible programmable systems of various electronics, as ground stations and a Box containing heaters as the reactor. The goal of this project is to learn various skills and to demonstrate the feasibility of a simulated reactor, utilizing a PID control and wireless connectivity.

42 - ENGINEERING↗

Uncertainty Quantification and Sensitivity Analysis of Non-Nuclear Advanced Controls Testbed Reactor Mockup

The research presented in this report describes our progress in applying stochastic methods and uncertainty quantification, parametric study, and variance-based sensitivity analysis (also known as Sobol sensitivity analysis) to a full-core model of a nuclear thermal propulsion (NTP) system simulated with Griffin, with the goal of developing a reduced order (surrogate) model which can be rapidly sampled while perturbing multiple input parameters. In this NTP system, reactivity and power feedback affect the rotation of control drums, which are controlled by a hybrid proportional, integral and derivative (PID) controller, actuated by the power demand and reactivity feedback from the numerical model. This model uses reactor kinetic feedback (mean generation time and $\beta$ from a transient Griffin simulation executed with the improved quasi-static method to provide the kinetic parameters) as inputs to functions which control the CD rotation angle. Using a number of stochastic method approaches, we developed a dual purpose training-surrogate model of the NTP system using polynomial regression. The trained model can be rapidly sampled while simultaneously perturbing various input parameters of the model, such as coefficients on the PID control, or temperature (directly affect the neutron cross section). The surrogate model delivers accurate results orders-of-magnitude faster (minutes, not days) than the base model. Once the base model has been trained, distributions of the uncertain parameters can be changed at will to investigate the effects of perturbing multiple inputs and their effect on the output. For example, coefficients used in the PID control system may vary due to some physical interference, or there may be uncertainty in the temperature of the neutron cross sections in various regions of the reactor. A distribution can be placed on these parameters and operational boundaries can be determined. The goal of this work is to support development of an advanced control system to operate CDs in a functioning NTP system.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Uncertainty quantification and sensitivity analysis of a nuclear thermal propulsion reactor startup sequence

The research presented in this article describes progress in applying stochastic methods, uncertainty quantification, parametric studies, and variance-based sensitivity analysis (also known as Sobol sensitivity analysis) to a full-core model of a nuclear thermal propulsion (NTP) system simulated via the radiation transport code Griffin to simulate neutronics. Our goal is to develop a reduced-order (surrogate) model that can be rapidly sampled with perturbations to multiple input parameters. In this NTP system, reactivity and power feedback affect the rotation of control drums (CDs), which is itself controlled by a hybrid proportional-integral-derivative (PID) controller actuated by the power demand and reactivity feedback from the numerical model. This model uses reactor kinetic feedback (mean generation time [Λ] and effective delayed neutron fraction [ β eff ] from a transient Griffin simulation executed via Griffin’s improved quasi-static solver to provide the kinetic parameters) as inputs to functions that control the CD rotation angle. By investigating numerous stochastic approaches, we developed a dual-purpose surrogate model of the NTP system, using polynomial regression in the Multiphysics Object-Oriented Simulation Environment (MOOSE) Stochastic Tools Module (STM). The trained model can be rapidly sampled while simultaneously perturbing various input parameters, such as coefficients on the PID control or temperature (directly affecting the neutron cross section). The surrogate model delivers accurate (within 5%) results at speeds orders of magnitude faster (minutes, not days of computational time) than the base model. Once the surrogate model has been trained, distributions of the uncertain parameters can be changed at will to investigate the effects of perturbing multiple inputs as well as the effects of these inputs on the model output. For example, coefficients used in the PID control system may vary due to some type of physical interference, or uncertainty may exist in the temperature of the neutron cross sections in various regions of the reactor. A distribution can be placed on these parameters, and operational boundaries can be determined. The goal of this work is to support development of an advanced control system for operating CDs in a functioning NTP system. This work is a scoping study of the MOOSE STM.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗

Counter-Current Flow Limitation Studies in Complex Geometries Utilizing Interface Capturing Simulations Coupled with PID Flow Rate Controller

In nuclear thermal-hydraulic studies, counter-current flow limitation (CCFL) typically refers to steam rising at a fast rate such that it prevents coolant from draining down within a confined channel. CCFL is a crucial issue in nuclear reactor safety analysis. This study investigates CCFL in debris bed channels using high-resolution interface-capturing simulations. A novel proportional-integral-derivative flow rate controller is developed to efficiently achieve the CCFL conditions. Verification studies confirm that CCFL occurs under the same conditions with or without the controller, demonstrating that PID control ensures accurate prediction. Three debris bed channel geometries were examined: a cylindrical channel, a channel with small obstacles, and a channel with large obstacles. Results show that obstacles significantly impact flow behavior, interfacial shear, wall shear, and pressure gradients required for CCFL. Furthermore, the comparison with experimental data confirmed that simulations incorporating geometric complexities align more closely with experimental CCFL conditions. A pressure gradient correlation was also developed for CCFL prediction.

Counter-current flow limitation↗

Third-integer Resonant Extraction Regulation System for Mu2e

A third-integer resonant slow extraction system is being developed for Fermilab's Delivery Ring to deliver protons to the upcoming Mu2e experiment. The timescale of the extraction (or spill) duration is 43 milliseconds, which is extremely short and unprecedented. Additionally, the experiment's strict and challenging requirements on the quality of the spill at this time scale has led to the development of a new Spill Regulation System (SRS) design. The SRS primarily consists of three components - slow regulation, fast regulation, and harmonic content suppressor. Contributions to the first two components of the SRS, i.e., Slow Regulation and Fast Regulation subsystems, will be presented in which new adaptive learning algorithm schemes for the slow regulation of the spill -- validated using particle tracking simulations -- shall be described. In addition to these novel methods for the enhancement of the spill regulation system, results of employing Machine Learning in enhancing the performance of the resonant extraction are also presented. At the forefront of applying ML techniques to solve non-linear accelerator control problems, this work includes optimizing the PID gains as well as the replacement of the traditional PID controller using Recurrent Neural Networks and Gated Recurrent Unit (GRU) ML models to achieve efficiencies greater than a PID controller. Cutting-edge on-going Reinforcement Learning efforts, including an actor-critic family of learning algorithms, to regulate the spill rate will be reviewed, as well as present analytical calculations pertaining the transit time of particles in a third-integer resonant extraction. Detailed numerical investigations and validations of such calculations, the model of which could be exported and reliably used in future analytical modeling of any resonant extraction, are discussed.

43 PARTICLE ACCELERATORS↗

Accelerated Aging Humidity Chamber for Nuclear Grade HEPA Filter Media

A unique accelerated aging humidity chamber for simulating the natural aging process of nuclear grade high efficiency particulate air (HEPA) filter media is being designed, implemented, and tested. The nuclear industry currently lacks information regarding the shelf life and service life of HEPA filter media. The Department of Energy recommends disposing of HEPA filters 10 years after the manufacture date for filters operated under dry conditions. Studies have shown the tensile strength and water repellency of HEPA filter media diminish with age. The chamber's principal function is to sustain an elevated relative humidity condition for an extended duration specified by the Arrhenius aging model. To accomplish this, various design parameters were established: (1) consistently achieve uniform humidity conditions within the chamber to equally expose media sheets, (2) ensure the chamber is sufficiently sealed to mitigate the uncontrolled infiltration of air, (3) confidently secure the media sheets to avert the introduction of unnecessary stressing/creasing, (4) fabricate the chamber from clear polycarbonate material to allow inspection of the media sheets, (5) provide a safe and ergonomic design for personnel, (6) ensure the chamber design is replicable and relatively easy to fabricate/assemble. A humidity source, fan, PID controller, humidity and temperature transmitters, and data loggers are required for proper operation and control of the chamber. The humidity source must accommodate target values up to 95% relative humidity throughout the chamber. The three humidity and temperature transmitters are strategically spaced throughout the chamber to ensure uniform conditions. The 10 cubic feet per minute (CFM) fan selected intends to accomplish the desired air change rate inside the chamber of approximately 20 air changes per hour (ACH). The 2 plenums on the upper and lower portions of the chamber utilize a perforated design to uniformly distribute air, promote air mixing, and control the air velocity entering the chamber. Industrial blueprint hanging clamps are being retrofitted to be seated within the chamber drawers and firmly secure the nuclear grade filter media. The aged media will be evaluated using autopsy methodology, and the results obtained intend to help clarify the useful life of nuclear grade HEPA filters. Nuclear grade high efficiency particulate air (HEPA) filters are defined as disposable, extended-media, dry-type filters with a rigid casing enclosing the full depth of the pleats and have a minimum particle removal efficiency of 99.97%. [1] - HEPA filters are constantly exposed to stressors and subsequently have been shown to degrade as the filter ages. Properties such as tensile strength and water repellency, along with others, are negatively impacted. [2] - Due to the scarcity of naturally aged HEPA filters available, it is crucial to develop accelerated aging methods which effectively mimic the natural degradation effects. - After accelerated aging, the filters are subject to testing in the Axial Flow Large Scale Test Stand (ALSTS) and other analysis techniques such as the same qualification tests performed directly after being manufactured. - Quality data is necessary for future decisions regarding the lifetime of nuclear grade HEPA filters. - The objective is to develop a prototype accelerated aging humidity chamber for nuclear grade HEPA filter media. The chamber should expose the media to commonly occurring stressors to expedite the degradation process caused by aging. PID yields reliable and reproducible results: - CFD model correlates well with actual performance; - Further analysis is needed to complete characterization; - Future work for the chamber is still to be completed. The data shown in Figure 5 and Table 1 indicate the system is functioning as designed. As can be determined from the system response evaluation in Table 1, the PID controller is yielding reliable and reproducible results at each set point. As the set point increases, the time constant and 20% settling time increase accordingly. From the results in Table 1, it stands to reason that the CFD model correlates well with the actual performance of the system. Each system response parameter for the 90%RH* CFD model is slightly lower than the parameters collected from the physical model tested at an equivalent set point. This is to be expected considering the inlet in the CFD model is a constant supply of water vapor, whereas the physical model uses a PID controller to monitor the amount of water vapor needed and reacts accordingly. More analysis is still to be done on the prototype chamber. A pressure test will be conducted in order to determine the hourly leak rate of the chamber. This test will involve constant air flow into the chamber until the desired pressure is reached. A uniformity test will also be conducted in order to ensure each filter medium is being exposed to equivalent relative humidity levels. This test will involve an array of humidity meters strategically oriented inside the chamber to record any possible gradients. Future work for the chamber includes dehumidification abilities, tests while fully loaded with filters, and temperature control. Once completed, the autopsy team at Institute of Clean Energy Technology will conduct necessary accelerated aging studies as needed.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Adaptive PID Gain Scheduling Control for Hydropower Turbine Using Neural CDE and Stochastic Distribution Shaping

This paper introduces a gain-scheduling PID controller design strategy for hydroturbine frequency control mode. This scheme first uses real data to learn the nonlinear dynamics of the hydroturbine using neural controlled differential equations and then perturbs the obtained nonlinear system at different equilibrium points, based on which a static output feedback adaptive dynamic programming algorithm is then used to optimize the PID gains for each equilibrium point. Moreover, a continuous-time version of stochastic distribution control is proposed to further fine-tune the optimized PID gains. Finally, the controller is obtained by implementing linear interpolation between the optimized PID control gains. The simulation results show that the proposed gain-scheduling PID controller can control a larger range of operation points compared with the given fixed PID controller and the baseline method. Compared with the given fixed PID controller, the proposed gain-scheduling PID controller can regulate hydroturbine frequency against disturbances induced by power-load variation with over 50% less overshoot for some operation points.

13 HYDRO ENERGY↗

Machine Learning for Slow Spill Regulation in the Fermilab Delivery Ring for Mu2e

A third-integer resonant slow extraction system is being developed for the Fermilab’s Delivery Ring to deliver protons to the Mu2e experiment. During a slow extraction process, the beam on target is liable to experience small intensity variations due to many factors. Owing to the experiment’s strict requirements in the quality of the spill, a Spill Regulation System (SRS) is currently under design. The SRS primarily consists of three components - slow regulation, fast regulation, and harmonic content tracker. In this presentation, we shall present the investigations of using Machine Learning (ML) in the fast regulation system, including further optimizations of PID controller gains for the fast regulation, prospects of an ML agent completely replacing the PID controller using supervised learning schemes such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) ML models, the simulated impact and limitation of machine response characteristics on the effectiveness of both PID and ML regulation of the spill. We also present here nascent results of Reinforcement Learning efforts, including continuous-action soft actor-critic methods, to regulate the spill rate.

43 PARTICLE ACCELERATORS↗

Nuclear microreactor transient and load-following control with deep reinforcement learning

The economic feasibility of nuclear microreactors will depend on minimizing operating costs through advancements in autonomous control, especially when these microreactors are operating alongside other types of energy systems (e.g., renewable energy). This study explores the application of deep reinforcement learning (RL) for real-time drum control in microreactors, exploring performance in regard to load-following scenarios. By leveraging a point kinetics model with thermal and xenon feedback, we first establish a baseline using a single-output RL agent, then compare it against a traditional proportional–integral–derivative (PID) controller. This study demonstrates that RL controllers, including both single- and multi-agent RL (MARL) frameworks, can achieve similar or even superior load-following performance as traditional PID control across a range of load-following scenarios. In short transients, the RL agent was able to reduce the tracking error rate in comparison to PID by one half to one third. Over extended 300-minute load-following scenarios in which xenon feedback becomes a dominant factor, PID maintained better accuracy, but RL still remained within a 1% error margin despite being trained only on short-duration scenarios. This highlights RL’s strong ability to generalize and extrapolate to longer, more complex transients, affording substantial reductions in training costs and reduced overfitting. Furthermore, when control was extended to multiple drums, MARL enabled independent drum control as well as maintained reactor symmetry constraints without sacrificing performance---an objective that standard single-agent RL could not learn. We also found that, as increasing levels of Gaussian noise were added to the power measurements, the RL controllers were able to maintain lower error rates than PID, and to do so with at least 10% and upwards of 150% less control effort. These findings illustrate RL's potential for autonomous nuclear reactor control, laying the groundwork for future integration into high-fidelity simulations and experimental validation efforts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗