SEARCH · Search NASA

Results for “optimization PID control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

51 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2025

A Reinforcement Learning Approach to Augment Conventional PID Control in Nuclear Power Plant Transient Operation

The ability of nuclear reactors to operate their power conversion cycles more flexibly will enhance their value to energy grids with variable pricing. Current nuclear control systems are typically classical controllers that are often based on proportional-integral-derivative (PID) control. This paper presents a method of augmenting the existing PID control for difficult transient operations in nuclear power plants using a reinforcement learning–derived feedforward signal applied in real time. The agents, which are trained on a test thermal load-following problem, are designed to improve steam generator outlet temperature control for a range of fast load-following scenarios covering ramp rates from 9%/min to 15%/min. Several reinforcement learning algorithms were initially investigated for the training of the feedforward agents with deep Q-learning (DQN) and proximal policy optimization (PPO) networks, which were found to be the most promising. The DQN controllers utilize discrete actions, giving them a better disturbance rejection at steady state but inconsistent response to initial temperature deviations. In contrast, PPO-trained agents, which take continuous actions except for a dead zone around zero, were shown to have the best combination of high disturbance rejection at steady state and good tracking of the desired temperature value. The ability of the PPO agent was also examined, with the average time of decision making found to be on the order of 1 ms. The fault properties of the controller under the loss of the reinforcement learning agent feedforward signal were also examined. The controller showed strong performance in situations of “no-signal” faults. but was less good at handling “stuck-at” faults, where the feedforward signal remains at a set value. In both cases, however, the PID was able to successfully maintain stability, eventually returning the system to a steady state. It is hoped that this work will allow for the proposed control architecture to be examined for more difficult control problems such that it may eventually be used to adapt existing nuclear plants for more aggressive load-following on grids of the future.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2024

Innovating the next generation of commercial smart building software

Nearly 30% of commercial building energy use is wasted due to equipment faults and HVAC controls problems. The result is increased emissions, compromised comfort and productivity, and less reliable coordination of building power needs with a clean grid. The energy impact alone represents $17 billion in potential savings. Today’s smart building software provides a robust solution to address these operational deficiencies. Energy management and information systems (EMIS) are saving up to 9% on average, with two-year paybacks. They are being incorporated into energy management processes, commissioning services, and utility programs. As effective as they are, two barriers prevent even deeper benefits; limited personnel to fix problems once they are identified, and the expense and time to manually implement changes in control systems. In partnership with the research community, the EMIS industry is developing new capabilities to overcome these barriers. Moving beyond siloed products for either fault detection and diagnostics, or optimal control, these new capabilities empower users to not only automatically identify faults, but also to push corrective action, and control improvements to their buildings. In this paper, several areas for enhancements are documented: ‘one-time’ correction of faults such as setpoints, schedules, and economizer lockouts; short-term active testing for automated proportional integral derivative (PID) loop tuning and functional testing; and continuous supervisory control for demand flexibility and year-round efficiency. Results are presented from a pair of partner implementations out of a dozen providers integrating these enhancements into their products, including field tests from across the country, and insights into operator acceptance and integration into operations and maintenance practices.

Casillas, Armando↗

NASA NTRS2001

Additive Runge-Kutta Schemes for Convection-Diffusion-Reaction Equations

Additive Runge-Kutta (ARK) methods are investigated for application to the spatially discretized one-dimensional convection-diffusion-reaction (CDR) equations. First, accuracy, stability, conservation, and dense output are considered for the general case when N different Runge-Kutta methods are grouped into a single composite method. Then, implicit-explicit, N = 2, additive Runge-Kutta ARK2 methods from third- to fifth-order are presented that allow for integration of stiff terms by an L-stable, stiffly-accurate explicit, singly diagonally implicit Runge-Kutta (ESDIRK) method while the nonstiff terms are integrated with a traditional explicit Runge-Kutta method (ERK). Coupling error terms are of equal order to those of the elemental methods. Derived ARK2 methods have vanishing stability functions for very large values of the stiff scaled eigenvalue, z(exp [I]) goes to infinity, and retain high stability efficiency in the absence of stiffness, z(exp [I]) goes to zero. Extrapolation-type stage-value predictors are provided based on dense-output formulae. Optimized methods minimize both leading order ARK2 error terms and Butcher coefficient magnitudes as well as maximize conservation properties. Numerical tests of the new schemes on a CDR problem show negligible stiffness leakage and near classical order convergence rates. However, tests on three simple singular-perturbation problems reveal generally predictable order reduction. Error control is best managed with a PID-controller. While results for the fifth-order method are disappointing, both the new third- and fourth-order methods are at least as efficient as existing ARK2 methods while offering error control and stage-value predictors.

Kennedy, Christopher A.↗

NASA NTRS2002

Additive Runge-Kutta Schemes for Convection-Diffusion-Reaction Equations

Additive Runge-Kutta (ARK) methods are investigated for application to the spatially discretized one- dimensional convection-diffusion-reaction (CDR) equations. Accuracy, stability, conservation, and dense-output are first considered for the general case when N different Runge-Kutta methods are grouped into a single composite method. Then, implicit-explicit, (N = 2), additive Runge-Kutta (ARK(sub 2)) methods from third- to fifth-order are presented that allow for integration of stiff terms by an L-stable, stiffly-accurate explicit, singly diagonally implicit Runge-Kutta (ESDIRK) method while the nonstiff terms are integrated with a traditional explicit Runge-Kutta method (ERK). Coupling error terms of the partitioned method are of equal order to those of the elemental methods. Derived ARK(sub 2) methods have vanishing stability functions for very large values of the stiff scaled eigenvalue, z['] yields -infinity, and retain high stability efficiency in the absence of stiffness, z['] yield 0. Extrapolation-type stage- value predictors are provided based on dense-output formulae. Optimized methods minimize both leading order ARK(sub 2) error terms and Butcher coefficient magnitudes as well as maximize conservation properties. Numerical tests of the new schemes on a CDR problem show negligible stiffness leakage and near classical order convergence rates. However, tests on three simple singular-perturbation problems reveal generally predictable order reduction. Error control is best managed with a PID-controller. While results for the fifth-order method are disappointing, both the new third- and fourth-order methods are at least as efficient as existing ARK(sub 2) methods.

Kennedy, Christopher A.↗

DOE OSTI2023

Model predictive control of mixing controlled compression ignition operation for low reactivity fuels

Using gasoline or other low reactivity fuels with a pilot injection or port fuel injection in a compression ignition engine has shown great potential in reducing NOx emissions while keeping high thermal efficiency compared to diesel. However, excessive combustion noise is caused by a high maximum pressure rise rate in the cylinder due to the higher fractions of premixed charge of the low-reactivity fuel. This noise can result in structural damage to engine components and as such, combustion noise limits the range of the operating parameters and makes the control of such engines challenging. In this study, a simulation environment was built up in MATLAB/Simulink leveraging a physics-based zero-dimension combustion model to capture the in-cylinder pressure time traces as well as metrics relevant to thermal efficiency and combustion noise. Here, in order to also facilitate the control of emissions, machine learning models were investigated to capture NOx emissions. A kernel-based extreme learning machine (K-ELM) performed best and had a coefficient of correlation (R-squared) of 0.998. The combustion and NOx emission models are valid for not only conventional gasoline fuel but also oxygenated alternative fuel blends at three different pilot injection strategies. In order to track key combustion metrics while keeping noise and emissions within constraints, a model predictive control (MPC) was applied for a compression ignition engine operating with a range of potential fuels and fuel injection strategies. The MPC is validated under different scenarios, including a load step change, fuel type change, and injection strategy change, with proportional–integral (PI) control as the baseline. The simulation results show that MPC reduces about 26% of ringing intensity in the transient process and 17% at the steady state for E30. Generally, MPC can optimize the overall performance through modifying the main injection timing, pilot fuel mass, and exhaust gas recirculation (EGR) fraction.

42 ENGINEERING↗

DOE OSTI2025

Real-Time Sea State Estimation for Wave Energy Converter Control via Machine Learning

Wave energy converters (WECs) harness the untapped power of ocean waves to generate renewable energy, offering a promising solution to sustainable energy. An optimal WEC control strategy is essential to maximize power capture that dynamically adjusts system parameters in response to rapidly changing sea states. This study presents a novel control approach that leverages neural networks to estimate sea states from onboard WEC measurements such as position, velocity, and force. Using a point absorber WEC device as a test platform, our proposed approach estimates sea states in real-time and subsequently adjusts PID controller gains to maximize energy extraction. Simulation results across diverse sea conditions demonstrate that our strategy eliminates the need for external wave monitoring equipment while maintaining power capture efficiency. The results show that our neural network-based control technique can improve power capture by 25.6% while significantly reducing system complexity. This approach offers a practical alternative for WEC deployments where direct wave measurements are either infeasible or cost prohibitive.

PIDcontrol↗

NASA NTRS2012

RESOLVE Project

The RESOLVE project is a lunar prospecting mission whose primary goal is to characterize water and other volatiles in lunar regolith. The Lunar Advanced Volatiles Analysis (LAVA) subsystem is comprised of a fluid subsystem that transports flow to the gas chromatograph- mass spectrometer (GC-MS) instruments that characterize volatiles and the Water Droplet Demonstration (WDD) that will capture and display water condensation in the gas stream. The LAVA Engineering Test Unit (ETU) is undergoing risk reduction testing this summer and fall within a vacuum chamber to understand and characterize C!Jmponent and integrated system performance. Ray will be assisting with component testing of line heaters, printed circuit heaters, pressure transducers, temperature sensors, regulators, and valves in atmospheric and vacuum environments. He will be developing procedures to guide these tests and test reports to analyze and draw conclusions from the data. In addition, he will gain experience with preparing a vacuum chamber with fluid and electrical connections. Further testing will include integrated testing of the fluid subsystem with the gas supply system, near-infrared spectrometer, WDD, Sample Delivery System, and GC-MS in the vacuum chamber. This testing will provide hands-on exposure to a flight forward spaceflight subsystem, the processes associated with testing equipment in a vacuum chamber, and experience working in a laboratory setting. Examples of specific analysis Ray will conduct include: pneumatic analysis to calculate the WOO's efficiency at extracting water vapor from the gas stream to form condensation; thermal analysis of the conduction and radiation along a line connecting two thermal masses; and proportional-integral-derivative (PID) heater control analysis. In this Research and Technology environment, Ray will be asked to problem solve real-time as issues arise. Since LAVA is a scientific subsystem, Ray will be utilizing his chemical engineering background to operate the near-infrared spectrometer and GC-MS instruments during ETU testing. Ray will be working with Modified Commercial off the Shelf (MCOTS) instruments and characterizing their analytical behavior for optimization. Ray will be offered the opportunity to suggest testing modifications or configuration changes at any time to improve the experimental effectiveness. He will gain many skills needed for working in a technical team setting requiring flexibility and critical thinking.

Parker, Ray O.↗

NASA NTRS2007

Identification and Control of Aircrafts using Multiple Models and Adaptive Critics

We compared two possible implementations of local linear models for control: one approach is based on a self-organizing map (SOM) to cluster the dynamics followed by a set of linear models operating at each cluster. Therefore the gating function is hard (a single local model will represent the regional dynamics). This simplifies the controller design since there is a one to one mapping between controllers and local models. The second approach uses a soft gate using a probabilistic framework based on a Gaussian Mixture Model (also called a dynamic mixture of experts). In this approach several models may be active at a given time, we can expect a smaller number of models, but the controller design is more involved, with potentially better noise rejection characteristics. Our experiments showed that the SOM provides overall best performance in high SNRs, but the performance degrades faster than with the GMM for the same noise conditions. The SOM approach required about an order of magnitude more models than the GMM, so in terms of implementation cost, the GMM is preferable. The design of the SOM is straight forward, while the design of the GMM controllers, although still reasonable, is more involved and needs more care in the selection of the parameters. Either one of these locally linear approaches outperform global nonlinear controllers based on neural networks, such as the time delay neural network (TDNN). Therefore, in essence the local model approach warrants practical implementations. In order to call the attention of the control community for this design methodology we extended successfully the multiple model approach to PID controllers (still today the most widely used control scheme in the industry), and wrote a paper on this subject. The echo state network (ESN) is a recurrent neural network with the special characteristics that only the output parameters are trained. The recurrent connections are preset according to the problem domain and are fixed. In a nutshell, the states of the reservoir of recurrent processing elements implement a projection space, where the desired response is optimally projected. This architecture trades training efficiency by a large increase in the dimension of the recurrent layer. However, the power of the recurrent neural networks can be brought to bear on practical difficult problems. Our goal was to implement an adaptive critic architecture implementing Bellman s approach to optimal control. However, we could only characterize the ESN performance as a critic in value function evaluation, which is just one of the pieces of the overall adaptive critic controller. The results were very convincing, and the simplicity of the implementation was unparalleled.

Principe, Jose C.↗

DOE OSTI2023

Data-Driven Modeling of a High Capacity Cryogenic System for Control Optimization

The Cryogenic Moderator System (CMS) is responsible for maintaining a steady flow of cold neutrons for numerous physics experiments at the Spallation Neutron Source (SNS) in Oak Ridge National Laboratory (ORNL). Sudden losses in beam power, known as beam trips, cause a major disturbance to the CMS due to large step changes in cooling demands. Ongoing efforts on upgrading the neutron beam power from 1.4 to 2.0MW are expected to generate larger transients that can further strain the CMS subsystems if they are not properly controlled. To manage such disturbances, four flow valves and one electric heater are adjusted by five decentralized proportional-integral-derivative (PID) controllers. However, the original PID gains were calibrated empirically based only on tracking performance and not based on disturbance rejection. To address this issue without compromising current CMS operations, a control-oriented model was developed to recalibrate the PID controllers offline. The zero-dimensional (0-D) model was based on simple physics-based principles and data-driven system identification techniques. The CMS was broken into several subsystems for analysis, each of which corresponds to a parametric model tied to the thermodynamic states of the working fluid. The model parameters were identified using the nonlinear least squares method where the residuals were calculated from available sensor data. Simulation results show that the proposed model can capture the dynamics of the CMS at steady state and during beam trips.

Maldonado Puente, Bryan↗

NASA NTRS1993

On identified predictive control

Self-tuning control algorithms are potential successors to manually tuned PID controllers traditionally used in process control applications. A very attractive design method for self-tuning controllers, which has been developed over recent years, is the long-range predictive control (LRPC). The success of LRPC is due to its effectiveness with plants of unknown order and dead-time which may be simultaneously nonminimum phase and unstable or have multiple lightly damped poles (as in the case of flexible structures or flexible robot arms). LRPC is a receding horizon strategy and can be, in general terms, summarized as follows. Using assumed long-range (or multi-step) cost function the optimal control law is found in terms of unknown parameters of the predictor model of the process, current input-output sequence, and future reference signal sequence. The common approach is to assume that the input-output process model is known or separately identified and then to find the parameters of the predictor model. Once these are known, the optimal control law determines control signal at the current time t which is applied at the process input and the whole procedure is repeated at the next time instant. Most of the recent research in this field is apparently centered around the LRPC formulation developed by Clarke et al., known as generalized predictive control (GPC). GPC uses ARIMAX/CARIMA model of the process in its input-output formulation. In this paper, the GPC formulation is used but the process predictor model is derived from the state space formulation of the ARIMAX model and is directly identified over the receding horizon, i.e., using current input-output sequence. The underlying technique in the design of identified predictive control (IPC) algorithm is the identification algorithm of observer/Kalman filter Markov parameters developed by Juang et al. at NASA Langley Research Center and successfully applied to identification of flexible structures.

Bialasiewicz, Jan T.↗

DOE OSTI2025

Reinforcement Learning for In-Spill Optimization of the Mu2e Resonant Extraction: Compensating Non-Stationarity

We present design considerations and challenges for the fast machine learning component of a third-order resonant beam extraction regulation system being commissioned to deliver steady beam rates to the mu2e experiment at Fermilab. Dedicated quadrupoles drive the tune toward the 29/3 resonance each spill, extracting beam at kV multiwire septa. The overall Spill Regulation System consists of (1) a “slow” process using ~100-spill averages to adjust the base quad ramp infrequently, (2) a feedforward harmonic content compensator, and (3) the “fast” ML agent reacting during each ongoing spill with on-the-fly additive corrections to the sum of (1) and (2). We have demonstrated improved beam-rate steadying for a fast ML agent compared to a PID controller using a quasi-physical spill simulation, and demonstrated distillation of that simulation into a predictive surrogate model. Current work includes a data-and-training pipeline to generate data-aware surrogates with real-world dynamics, even as the dynamics shift unpredictably. The surrogates are to act as RL environments against which to train our fast ML control agents before deploying them on FPGA in the live system. Further current efforts focus on modeling and controlling beam loss around the storage ring, understanding additional available hardware inputs to the model, and the interplay of these with beam-steadying performance.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

Comparing Outdoor to Indoor Performance for Bifacial Modules Affected by Polarization-Type Potential-Induced Degradation

Bifacial photovoltaic (PV) modules have the advantage of using light reflected off of the ground to contribute to power production. Predicting the energy gain is challenging and requires complex models to do so accurately. Often, module degradation over time is neglected in models for the sake of simplicity or is underestimated. Comparing outdoor and indoor current–voltage (I–V) performance for bifacial modules is more challenging than for monofacial modules, as there are additional variables to consider such as rear albedo non-uniformity, cell mismatch, and their effects on temperature. This challenge is compounded when heterogeneous degradation modes occur, such as polarization-type potential-induced degradation (PID-p). To examine the effects of PID-p on I–V predictions using an empirical data-driven approach, 16 bifacial PERC modules are installed outdoors on racks with different albedo conditions. A subset is exposed to high-voltage biases of −1500 V or +1500 V. Outdoor data are traced at irradiance ranges of 150–250 W/m 2 , 500–600 W/m 2 , and 900–1000 W/m 2 . These curves are corrected using control module temperature, wire resistivity, and module resistance measured indoors. We examine several methods to transform indoor I–V curves to accurately, and more simply than existing methods, approximate outdoor performance for bifacial modules without and with varying levels of PID-p degradation. This way, bifacial performance modeling can be more accessible and informed by fielded, degraded modules. Distributions of percent errors between indoor and outdoor performance parameters and Mean Absolute Percent Errors (MAPEs) are used to assess method quality. Results including low-irradiance data (150–250 W/m 2 ) are discussed but are filtered for quantifying method quality as these data introduce substantial errors. The method with the most optimal tradeoff between low MAPE and analysis simplicity involves measuring the front side of a module indoors at an irradiance equal to plane-of-array irradiance plus the product of module bifaciality and albedo irradiance. This method gives MAPE values of 1–6.5% for non-degraded and 1.6–5.9% for PID-p degraded module performance.

14 SOLAR ENERGY↗

DOE OSTI2025

Nuclear microreactor transient and load-following control with deep reinforcement learning

The economic feasibility of nuclear microreactors will depend on minimizing operating costs through advancements in autonomous control, especially when these microreactors are operating alongside other types of energy systems (e.g., renewable energy). This study explores the application of deep reinforcement learning (RL) for real-time drum control in microreactors, exploring performance in regard to load-following scenarios. By leveraging a point kinetics model with thermal and xenon feedback, we first establish a baseline using a single-output RL agent, then compare it against a traditional proportional–integral–derivative (PID) controller. This study demonstrates that RL controllers, including both single- and multi-agent RL (MARL) frameworks, can achieve similar or even superior load-following performance as traditional PID control across a range of load-following scenarios. In short transients, the RL agent was able to reduce the tracking error rate in comparison to PID by one half to one third. Over extended 300-minute load-following scenarios in which xenon feedback becomes a dominant factor, PID maintained better accuracy, but RL still remained within a 1% error margin despite being trained only on short-duration scenarios. This highlights RL’s strong ability to generalize and extrapolate to longer, more complex transients, affording substantial reductions in training costs and reduced overfitting. Furthermore, when control was extended to multiple drums, MARL enabled independent drum control as well as maintained reactor symmetry constraints without sacrificing performance---an objective that standard single-agent RL could not learn. We also found that, as increasing levels of Gaussian noise were added to the power measurements, the RL controllers were able to maintain lower error rates than PID, and to do so with at least 10% and upwards of 150% less control effort. These findings illustrate RL's potential for autonomous nuclear reactor control, laying the groundwork for future integration into high-fidelity simulations and experimental validation efforts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

Creating a Simulation Platform for Research and Development of Advanced Control Methods

Advanced nuclear reactors are essential to meet the changing energy requirements throughout both the United States and the rest of world. In addition to other features, they are designed to enable deployment in remote locations and operate in a fully (or near-fully) autonomous manner, which will require a new control paradigm. To realize autonomously operating reactors, the U.S. Department of Energy’s Nuclear Energy Enabling Technologies Advanced Sensors and Instrumentation (NEET ASI) program conducts research and development into the enabling technologies and methods needed, including digital twins, machine learning, and risk modeling, in addition to various types of control methods. These technologies and methods are the key foundations needed to achieve fully autonomous systems. To develop and evaluate the technologies and methods necessary for achieving autonomous operations, it is critical to identify a software tool capable of integrating all the required elements. In surveying the available solutions, no software platforms were identified that could accomplish what was needed without introducing drawbacks. This challenge was the motivation for the current effort: to develop a software platform that can seamlessly integrate autonomouscontrol-enabling technologies and methods, allowing for accelerated research and development and transfer of ideas. The resulting platform, known as the Control and Optimization Modular Modeling Application for Nuclear Deployment (COMMAND), is Python-based, and leverages open-source tools to provide flexibility and facilitate building upon prior research. It is designed to enable advanced reactor developers to deploy and test advanced control technologies and methods coupled with their own models, solutions, and hardware. Given the substantial undertaking of developing such a platform, the current effort focused on laying down scalable, flexible software foundations and infrastructure, then demonstrating the platform via a use case. These foundations included developing generic modules, which contain the base variable and system blocks (the information and functional building blocks, respectively, that can be used to design a simulation) and the data handling and storage blocks needed to exchange information between the various blocks; as well as enablingtechnology-specific modules. This platform was evaluated via a use case, which was to simulate and control a process for the Microreactor Automated Control System (MACS) test bed. While MACS is not currently directly coupled to any specific microreactor physics, it was initially developed in concert with the Microreactor Applications Research Validation and Evaluation (MARVEL) microreactor, and so the MARVEL physics are used here. As part of this use case, several enabling-technology-specific blocks within COMMAND were integrated, including a proportional integral derivative (PID) control block, a Reactor Excursion and Leak Analysis Program (RELAP5-3D) block, and an anomaly detection block. The COMMAND software platform was successfully demonstrated to achieve the scalability and flexibility objectives of this effort and will be leveraged by the program’s research efforts to advance state of the art control methodologies towards autonomous operations of advanced reactors. As new use cases are created and implemented, it is anticipated that COMMAND will continue to grow and evolve to meet new requirements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2020

Defect Kinetics and Control for Module Reliability

Potential induced degradation is currently one of the most important module degradation mechanisms. It has been suggested that stacking faults decorated with sodium from the module glass are responsible for this effect and authors have also shown the reversibility of this effect upon reverse biasing of the module. The importance of sodium in the failure mechanism is clear, however, little is known regarding the factors that control its diffusion into the wafer, making it nearly impossible to predict the performance of a given module and engineer it to be better. Sodium migration from module glass into silicon cells and the resulting module degradation is a clear example of how defect kinetics can determine overall module performance and long-term reliability. To the detriment of the industry and its bankability, no quantitative models yet exist to predict defect-assisted module degradation, limiting the progress in improving reliability. In particular, the understanding of defect behavior under high electric fields, under stresses imparted by encapsulation or temperature, and under real operating conditions over long periods of time is a crucial gap in the current state-of-the-art. In this work we developed a Defect-Device-Degradation model to predict defect behavior and its impact on device performance over the module operational lifetime using experimentally-determined defect parameterizations. The validated model will provide a platform for manufacturing process optimization across input materials and architectures to avoid deleterious defects upstream and enable enhanced module robustness.

36 MATERIALS SCIENCE↗