Search NASA⌕ Search

SEARCH · Search NASA

Results for “Differentiable predictive control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Learning Constrained Parametric Differentiable Predictive Control Policies With Guarantees

We present differentiable predictive control (DPC), a method for offline learning of constrained neural control policies for nonlinear dynamical systems with performance guarantees. We show that the sensitivities of the parametric optimal control problem can be used to obtain direct policy gradients. Specifically, we employ automatic differentiation (AD) to efficiently compute the sensitivities of the model predictive control (MPC) objective function and constraints penalties. To guarantee safety upon deployment, we derive probabilistic guarantees on closed-loop stability and constraint satisfaction based on indicator functions and Hoeffding’s inequality. We empirically demonstrate that the proposed method can learn neural control policies for various parametric optimal control tasks. In particular, we show that the proposed DPC method can stabilize systems with unstable dynamics, track time-varying references, and satisfy nonlinear state and input constraints. Our DPC method has practical time savings compared to alternative approaches for fast and memory-efficient controller design. Specifically, DPC does not depend on a supervisory controller as opposed to approximate MPC based on imitation learning. We demonstrate that, without losing performance, DPC is scalable with greatly reduced demands on memory and computation compared to implicit and explicit MPC while being more sample efficient than model-free reinforcement learning (RL) algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Neural Lyapunov Differentiable Predictive Control

We present a learning-based predictive control methodology using the differentiable programming framework with sufficient Lyapunov-based guarantees. The neural Lyapunov differentiable predictive control (NLDPC) learns the policy by constructing a computational graph encompassing the system dynamics, state, and input constraints, and the necessary Lyapunov certification constraints, and thereafter using the automatic differentiation to update the neural policy parameters. In conjunction, our approach jointly learns a Lyapunov function that certifies the regions of state-space with stable dynamics. We also provide an initial condition sampling-based statistical guarantee for the training of NLDPC. Our offline training approach provides a computationally efficient and scalable alternative to classical model predictive control solutions, which now directly comes with Lyapunov based guarantees as shown in this paper. We substantiate the advantages of the proposed approach with simulations to stabilize the double integrator model and on an example of controlling an aircraft model.

Mukherjee, Sayak↗
DOE OSTI2022

Differentiable Predictive Control with Safety Guarantees: A Control Barrier Function Approach

In this paper, we develop a novel form of differentiable predictive control (DPC) with safety and robustness guarantees. DPC is a form of approximate model predictive control (MPC), wherein the control policy is a neural network that learns a receding horizon, optimal control law. The proposed approach exploits a new form of sampled-data barrier function to enforce safety, while only interrupting the neural network-based controller near the boundary of the safe set. The effectiveness of the proposed approach is demonstrated in simulation.

Shaw Cortez, Wenceslao E.↗
DOE OSTI2022

Learning Stochastic Parametric Differentiable Predictive Control Policies

We present a scalable unsupervised learning-based method for obtaining explicit control policies for model predictive control problems for stochastic linear systems with additive uncertainties subject to nonlinear chance constraints. We call the proposed method stochastic parametric differentiable predictive control (SP-DPC), which extends the recently proposed deterministic DPC policy optimization algorithm. We formulate the SP-DPC as a deterministic approximation to the stochastic parametric constrained optimal control problem via independent sampling of the problem's parameters and uncertainties. This formulation allows us to directly compute the policy gradients via automatic differentiation of the problem's value function, evaluated over sampled parameters and uncertainties. In particular, the computed expectation of the problem's value function is backpropagated through the finite-time closed-loop system rollouts parametrized by a known nominal system dynamics model and neural control policy. We also provide theoretical probabilistic guarantees on closed-loop stability and chance constraints satisfaction for systems controlled by learned neural policies. We demonstrate the computational efficiency and scalability of the proposed policy optimization algorithm in three numerical examples, including systems with a large number of states or subject to nonlinear constraints.

Drgona, Jan↗
DOE OSTI2022

Koopman-based Differentiable Predictive Control for the Dynamics-Aware Economic Dispatch Problem

The dynamics-aware economic dispatch (DED) problem embeds low-level generator dynamics and operational constraints to enable near real-time scheduling of generation units in a power network. DED produces a more dynamic supervisory control policy than traditional economic dispatch (T-ED) that reduces overall generation costs. However, the incorporation of differential equations that govern the system dynamics makes DED an optimization problem that is computationally prohibitive to solve. In this work, we present a new data-driven approach based on differentiable programming to efficiently obtain offline parametric solutions to the underlying DED problem. In particular, we employ the recently proposed differentiable predictive control (DPC) for offline learning of explicit neural control policies based on identified Koopman operator (KO) model of the system dynamics. We demonstrate the high solution quality and five orders of magnitude computational-time savings of the DPC method over the original optimization-based DED approach on a 9-bus test power grid network.

King, Ethan↗
DOE OSTI2022

LBC (Learning Building Control)

LBC encompasses the source code and data to reproduce and extend results for a manuscript that compares demand responsive control schemes for multi-zone buildings. It includes examples of model predictive control (MPC), value function approximation via CVXPYLAYERS, and differentiable predictive control (DPC). The goal of the study is to evaluate state-of-the-art controllers and establish the efficacy (if any) of learning-based approaches that leverage deep neural networks in one way or another.

Zhang, Xiangyu↗
DOE OSTI2022

Dynamic Decarbonization through Autonomous Physics-Centric Deep Learning and Optimization of Building Operations (Abstract only)

This project directly addresses the primary goal of Area of Interest 2 in the CRADA call: to advance optimization-based integrated energy management systems in commercial and residential buildings. Pacific Northwest National Laboratory (PNNL) and its industry partner PassiveLogic aim to accomplish this by reaching three key objectives. First, to ensure a broad impact in the building controls industry, PNNL will extend its open-source library for predictive control synthesis by augmenting its capabilities with data-driven self-learning of building models and auto-calibration of predictive controllers. The effort will focus on building use cases selected in collaboration with PassiveLogic. The team will specifically address the development of methods for data-driven adaptation of building models, investigation of model architectures that best address specific building types, and automated synthesis of differentiable predictive controllers that optimize diverse objectives. Second, PNNL will collaborate with PassiveLogic to integrate the aforementioned methods with PasiveLogic’s advanced controls platform. The collaborative integration effort will inform the developments under the first objective by providing specific data on the attainable performance of model learning on resource-constrained edge computing platforms. This software integration effort will increase the technical maturity of the developed libraries by exploring the use of software integration tools and methods. Third, PNNL and PassiveLogic will work to improve the technology readiness of the developed predictive controllers by testing their performance in relevant test environments, such as high-fidelity simulation, hardware in the loop, and actual test buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Preliminary Development of Heat Transfer Model-Based Control Algorithms of Liquid Sodium Purification System: Advanced Sensors and Instrumentation Advanced Controls

Monitoring the operation of sodium purification system is essential for efficient operation of sodium fast reactors. In this work, a heat transfer model has been developed for monitoring the plugging meter and cold trap systems at the Mechanisms Engineering Test Loop (METL) liquid sodium facility at Argonne National Laboratory. The model of the purification system was developed by treating the respective aspects of the cold trap purification loop and plugging meter diagnostic loop as two separate control volumes using information from the METL piping and instrumentation diagram (P&ID). A model predictive controller was designed using first order differential equations with the specified boundary conditions. The system behavior was studied with a tuned optimized procedure using the internal cold trap temperature and plugging meter outlet temperature as control variables, and the air blower temperature as an independent variable respectively. Results of computer simulations obtained in this study compared favorably with experimental data showing very good reference tracking response with negligible overshoot as both plugging meter and cold trap physical models approach the setpoint.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Phosphoproteomics Modifications in Women with Rheumatoid Arthritis─Application of Web-Based Software to Enhance Data Visualization

Individuals with rheumatoid arthritis (RA) are at increased risk of functional disability, cardiovascular disease, and obesity, all of which are influenced by dysregulated skeletal muscle. Here, this pilot study aims to identify phosphoproteomics changes in RA skeletal muscle and visualize modifications through development of a web-based app designed to promote user-friendly data interpretation and visualization. NanoLC–MS/MS analysis was performed on vastus lateralis biopsies from three women with RA and matched healthy controls. Differential analysis was performed using the Limma R package. Kinase substrate enrichment analysis (KSEA) predicted changes in kinase activity. RA muscle displayed 35 upregulated and 60 downregulated phosphosites, including the cytoskeletal proteins TTN (Ser33201, Ser33013, Ser20925), NEB (Ser2219, Thr254, Ser33013, Ser20925), FLNA (Ser1459), and LASP1 (Ser146). Compared to healthy controls, KSEA predicted decreased activity of several kinases in RA muscle, including PRKACA and CDKs. All such changes were visualized by use of our web-based app. Overall, phosphoproteome analysis reveals signaling alterations in RA skeletal muscle linked to cytoskeletal proteins, representing candidate disease biomarkers; these modifications can be explored through use of our web-based software.

phosphoproteomics↗
DOE OSTI2026

Differentiable multiphase flow model for physics-informed machine learning in reservoir pressure management

Accurate subsurface reservoir pressure control is extremely challenging due to geological heterogeneity and multiphase fluid-flow dynamics. Predicting behavior in this setting relies on high-fidelity physics-based simulations that are computationally expensive. Yet, the uncertain, heterogeneous properties that control these flows make it necessary to perform many of these expensive simulations, which is often prohibitive. To address these challenges, we introduce a physics-informed machine learning workflow that couples a fully differentiable multiphase flow simulator, which is implemented in the DPFEHM framework with a convolutional neural network (CNN). The CNN learns to predict fluid extraction rates from heterogeneous permeability fields to enforce pressure limits at critical reservoir locations. By incorporating transient multiphase flow physics into the training process, our method enables more practical and accurate predictions for realistic injection-extraction scenarios compared to previous works. To speed up training, we pretrain the model on single-phase, steady-state simulations and then finetune it on full multiphase scenarios, which dramatically reduces the computational cost. We demonstrate that high-accuracy training can be achieved with fewer than three thousand full-physics multiphase flow simulations – compared to previous estimates requiring up to ten million. This drastic reduction in the number of simulations is achieved by leveraging transfer learning from much less expensive single phase simulations.

25 ENERGY STORAGE↗
DOE OSTI2023

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

The ability of recurrent neural networks (RNN) to model nonlinear dynamics of high dimensional process data has enabled data-driven RNN-based fault detection algorithms. Previous studies have focused on detecting faults by identifying the discrepancies in data distribution between the faulty and normal data, as reflected in prediction errors generated by RNN models. However, in industrial processes, variations in data distribution can also result from changes in normal control setpoints and compensatory control adjustments in response to disturbances, making it hard to differentiate between normal and faulty conditions. This paper proposes a fault detection method utilizing a long short-term memory (LSTM) encoder–decoder structure with regularized hidden dynamics and reversible instance normalization (RevIN) to compactly represent high-dimensional measurements for effective monitoring. During training, the hidden states of the model are regularized to form a low-dimensional latent space representation of the original multivariate time series data. As a result, the prediction errors of the latent states can be used to monitor the abnormal dynamic variations, while the reconstruction errors of the measured variables are used to monitor the abnormal static variations. Furthermore, the proposed indices can reflect operating conditions, even when the distribution of test data changes, which helps distinguish faults from normal adjustments and disturbances that controllers can settle. Here, data from numerical simulation and the Tennessee Eastman process are used to illustrate the effectiveness of the proposed fault detection method.

42 ENGINEERING↗
DOE OSTI2024

Power System Frequency Dynamics Modeling, State Estimation, and Control using Neural Ordinary Differential Equations (NODEs) and Soft Actor-Critic (SAC) Machine Learning Approaches

With the global energy transition of the electric power system, grid control, supervision, and protection is becoming more challenging. With the increasing integration of renewable energy sources (RES), the system dynamics are changing, causing traditional power system dynamic modeling with swing equation-based modeling approaches to fail. Additionally, the converter-dominated power grid is decreasing the system inertia, making the power system more fragile to the frequency swings. This paper first investigates and compares the application of a model-based Kalman filter state estimation approach with (i) a model-free machine learning approach --- neural ordinary differential equations (NODEs) --- and (ii) a data-driven system identification (SysId) approach to model and infer critical state values of the power system frequency dynamics. Then a model predictive control (MPC) framework is compared to a model-free Soft Actor-Critic (SAC) reinforcement learning (RL) control algorithm in providing efficient fast frequency response (FFR) to the power system frequency dynamics. The approaches are compared in terms of their performance goals as well as their per-timestep computational efficiency. Furthermore, the comparative study for state estimation shows that for the model-free requirement, both NODEs and SysId can provide accurate state estimates; however, with increasing model complexity, NODEs can be a better choice for model identification. Similarly, the results from the FFR comparative study show that the SAC RL-based FFR, once trained, outperforms MPC with better control signals and faster computation time, making the SAC RL-based FFR better option for providing FFR to the power system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Latent Neural ODE for Integrating Multi-Timescale Measurements in Smart Distribution Grids

Under a smart grid paradigm, there has been an increase in sensor installations to enhance situational awareness. The measurements from these sensors can be leveraged for real-time monitoring, control, and protection. However, these measurements are typically irregularly sampled. These measure-ments may also be intermittent due to communication bandwidth limitations. To tackle this problem, this paper proposes a novel latent neural ordinary differential equations (LODE) approach to aggregate the unevenly sampled multivariate time-series measurements. The proposed approach is flexible in performing both imputations and predictions while being computationally efficient. Simulation results on IEEE 37 bus test systems illustrate the efficiency of the proposed approach.

multi time-scale measurements↗
DOE OSTI2025

Irradiation Vehicles for Evaluating SiC/SiC Cladding Bowing Under Neutron Flux Gradients

Silicon carbide fiber–reinforced silicon carbide matrix (SiC/SiC) composites are among the most promising candidates for long term accident-tolerant nuclear fuel cladding. A key challenge related to their deployment is lateral bowing caused by differential radiation-induced swelling under dose or temperature gradients, which could obstruct coolant flow or interfere with control rod/blade movements. Although previous modeling efforts have predicted bowing behavior in light-water reactor (LWR) environments, experimental validation remains limited, especially at prototypic LWR temperatures. This study addresses that gap by irradiating six reduced-length SiC/SiC cladding tubes (~600 mm) in the High Flux Isotope Reactor (HFIR) at ~300°C, which is representative of LWR conditions. The tubes will be housed in a sealed vessel with an inert gas gap to maintain target temperatures and prevent direct coolant contact. Arranged in three pairs, each set will receive a different radiation dose (low, medium, high), with the central pair receiving ~0.1 displacements per atom (dpa)—the expected dose for peak bowing. The experiment will determine the dose-dependent bowing behavior and validate predictive finite element models. In this work, the tubes are freely suspended from pins to allow for unconstrained bowing; however, we present a concept for introducing localized constraints to represent grid spacer effects. Post-irradiation examination will include swelling measurements and profilometry to assess bowing and compare the results with model predictions. This work aims to confirm the conditions under which maximum bowing occurs so as to improve the reliability of SiC/SiC performance models in nuclear applications.

Russell, Nick [ORNL] (ORCID:0000000296099820)↗