Search NASA⌕ Search

SEARCH · Search NASA

Results for “approximate dynamic programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

NO-LESS: Near optimal curtailment strategy selection for net load balancing in micro grids

The integration of renewable energy sources such as solar PVs into micro grids has increased in recent years. However, the intermittent and unpredictable nature of renewable energy generation makes supply uncertain, in turn requiring continuous net load balancing to ensure grid stability. One technique to achieve net load balancing is curtailment, however, selection of the optimal set of curtailment strategies to achieve a targeted curtailment is NP-hard. State-of-the-art curtailment selection techniques approach this problem by either developing computationally expensive optimal curtailment strategies or reduce accuracy to generate sub-optimal solutions quickly. In this work, we develop NO-LESS: a Near OptimaL CurtailmEnt Strategy Selection algorithm. NO-LESS is a novel Fully Polynomial Time Approximation Scheme (FPTAS) which determines bounded (near optimal) curtailment strategies in a small amount of time while simultaneously satisfying practical constraints on strategy switching overhead and curtailment fairness. We perform both theoretical analysis and practical evaluation to show that NO-LESS is a scalable solution for performing net load balancing through curtailment strategy selection in Micro Grids.

14 SOLAR ENERGY↗
DOE OSTI2023

A Fast Temporal Decomposition Procedure for Long-Horizon Nonlinear Dynamic Programming

We propose a fast temporal decomposition procedure for solving long-horizon nonlinear dynamic programs. The core of the procedure is sequential quadratic programming (SQP) that utilizes a differentiable exact augmented Lagrangian as the merit function. Within each SQP iteration, we approximately solve the Newton system using an overlapping temporal decomposition strategy. We show that the approximate search direction is still a descent direction of the augmented Lagrangian provided the overlap size and penalty parameters are suitably chosen, which allows us to establish the global convergence. Moreover, we show that a unit step size is accepted locally for the approximate search direction and further establish a uniform, local linear convergence over stages. This local convergence rate matches the rate of the recent Schwarz scheme (Na et al. 2022). However, the Schwarz scheme has to solve nonlinear subproblems to optimality in each iteration, whereas we only perform a single Newton step instead. Numerical experiments validate our theories and demonstrate the superiority of our method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

TU-CD-304-01: FEATURED PRESENTATION and BEST IN PHYSICS (THERAPY): Trajectory Modulated Arc Therapy: Development of Novel Arc Delivery Techniques Integrating Dynamic Table Motion for Extended Volume Treatments

Purpose: Integration of coordinated robotic table motion with inversely-planned arc delivery has the potential to resolve table-top delivery limitations of large-field treatments such as Total Body Irradiation (TBI), Total Lymphoid Irradiation (TLI), and Cranial-Spinal Irradiation (CSI). We formulate the foundation for Trajectory Modulated Arc Therapy (TMAT), and using Varian Developer Mode capabilities, experimentally investigate its practical implementation for such techniques. Methods: A MATLAB algorithm was developed for inverse planning optimization of the table motion, MLC positions, and gantry motion under extended-SSD geometry. To maximize the effective field size, delivery trajectories for TMAT TBI were formed with the table rotated at 270° IEC and dropped vertically to 152.5cm SSD. Preliminary testing of algorithm parameters was done through retrospective planning analysis. Robotic delivery was programmed using custom XML scripting on the TrueBeam Developer Mode platform. Final dose was calculated using the Eclipse AAA algorithm. Initial verification of delivery accuracy was measured using OSLDs on a solid water phantom of varying thickness. Results: A comparison of DVH curves demonstrated that dynamic couch motion irradiation was sufficiently approximated by static control points spaced in intervals of less than 2cm. Optimized MLC motion decreased the average lung dose to 68.5% of the prescription dose. The programmed irradiation integrating coordinated table motion was deliverable on a TrueBeam STx linac in 6.7 min. With the couch translating under an open 10cmx20cm field angled at 10°, OSLD measurements along the midline of a solid water phantom at depths of 3, 5, and 9cm were within 3% of the TPS AAA algorithm with an average deviation of 1.2%. Conclusion: A treatment planning and delivery system for Trajectory Modulated Arc Therapy of extended volumes has been established and experimentally demonstrated for TBI. Extension to other treatment techniques such as TLI and CSI is readily achievable through the developed platform. Grant Funding by Varian Medical Systems.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

Solving Stochastic Dynamic Programming Problems: A Mixed Complementarity Approach

We detail a mixed complementarity problem (MCP) formulation of continuous state dynamic programming problems (DP-MCP). We write the solution to projection methods in value function iteration (VFI) as a joint set of optimality conditions that characterize maximization of the Bellman equation; and approximation of the value function. The MCP approach replaces the iterative component of projection based VFI with a one-shot solution to a square system of complementary conditions. We provide three numerical examples to illustrate our approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-driven modeling and control of dynamical systems using Koopman and Perron-Frobenius operators

This dissertation studies the data-driven modeling and control problem of nonlinear systems by exploiting the linear operator theoretic framework involving Koopman and Perro-Frobenius operator. A systematic linear-operator based controller design procedure has been established, which can be used to solve a variety of nonlinear control problems, including feedback stabilization using control Lyapunov functions, optimal quadratic regulation using Koopman eigenfunctions and convex optimization formulation of optimal control problem using P-F and Koopman operator approximation. As the core of data-driven modeling, we first propose a new algorithm for the finite-dimensional approximation of the linear transfer Koopman and Perron-Frobenius operator from time-series data. We argue that the existing approach for the finite-dimensional approximation of these transfer operators such as Dynamic Mode Decomposition (DMD) and Extended Dynamic Mode Decomposition (EDMD) do not capture two important properties of these operators, namely positivity and Markov property. The algorithm we propose preserves these two properties. We call the proposed algorithm as naturally structured DMD (NSDMD) since it retains the inherent properties of these operators. Naturally structured DMD algorithm leads to a better approximation of the steady-state dynamics of the system regarding computing Koopman and Perron- Frobenius operator eigenfunctions and eigenvalues. However, preserving positivity property is critical for capturing the real transient dynamics of the system. This positivity property of the transfer operators and it's finite-dimensional approximation play an important role for controller and estimator design of nonlinear systems. To solve the feedback stabilization problem for nonlinear control systems, we tried to take advantage of the Koopman operator framework. The Koopman operator approach provides a linear representation for a nonlinear dynamical system and a bilinear representation for a nonlinear control system. The problem of feedback stabilization of a nonlinear control system is then transformed to the stabilization of a bilinear control system. We propose a control Lyapunov function (CLF)-based approach for the design of stabilizing feedback controllers for the bilinear system. The search for finding a CLF for the bilinear control system is formulated as a convex optimization problem. This leads to a schematic procedure for designing CLF-based stabilizing feedback controllers for the bilinear system and hence the original nonlinear system. Another advantage of the proposed controller design approach outlined in this dissertation is that it does not require explicit knowledge of system dynamics. In particular, the bilinear representation of a nonlinear control system in the Koopman eigenfunction space can be obtained from time-series data. Next, we study the optimal quadratic regulation problem for nonlinear systems. The linear operator theoretic framework involving the Koopman operator is used to lift the dynamics of nonlinear control system to an infinite-dimensional bilinear system. The optimal quadratic regulation problem for nonlinear system is formulated in terms of the finite-dimensional approximation of the bilinear system. A convex optimization-based approach is proposed for solving the quadratic regulator problem for bilinear system. We applied a variety of examples and compared the simulation results between our framework and conventional LQR control using linearized model. For more general optimal control problems, we provide a density-function based convex formulation for the optimal control problem of the nonlinear system. The convex formulation relies on the duality result in the stability theory of a dynamical system involving density function and Perron-Frobenius operator. The optimal control problem is formulated as an infinite-dimensional convex optimization program. The finite-dimensional approximation of the optimization problem relies on the recent advances made in the data-driven computation of the Koopman operator, which is dual to the Perron-Frobenius operator. Simulation results are presented to demonstrate the application of the developed framework.

Huang, Bowen↗
DOE OSTI2022

Multi-Fidelity Stochastic Economic Dispatch for Operating Low Carbon Power Grids

Grid operators can address the inherently stochastic nature of renewables by solving a two-stage stochastic programming model that minimizes the cost of dispatch decisions while accounting for the complex grid dynamics. It is common to use a sample average approximation to estimate the expectation of the second stage costs in this model. However, the large sample count needed for numerical accuracy makes effective modeling large-scale electric grids computationally intractable. We introduce a control variate multi-fidelity estimator for the second-stage recourse that enables high quality dispatch decisions in real-time with a reduced computational burden. We obtain a hierarchy of model fidelities by linearizing the AC power flow system representation to DC power flow, and by relaxing transmission and voltage network constraints. We evaluate the performance of our proposed method on a synthetic grid with 73 buses against a deterministic baseline with persistence forecast and a high-fidelity reference. Our analysis shows a computational speed-up of 7.62x with a minimal loss in accuracy. The multi-fidelity method is well suited to fidelity combinations that use a simplified network topology in their lower fidelity model and is an attractive option for applications where accurate grid modeling needed on a limited computational budget.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Deep Learning Explicit Differentiable Predictive Control Laws for Buildings

We present a differentiable predictive control (DPC) methodology for learning constrained control laws for unknown nonlinear systems. DPC poses an approximate solution to multiparametric programming problems emerging from explicit nonlinear model predictive control (MPC). Contrary to approximate MPC, DPC does not require supervision by an expert controller. Instead, a system dynamics model is learned from a small dataset of recorded observations of the perturbed system's dynamics and the control law is optimized offline by interaction with the learned system model. The DPC method is based on two sequential steps, i) system identification using a constrained neural state-space model, and ii) optimization of an explicit control law parametrized by another neural network in closed-loop simulation with the identified neural state-space model. The combination of a differentiable closed-loop system and penalty methods for constraint handling of system outputs and inputs allows us to optimize the control law's parameters directly by backpropagating economic MPC loss through the learned system model. By incorporating domain knowledge and leveraging established techniques from optimal control, our method leverages deep neural networks as nonlinear function approximators for system identification and control while avoiding concomitant costs of intractably large datasets, and computationally expensive over-parametrized models. The scalability, data efficiency, and constrained optimal control capability of the proposed DPC method are demonstrated in simulation using a multi-zone building emulator.

Drgona, Jan↗
DOE OSTI2019

Multi-Stage Stochastic Programming to Joint Economic Dispatch for Energy and Reserve With Uncertain Renewable Energy

To address the uncertain renewable energy in the day-ahead optimal dispatch of energy and reserve, a multi-stage stochastic programming model is established in this paper to minimize the expected total costs. The uncertainties over the multiple stages are characterized by a scenario tree and the optimal dispatch scheme is cast as a decision tree which guarantees the flexibility to decide the reasonable outputs of generation and the adequate reserves accounting for different realizations of renewable energy. Most importantly, to deal with the “Curse of Dimensionality” of stochastic programming, stochastic dual dynamic programming (SDDP) is employed, which decomposes the original problem into several sub-problems according to the stages. Specifically, the SDDP algorithm performs forward pass and backward pass repeatedly until the convergence criterion is satisfied. At each iteration, the original problem is approximated by creating a linear piecewise function. Besides, an improved convergence criterion is adopted to narrow the optimization gaps. The results on the IEEE 118-bus system and real-life provincial power grid show the effectiveness of the proposed model and method.

economic dispatch↗
DOE OSTI2025

GPR_calculator: An on-the-fly surrogate model to accelerate massive nudged elastic band calculations

We present GPR_calculator, a package based on Python and C++ programming languages to build an on-the-fly surrogate model using Gaussian Process Regression (GPR) to approximate computationally expensive electronic structure calculations. The key idea is to dynamically train a GPR model during the simulation that can accurately predict energies and forces with uncertainty quantification. When the uncertainty is high, the costly electronic structure calculation is performed to obtain the ground truth data, which is then used to update the GPR model. To illustrate the effectiveness of GPR_calculator, we demonstrate its application in Nudged Elastic Band (NEB) simulations of surface diffusion and reactions, achieving 3-10 times acceleration compared to pure ab initio calculations. The source code is available at https://github.com/MaterSim/GPR_calculator.

Gaussian process regression↗
DOE OSTI2020

A Deep Generative Model for Non-Intrusive Identification of EV Charging Profiles

The proliferation of electric vehicles (EVs) brings environmental benefits and technical challenges to power grids. An identification algorithm which can accurately extract individual EV charging profiles out of widely available smart meter measurements has attracted great interests. This paper proposes a non-intrusive identification framework for EV charging profile extraction, which is driven by deep generative models (DGM). First, the proposed DGM is designed as a representation layer embedded into the Markov process and used to model the joint probability distribution of available time-series data. A novel contribution is to approximate posterior distributions by neural networks whose parameters are obtained by variational inference and supervised learning. Second, the EV charging status is inferred from the DGM via dynamic programming. Lastly, the desired EV charging profile can be reconstructed by the rated power of EV models and inferred status. Compared with the benchmark Hidden Markov Models, the proposed framework can better handle noise in data with less computational complexity and better overall accuracy performances with smaller recall. The proposed framework is validated by numerical experiments on the Pecan Street dataset.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2016

Graphical models for optimal power flow

Optimal power flow (OPF) is the central optimization problem in electric power grids. Although solved routinely in the course of power grid operations, it is known to be strongly NP-hard in general, and weakly NP-hard over tree networks. In this paper, we formulate the optimal power flow problem over tree networks as an inference problem over a tree-structured graphical model where the nodal variables are low-dimensional vectors. We adapt the standard dynamic programming algorithm for inference over a tree-structured graphical model to the OPF problem. Combining this with an interval discretization of the nodal variables, we develop an approximation algorithm for the OPF problem. Further, we use techniques from constraint programming (CP) to perform interval computations and adaptive bound propagation to obtain practically efficient algorithms. Compared to previous algorithms that solve OPF with optimality guarantees using convex relaxations, our approach is able to work for arbitrary tree-structured distribution networks and handle mixed-integer optimization problems. Further, it can be implemented in a distributed message-passing fashion that is scalable and is suitable for “smart grid” applications like control of distributed energy resources. In conclusion, numerical evaluations on several benchmark networks show that practical OPF problems can be solved effectively using this approach.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2011

A design approach for improving the performance of single-grid planar retarding potential analyzers

Planar retarding potential analyzers (RPAs) have a long flight history and have been included on numerous spaceflight missions including Dynamics Explorer, the Defense Meteorological Satellite Program, and the Communications/Navigation Outage Forecast System. RPAs allow for simultaneous measurement of plasma composition, density, temperature, and the component of the velocity vector normal to the aperture plane. Internal conductive grids are used to approximate ideal potential planes within the instrument, but these grids introduce perturbations to the potential map inside the RPA and cause errors in the measurement of the parameters listed above. A numerical technique is presented herein for minimizing these grid errors for a specific mission by varying the depth and spacing of the grid wires. The example mission selected concentrates on plasma dynamics near the sunset terminator in the equatorial region. The international reference ionosphere model is used to discern the average conditions expected for this mission, and a numerical model of the grid-particle interaction is used to choose a grid design that will best fulfill the mission goals.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2018

Synchronous acceleration with tapered dielectric-lined waveguides

Here, we present a general concept to accelerate non-relativistic charged particles. Our concept employs an adiabatically-tapered dielectric-lined waveguide which supports accelerating phase velocities for synchronous acceleration. We propose an ansatz for the transient field equations, show it satisfies Maxwell's equations under an adiabatic approximation and find excellent agreement with a finite-difference time-domain computer simulation. The fields were implemented into the particle-tracking program {\sc astra} and we present beam dynamics results for an accelerating field with a 1-mm-wavelength and peak electric field of 100~MV/m. The numerical simulations indicate that a $\sim 200$-keV electron beam can be accelerated to an energy of $\sim10$~MeV over $\sim 10$~cm. The novel scheme is also found to form electron beams with parameters of interest to a wide range of applications including, e.g., future advanced accelerators, and ultra-fast electron diffraction.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Plant-specific Model and Data Analysis using Dynamic Security Modeling and Simulation

The requirements for U.S. nuclear power plants to maintain a large on-site physical security force contribute to their high operational costs. The cost of maintaining the current physical security posture is approximately 10% of the overall operation and maintenance budget for commercial nuclear power plants. The goal of the Light Water Reactor Sustainability (LWRS) program’s physical security pathway is to develop tools, methods, and technologies and provide the technical basis for an optimized physical security posture. The conservatisms built into current security postures may be analyzed and minimized in order to reduce security costs while still ensuring adequate security and operational safety. The research performed at Idaho National Laboratory within LWRS program’s physical security pathway has successfully developed a dynamic force-on-force modeling framework using various computer simulation tools and integrating them with the dynamic assessment Event Modeling Risk Assessment using Linked Diagrams (EMRALD) tool. This document provides an update on the progress in applying a dynamic computational framework that links results from a commercially available force-on-force simulation tool, a commercially available thermal-hydraulic tool, and EMRALD to an operating commercial nuclear power plant. This report is only a summary of the progress and does not contain specific modeling results as those contain sensitive security information. This process of including plant procedures and multiple analysis results is being called Modeling and Analysis for Safety Security using Dynamic EMRALD Framework or MASS-DEF. Previous reports described how a user could integrate their plant-specific force-on-force models with the dynamic simulation tool EMRALD, model operator actions, integrate with probabilistic risk assessment tools, such as CAFTA (Computer Aided Fault Tree Analysis System) or SAPHIRE (Systems Analysis Programs for Hands-on Integrated Reliability Evaluations), and with thermal-hydraulic tools, such as RELAP-5. Previous reports applied various combinations of available simulations codes with EMRALD using generic plant models to demonstrate how to perform the analysis. This report documents the results of applying the dynamic computational framework to an actual nuclear facility using their security scenarios and timelines. This report does not contain any plant's sensitive information and/or Safeguards Information. The purpose of this study was to verify that results achieved using generic models are similar to actual plant results and to refine our guidance on the use of the framework. This assessment enables further analysis, such as what-if scenarios and staff-reduction evaluation, thereby optimizing physical security at plants.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Efficient Region of Attraction Characterization for Control and Stabilization of Load Tap Changer Dynamics

In this article, we study the monitoring and control of long-term voltage stability considering load tap changer (LTC) dynamics. We show that under generic conditions, the LTC dynamics admit a unique stable equilibrium. For the stable equilibrium, we characterize an explicit inner approximation of its largest region of attraction (ROA). Compared to existing results, the computational complexity of the ROA characterization is drastically reduced. We propose a quadratically constrained linear program formulation for the ROA characterization problem. In addition, we formulate a second-order cone program for online voltage stability monitoring and control exploiting the proposed ROA characterization. Finally, we demonstrate the efficacy of the proposed formulations on the ROA characterization and stability monitoring and control using a standard IEEE test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

PredCom: A Predictive Approach to Collecting Approximated Communication Traces

Communication traces collected from MPI applications are an important source of information for performance optimization as they can help analysts determine communication patterns and identify inefficiencies. However, their collection, especially at scale, is time consuming, since it usually requires running the complete target application on a large number of nodes. In this work, we present PredCom, a tool-chain to generate a predictive communication proxy based on information gathered from a few small scale runs, which allows us to extract approximate communication traces with an accuracy high enough for most analysis goals. For this, we combine LLVM passes on the original source code (to capture static program structure) with parameter prediction (to capture dynamic and scaling behavior). This approach drastically reduces the time needed for collecting the communication traces, even for traces on large numbers of MPI processes. Here, we demonstrate that PredCom generates communication traces of various applications up to 1612x faster with an accuracy loss of 0.11 on average compared to the original large-scale traces, and we show that the generated traces can be used to optimize process placement.

97 MATHEMATICS AND COMPUTING↗