Search NASA⌕ Search

SEARCH · Search NASA

Results for “dynamic programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Comparison of Real-Time Pressure Rail Selection Algorithms for the Hybrid Hydraulic Electric Architecture: Case Study on a Track Loader

Abstract The hybrid hydraulic electric architecture (HHEA) seeks to combine the high power/torque/force density of hydraulics with the efficiency of electric machines. A set of common pressure rails is used to provide a majority of the power and this power is modulated by small electric machines to provide precise control for the operator. The HHEA has been studied in previous work using off-line dynamic programming optimization to determine energy efficient pressure rail selections, but this approach requires drive cycle information apriori. A Lagrange multiplier method has also been investigated where a set of gains (Lagrange multipliers) are optimized off-line with the idea the these gains, once determined, could be used for real-time operation. In this work, three new real-time pressure rail selection algorithms that do not require future drive cycle information are investigated; greedy, torque minimizing, and thresholding. The greedy control is found to only use 1% more energy than the globally optimal dynamic programming solution; but a model of energy loss is required.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

How good are learning-based control v.s. model-based control for load shifting? Investigations on a single zone building energy system

Both model predictive control (MPC) and deep reinforcement learning control (DRL) have been presented as a way to approximate the true optimality of a dynamic programming problem, and these two have shown significant operational cost saving potentials for building energy systems. Furthermore, there is still a lack of in-depth quantitative studies on their approximation levels to the true optimality, especially in the building energy domain. To fill in the gap, this paper provides a numerical framework that enables the evaluation of the optimality levels of different controllers for building energy systems. This framework is then used to comprehensively compare the optimal control performance of both MPC and DRL controllers with given computation budgets for a single zone fan coil unit system. Note the optimality is estimated based on a user-specific selection of trade-off weights among energy costs, thermal comfort and control slew rates. Compared with the best optimality we can find through expensive optimization simulations, the best DRL agent can maximally approximate the optimality by 96.54%, which outperforms the best MPC whose optimality level is 90.11%. However, due to the stochasticity, the DRL agent is only expected to approximate the optimality by 90.42%, which is almost equivalent to the best MPC. Except for Proximal Policy Optimization (PPO), all DRL agents can have a better approximation to the optimality than the best MPC, and are expected to have better approximation than the MPC with a prediction horizon of 32 steps (15 min per step). In terms of reducing energy cost and thermal discomfort, MPC can outperform the rule-based control (RBC) by 18.47%–25.44%. DRL can be expected to outperform RBC by 18.95%–25.65% ,and the best DRL control policy can outperform RBC by 20.29%–29.72%. Although the comparison of the optimality level is performed in a perfect setting, e.g., MPC assumes perfect models, and DRL assumes a perfect offline training process and online deployment process, this can shed insight on their capabilities of approximating to the original dynamic programming problem.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

Utilising demand response for distribution service restoration to achieve grid resiliency against natural disasters

The increased frequency of power outages due to natural disasters in recent years has highlighted the urgency of enhancing distribution grid resilience. The effective distribution service restoration (DSR) is an important measure for a resilient distribution grid. In this work, the authors demonstrate that DSR can be significantly improved by leveraging the flexibility provided by the inclusion of demand response (DR). The authors propose a framework for this by considering integrated control of household-level flexible appliances to vary the load demand at the distribution-grid level to improve DSR. The overall framework of the proposed system is modelled as a three-step method considering three optimization problems to (i) calculate feasible controllable aggregated load range for each bus, (ii) determine candidate buses to perform DR and their target load demand, and (iii) maintain the load level in each house through home energy management during the DSR, considering uncertainties in load and solar generation sequentially. The optimization problems are formulated as linear programming, mixed-integer linear programming, and multistage stochastic programming (solved using the stochastic dual dynamic programming) models. Finally, case studies performed in the IEEE 123-node test feeder show improvements in resilience in terms of energy restored compared to the restoration process without DR.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

MultiSector Dynamics: 2023 Inaugural Workshop Report

Preface The MultiSector Dynamics (MSD) Community of Practice (CoP) hosted an inaugural workshop on October 3-5, 2023 at the University of California, Davis, to bring together members of the MSD community of practice to advance understanding of the co-evolution of human and natural systems, and to build the next generation of tools that bridge sectors, scales, and systems to realize a more resilient and equitable future. The theme of the workshop was "Advancing Complex Adaptive Human-Earth Systems Science in a World of Interconnected Risks". This document outlines the motivation for the workshop, its goals and objectives, the application process, the agenda, overviews of the training sessions offered to the workshop participants and a summary of each breakout session. The MSD workshop report further discusses the feedback from workshop participants and presents some reflections and next steps. The MSD Workshop organizers thank the DOE Office of Science, Earth and Environmental System Modeling, MultiSector Dynamics program area for financial support of its activities through the Integrated Multisector Multiscale Modeling (IM3) project. For more information related to the broader DOE MultiSector Dynamics Program please see https://climatemodeling.science.energy.gov/program-area/multisector-dynamics. D.L.M. and C.M.B. acknowledge support from the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory (ORNL), managed by UT-Battelle, LLC, for the US Department of Energy (DOE). Disclaimer This report was prepared as an account of work sponsored by an agency of the United States Government. Neither theUnited States Government nor any agency thereof, nor Battelle Memorial Institute, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, complete- ness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial products, process, or service by trade name,trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or Battelle Memorial Institute. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. Pacific Northwest National Laboratory operated by Battelle for the United States Department of Energy Available from:Office of Scientific and Technical Information http://www.OSTI.gov multisectordynamics.org This work is made available under the terms of the Creative Commons Attribution- NonCommercial 4.0 International (CC BY-NC 4.0) https://creativecommons.org/licenses/by-nc/4 Suggested citation: Monier, E., Reed, P.M., Vernon, C.R., Hadjimichael, A., Brelsford, C.M., Burleyson, C.B., Dyreson, A.R., Fletcher, S.M., Giang, A., Gupta, R.S., Jackson, N.D., Jones, A.D., Lamontagne, J.R., McCollum, D.L., Morris, J.F., Moss, R.H., Peng, W., Saari, R.K., Srikrishnan, V., Szinai, J.K., Yoon, J. (2024) MultiSector Dynamics: 2023 Inaugural Workshop Report. MSD-LIVE Data Repository. doi:10.57931/2371710.

Monier, Erwan↗
DOE OSTI2022

Deciphering the Dynamic Molecular Program of Radiation-Induced Endothelial Senescence

Radiation-induced cellular senescence is a double-edged sword, acting as both a tumor suppression process limiting tumor proliferation, and a crucial process contributing to normal tissue injury. Endothelial cells play a role in normal tissue injury after radiation therapy. Recently, a study observed an accumulation of senescent endothelial cells (ECs) around radiation-induced lung focal lesions following stereotactic radiation injury in mice. However, the effect of radiation on EC senescence remains unclear because it depends on dose and fractionation, and because the senescent phenotype is heterogeneous and dynamic.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2021

Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters

In this report, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Achieving designed texture and flows in bulk active nematics using optimal control theory

Being intrinsically nonequilibrium, active materials can potentially perform functions that would be thermodynamically forbidden in passive materials. However, active systems have diverse local attractors that correspond to distinct dynamical states, many of which exhibit chaotic turbulent-like dynamics and thus cannot perform work or useful functions. Designing such a system to choose a specific dynamical state is a formidable challenge. Motivated by recent advances enabling optogenetic control of experimental active materials, we describe an optimal control theory framework that identifies a spatiotemporal sequence of light-generated activity that drives an active nematic system toward a prescribed dynamical steady state. Active nematics are unstable to spontaneous defect proliferation and chaotic streaming dynamics in the absence of control. We demonstrate that optimal control theory can compute activity fields that redirect the dynamics into a variety of alternative dynamical programs and functions. This includes dynamically reconfiguring between states, selecting and stabilizing emergent behaviors that do not correspond to attractors, and are hence unstable in the uncontrolled system. Furthermore, our results provide a roadmap to leverage optical control methods to rationally design structure, dynamics, and function in a wide variety of active materials.

Complex systems theory↗
DOE OSTI2021

Overvoltage Prevention and Curtailment Reduction Using Adaptive Droop-Based Supplementary Control in Smart Inverters

Recent developments in the renewable energy sector have seen an unprecedented growth in residential photovoltaic (PV) installations. However, high PV penetration levels often lead to overvoltage problems in low-voltage (LV) distribution feeders. Smart inverter control such as active power curtailment (APC)-based overvoltage control can be implemented to overcome these challenges. The APC technique utilizes a constant droop-based approach which curtails power rigidly, which can lead to significant energy curtailment in the LV distribution feeders. In this paper, different variations of the APC technique with linear, quadratic, and exponential droops have been analyzed from the point-of-view of energy curtailment for a LV distribution network in North America. Further, a combinatorial approach using various droop-based APC methods in conjunction with adaptive dynamic programming (ADP) as a supplementary control scheme has also been proposed. The proposed approach minimizes energy curtailment in the LV distribution network by adjusting the droop gains. Simulation results depict that ADP in conjunction with exponential droop reduces the energy curtailment to approximately 50% compared to using the standard linear droop.

42 ENGINEERING↗
DOE OSTI2022

Nonadiabatic Molecular Dynamics by Multiconfiguration Pair-Density Functional Theory

Herein, we present the first implementation of multiconfiguration pair-density functional theory (MC-PDFT) ab initio molecular dynamics. MC-PDFT is a multireference electronic structure method that in many cases has a similar accuracy (or even better accuracy) the complete active space second-order perturbation theory (CASPT2) at a significantly lower computational cost. In this study, we introduced MC-PDFT analytical gradients into the SHARC molecular dynamics program for ab initio, nonadiabatic molecular dynamics simulations. We verify our implementation by examining the intersystem crossing dynamics of thioformaldehyde, and we observe excellent agreement with recent CASPT2 and experimental findings. Moreover, with MC-PDFT, we could perform dynamics simulations with the 12 electron in 10 orbitals active space that was computationally too expensive for direct dynamics with CASPT2.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Connected Vehicle-Based Traffic Signal Coordination

This study presents a connected vehicles (CVs)-based traffic signal optimization framework for a coordinated arterial corridor. The signal optimization and coordination problem are first formulated in a centralized scheme as a mixed-integer nonlinear program (MINLP). The optimal phase durations and offsets are solved together by minimizing fuel consumption and travel time considering an individual vehicle’s trajectories. Due to the complexity of the model, we decompose the problem into two levels: an intersection level to optimize phase durations using dynamic programming (DP), and a corridor level to optimize the offsets of all intersections. In order to solve the two-level model, a prediction-based solution technique is developed. The proposed models are tested using traffic simulation under various scenarios. Compared with the traditional actuated signal timing and coordination plan, the signal timing plans generated by solving the MINLP and the two-level model can reasonably improve the signal control performance. When considering varies vehicle types under high demand levels, the proposed two-level model reduced the total system cost by 3.8% comparing to baseline actuated plan. MINLP reduced the system cost by 5.9%. It also suggested that coordination scheme was beneficial to corridors with relatively high demand levels. For intersections with major and minor street, coordination conducted for major street had little impacts on the vehicles at the minor street.

42 ENGINEERING↗
DOE OSTI2023

DP-Based Optimization of BESS to Substitute RICE Reserves for Improved Economic Benefits

In regions with a significant proportion of renewable energy sources, reciprocating internal combustion engines (RICE or IC) are frequently employed as grid reserve solutions. This study suggests a scheduling technique for battery energy storage systems (BESS) that utilizes dynamic programming (DP) to replace IC units, adding versatility to the total reserve capacity and resulting in higher BESS earnings compared to traditional scheduling approaches. The capacity sweeping approach to optimize BESS’s revenue sometimes leads to operation which is counter intuitive to conventional battery sizing approaches.

25 ENERGY STORAGE↗
DOE OSTI2021

Incorporating Valve Switching Losses Into a Static Optimal Control Algorithm for the Hybrid Hydraulic-Electric Architecture (HHEA)

The Hybrid Hydraulic-Electric Architecture (HHEA) has, in recent years, been proposed as an energy efficient alternative to conventional load-sensing architectures in mobile machines such as excavators and wheel-loaders. HHEA leverages the advantages of hydraulic power and electric power to eliminate throttling valves while also improving the energy and control performance of the system. The architecture utilizes a set of common pressure rails to provide a majority of power and and a small electric motor driven pump to modulate this power to meet the exact demand. Previous work has developed a computationally efficient Lagrange Multiplier approach for determining the optimal pressure rail selections that minimizes the energy losses in the system. The static model used considers only the energy use for each pressure rail selection but not the losses associated with the valves during the transition. This paper presents an approach to include the switching losses in the model and in the optimization procedure. To capture the switching losses, switching events between different rails and at various input and output flow rates were simulated with consideration of valve spool dynamics. A parameterized model that summarizes the losses is then obtained, allowing switching losses to be added to the previous energy analysis. The performance of the switching loss model was compared with reference data obtained from a high-fidelity simulation model. To incorporate the switching losses into optimal control algorithm, an efficient dynamic programming approach that prevents frequent switching is adopted in place of the Lagrange multiplier approach. The overall effect of switching losses on the energy consumption and optimal control decisions is presented. In general, switching losses contribute to about 9–10% of input energy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Data-based and secure switched cyber–physical systems

In this work, we develop a completely model-free moving target defense framework for the detection and mitigation of sensor and/or actuator attacks in cyber–physical systems with dynamics that evolve in discrete-time. We incorporate an intrusion detection mechanism based on an approximate dynamic programming technique that learns the policies for optimal regulation and optimal tracking while simultaneously defending against actuator and sensor attacks in a model-free fashion. Switching rules are leveraged to force proactive and reactive defense mechanisms as well as, guarantee the stability of the equilibrium point. Finally, as a case study, we apply the proposed moving target defense framework to a DC–DC converter that is used in electric vehicles.

42 ENGINEERING↗
DOE OSTI2019

Machine Learning-Based Anomaly Detection for Load Forecasting Under Cyberattacks

Accurate load forecasting can create both economic and reliability benefits for power system operators. However, the cyberattack on load forecasting may mislead operators to make unsuitable operational decisions for the electricity delivery. To effectively and accurately detect these cyberattacks, this paper develops a machine learning-based anomaly detection (MLAD) methodology. First, load forecasts provided by neural networks are used to reconstruct the benchmark and scaling data by using the k-means clustering. Second, the cyberattack template is estimated by the naive Bayes classification based on the cumulative distribution function and statistical features of the scaling data. Finally, the dynamic programming is utilized to calculate both the occurrence and parameter of one cyberattack on load forecasting data. Here, a widely used symbolic aggregation approximation method is compared with the developed MLAD method. Numerical simulations on the publicly load data show that the MLAD method can effectively detect cyberattacks for load forecasting data with relatively high accuracy. Also, the robustness of MLAD is verified by thousands of attack scenarios based on Monte Carlo simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Optimal economic dispatch policy for prosumer with energy storage considering self-consumption demand

This paper analyzed the effects of self-consumption demand on the joint economic dispatch of prosumers (energy consumers who are also producers), particularly for prosumers with both energy storage and distributed energy sources (DERs). Studies in the existing literature on the economic dispatch scheduling policy of energy storage, mostly from the perspective of electricity merchants, do not address the impacts of self-consumption demand. However, due to the intermittent and high levels of uncertainty regarding DERs generation and the dynamic demand of the prosumer, production and consumption are not always simultaneous; there are two possible scenarios in each period depending on whether DERs generation can meet prosumers' self-consumption or not. Incorporating the self-consumption demand will pose modeling challenges since these two scenarios cannot occur simultaneously in each period, and different scenarios require different decisions for prosumers. Further, this paper analyzed the two scenarios separately to find the optimal storage scheduling strategy, and the results were combined to get the optimal global solution. We focused on prosumers' economic decision-making while considering self-consumption demand and the physical constraints of a battery based on dynamic programming. Our study showed that the feasible state of charge (SOC) range of storage can be segmented into several sub-ranges by SOC reference points under the above two scenarios. As a result, a prosumer's optimal scheduling can be uniquely and conveniently selected based on the sub-ranges within which the current SOC falls. The results, therefore, provided multistage decision-making guidance for prosumers with energy storage.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Optimal checkpointing for adjoint multistage time-stepping schemes

Here, we consider checkpointing strategies that minimize the number of recomputations needed when performing discrete adjoint computations using multistage time-stepping schemes that require computing several substeps within one complete time step. Specifically, we propose two algorithms that can generate optimal checkpoint-ing schedules under weak assumptions. The first is an extension of the seminal Revolve algorithm adapted to multistage schemes. The second algorithm, named CAMS, is developed based on dynamic programming, and it requires the least number of recomputations when compared with other algorithms. The CAMS algorithm is made publicly available in a library with bindings to C and Python. Numerical results show that the proposed algorithms can deliver up to two times the speedup compared with that of classical Revolve. Moreover, we discuss the utilization of the CAMS library in mature scientific computing libraries and demonstrate the ease of using it in an adjoint workflow. The proposed algorithms have been adopted by the PETSc TSAdjoint library. Their performance has been demonstrated with a large-scale PDE-constrained optimization problem on a leadership-class supercomputer. This work is a significant extension of the authors' conference paper.

97 MATHEMATICS AND COMPUTING↗