Search NASA⌕ Search

SEARCH · Search NASA

Results for “Optimal Control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Multistage Stabilized Continuation for Indirect Optimal Control of Three-Dimensional Hypersonic Trajectories

This work presents the development of a multistaged stabilized continuation for the three-dimensional unpowered hypersonic trajectory planning problem using indirect optimal control methods. The stabilized continuation method is noniterative and guaranteed to terminate within a finite number of floating point operations, thereby making it well suited for onboard autonomous implementations. Here, we present a multistage formulation of the stabilized continuation scheme that involves starting with a “loose” integration tolerance during the first stage and ramping up toward a “strict” integration tolerance through subsequent stages. An important benefit of this approach is that even when the solution to the underlying optimal control problem is numerically unstable, such as with the hypersonic vehicle footprint generation problem, the stabilized continuation algorithm is shown to be successful in finding a solution while providing some additional insights into the underlying cause of the numerical instability.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Optimal control of large quantum systems: assessing memory and runtime performance of GRAPE

Abstract Gradient Ascent Pulse Engineering (GRAPE) is a popular technique in quantum optimal control, and can be combined with automatic differentiation (AD) to facilitate on-the-fly evaluation of cost-function gradients. We illustrate that the convenience of AD comes at a significant memory cost due to the cumulative storage of a large number of states and propagators. For quantum systems of increasing Hilbert space size, this imposes a significant bottleneck. We revisit the strategy of hard-coding gradients in a scheme that fully avoids propagator storage and significantly reduces memory requirements. Separately, we present improvements to numerical state propagation to enhance runtime performance. We benchmark runtime and memory usage and compare this approach to AD-based implementations, with a focus on pushing towards larger Hilbert space sizes. The results confirm that the AD-free approach facilitates the application of optimal control for large quantum systems which would otherwise be difficult to tackle.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Optimal Control of SOEC-Based Hydrogen Production Systems for Demand Response Using Deep Reinforcement Learning in Smart Grids

Solid oxide electrolysis cell (SOEC) hydrogen production technology can range in size from small, appliance-size equipment to large-scale, central production facilities that can be tied directly to renewable or non-greenhouse-gas-emitting forms of electricity production, making it an ideal resource for demand response (DR). The SOEC hydrogen production system is a complex integrated system that encompasses fluid dynamics, electrical dynamics, and electrochemical and thermal dynamics, all of which involve non-linearity and non-convexity. Proper control of the SOEC hydrogen production system is crucial to enable its participation in the DR program. Here, to overcome the difficulty of designing an explicit control law for such nonlinear systems with nonconvex optimization features in DR applications, deep reinforcement learning (DRL) is explored to achieve the optimal control of the SOEC system for DR participation. Specifically, a twin delayed deterministic policy gradient (TD3) control framework is applied to achieve optimal response performance during DR events by considering power tracking error and hydrogen production efficiency with a suitable reward function. Two case studies with grid connections for tracking different DR commands were investigated. The first case study involved operating conditions reaching the boundaries, while the second involved operating conditions within the boundaries. The results showed that the proposed DRL-based control for SOEC can track the DR signal in a timely manner while maintaining high energy efficiency.

08 HYDROGEN↗
DOE OSTI2022

State elimination for mixed‐integer optimal control of partial differential equations by semigroup theory

Abstract Mixed‐integer optimal control problems governed by partial differential equations (MIPDECOs) are powerful modeling tools but also challenging in terms of theory and computation. We propose a highly efficient state elimination approach for MIPDECOs that are governed by partial differential equations that have the structure of an abstract ordinary differential equation in function space. This allows us to avoid repeated calculations of the states for all time steps, and our approach is applied only once before starting the optimization. The presentation of theoretical results is complemented by numerical experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Data-driven Optimal Control Strategy for Virtual Synchronous Generator via Deep Reinforcement Learning Approach

This paper aims at developing a data-driven optimal control strategy for virtual synchronous generator (VSG) in the scenario where no expert knowledge or requirement for system model is available. Firstly, the optimal and adaptive control problem for VSG is transformed into a reinforcement learning task. Specifically, the control variables, i.e., virtual inertia and damping factor, are defined as the actions. Meanwhile, the active power output, angular frequency and its derivative are considered as the observations. Moreover, the reward mechanism is designed based on three preset characteristic functions to quantify the control targets: (1) maintaining the deviation of angular frequency within special limits; (2) preserving well-damped oscillations for both the angular frequency and active power output; (3) obtaining slow frequency drop in the transient process. Next, to maximize the cumulative rewards, a decentralized deep policy gradient algorithm, which features model-free and faster convergence, is developed and employed to find the optimal control policy. With this effort, a data-driven adaptive VSG controller can be obtained. By using the proposed controller, the inverter-based distributed generator can adaptively adjust its control variables based on current observations to fulfill the expected targets in model-free fashion. Finally, simulation results validate the feasibility and effectiveness of the proposed approach.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

State preparation of lattice field theories using quantum optimal control

Here, we explore the application of quantum optimal control (QOC) techniques to state preparation of lattice field theories on quantum computers. As a first example, we focus on the Schwinger model, quantum electrodynamics in 1+1 dimensions. We demonstrate that QOC can significantly speed up the ground state preparation compared to gate-based methods, even for models with long-range interactions. Using classical simulations, we explore the dependence on the interqubit coupling strength and the device connectivity, and we study the optimization in the presence of noise. While our simulations indicate potential speedups, the results strongly depend on the device specifications. In addition, we perform exploratory studies on the preparation of thermal states. Our results motivate further studies of QOC techniques in the context of quantum simulations for fundamental physics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Optimal control of coupled quantum systems based on the first-order Magnus expansion: Application to multiple dipole-dipole-coupled molecular rotors

This paper presents a method for performing approximate optimal control simulations for quantum systems with multiple coupled degrees of freedom. In this work, the time evolution is simulated using the first-order Magnus expansion in the interaction picture, where the couplings between different degrees of freedom are treated as the perturbation. A numerical implementation procedure is presented that leverages upon pairwise couplings and the separability of the zeroth-order time evolution operator to achieve a reduced computational cost, which is analyzed with respect to the number of degrees of freedom. The formulation is compatible with gradient-free methods to optimize the control field, and a stochastic hill climbing algorithm is adopted for this purpose. As illustrations, optimal control simulations are performed for systems of two and three dipole-dipole-coupled molecular rotors under the influence of a control field. For the two-rotor system, the field is optimized to achieve either orientation or entanglement objectives. For the three-rotor system, the field is optimized either to orient all three rotors in the same direction or to orient one rotor in a particular direction while the other two rotors point in the opposite direction.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Optimal Control for Fast Frequency Response and Black-Start using Embedded Storages with Grid-Forming Control

The grid-forming inverter is regarded as the solution for integrating high levels of renewable resources into future power systems. Ensuring the stable operation of grids necessitates that grid-forming inverters offer fast frequency response. This report introduces an optimal control method that coordinates the embedded storage within the grid-forming control model with conventional synchronous generators. The optimized active power reference for the embedded storage is generated using receding horizon optimization control, aiming to keep the center of inertia frequency within acceptable limits. The effectiveness of the proposed control is verified through testing in the IEEE 39-bus system. In addition, with grid-forming capability, we will also investigate the application of using mobile embedded storages as black-start units to provide cranking power to energize non-blackstart generators in a black-start process.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

OPTIMAL CONTROL AND ENERGY-SAVING ANALYSIS OF COMMON PRESSURE RAIL ARCHITECTURES: HHEA & STEAM

In recent years several novel hydraulic architectures have been proposed with the intention of significantly increasing system efficiency. Two of these architectures, Steigerung der Energieeffizienz in der Arbeitshydraulik mobiler Arbeitsmaschinen (STEAM), and the Hybrid Hydraulic-Electric Architecture (HHEA), use a system of multiple common pressure rails (CPRs) to serve the multiple degrees-of-freedom of the machine. The key difference is that STEAM throttles hydraulic power from these rails while HHEA combines electric and hydraulic power to meet actuator demands. As a throttle-less architecture, HHEA is expected to save more energy than STEAM at the expense of added complexity. Therefore, it is useful to quantify this additional energy saving. Both systems have discrete operational choices corresponding to how the CPRs are utilized for each actuator. It is necessary to determine optimal operation for each of these architectures for analysis and fair comparison. Techniques for optimal operation of the HHEA have been developed previously from the Langrange multiplier method. Applying the same optimal control method to STEAM encountered some technical challenge leading to the optimal control algorithm not being able to satisfy certain constraints. The issue is analyzed and solved by adding noise to the optimization. Using this proposed algorithm, case studies are performed to compare the energy-saving potentials of STEAM and HHEA for two sizes of excavators and a wheel-loader performing representative duty cycles. The baseline is a standard load-sensing architecture. Results show that STEAM and HHEA can reduce energy consumption between 35-65% and 50-80% respectively

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Chapter Nine - Automated Optimal Control in Energy Systems: The Reinforcement Learning Approach

With the development of smart grid technologies an increasing number of new devices and participants have joined modern energy systems and are inevitably making them more complicated and interdependent than ever. Optimally controlling such a complex energy system and maintaining its operation in a high-efficient, secure, and resilient manner are challenging tasks to the system operators. Fortunately, the revolution in deep learning and artificial intelligence (AI), both from hardware and algorithms perspectives, has provided new ideas and solutions to many previously intractable problems. As a result, this advance in computer science also sparked great research interests in utilizing AI in solving engineering problems related to the modern energy systems. Among many AI techniques, deep reinforcement learning (DRL) has demonstrated great potential for solving sequential optimization problems, which are very common in the engineering domains. Its ability to handle nonlinearity and stochasticity in controlled systems has out-competed many traditional optimal control algorithms. Therefore in this chapter, we focus on the state-of-the-art of DRL concepts and related algorithms, compare their pros and cons with traditional optimal control approaches and discuss the typical workflow for leveraging RL in solving complex problems in modern energy systems.

artificial intelligence↗
DOE OSTI2020

Extremum seeking for optimal control problems with unknown time-varying systems and unknown objective functions

We consider the problem of optimal feedback control of an unknown, noisy, time-varying, dynamic system that is initialized repeatedly. Examples include a robotic manipulator which must perform the same motion, such as assisting a human, repeatedly and accelerating cavities in particle accelerators which are turned on for a fraction of a second with given initial conditions and vary slowly due to temperature fluctuations. In this paper, we present an approach that applies to systems of practical interest. The method presented here is model independent; does not require knowledge of the objective function; is robust to measurement noise; is applicable for any set of initial conditions; is applicable to simultaneously controlling an arbitrary number of parameters; and may be implemented with a broad range of continuous or discontinuous functions such as sine or square waves. For systems with convex cost functions we prove that our algorithm will produce controllers that approach the minimal cost. For linear systems we reproduce the cost minimizing linear quadratic regulator optimal controller that could have been designed analytically had the system and cost function been known. We demonstrate the effectiveness of the algorithm with simulation studies of noisy and time-varying systems.

42 ENGINEERING↗
DOE OSTI2024

Optimal control of polar sea-ice near its tipping points

Abstract Several Earth system components are at a high risk of undergoing rapid, irreversible qualitative changes or “tipping” with increasing climate warming. It is therefore necessary to investigate the feasibility of arresting or even reversing the crossing of tipping thresholds. Here, we study feedback control of an idealized energy balance model (EBM) for Earth’s climate, which exhibits a “small icecap” instability responsible for a rapid transition to an ice-free climate under increasing greenhouse gas forcing. We develop an optimal control strategy for the EBM under different forcing scenarios to reverse sea-ice loss while minimizing costs. Control is achievable for this system, but the cost nearly quadruples once the system tips. While thermal inertia may delay tipping, leading to an overshoot of the critical forcing threshold, this leeway comes with a steep rise in requisite control once tipping occurs. Additionally, we find that the optimal control is localized in the polar region.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Extending XACC for Quantum Optimal Control

Quantum computing vendors are beginning to open up application programming interfaces for direct pulse-level quantum control. With this, programmers can begin to describe quantum kernels of execution via sequences of arbitrary pulse shapes. This opens new avenues of research and development with regards to smart quantum compilation routines that enable direct translation of higher-level digital assembly representations to these native pulse instructions. In this work, we present an extension to the XACC system-level quantum-classical software framework that directly enables this compilation lowering phase via user-specified quantum optimal control techniques. This extension enables the translation of digital quantum circuit representations to equivalent pulse sequences that are optimal with respect to the backend system dynamics. Our work is modular and extensible, enabling third party optimal control techniques and strategies in both C++ and Python. We demonstrate this extension with familiar gradient-based methods like gradient ascent pulse engineering (GRAPE), gradient optimization of analytic controls (GOAT), and Krotov's method. Our work serves as a foundational component of future quantum-classical compiler designs that lower high-level programmatic representations to low-level machine instructions.

Nguyen, Thien↗
DOE OSTI2020

Risk-averse optimal control of semilinear elliptic PDEs

In this paper, we consider the optimal control of semilinear elliptic PDEs with random inputs. These problems are often nonconvex, infinite-dimensional stochastic optimization problems for which we employ risk measures to quantify the implicit uncertainty in the objective function. In contrast to previous works in uncertainty quantification and stochastic optimization, we provide a rigorous mathematical analysis demonstrating higher solution regularity (in stochastic state space), continuity and differentiability of the control-to-state map, and existence, regularity and continuity properties of the control-to-adjoint map. Our proofs make use of existing techniques from PDE-constrained optimization as well as concepts from the theory of measurable multifunctions. We illustrate our theoretical results with two numerical examples motivated by the optimal doping of semiconductor devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Centralized and Decentralized Optimal Control of Variable Speed Heat Pumps

Utility service providers are often challenged with the synchronization of thermostatically controlled loads. Load synchronization, as a result of naturally occurring and demand-response events, has the potential to damage power distribution equipment. Because thermostatically controlled loads constitute most of the power consumed by the grid at any given time, the proper control of such devices can lead to significant energy savings and improved grid stability. The contribution of this paper is the development of an optimal control algorithm for commonly used variable speed heat pumps. By means of selective peer-to-peer communication, our control architecture allows for the regulation of home temperatures while simultaneously minimizing aggregate power consumption, and aggregate load volatility. An optimal centralized controller is also explored and compared against its decentralized counterpart.

30 DIRECT ENERGY CONVERSION↗
DOE OSTI2025

Fast and High-Fidelity SNAP Gate Enabled by Optimal Control on Floquet-Engineered Systems

Superconducting cavities with high quality factors, coupled to nonlinear ancilla, provide a promising platform for quantum information storage and manipulation. However, the commonly used selective number-dependent arbitrary phase (SNAP) gate faces significant challenges in ultra-high-coherence cavities, where weak dispersive shifts result in prolonged gate times. Here, we propose a protocol to achieve high-fidelity SNAP gates that are orders of magnitude faster than the standard implementation, breaking the speed limit set by the bare dispersive shift. This enhancement is achieved by dynamically amplifying dispersive coupling via sideband interactions, followed by quantum optimal control on Floquet-engineered systems. Additionally, we present a unified perturbation theory that explains both the gate acceleration and associated benign drive-induced decoherence, corroborated by Floquet\textendash Markov simulations. These results pave the way for the experimental realization of high-fidelity control of weakly coupled, high-coherence cavities, and expanding the scope of optimal control techniques in Floquet quantum systems.

You, Xinyuan [Fermilab]↗
DOE OSTI2021

Incorporating Valve Switching Losses Into a Static Optimal Control Algorithm for the Hybrid Hydraulic-Electric Architecture (HHEA)

The Hybrid Hydraulic-Electric Architecture (HHEA) has, in recent years, been proposed as an energy efficient alternative to conventional load-sensing architectures in mobile machines such as excavators and wheel-loaders. HHEA leverages the advantages of hydraulic power and electric power to eliminate throttling valves while also improving the energy and control performance of the system. The architecture utilizes a set of common pressure rails to provide a majority of power and and a small electric motor driven pump to modulate this power to meet the exact demand. Previous work has developed a computationally efficient Lagrange Multiplier approach for determining the optimal pressure rail selections that minimizes the energy losses in the system. The static model used considers only the energy use for each pressure rail selection but not the losses associated with the valves during the transition. This paper presents an approach to include the switching losses in the model and in the optimization procedure. To capture the switching losses, switching events between different rails and at various input and output flow rates were simulated with consideration of valve spool dynamics. A parameterized model that summarizes the losses is then obtained, allowing switching losses to be added to the previous energy analysis. The performance of the switching loss model was compared with reference data obtained from a high-fidelity simulation model. To incorporate the switching losses into optimal control algorithm, an efficient dynamic programming approach that prevents frequent switching is adopted in place of the Lagrange multiplier approach. The overall effect of switching losses on the energy consumption and optimal control decisions is presented. In general, switching losses contribute to about 9–10% of input energy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

On the Traffic Impacts of Optimally Controlled Connected and Automated Vehicles

The implementation of connected and automated vehicle (CAV) technologies enables a novel computational framework for real-time control actions aimed at optimizing energy consumption and associated benefits. Several research efforts reported in the literature to date have proposed decentralized control algorithms to coordinate CAVs in various traffic scenarios, e.g., highway on-ramps, intersections, and roundabouts. However, the impact of optimally coordinating CAVs on the performance of a transportation network has not been thoroughly analyzed yet. In this paper, we apply a decentralized optimal control framework in a transportation network and compare its performance to a baseline scenario consisting of human-driven vehicles. We show that introducing of CAVs yields radically improved roadway capacity and network performance.

Zhao, Liuhui↗