Search NASA⌕ Search

SEARCH · Search NASA

Results for “stochastic control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Laboratory Evaluation of Federated, Hierarchical Controls for Distribution Power System Management: Preprint

The connection of more loads and distributed energy resources (DERs) to the distribution power system brings both challenges and opportunities to system operators. There are opportunities to aggregate flexible loads and DERs to provide transmission grid services, but the coordinated actions of DERs being managed by independent, third-party DER aggregators to support transmission system operations can present challenges. We developed a federated DER management architecture and control framework that aims to manage heterogeneous DERs to deliver reliable transmission grid services while respecting distribution system constraints. The controls include stochastic day-ahead optimization, model predictive control, and a simple real-time management scheme. We present simulation results obtained from a realistic laboratory test bed of federated controls managing DERs within a substation service area to make the substation net power follow the optimal net power determined by the day-ahead optimization based on cost and limiting reverse power flow.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Survey on stochastic distribution systems: A full probability density function control theory with potential applications

Complex systems seen either in general engineering practice or economics are subjected to ever increased uncertainties that are mostly represented as random variables or parameters, and the characteristics of random variables are represented by their probability density functions (PDFs). Controlling their PDFs means to shape their stochastic distributions and in general it would provide a full treatment for system analysis and operational control and optimization. This leads to the development of stochastic distribution control (SDC) systems theory in the past decades, where the original aim of the controller design is to realize a shape control of the distributions of certain random variables in their PDFs sense for some engineering processes. Indeed, once the PDFs of these random variables or parameters are used to describe their distribution characters, the control task is to obtain control signals so that the output PDFs of stochastic systems are made to follow their target PDFs. The subject of SDC was initially originated for non-Gaussian stochastic control systems design but has found a wide spectrum of applications in general systems in terms of data-driven modeling, analysis, signal processing (filtering), data mining via multivariable statistics, decision-making (optimization) for systems subjected to uncertainties and even in economics. In this context, SDC constitutes an effective primer tool for complex system analysis, control and operational optimizations. In this review paper, a detailed survey of the developments on the research of SDC systems will be made together with their wide spectrum applications and future perspectives.

42 ENGINEERING↗
DOE OSTI2021

Discrete-Time Causal Control of a Wave Energy Converter With Finite Stroke in Stochastic Waves

We consider feedback control design for a wave energy converter (WEC) for which the power takeoff (PTO) system has a finite stroke limit. Stationary stochastic wave loading is assumed, with a known spectrum, and the plant dynamics are assumed to be linear. We develop a technique for the design of a discrete-time controller, which has three design stages. In the first stage, a linear controller is optimized while imposing a relaxation of the maximum stroke constraint on the design. In the second stage, a tandem nonlinear feedback loop is designed for the purpose of stroke protection. In the third stage, the two designs (linear and nonlinear) are fused in a manner that preserves the stability of the overall system. The technique is demonstrated in a simulation of a simple cylindrical buoy. We show that the controller may be tuned through the adjustment of scalar design parameters, which adjust the tradeoff between the mean generated power and the force levels required to protect the stroke.

16 TIDAL AND WAVE POWER↗
DOE OSTI2021

Stochasticity in the Switching of Nanodisks for Probabilistic Computing

Stochasticity in magnetic nanodevices is an essential characteristic for harnessing these devices to computing based on population coding or the building blocks of probabilistic computing, p-bits. A magnetic tunneling junction (MTJ) consisting of a patterned magnetic element is considered a promising computing unit in the concept of artificial neurons and p-bits. A comprehensive understanding of the stochasticity in the switching of patterned magnetic elements is crucial for realizing MTJ-based probabilistic computing technology. In the present work, the stochastic behavior in the switching process of a perpendicularly magnetized Co/Pt disk within an array was directly observed utilizing full-field soft X-ray microscopy. Within 50 repeated hysteretic cycles, the stochastic magnetization switching of individual Co/Pt disks within disk arrays is identified. We found that the stochasticity in the magnetization switching of disks considerably depends on the disk size. The stochasticity initially decreases as the disk radius gets bigger from 125 to 375 nm (region I), then increases with further enlarging the disk size to 625 nm (region II). The variance of thermal fluctuation relevant to the disk size and the multilevel switching within a disk are severely involved in the observed size-dependent stochasticity. Furthermore, this work provides the way for controlling the stochasticity in the switching of nanopatterned elements, which is a key aspect of MTJ-based probabilistic computing.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Fault isolation and fault-tolerant control for Takagi-Sugeno fuzzy time-varying delay stochastic distribution systems

A fault isolation, estimation, and fault-tolerant control (FTC) scheme for nonlinear time-varying delay stochastic distribution control systems was presented in this paper. The Takagi-Sugeno fuzzy model was adopted to approach the nonlinear dynamics of time-varying delay systems. According to the output equivalence principle and Laplace transformation, an augmented state vector was given to solve the time-varying delay problem. When multiple actuator faults and interference occur simultaneously, fault detection, isolation and fault estimation was designed to obtained the fault information. To decouple faults and obtain the value and location information of the fault, the system was separated into two parts through the designed multiple conversion matrices, in which one subsystem was only affected by one actuator fault. This has simplified the design of fault isolation and estimation. A adaptive observer for fault estimation was given. Then, fault information such as the time, location, and size was determined. The observer gain matrices were calculated using linear matrix inequality (LMI). When a fault was detected and diagnosed, a FTC algorithm was devised using the proportional-integral control scheme to compensate the fault as much as possible. It has been shown that even if multiple faults actuator occurred simultaneously, the FTC controller still ensured the output probability density function of the system traced the desired probability density function when a fault occurred. Finally, the expected results were obtained through the simulation example, which confirmed the effectiveness of the method.

42 ENGINEERING↗
DOE OSTI2020

Stochastic Fault Detection

This entry describes the state-of-the-art and future perspectives on stochastic fault detection, namely, stochastic fault detection and diagnosis (FDD). Both model-based and data-driven FDD methods for stochastic signals and systems have been included, where the use of hypothesis testing, Kalman filtering, system estimation, principal component analysis (PCA), and stochastic distribution control has been discussed for the construction of effective FDD algorithms. Indeed, stochastic FDD constitute an important and integrated part in developing fault-tolerant controls (FTC) for guaranteed safe operation of control systems, of which increased penetration of random factors is inevitable nowadays.

Wang, Aiping↗
DOE OSTI2026

Measurement and feedback-driven adaptive dynamics in the classical and quantum kicked top

In classical dynamical systems, stochastic feedback can stabilize otherwise unstable periodic orbits, giving rise to distinct controlled and uncontrolled phases as the rate of control application is varied. In this work, we apply these control protocols in classical, semiclassical, and quantum regimes to the kicked top, a paradigmatic model of quantum chaos. The quantum kicked top, modeled as the dynamics of a spin-S object, naturally interpolates between these regimes with the spin size S acting as an effective Planck constant. We show that the dynamics of the kicked top in classical, semiclassical, and fully quantum limits can all be controlled using stochastic feedback protocols. Comparing the full quantum dynamics to a truncated Wigner approximation that captures quantum noise but neglects interference beyond the Ehrenfest time, we find that low-moment observables are largely accounted for semiclassically, while the remaining discrepancy in higher moments is consistent with contributions from interference and possibly nonlinearities in rare trajectories that explore the compact phase space. We also find rapid purification in the numerics studied for all rates of control considered, suggesting that control quenches the top's ability to encode a qubit of quantum information even in the uncontrolled phase.

Prasad, Mahaveer [Tata Inst.; Singapore U. Tech. D↗
DOE OSTI2025

Adaptive PID Gain Scheduling Control for Hydropower Turbine Using Neural CDE and Stochastic Distribution Shaping

This paper introduces a gain-scheduling PID controller design strategy for hydroturbine frequency control mode. This scheme first uses real data to learn the nonlinear dynamics of the hydroturbine using neural controlled differential equations and then perturbs the obtained nonlinear system at different equilibrium points, based on which a static output feedback adaptive dynamic programming algorithm is then used to optimize the PID gains for each equilibrium point. Moreover, a continuous-time version of stochastic distribution control is proposed to further fine-tune the optimized PID gains. Finally, the controller is obtained by implementing linear interpolation between the optimized PID control gains. The simulation results show that the proposed gain-scheduling PID controller can control a larger range of operation points compared with the given fixed PID controller and the baseline method. Compared with the given fixed PID controller, the proposed gain-scheduling PID controller can regulate hydroturbine frequency against disturbances induced by power-load variation with over 50% less overshoot for some operation points.

13 HYDRO ENERGY↗
DOE OSTI2022

Next-Cycle Optimal Dilute Combustion Control via Online Learning of Cycle-to-Cycle Variability Using Kernel Density Estimators

Dilute combustion using exhaust gas recirculation (EGR) presents a cost-effective method for increasing the efficiency of spark-ignition (SI) engines. However, the maximum amount of EGR that can be used at a given condition is limited by a rapid increment of cycle-to-cycle variability (CCV). This study describes a methodology to design a model-based stochastic optimal controller to adjust the cycle-to-cycle fuel injection quantity in order to reduce CCV and further extend the dilute limit. Given the complexity and chaotic nature of combustion events, the controller was enhanced with online learning in order to identify the statistical properties of combustion efficiency, which are needed to generate predictions for next-cycle events. This study showed that a kernel density estimator (KDE) can be used to learn the combustion properties in real time and can be incorporated into the feedback policy in order to calculate the optimal control command. Experimental results suggested that the dilute limit can be extended from 18.5% to 21% EGR fraction at an operating condition relevant for highway cruising. Additionally, the proposed controller can achieve a large CCV reduction with less fuel enrichment compared to previous methods, overall contributing to an increase in 0.2% indicated fuel conversion efficiency.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Reducing Urban Traffic Congestion Using Deep Learning and Model Predictive Control

This article proposes a deep learning (DL)-based control algorithm—DL velocity-based model predictive control (VMPC)—for reducing traffic congestion with slowly time-varying traffic signal controls. This control algorithm consists of system identification using DL and traffic signal control using VMPC. For the training process of DL, we established a modeling error entropy loss as the criteria inspired by the theory of stochastic distribution control (SDC) originated by the fourth author. Herein, simulation results show that the proposed algorithm can reduce traffic congestion with a slowly varying traffic signal control input. Results of an ablation study demonstrate that this algorithm compares favorably to other model-based controllers in terms of prediction error, signal varying speed, and control effectiveness.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Variational design principles for nonequilibrium colloidal assembly

Using large deviation theory and principles of stochastic optimal control, we show that rare molecular dynamics trajectories conditioned on assembling a specific target structure encode a set of interactions and external forces that lead to enhanced stability of that structure. Such a relationship can be formulated into a variational principle, for which we have developed an associated optimization algorithm and have used it to determine optimal forces for targeted self-assembly within nonequilibrium steady-states. We illustrate this perspective on inverse design in a model of colloidal cluster assembly within linear shear flow. We find that colloidal clusters can be assembled with high yield using specific short-range interactions of tunable complexity. Shear decreases the yields of rigid clusters, while small values of shear increase the yields of nonrigid clusters. The enhancement or suppression of the yield due to shear is rationalized with a generalized linear response theory. Furthermore, by studying 21 unique clusters made of six, seven, or eight particles, we uncover basic design principles for targeted assembly out of equilibrium.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Sensitivity Analysis in the Presence of Intrinsic Stochasticity for Discrete Fracture Network Simulations

Abstract Large‐scale discrete fracture network (DFN) simulators are standard fare for studies involving the sub‐surface transport of particles since direct observation of real world underground fracture networks is generally infeasible. While these simulators have successfully been used in several engineering applications, estimates of output quantities of interest (QoI) — such as breakthrough time of particles reaching the edge of the system — suffer from two distinct types of uncertainty. A run of a DFN simulator requires several parameters to be set that dictate the placement and size of fractures, the density of fractures, and the overall permeability of the system; uncertainty on the proper parameters will lead to uncertainty in the QoI, called epistemic uncertainty. Furthermore, since these input settings to DFN simulators control the stochastic processes which place fractures and govern flow, understanding how this randomness affects the QoI requires several runs of the simulator at distinct random seeds. The uncertainty in the QoI attributed to different realizations (i.e., different seeds) of the same random process (i.e., identical input parameters) leads to a second type of uncertainty, called aleatoric uncertainty. In this paper, we perform a Sensitivity Analysis, which directly attributes the uncertainty observed in the QoI to the epistemic uncertainty from each input parameter and to the aleatoric uncertainty. Beyond the specific takeaways on which input variables influence uncertainty in the QoI the most, a major contribution of this paper is the introduction of a statistically rigorous workflow for characterizing the uncertainty in DFN flow simulations that exhibit heteroskedasticity.

58 GEOSCIENCES↗
DOE OSTI2023

Optimal mitigation and control over power system dynamics for stochastic grid resilience

Optimal mitigation planning for highly disruptive contingencies to a transmission-level power system requires optimization with dynamic power system constraints, due to the key role of dynamics in system stability to major perturbations. We formulate a generalized disjunctive program to determine optimal grid component hardening choices for protecting against major failures, with differential algebraic constraints representing system dynamics (specifically, differential equations representing generator and load behavior and algebraic equations representing instantaneous power balance over the transmission system). We optionally allow stochastic optimal pre-positioning across all considered failure scenarios, and optimal emergency control within each scenario. This novel formulation allows, for the first time, analyzing the resilience interdependencies of mitigation planning, preventive control, and emergency control. Using all three strategies in concert is particularly effective at maintaining robust power system operation under severe contingencies, as we demonstrate on the western system coordinating council 9-bus test system using synthetic multi-device outage scenarios.

30 DIRECT ENERGY CONVERSION↗
DOE OSTI2020

Chapter Nine - Automated Optimal Control in Energy Systems: The Reinforcement Learning Approach

With the development of smart grid technologies an increasing number of new devices and participants have joined modern energy systems and are inevitably making them more complicated and interdependent than ever. Optimally controlling such a complex energy system and maintaining its operation in a high-efficient, secure, and resilient manner are challenging tasks to the system operators. Fortunately, the revolution in deep learning and artificial intelligence (AI), both from hardware and algorithms perspectives, has provided new ideas and solutions to many previously intractable problems. As a result, this advance in computer science also sparked great research interests in utilizing AI in solving engineering problems related to the modern energy systems. Among many AI techniques, deep reinforcement learning (DRL) has demonstrated great potential for solving sequential optimization problems, which are very common in the engineering domains. Its ability to handle nonlinearity and stochasticity in controlled systems has out-competed many traditional optimal control algorithms. Therefore in this chapter, we focus on the state-of-the-art of DRL concepts and related algorithms, compare their pros and cons with traditional optimal control approaches and discuss the typical workflow for leveraging RL in solving complex problems in modern energy systems.

artificial intelligence↗
DOE OSTI2026

Optimisation of the Kaplan hydropower system via PID 2 and digital twin

Here, this paper proposes a proportional–integral-double–derivative (PID 2 ) optimisation method for the Kaplan hydropower system by building a digital twin. The study first uses one multilayer perceptron (MLP) to model the hydroturbine dynamic and then adopts three connected MLPs to model the generator dynamic, both in an open-loop fashion. Inspired by stochastic distribution control (SDC) theory, we regard the training of the turbine's neural network model as a process control problem, and we propose minimising entropy loss to update the network parameters. The next step is to build the digital twin by connecting the neural network models with a PID 2 controller and a lead-lag exciter and run the whole model in a closed-loop fashion. After that, a binary search approach is applied to optimise the PID 2 parameters based on the obtained digital twin model. The simulation results show that the proposed method can reduce the mean square tracking error by more than 90%. Furthermore, the method is extended to jointly optimise the PID 2 controller and excitation system gains through multiobjective optimisation, leveraging Pareto frontier analysis to balance active power and voltage tracking performance. Simulation results confirm the effectiveness of the proposed method, achieving a 83.46% reduction in relative mean square error of active power, a 47.13% reduction in terminal voltage tracking error, and an 82.78% improvement in the overall scalarized objective.

Hydropower system↗
DOE OSTI2021

Sparse Control Synthesis for Uncertain Responsive Loads With Stochastic Stability Guarantees

In this report, recent studies have demonstrated the potential of flexible loads in providing frequency response services, predominantly due to their availability and cost-effectiveness. However, uncertainty and variability in various weather-related and end-use behavioral factors often impact the reliability of demand-side control performance. This work addresses this problem with the design of a demand-side control to achieve frequency response under load uncertainties. Our approach involves modeling the load uncertainties via stochastic processes that appear as both multiplicative and additive in the power system dynamics. Recently developed mean square exponential stability (MSES) results for continuous-time linear stochastic systems are applied to pose the control synthesis problem which results in an LMI-based optimization problem. Additional costs and constraints are added to the LMI-based controller synthesis to ensure MSES, improve closed-loop transient performance, maximize tolerable uncertainties, and promote sparsity in the controller. Additionally, the fundamental limitations between the tolerable uncertainties and control efforts while ensuring MSES are discussed. Further, the control synthesis problem for the case of the full-state measurement is generalized to the case of partial-state measurements. The proposed control synthesis is illustrated on an IEEE 39 bus system with rigorous studies to demonstrate the role of sparsity, closed-loop transient performance, tolerable uncertainties, and control efforts while ensuring MSES and achieving frequency response.

42 ENGINEERING↗
DOE OSTI2024

Convergence Analysis for an Online Data-Driven Feedback Control Algorithm

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system’s state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings.

97 MATHEMATICS AND COMPUTING↗