SEARCH · Search NASA

Results for “stochastic control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗

NASA NTRS1980

Wheat forecast economics effect study

A model to assess the value of improved information regarding the inventories, productions, exports, and imports of crop on a worldwide basis is discussed. A previously proposed model is interpreted in a stochastic control setting and the underlying assumptions of the model are revealed. In solving the stochastic optimization problem, the Markov programming approach is much more powerful and exact as compared to the dynamic programming-simulation approach of the original model. The convergence of a dual variable Markov programming algorithm is shown to be fast and efficient. A computer program for the general model of multicountry-multiperiod is developed. As an example, the case of one country-two periods is treated and the results are presented in detail. A comparison with the original model results reveals certain interesting aspects of the algorithms and the dependence of the value of information on the incremental cost function.

Mehra, R. K.↗

NASA NTRS1976

Attitude control of Mariner Jupiter-Saturn spacecraft

A major challenge of the Mariner Jupiter-Saturn '77 spacecraft was devising a suboptimal attitude controller that could meet the demanding mission requirements. The challenge was met by implementing a discrete stochastic controller for a specially designed onboard computer. The paper describes the design and operation of the controller, based on a simple model of spacecraft dynamics. Two types of cruising modes are considered: inertial cruise, where spacecraft attitude is determined from gyro position outputs, and celestial cruise, where position information is obtained from sun sensors and a star tracker. These two cruise modes under conditions of disturbances were simulated on computer, and the results showed that the controller maintained the spacecraft attitude with low rates. An appendix gives details on the single-step predictor.

Bahrami, K. A.↗

NASA NTRS1972

Parameter uncertainties in control system design.

Development of a design method for including the effects of parameter uncertainties in the design of linear control systems. The approach taken to this problem may be classified as a special case of the stochastic control problem. Thus the formulation is based on the minimization of the expected value of a quadratic performance index defined in terms of the system state vector. The uncertainty in the value of the performance index is the result of the statistical nature of the system parameters rather than a random input signal. It is shown that the expected value of the performance index may be written as a sum of two terms under the assumption of first-order variations of the system state. The first of these terms expresses the nominal performance of the system when the system parameters assume their mean values. The second term represents the effect of the uncertainties on the expected value of the performance index, and is interpreted as an index of system sensitivity.

Palsson, T.↗

DOE OSTI2022

Learning Stochastic Parametric Differentiable Predictive Control Policies

We present a scalable unsupervised learning-based method for obtaining explicit control policies for model predictive control problems for stochastic linear systems with additive uncertainties subject to nonlinear chance constraints. We call the proposed method stochastic parametric differentiable predictive control (SP-DPC), which extends the recently proposed deterministic DPC policy optimization algorithm. We formulate the SP-DPC as a deterministic approximation to the stochastic parametric constrained optimal control problem via independent sampling of the problem's parameters and uncertainties. This formulation allows us to directly compute the policy gradients via automatic differentiation of the problem's value function, evaluated over sampled parameters and uncertainties. In particular, the computed expectation of the problem's value function is backpropagated through the finite-time closed-loop system rollouts parametrized by a known nominal system dynamics model and neural control policy. We also provide theoretical probabilistic guarantees on closed-loop stability and chance constraints satisfaction for systems controlled by learned neural policies. We demonstrate the computational efficiency and scalability of the proposed policy optimization algorithm in three numerical examples, including systems with a large number of states or subject to nonlinear constraints.

Drgona, Jan↗

NASA NTRS1974

Modern control concepts in hydrology

Two approaches to an identification problem in hydrology are presented based upon concepts from modern control and estimation theory. The first approach treats the identification of unknown parameters in a hydrologic system subject to noisy inputs as an adaptive linear stochastic control problem; the second approach alters the model equation to account for the random part in the inputs, and then uses a nonlinear estimation scheme to estimate the unknown parameters. Both approaches use state-space concepts. The identification schemes are sequential and adaptive and can handle either time invariant or time dependent parameters. They are used to identify parameters in the Prasad model of rainfall-runoff. The results obtained are encouraging and conform with results from two previous studies; the first using numerical integration of the model equation along with a trial-and-error procedure, and the second, by using a quasi-linearization technique. The proposed approaches offer a systematic way of analyzing the rainfall-runoff process when the input data are imbedded in noise.

Duong, N.↗

NASA NTRS1975

Modern control concepts in hydrology

Two approaches to an identification problem in hydrology are presented, based upon concepts from modern control and estimation theory. The first approach treats the identification of unknown parameters in a hydrologic system subject to noisy inputs as an adaptive linear stochastic control problem; the second approach alters the model equation to account for the random part in the inputs, and then uses a nonlinear estimation scheme to estimate the unknown parameters. Both approaches use state-space concepts. The identification schemes are sequential and adaptive and can handle either time-invariant or time-dependent parameters. They are used to identify parameters in the Prasad model of rainfall-runoff. The results obtained are encouraging and confirm the results from two previous studies; the first using numerical integration of the model equation along with a trial-and-error procedure, and the second using a quasi-linearization technique. The proposed approaches offer a systematic way of analyzing the rainfall-runoff process when the input data are imbedded in noise.

Duong, N.↗

NASA NTRS1977

Control of nonlinear stochastic systems using adaptive estimation

A general procedure for the design of a digital controller for nonlinear stochastic systems is presented; the emphasis is on development of partitioned-adaptive control techniques which provide rapid tracking of the system state vector to follow rapid trajectory variations. Tests involving the control of a low-order nonlinear oscillator suggest that the procedure yields an effective method for achieving closed-loop performance, despite the difficulty of the control problem

Vanlandingham, H. F.↗

DOE OSTI2025

Stochastic Adaptive Droop Control in Frequency Regulation of Power Systems With Intermittent Generators

Modern power systems (MPSs), including microgrids (MGs), are increasingly incorporating multiple renewable energy sources (RESs) such as wind and solar power, as well as battery storage and controllable loads. While environmentally beneficial, these sources pose challenges for control and management due to their intermittent and stochastic nature, especially in maintaining frequency stability with multiple interconnected generators of varying capacities. Traditional droop control methods are effective in systems with generators that are dispatchable and have fixed generation capacities, but they fall short when applied to systems with RESs, where generation capacities are dynamic and affected by unpredictable environmental conditions. To address these challenges, this paper introduces a novel stochastic adaptive droop control (SADC) method for load frequency control (LFC). The proposed method adapts droop coefficients in real time, based on the measured stochastic data of power generation capacities, enabling more effective frequency regulation in systems with variable and intermittent power generation. Unlike traditional adaptive control methods, which assume constant or slowly-varying system parameters, this approach accounts for stochastic processes by modeling them as Markov chains, enabling robust performance under highly dynamic and unpredictable conditions. The key contributions of this work include the development of real-time droop coefficient adaptation algorithms, derivation of their stability and convergence properties, and the demonstration of the advantages of the method through simulations. Case studies highlight the improved performance of frequency regulation, particularly in addressing the impact of stochastic weather conditions and the benefits of reducing dependence on battery reserves in dealing with intermittency of RESs. Finally, this paper provides a comprehensive analysis of the theoretical foundations of the method, as well as practical implementation insights for future power systems with high penetration of RESs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1973

Control optimization, stabilization and computer algorithms for aircraft applications

Computationally useful algorithms are considered that can aid the control engineer in designing systems control in linear time invariant dynamics for aircraft applications. Structural aspects of system identification, matrix parameterization, and the effect of feedback on identifiability of systems. Adaptive and stochastic control model constructions are projected, and a method for approximate identification of aircraft characteristics and subsequent generation of control signals is outlined.

Mitter, S. K.↗

NASA NTRS1979

On reliable control system designs with and without feedback reconfigurations

This paper contains an overview of a theoretical framework for the design of reliable multivariable control systems, with special emphasis on actuator failures and necessary actuator redundancy levels. Using a linear model of the system, with Markovian failure probabilities and quadratic performance index, an optimal stochastic control problem is posed and solved. The solution requires the iteration of a set of highly coupled Riccati-like matrix difference equations; if these converge one has a reliable design; if they diverge, the design is unreliable, and the system design cannot be stabilized. In addition, it is shown that the existence of a stabilizing constant feedback gain and the reliability of its implementation is equivalent to the convergence properties of a set of coupled Riccati-like matrix difference equations. In summary, these results can be used for offline studies relating the open loop dynamics, required performance, actuator mean time to failure, and functional or identical actuator redundancy, with and without feedback gain reconfiguration strategies.

Birdwell, J. D.↗

NASA NTRS1975

Survey of decentralized control methods

An overview is presented of the types of problems that are being considered by control theorists in the area of dynamic large scale systems with emphasis on decentralized control strategies. Approaches that deal directly with decentralized decision making for large scale systems are discussed. It is shown that future advances in decentralized system theory are intimately connected with advances in the stochastic control problem with nonclassical information pattern. The basic assumptions and mathematical tools associated with the latter are summarized, and recommendations concerning future research are presented.

Athans, M.↗

NASA NTRS1975

Limited variance control in statistical low thrust guidance analysis

Difficulties arise in guiding a solar electric propulsion spacecraft due to nongravitational accelerations caused by random fluctuations in the magnitude and direction of the thrust vector. These difficulties may be handled by using a low thrust guidance law based on the linear-quadratic-Gaussian problem of stochastic control theory with a minimum terminal miss performance criterion. Explicit constraints are imposed on the variances of the control parameters, and an algorithm based on the Hilbert space extension of a parameter optimization method is presented for calculation of gains in the guidance law. The terminal navigation of a 1980 flyby mission to the comet Encke is used as an example.

Jacobson, R. A.↗

NASA NTRS1993

Adaptive nonlinear polynomial neural networks for control of boundary layer/structural interaction

The acoustic pressures developed in a boundary layer can interact with an aircraft panel to induce significant vibration in the panel. Such vibration is undesirable due to the aerodynamic drag and structure-borne cabin noises that result. The overall objective of this work is to develop effective and practical feedback control strategies for actively reducing this flow-induced structural vibration. This report describes the results of initial evaluations using polynomial, neural network-based, feedback control to reduce flow induced vibration in aircraft panels due to turbulent boundary layer/structural interaction. Computer simulations are used to develop and analyze feedback control strategies to reduce vibration in a beam as a first step. The key differences between this work and that going on elsewhere are as follows: that turbulent and transitional boundary layers represent broadband excitation and thus present a more complex stochastic control scenario than that of narrow band (e.g., laminar boundary layer) excitation; and secondly, that the proposed controller structures are adaptive nonlinear infinite impulse response (IIR) polynomial neural network, as opposed to the traditional adaptive linear finite impulse response (FIR) filters used in most studies to date. The controllers implemented in this study achieved vibration attenuation of 27 to 60 dB depending on the type of boundary layer established by laminar, turbulent, and intermittent laminar-to-turbulent transitional flows. Application of multi-input, multi-output, adaptive, nonlinear feedback control of vibration in aircraft panels based on polynomial neural networks appears to be feasible today. Plans are outlined for Phase 2 of this study, which will include extending the theoretical investigation conducted in Phase 2 and verifying the results in a series of laboratory experiments involving both bum and plate models.

Parker, B. Eugene, Jr.↗

DOE OSTI2025

Stochastic Model Predictive Control With Gaussian Wind Direction Preview for Wake Steering

This article addresses the problem of wake steering control for wind farms that explicitly consider the tradeoff between farm-level power generation and yaw duty cycle under variable and uncertain wind conditions. A novel stochastic model predictive control (MPC) algorithm is presented, which utilizes a stochastic model of the freestream wind field components in a receding horizon framework to compute optimal yaw set points that maximize the expected value of the farm power while constraining the yaw actuation. Different configurations of the algorithm are evaluated using a steady-state wind farm simulator. The proposed stochastic MPC algorithm can plan control actions over a future prediction horizon based on probabilistic estimates of the incoming wind magnitude and direction.

17 WIND ENERGY↗

DOE OSTI2024

Evaluation of a Stochastic Mixing Scheme in the Deep Convective Gray Zone Using a Tropical Oceanic Deep Convection Case Study

Abstract A stochastic horizontal subgrid‐scale mixing scheme is evaluated in ensemble simulations of a tropical oceanic deep convection case using a horizontal grid spacing (Δ h ) of 3 km. The stochastic scheme, which perturbs the horizontal mixing coefficient according to a prescribed spatiotemporal autocorrelation scale, is found to generally increase mesoscale organization and convective intensity relative to a non‐stochastic control simulation. Perturbations applied at relatively short autocorrelation scales induce differences relative to the control that are more systematic than those from perturbations applied at relatively long scales that yield more variable outcomes. A simulation with mixing enhanced by a constant factor of 4 significantly increases mesoscale organization and convective intensity, while turning off horizontal subgrid‐scale mixing decreases both. Total rainfall is modulated by a combination of mesoscale organization, areal coverage of convection, and convective intensity. The stochastic simulations tend to behave more similarly to the constant enhanced mixing simulation owing to greater impacts from enhanced mixing as compared to reduced mixing. The impacts of stochastic mixing are robust, ascertained by comparing the stochastic mixing ensembles with a non‐stochastic mixing ensemble that has grid‐scale noise added to the initial thermodynamic field. Compared to radar observations and a higher resolution Δ h = 1 km simulation, stochastic mixing seemingly degrades the simulation performance. These results imply that stochastic mixing produces non‐negligible impacts on convective system properties and evolution but does not lead to an improved representation of convective cloud characteristics in the case studied here.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1982

An identification algorithm for linear stochastic systems with time delays

Linear discrete stochastic control systems containing unknown multiple time delays, plant parameters and noise variances are considered. An algorithm is established which uses the maximum-likelihood technique to identify the unknown parameters. An estimated likelihood function is evaluated based on the previous parameter estimates, which in turn generates a new descent direction vector to update the unknown parameters. The delays and plant parameters are identified in their respective parameter spaces. An example of a second-order stochastic system has been implemented by digital simulation to demonstrate the applicability of the algorithm.

Leondes, C. T.↗

NASA NTRS1976

Trends in modern system theory

The topics considered are related to linear control system design, adaptive control, failure detection, control under failure, system reliability, and large-scale systems and decentralized control. It is pointed out that the design of a linear feedback control system which regulates a process about a desirable set point or steady-state condition in the presence of disturbances is a very important problem. The linearized dynamics of the process are used for design purposes. The typical linear-quadratic design involving the solution of the optimal control problem of a linear time-invariant system with respect to a quadratic performance criterion is considered along with gain reduction theorems and the multivariable phase margin theorem. The stumbling block in many adaptive design methodologies is associated with the amount of real time computation which is necessary. Attention is also given to the desperate need to develop good theories for large-scale systems, the beginning of a microprocessor revolution, the translation of the Wiener-Hopf theory into the time domain, and advances made in dynamic team theory, dynamic stochastic games, and finite memory stochastic control.

Athans, M.↗