Search NASA⌕ Search

SEARCH · Search NASA

Results for “Stochastic optimal control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Preliminary study of auto-differentiation algorithm in beam dynamics with stochastic process

Modern particle accelerator optimization requires sophisticated computational methods to address the inherently stochastic nature of beam dynamics. This research develops a framework applying AD to SDEs that specifically addresses beam dynamics challenges in particle accelerators, focusing on accurately modeling and optimizing beam behavior in regimes dominated by stochastic processes. By incorporating key physical phenomena such as synchrotron radiation, wakefield effects, and quantum excitation, the framework aims to provide auto differentiation on the figure of merit of the phase space evolution and beam dynamics. The methodology will enable effective optimization method in a dynamic system with stochastic process.

Accelerator Physics↗
DOE OSTI2024

Reinforcement learning pulses for transmon qubit entangling gates

The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1987

Improving stability margins in discrete-time LQG controllers

Some of the problems are discussed which are encountered in the design of discrete-time stochastic controllers for problems that may adequately be described by the Linear Quadratic Gaussian (LQG) assumptions; namely, the problems of obtaining acceptable relative stability, robustness, and disturbance rejection properties. A dynamic compensator is proposed to replace the optimal full state feedback regulator gains at steady state, provided that all states are measurable. The compensator increases the stability margins at the plant input, which may possibly be inadequate in practical applications. Though the optimal regulator has desirable properties the observer based controller as implemented with a Kalman filter, in a noisy environment, has inadequate stability margins. The proposed compensator is designed to match the return difference matrix at the plant input to that of the optimal regulator while maintaining the optimality of the state estimates as directed by the measurement noise characteristics.

Oranc, B. Tarik↗
DOE OSTI2025

Deep reinforcement learning control for co-optimizing energy consumption, thermal comfort, and indoor air quality in an office building

With the recent demand for decarbonization and energy efficiency, advanced HVAC control using Deep Reinforcement Learning (DRL) becomes a promising solution. Due to its flexible structures, DRL has been successful in energy reduction for many HVAC systems. However, only a few researches applied DRL agents to manage the entire central HVAC system and control multiple components in both the water loop and the air loop, owing to its complex system structures. Moreover, those researches have not extended their applications by incorporating the indoor air quality, especially both CO2 and PM2.5concentrations, on top of energy saving and thermal comfort, as achieving those objectives simultaneously can cause multiple control conflicts. What's more, DRL agents are usually trained on the simulation environment before deployment, so another challenge is to develop an accurate but relatively simple simulator. Therefore, we propose a DRL algorithm for a central HVAC system to co-optimize energy consumption, thermal comfort, indoor CO2 level, and indoor PM2.5 level in an office building. To train the controller, we also developed a hybrid simulator that decoupled the complex system into multiple simulation models, which are calibrated separately using laboratory test data. The hybrid simulator combined the dynamics of the HVAC system, the building envelope, as well as moisture, CO2, and particulate matter transfer. Three control algorithms (rule-based, MPC, and DRL) are developed, and their performances are evaluated on the hybrid simulator environment with a realistic scenario (i.e., with stochastic noises). The test results showed that, the DRL controller can save 21.4 % of energy compared to a rule-based controller, and has improved thermal comfort, reduced indoor CO2 concentration. The MPC controller showed an 18.6 % energy saving compared to the DRL controller, mainly due to savings from comfort and indoor air quality boundary violations caused by unmeasured disturbances, and it also highlights computational challenges in real-time control due to non-linear optimization. Finally, we provide the practical considerations for designing and implementing the DRL and MPC controllers based on their respective pros and cons.

Guo, Fangzhou↗
NASA NTRS1973

Control optimization, stabilization and computer algorithms for aircraft applications

Computationally useful algorithms are considered that can aid the control engineer in designing systems control in linear time invariant dynamics for aircraft applications. Structural aspects of system identification, matrix parameterization, and the effect of feedback on identifiability of systems. Adaptive and stochastic control model constructions are projected, and a method for approximate identification of aircraft characteristics and subsequent generation of control signals is outlined.

Mitter, S. K.↗
NASA NTRS1984

Tradeoff methods in multiobjective insensitive design of airplane control systems

The latest results of an ongoing study of computer-aided design of airplane control systems are given. Constrained minimization algorithms are used, with the design objectives in the constraint vector. The concept of Pareto optimiality is briefly reviewed. It is shown how an experienced designer can use it to find designs which are well-balanced in all objectives. Then the problem of finding designs which are insensitive to uncertainty in system parameters are discussed, introducing a probabilistic vector definition of sensitivity which is consistent with the deterministic Pareto optimal problem. Insensitivity is important in any practical design, but it is particularly important in the design of feedback control systems, since it is considered to be the most important distinctive property of feedback control. Methods of tradeoff between deterministic and stochastic-insensitive (SI) design are described, and tradeoff design results are presented for the example of the a Shuttle lateral stability augmentation system. This example is used because careful studies have been made of the uncertainty in Shuttle aerodynamics. Finally, since accurate statistics of uncertain parameters are usually not available, the effects of crude statistical models on SI designs are examined.

Schy, A. A.↗
NASA NTRS1988

A Mathematical Formulation of the SCOLE Control Problem. Part 2: Optimal Compensator Design

The study initiated in Part 1 of this report is concluded and optimal feedback control (compensator) design for stability augmentation is considered, following the mathematical formulation developed in Part 1. Co-located (rate) sensors and (force and moment) actuators are assumed, and allowing for both sensor and actuator noise, stabilization is formulated as a stochastic regulator problem. Specializing the general theory developed by the author, a complete, closed form solution (believed to be new with this report) is obtained, taking advantage of the fact that the inherent structural damping is light. In particular, it is possible to solve in closed form the associated infinite-dimensional steady-state Riccati equations. The SCOLE model involves associated partial differential equations in a single space variable, but the compensator design theory developed is far more general since it is given in the abstract wave equation formulation. The results thus hold for any multibody system so long as the basic model is linear.

Balakrishnan, A. V.↗
NASA NTRS1997

Integrator Windup Protection-Techniques and a STOVL Aircraft Engine Controller Application

Integrators are included in the feedback loop of a control system to eliminate the steady state errors in the commanded variables. The integrator windup problem arises if the control actuators encounter operational limits before the steady state errors are driven to zero by the integrator. The typical effects of windup are large system oscillations, high steady state error, and a delayed system response following the windup. In this study, methods to prevent the integrator windup are examined to provide Integrator Windup Protection (IW) for an engine controller of a Short Take-Off and Vertical Landing (STOVL) aircraft. An unified performance index is defined to optimize the performance of the Conventional Anti-Windup (CAW) and the Modified Anti-Windup (MAW) methods. A modified Genetic Algorithm search procedure with stochastic parameter encoding is implemented to obtain the optimal parameters of the CAW scheme. The advantages and drawbacks of the CAW and MAW techniques are discussed and recommendations are made for the choice of the IWP scheme, given some characteristics of the system.

KrishnaKumar, K.↗
NASA NTRS1978

Estimation and filter stability of stochastic delay systems

Linear and nonlinear filtering for stochastic delay systems are studied. A representation theorem for conditional moment functionals is obtained, which, in turn, is used to derive stochastic differential equations describing the optimal linear or nonlinear filter. A complete characterization of the optimal filter is given for linear systems with Gaussian noise. Stability of the optimal filter is studied in the case where there are no delays in the observations. Using the duality between linear filtering and control, asymptotic stability of the optimal filter is proved. Finally, the cascade of the optimal filter and the deterministic optimal quadratic control system is shown to be asymptotically stable as well.

Kwong, R. H.↗
NASA NTRS2012

Improving Balance Function Using Low Levels of Electrical Stimulation of the Balance Organs

Crewmembers returning from long-duration space flight face significant challenges due to the microgravity-induced inappropriate adaptations in balance/sensorimotor function. The Neuroscience Laboratory at JSC is developing a method based on stochastic resonance to enhance the brain's ability to detect signals from the balance organs of the inner ear and use them for rapid improvement in balance skill, especially when combined with balance training exercises. This method involves a stimulus delivery system that is wearable/portable and provides imperceptible electrical stimulation to the balance organs of the human body. Stochastic resonance (SR) is a phenomenon whereby the response of a nonlinear system to a weak periodic input signal is optimized by the presence of a particular non-zero level of noise. This phenomenon of SR is based on the concept of maximizing the flow of information through a system by a non-zero level of noise. Application of imperceptible SR noise coupled with sensory input in humans has been shown to improve motor, cardiovascular, visual, hearing, and balance functions. SR increases contrast sensitivity and luminance detection; lowers the absolute threshold for tone detection in normal hearing individuals; improves homeostatic function in the human blood pressure regulatory system; improves noise-enhanced muscle spindle function; and improves detection of weak tactile stimuli using mechanical or electrical stimulation. SR noise has been shown to improve postural control when applied as mechanical noise to the soles of the feet, or when applied as electrical noise at the knee and to the back muscles. SR using imperceptible stochastic electrical stimulation of the vestibular system (stochastic vestibular stimulation, SVS) applied to normal subjects has shown to improve the degree of association between the weak input periodic signals introduced via venous blood pressure receptors and the heart-rate responses. Also, application of SVS over 24 hours improves the long-term heart-rate dynamics and motor responsiveness as indicated by daytime trunk activity measurements in patients with multi-system atrophy, Parkinson s disease, or both, including patients who were unresponsive to standard therapy for Parkinson s disease. Recent studies conducted at the NASA JSC Neurosciences Laboratories showed that imperceptible SVS, when applied to normal young healthy subjects, leads to significantly improved balance performance during postural disturbances on unstable compliant surfaces. These studies have shown the benefit of SR noise characteristic optimization with imperceptible SVS in the frequency range of 0-30 Hz, and amplitudes of stimulation have ranged from 100 to 400 microamperes.

Bloomberg, Jacob↗
NASA NTRS1991

A statistical rain attenuation prediction model with application to the advanced communication technology satellite project. 3: A stochastic rain fade control algorithm for satellite link power via non linear Markow filtering theory

The dynamic and composite nature of propagation impairments that are incurred on Earth-space communications links at frequencies in and above 30/20 GHz Ka band, i.e., rain attenuation, cloud and/or clear air scintillation, etc., combined with the need to counter such degradations after the small link margins have been exceeded, necessitate the use of dynamic statistical identification and prediction processing of the fading signal in order to optimally estimate and predict the levels of each of the deleterious attenuation components. Such requirements are being met in NASA's Advanced Communications Technology Satellite (ACTS) Project by the implementation of optimal processing schemes derived through the use of the Rain Attenuation Prediction Model and nonlinear Markov filtering theory.

Manning, Robert M.↗
NASA NTRS1994

Optimal active control for Burgers equations

A method for active fluid flow control based on control theory is discussed. Dynamic programming and fixed point successive approximations are used to accommodate the nonlinear control problem. The long-term goal of this project is to establish an effective method applicable to complex flows such as turbulence and jets. However, in this report, the method is applied to stochastic Burgers equation as an intermediate step towards this goal. Numerical results are compared with those obtained by gradient search methods.

Ikeda, Yutaka↗
NASA NTRS1965

On stochastic extremum problems - calculus.

Lagrange multiplier technique for determining stationary points /of functions/ or stationary functions /of integrals/ of expected value of random functions with certain random constraints

LAGRANGE MULTIPLIER↗
NASA NTRS2008

Robust Path Planning and Feedback Design Under Stochastic Uncertainty

Autonomous vehicles require optimal path planning algorithms to achieve mission goals while avoiding obstacles and being robust to uncertainties. The uncertainties arise from exogenous disturbances, modeling errors, and sensor noise, which can be characterized via stochastic models. Previous work defined a notion of robustness in a stochastic setting by using the concept of chance constraints. This requires that mission constraint violation can occur with a probability less than a prescribed value.In this paper we describe a novel method for optimal chance constrained path planning with feedback design. The approach optimizes both the reference trajectory to be followed and the feedback controller used to reject uncertainty. Our method extends recent results in constrained control synthesis based on convex optimization to solve control problems with nonconvex constraints. This extension is essential for path planning problems, which inherently have nonconvex obstacle avoidance constraints. Unlike previous approaches to chance constrained path planning, the new approach optimizes the feedback gain as wellas the reference trajectory.The key idea is to couple a fast, nonconvex solver that does not take into account uncertainty, with existing robust approaches that apply only to convex feasible regions. By alternating between robust and nonrobust solutions, the new algorithm guarantees convergence to a global optimum. We apply the new method to an unmanned aircraft and show simulation results that demonstrate the efficacy of the approach.

autonomuys vehicles↗
NASA NTRS1981

Optimal interpolation and the Kalman filter

The estimation theory of stochastic-dynamic systems is described and used in a numerical study of optimal interpolation. The general form of data assimilation methods is reviewed. The Kalman-Bucy, KB filter, and optimal interpolation (OI) filters are examined for effectiveness in performance as gain matrices using a one-dimensional form of the shallow-water equations. Control runs in the numerical analyses were performed for a ten-day forecast in concert with the OI method. The effects of optimality, initialization, and assimilation were studied. It was found that correct initialization is necessary in order to localize errors, especially near boundary points. Also, the use of small forecast error growth rates over data-sparse areas was determined to offset inaccurate modeling of correlation functions near boundaries.

Cohn, S.↗
NASA NTRS1985

Stochastic response analysis, order reduction, and output feedback controllers for flexible spacecraft

Real disturbances and real sensors have finite bandwidths. The first objective of this paper is to incorporate this finiteness in the 'open-loop modal cost analysis' as applied to a flexible spacecraft. Analysis based on residue calculus shows that among other factors, significance of a mode depends on the power spectral density of disturbances and the response spectral density of sensors at the modal frequency. The second objective of this article is to compare performances of an optimal and a suboptimal output feedback controller, the latter based on 'minimum error excitation' of Kosut. Both the performances are found to be nearly the same, leading us to favor the latter technique because it entails only linear computations. Our final objective is to detect an instability due to truncated modes by representing them as a multiplicative and an additive perturbation in a nominal transfer function. In an example problem it is found that this procedure leads to a narrow range of permissible controller gains, and that it labels a wrong mode as a cause of instability. A free beam is used to illustrate the analysis in this work.

Hablani, H. B.↗