Search NASA⌕ Search

SEARCH · Search NASA

Results for “Hamilton-Jacobi-Bellman equation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

15 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

Matched asymptotic expansion of the Hamilton-Jacobi-Bellman equation for aeroassisted plane-change maneuvers

In this paper we develop a general procedure for constructing a matched asymptotic expansion of the Hamilton-Jacobi-Bellman equation based on the method of characteristics. The development is for a class of perturbation problems whose solution exhibits two-time-scale behavior. A regular expansion for problems of this type is inappropriate since it is not uniformly valid over a narrow range of the independent variable. Of particular interest here is the manner in which matching and boundary conditions are enforced when the expansion is carried out to first order. Two cases are distinguished - one where the left boundary condition coincides with or lies to the right of the singular region and one where the left boundary condition lies to the left of the singular region. A simple example is used to illustrate the procedure, and its potential application to aeroassisted plane change is described.

Calise, Anthony J.↗
NASA NTRS1993

Guidance law development for aeroassisted transfer vehicles using matched asymptotic expansions

This report addresses and clarifies a number of issues related to the Matched Asymptotic Expansion (MAE) analysis of skip trajectories, or any class of problems that give rise to inner layers that are not associated directly with satisfying boundary conditions. The procedure for matching inner and outer solutions, and using the composite solution to satisfy boundary conditions is developed and rigorously followed to obtain a set of algebraic equations for the problem of inclination change with minimum energy loss. A detailed evaluation of the zeroth order guidance algorithm for aeroassisted orbit transfer is performed. It is shown that by exploiting the structure of the MAE solution procedure, the original problem, which requires the solution of a set of 20 implicit algebraic equations, can be reduced to a problem of 6 implicit equations in 6 unknowns. A solution that is near optimal, requires a minimum of computation, and thus can be implemented in real time and on-board the vehicle, has been obtained. Guidance law implementation entails treating the current state as a new initial state and repetitively solving the zeroth order MAE problem to obtain the feedback controls. Finally, a general procedure is developed for constructing a MAE solution up to first order, of the Hamilton-Jacobi-Bellman equation based on the method of characteristics. The development is valid for a class of perturbation problems whose solution exhibits two-time-scale behavior. A regular expansion for problems of this type is shown to be inappropriate since it is not valid over a narrow range of the independent variable. That is, it is not uniformly valid. Of particular interest here is the manner in which matching and boundary conditions are enforced when the expansion is carried out to first order. Two cases are distinguished-one where the left boundary condition coincides with, or lies to the right of, the singular region, and another one where the left boundary condition lies to the left of the singular region. A simple example is used to illustrate the procedure where the obtained solution is uniformly valid to O(Epsilon(exp 2)). The potential application of this procedure to aeroassisted plane change is also described and partially evaluated.

Calise, Anthony J.↗
DOE OSTI2025

Whitepaper: Optimal Control from a Fluid Dynamics Perspective

An optimal control problem described by the Hamilton-Jacobi-Bellman equation can be developed into a problem that can be solved by general computational fluid dynamics packages. We describe how this formulation would allow a classical problem in optimal control, Zermelo’s problem, to be treated as a multi-fluid problem. This approach has the advantage of allowing optimal navigation problems to be conducted over large areas, as well as to include moderately larger numbers of ships. We draw comparisons between this approach and the field of fluid control for fluid animations in movies.

42 ENGINEERING↗
NASA NTRS1986

Optimal control of aeroassisted plane change maneuver using feedback expansions

A guidance law for an aeroassisted plane change maneuver is developed by an asymptotic expansion technique using a small parameter which essentially represents the ratio of the inertial forces to the atmospheric forces. This guidance law minimizes the energy loss while meeting terminal constraints on the altitude, flight path angle, and heading angle. By neglecting the inertial forces, the resulting optimization problem is integrable and can be determined in closed form. This zeroth-order solution is the first term in an asymptotic series solution of the Hamilton-Jacobi-Bellman equation. The remaining terms are determined from the solution of a first-order, linear partial differential equation whose solution requires only quadrature integration. Our initial results in using this guidance scheme are encouraging.

Mishne, D.↗
NASA NTRS1986

A guidance law for the aeroassisted plane change maneuver in the presence of atmospheric uncertainties

A stochastic feedback control law for a space vehicle performing an aeroassisted plane-change maneuver is developed. The stochastic control law is designed to minimize the energy loss while taking into consideration the uncertainty in the atmospheric density. The solution is based on expansion of the stochastic Hamilton-Jacobi-Bellman equation (or dynamic programming) about a zeroth-order known integrable solution. The resulting guidance law is expressed as a series expansion in the noise power spectral densities. A numerical example indicates the potential improvement of this method.

Mishne, D.↗
NASA NTRS1988

An approximate atmospheric guidance law for aeroassisted plane change maneuvers

An approximate optimal guidance law for the aeroassisted plane change problem is presented which is based upon an expansion of the Hamilton-Jacobi-Bellman equation with respect to the small parameter of Breakwell et al. (1985). The present law maximizes the final velocity of the reentry vehicle while meeting terminal constraints on altitude, flight path angle, and heading angle. The integrable zeroth-order solution found when the small parameter is set to zero corresponds to a solution of the problem where the aerodynamic forces dominate the inertial forces. Higher order solutions in the expansion are obtained from the solution of linear partial differential equations requiring only quadrature integration.

Speyer, Jason L.↗
NASA NTRS1989

A new adaptive control approach for aerospace vehicles with parameter uncertainties

A new stochastic adaptive control structure is developed for the problem of combined parameter estimation and control of aerospace vehicles with changing parameters. Parameter uncertainties are modeled as first-order Gauss-Markov processes, and are introduced to the system dynamics through a small parameter. It is assumed that an accurate inertial measurement unit gives perfect measurements of the state variables. Since the stochastic system is assumed to be Gauss-Markov, the density function of the parameters given these measurements is conditionally Gaussian. Based on this conditionally Gaussian density, the problem of minimizing a quadratic cost over an infinite time horizon can be set up within the framework of stochastic optimal control theory. The optimal feedback control law is derived from a straightforward expansion of the Hamilton-Jacobi-Bellman equation, based on the LQG solution. The resulting nonlinear controller is applied to the pitch axis control of a space platform with uncertain moments of inertia and is shown to produce marked improvement over a fixed controller.

Hahn, Yungsun↗
NASA NTRS1989

Nonlinear longitudinal control of a supermaneuverable aircraft

A technique is described which can be used for design of feedback controllers for high-performance aircraft operating in flight conditions in which nonlinearities significantly affect performance. Designs are performed on a mathematical model of the longitudinal dynamics of a hypothetical aircraft similar to proposed supermaneuverable flight test vehicles. Nonlinear controller designs are performed using truncated solutions of the Hamilton-Jacobi-Bellman equation. Preliminary results show that the method yields promising results.

Garrard, William L.↗
NASA NTRS1992

Nonlinear feedback control of highly manoeuvrable aircraft

This paper describes the application of nonlinear quadratic regulator (NLQR) theory to the design of control laws for a typical high-performance aircraft. The NLQR controller design is performed using truncated solutions of the Hamilton-Jacobi-Bellman equation of optimal control theory. The performance of the NLQR controller is compared with the performance of a conventional P + I gain scheduled controller designed by applying standard frequency response techniques to the equations of motion of the aircraft linearized at various angles of attack. Both techniques result in control laws which are very similar in structure to one another and which yield similar performance. The results of applying both control laws to a high-g vertical turn are illustrated by nonlinear simulation.

Garrard, William L.↗
DOE OSTI2025

Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker–Planck Equations

The Fokker-Planck (FP) equation is a foundational partial differential equation (PDE) in stochastic processes involving Brownian motions. However, the curse of dimensionality (CoD) poses a formidable challenge when dealing with high-dimensional FP equations. Although Monte Carlo simulation and (vanilla) Physics-Informed Neural Networks (PINNs) have shown the potential to tackle CoD, both methods exhibit significant numerical errors in high dimensions when dealing with the probability density function (PDF) associated with Brownian motion. The point-wise PDF values tend to decrease exponentially as dimensionality increases, surpassing the precision of numerical simulations and resulting in substantial errors. In addition, due to its massive sampling, Monte Carlo fails to offer fast sampling. Modeling the logarithm likelihood (LL) via vanilla PINNs transforms the FP equation into a notoriously difficult Hamilton-Jacobi-Bellman (HJB) equation, which is impractical for PINN learning, whose error grows rapidly with dimension. To this end, we propose a novel approach utilizing a score-based solver to fit the score function in stochastic differential equations (SDEs). The score function, defined as the gradient of the LL, plays a fundamental role in inferring LL and PDF and enables fast SDE sampling, offering an effective means to overcome the CoD. Three fitting methods, Score Matching (SM), Sliced Score Matching (SSM), and Score-PINN, are introduced, each contributing unique advantages in computational complexity, accuracy, and generality. The proposed score-based SDE solver operates in two stages: first, employing score matching or Score-PINN to acquire the score function; and second, solving the LL via an ordinary differential equation (ODE) using the obtained score function. Comparative evaluations across these methods showcase varying trade-offs. The proposed methodology is evaluated across diverse SDEs, including anisotropic Ornstein-Uhlenbeck processes, geometric Brownian motion, and Brownian motion with varying eigenspace. We also test various distributions, including Gaussian, Log-normal, Laplace, and Cauchy distributions. The numerical results demonstrate the score-based SDE solver’s stability, speed, and performance across different experimental settings, solidifying its potential as a solution to CoD for high-dimensional FP equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Neural network approaches for parameterized optimal control

Here, we consider numerical approaches for deterministic, finite-dimensional optimal control problems whose dynamics depend on unknown or uncertain parameters. We seek to amortize the solution over a set of relevant parameters in an offline stage to enable rapid decision-making and be able to react to changes in the parameter in the online stage. To tackle the curse of dimensionality arising when the state and/or parameter are high-dimensional, we represent the policy using neural networks. We compare two training paradigms: First, our model-based approach leverages the dynamics and definition of the objective function to learn the value function of the parameterized optimal control problem and obtain the policy using a feedback form. Second, we use actor-critic reinforcement learning to approximate the policy in a data-driven way. Using an example involving a two-dimensional convection-diffusion equation, which features high-dimensional state and parameter spaces, we investigate the accuracy and efficiency of both training paradigms. While both paradigms lead to a reasonable approximation of the policy, the model-based approach is more accurate and considerably reduces the number of PDE solves.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1989

Real-time approximate optimal guidance laws for the advanced launch system

An approach to optimal ascent guidance for a launch vehicle is developed using an expansion technique. The problem is to maximize the payload put into orbit subject to the equations of motion of a rocket over a rotating spherical earth. It is assumed that the thrust and gravitational forces dominate over the aerodynamic forces. It is shown that these forces can be separated by a small parameter epsilon, where epsilon is the ratio of the atmospheric scale height to the radius of the earth. The Hamilton-Jacobi-Bellman or dynamic programming equation is expanded in a series where the zeroth-order term (epsilon = 0) can be obtained in closed form. The zeroth-order problem is that of putting maximum payload into orbit subject to the equations of motion of a rocket in a vacuum over a flat earth. The neglected inertial and aerodynamic terms are included in higher order terms of the expansion, which are determined from the solution of first-order linear partial differential equations requiring only quadrature integrations. These quadrature integrations can be performed rapidly, so that real-time approximate optimization can be used to construct the launch guidance law.

Speyer, Jason L.↗
NASA NTRS1990

A real-time approximate optimal guidance law for flight in a plane

A real-time guidance scheme is presented for the problem of maximizing the payload into orbit subject to the equations of motion of a rocket over a nonrotating spherical earth. The flight is constrained to a path in the equatorial plane while reaching an orbital altitude at orbital injection speeds. The dynamics of the problem can be separated into primary and perturbation effects by a small parameter, epsilon, which is the ratio of the atmospheric scale height to the radius of the earth. The Hamilton-Jacobi-Bellman or dynamic programming equation is expanded in an asymptotic series where the zeroth-order term (epsilon = 0) can be obtained in closed form. The neglected perturbation terms are included in the higher-order terms of the expansion, which are determined from the solution of first-order linear partial differential equations requiring only integrations which are quadratures. The quadratures can be performed rapidly with emerging computer capability, so that real-time approximate optimization can be used to construct the launch guidance law. The application of this technique to flight in three-dimensions is made apparent from the solution presented.

Feeley, Timothy S.↗
NASA NTRS1991

Techniques for developing approximate optimal advanced launch system guidance

An extension to the authors' previous technique used to develop a real-time guidance scheme for the Advanced Launch System is presented. The approach is to construct an optimal guidance law based upon an asymptotic expansion associated with small physical parameters, epsilon. The trajectory of a rocket modeled as a point mass is considered with the flight restricted to an equatorial plane while reaching an orbital altitude at orbital injection speeds. The dynamics of this problem can be separated into primary effects due to thrust and gravitational forces, and perturbation effects which include the aerodynamic forces and the remaining inertial forces. An analytic solution to the reduced-order problem represented by the primary dynamics is possible. The Hamilton-Jacobi-Bellman or dynamic programming equation is expanded in an asymptotic series where the zeroth-order term (epsilon = 0) can be obtained in closed form.

Feeley, Timothy S.↗
DOE OSTI2025

Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Physics-Informed Neural Networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD) algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, Hamilton-Jacobi-Bellman (HJB), viscous Burgers’, Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

97 MATHEMATICS AND COMPUTING↗