Search NASA⌕ Search

SEARCH · Search NASA

Results for “Approximate dynamic programming problem”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Data-driven modeling and control of dynamical systems using Koopman and Perron-Frobenius operators

This dissertation studies the data-driven modeling and control problem of nonlinear systems by exploiting the linear operator theoretic framework involving Koopman and Perro-Frobenius operator. A systematic linear-operator based controller design procedure has been established, which can be used to solve a variety of nonlinear control problems, including feedback stabilization using control Lyapunov functions, optimal quadratic regulation using Koopman eigenfunctions and convex optimization formulation of optimal control problem using P-F and Koopman operator approximation. As the core of data-driven modeling, we first propose a new algorithm for the finite-dimensional approximation of the linear transfer Koopman and Perron-Frobenius operator from time-series data. We argue that the existing approach for the finite-dimensional approximation of these transfer operators such as Dynamic Mode Decomposition (DMD) and Extended Dynamic Mode Decomposition (EDMD) do not capture two important properties of these operators, namely positivity and Markov property. The algorithm we propose preserves these two properties. We call the proposed algorithm as naturally structured DMD (NSDMD) since it retains the inherent properties of these operators. Naturally structured DMD algorithm leads to a better approximation of the steady-state dynamics of the system regarding computing Koopman and Perron- Frobenius operator eigenfunctions and eigenvalues. However, preserving positivity property is critical for capturing the real transient dynamics of the system. This positivity property of the transfer operators and it's finite-dimensional approximation play an important role for controller and estimator design of nonlinear systems. To solve the feedback stabilization problem for nonlinear control systems, we tried to take advantage of the Koopman operator framework. The Koopman operator approach provides a linear representation for a nonlinear dynamical system and a bilinear representation for a nonlinear control system. The problem of feedback stabilization of a nonlinear control system is then transformed to the stabilization of a bilinear control system. We propose a control Lyapunov function (CLF)-based approach for the design of stabilizing feedback controllers for the bilinear system. The search for finding a CLF for the bilinear control system is formulated as a convex optimization problem. This leads to a schematic procedure for designing CLF-based stabilizing feedback controllers for the bilinear system and hence the original nonlinear system. Another advantage of the proposed controller design approach outlined in this dissertation is that it does not require explicit knowledge of system dynamics. In particular, the bilinear representation of a nonlinear control system in the Koopman eigenfunction space can be obtained from time-series data. Next, we study the optimal quadratic regulation problem for nonlinear systems. The linear operator theoretic framework involving the Koopman operator is used to lift the dynamics of nonlinear control system to an infinite-dimensional bilinear system. The optimal quadratic regulation problem for nonlinear system is formulated in terms of the finite-dimensional approximation of the bilinear system. A convex optimization-based approach is proposed for solving the quadratic regulator problem for bilinear system. We applied a variety of examples and compared the simulation results between our framework and conventional LQR control using linearized model. For more general optimal control problems, we provide a density-function based convex formulation for the optimal control problem of the nonlinear system. The convex formulation relies on the duality result in the stability theory of a dynamical system involving density function and Perron-Frobenius operator. The optimal control problem is formulated as an infinite-dimensional convex optimization program. The finite-dimensional approximation of the optimization problem relies on the recent advances made in the data-driven computation of the Koopman operator, which is dual to the Perron-Frobenius operator. Simulation results are presented to demonstrate the application of the developed framework.

Huang, Bowen↗
DOE OSTI2021

Deep Learning Explicit Differentiable Predictive Control Laws for Buildings

We present a differentiable predictive control (DPC) methodology for learning constrained control laws for unknown nonlinear systems. DPC poses an approximate solution to multiparametric programming problems emerging from explicit nonlinear model predictive control (MPC). Contrary to approximate MPC, DPC does not require supervision by an expert controller. Instead, a system dynamics model is learned from a small dataset of recorded observations of the perturbed system's dynamics and the control law is optimized offline by interaction with the learned system model. The DPC method is based on two sequential steps, i) system identification using a constrained neural state-space model, and ii) optimization of an explicit control law parametrized by another neural network in closed-loop simulation with the identified neural state-space model. The combination of a differentiable closed-loop system and penalty methods for constraint handling of system outputs and inputs allows us to optimize the control law's parameters directly by backpropagating economic MPC loss through the learned system model. By incorporating domain knowledge and leveraging established techniques from optimal control, our method leverages deep neural networks as nonlinear function approximators for system identification and control while avoiding concomitant costs of intractably large datasets, and computationally expensive over-parametrized models. The scalability, data efficiency, and constrained optimal control capability of the proposed DPC method are demonstrated in simulation using a multi-zone building emulator.

Drgona, Jan↗
DOE OSTI2023

Integrated Optimization of Powertrain Energy Management and Vehicle Motion Control for Autonomous Hybrid Electric Vehicles

Hybrid Electric Vehicles (HEVs) and autonomous vehicles have been widely studied recently for on-road transportation. In the study of autonomous HEVs, the control of the vehicle's external dynamics and powertrain dynamics are often treated separately. Optimizing these two problems together can significantly improve fuel economy. In this paper, an autonomous HEV following a leader is considered. First, the augmented model to integrate the abovementioned dynamics is presented. Second, the optimization problem is defined to find the optimum fuel consumption of the follower in pursuit of a leader in a drive cycle. A customized control strategy based on Approximate Dynamic Programming (ADP) is then explored in which the optimal cost-to-go at each time step is approximated using neural networks. Also, the accuracy of the optimization solution is enhanced by applying the concept of the reachable sets. At last, three case studies show that the examined integrated control strategy outperforms the one with the separated optimization method by an additional 7.4%, 4.6%, and 11.8% improvement in fuel consumption, respectively.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1979

Dynamic loads analysis system (DYLOFLEX) summary. Volume 1: Engineering formulation

The DYLOFLEX computer program system expands the aeroelastic cycle from that in the FLEXSTAB computer program system to include dynamic loads analyses involving active controls. Two aerodynamic options exist within DYLOFLEX. The analyst can formulate the problem with unsteady aerodynamics calculated using the doublet lattice method or with quasi-steady aerodynamics formulated from either FLEXSTAB or doublet lattice steady state aerodynamics with unsteady effects approximated by indicial lift growth functions. The equations of motion are formulated assuming straight and level flight and small motions. Loads are calculated using the force summation technique. DYLOFLEX consists of nine standalone programs which can be linked with each other by magnetic files used to transmit the required data between programs.

Miller, R. D.↗
NASA NTRS1989

An experiment-based comparative study of fuzzy logic control

An approach is presented to the control of a dynamic physical system through the use of approximate reasoning. The approach has been implemented in a program named POLE, and the authors have successfully built a prototype hardware system to solve the cartpole balancing problem in real-time. The approach provides a complementary alternative to the conventional analytical control methodology and is of substantial use when a precise mathematical model of the process being controlled is not available. A set of criteria for comparing controllers based on approximate reasoning and those based on conventional control schemes is furnished.

Berenji, Hamid R.↗
NASA NTRS1973

Solution of the stochastic control problem in unbounded domains.

Bellman's dynamic programming equation for the optimal index and control law for stochastic control problems is a parabolic or elliptic partial differential equation frequently defined in an unbounded domain. Existing methods of solution require bounded domain approximations, the application of singular perturbation techniques or Monte Carlo simulation procedures. In this paper, using the fact that Poisson impulse noise tends to a Gaussian process under certain limiting conditions, a method which achieves an arbitrarily good approximate solution to the stochastic control problem is given. The method uses the two iterative techniques of successive approximation and quasi-linearization and is inherently more efficient than existing methods of solution.

Robinson, P.↗
DOE OSTI2022

Efficient Region of Attraction Characterization for Control and Stabilization of Load Tap Changer Dynamics

In this article, we study the monitoring and control of long-term voltage stability considering load tap changer (LTC) dynamics. We show that under generic conditions, the LTC dynamics admit a unique stable equilibrium. For the stable equilibrium, we characterize an explicit inner approximation of its largest region of attraction (ROA). Compared to existing results, the computational complexity of the ROA characterization is drastically reduced. We propose a quadratically constrained linear program formulation for the ROA characterization problem. In addition, we formulate a second-order cone program for online voltage stability monitoring and control exploiting the proposed ROA characterization. Finally, we demonstrate the efficacy of the proposed formulations on the ROA characterization and stability monitoring and control using a standard IEEE test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Koopman Model Predictive Control for Eco-Driving of Automated Vehicles

In this paper, we develop a data-driven process for building a model predictive control (MPC) for eco-driving of automated vehicles. The process involves performing system identification in which the non-linear vehicle dynamics model is approximated by the Koopman operator, a linear predictor of higher state-dimension, in a data-driven framework. This approach allows us to formulate the eco-driving problem in a constrained quadratic program that leads to a computationally fast MPC. The MPC is then implemented as a closed-loop control of an electric vehicle in numerical simulations for demonstration.

autonomous vehicle↗
NASA NTRS1993

Control design variable linking for optimization of structural/control systems

A method is presented to integrate the design space of structural/control system optimization problems in the case of linear state feedback control. Conventional structural sizing variables and elements of the feedback gain matrix are both treated as strictly independent design variables in optimization by extending design variable linking concepts to the control gains. Several approximation concepts including new control design variable linking schemes are used to formulate the integrated structural/control optimization problem as a sequence of explicit nonlinear mathematical programming problems. Examples which involve a variety of behavior constraints, including constraints on dynamic stability, damped frequencies, control effort, peak transient displacement, acceleration, and control force limits, are effectively solved by using the method presented.

Jin, Ik Min↗
NASA NTRS1986

Multiple grid problems on concurrent-processing computers

Three computer codes were studied which make use of concurrent processing computer architectures in computational fluid dynamics (CFD). The three parallel codes were tested on a two processor multiple-instruction/multiple-data (MIMD) facility at NASA Ames Research Center, and are suggested for efficient parallel computations. The first code is a well-known program which makes use of the Beam and Warming, implicit, approximate factored algorithm. This study demonstrates the parallelism found in a well-known scheme and it achieved speedups exceeding 1.9 on the two processor MIMD test facility. The second code studied made use of an embedded grid scheme which is used to solve problems having complex geometries. The particular application for this study considered an airfoil/flap geometry in an incompressible flow. The scheme eliminates some of the inherent difficulties found in adapting approximate factorization techniques onto MIMD machines and allows the use of chaotic relaxation and asynchronous iteration techniques. The third code studied is an application of overset grids to a supersonic blunt body problem. The code addresses the difficulties encountered when using embedded grids on a compressible, and therefore nonlinear, problem. The complex numerical boundary system associated with overset grids is discussed and several boundary schemes are suggested. A boundary scheme based on the method of characteristics achieved the best results.

Eberhardt, D. S.↗
NASA NTRS1988

Dual adaptive control: Design principles and applications

The design of an actively adaptive dual controller based on an approximation of the stochastic dynamic programming equation for a multi-step horizon is presented. A dual controller that can enhance identification of the system while controlling it at the same time is derived for multi-dimensional problems. This dual controller uses sensitivity functions of the expected future cost with respect to the parameter uncertainties. A passively adaptive cautious controller and the actively adaptive dual controller are examined. In many instances, the cautious controller is seen to turn off while the latter avoids the turn-off of the control and the slow convergence of the parameter estimates, characteristic of the cautious controller. The algorithms have been applied to a multi-variable static model which represents a simplified linear version of the relationship between the vibration output and the higher harmonic control input for a helicopter. Monte Carlo comparisons based on parametric and nonparametric statistical analysis indicate the superiority of the dual controller over the baseline controller.

Mookerjee, Purusottam↗
NASA NTRS1974

Static, stability, and dynamic analysis of shells of revolution by numerical integration - A comparison

Recent innovations in digital computer technology have enabled engineers to analyze shell structures of complex configurations without unduly restrictive approximations. An attempt is made to compare the various programs now generally available from the point of view of the advantages of the relative technique utilized, as well as the programmed state of the art. Many of the comparisons are based on the sample problems solved by the STARS-2 system of programs. These examples indicate both the structural detail which can be analyzed by, and the analytical capabilities available in, the numerical shell-of-revolution programs. All advantages and differences are demonstrated by use of solutions for realistic shell problems in the areas of statics, stability, vibrations, and dynamic response of shells subjected to time-dependent loadings.

Svalbonas, V.↗
NASA NTRS1988

A feedback linearization approach to spacecraft control using momentum exchange devices

Recent developments in the area of nonlinear control theory have shown how coordiante changes in the state and input spaces can be used with nonlinear feedback to transform certain nonlinear ordinary differential equations into equivalent linear equations. These feedback linearization techniques are applied to resolve two problems arising in the control of spacecraft equipped with control moment gyroscopes (CMGs). The first application involves the computation of rate commands for the gimbals that rotate the individual gyroscopes to produce commanded torques on the spacecraft. The second application is to the long-term management of stored momentum in the system of control moment gyroscopes using environmental torques acting on the vehicle. An approach to distributing control effort among a group of redundant actuators is described that uses feedback linearization techniques to parameterize sets of controls which influence a specified subsystem in a desired way. The approach is adapted for use in spacecraft control with double-gimballed gyroscopes to produce an algorithm that avoids problematic gimbal configurations by approximating sets of gimbal rates that drive CMG rotors into desirable configurations. The momentum management problem is stated as a trajectory optimization problem with a nonlinear dynamical constraint. Feedback linearization and collocation are used to transform this problem into an unconstrainted nonlinear program. The approach to trajectory optimization is fast and robust. A number of examples are presented showing applications to the proposed NASA space station.

Dzielski, John Edward↗
NASA NTRS1976

Indirect synthesis of multidegree-of-freedom transient systems

The indirect synthesis method is developed and shown to be capable of leading a near-optimal design of multidegree-of-freedom and multidesign-element transient nonlinear dynamical systems. The basis of the approach is to select the open design parameters such that the response of the portion of the system being designed approximates the limiting performances solution. The limiting performance problem can be formulated as one of linear programming by replacing all portions of the system subject to transient disturbances by control forces and supposing that the remaining portions are linear as are the overall kinematic constraints. One then selects the design parameters that respond most closely to the limiting performance solution, which can be achieved by unconstrained curve-fitting techniques.

Chen, Y. H.↗
NASA NTRS1985

Probabilistic Structural Analysis Theory Development

The objective of the Probabilistic Structural Analysis Methods (PSAM) project is to develop analysis techniques and computer programs for predicting the probabilistic response of critical structural components for current and future space propulsion systems. This technology will play a central role in establishing system performance and durability. The first year's technical activity is concentrating on probabilistic finite element formulation strategy and code development. Work is also in progress to survey critical materials and space shuttle mian engine components. The probabilistic finite element computer program NESSUS (Numerical Evaluation of Stochastic Structures Under Stress) is being developed. The final probabilistic code will have, in the general case, the capability of performing nonlinear dynamic of stochastic structures. It is the goal of the approximate methods effort to increase problem solving efficiency relative to finite element methods by using energy methods to generate trial solutions which satisfy the structural boundary conditions. These approximate methods will be less computer intensive relative to the finite element approach.

Burnside, O. H.↗
NASA NTRS1986

A comparative analysis of static and dynamic load balancing strategies

The problem of uniformly distributing the load of a parallel program over a multiprocessor system was considered. A program was analyzed whose structure permits the computation of the optimal static solution. Then four strategies for load balancing were described and their performance compared. The strategies are: (1) the optimal static assignment algorithm which is guaranteed to yield the best static solution, (2) the static binary dissection method which is very fast but suboptimal, (3) the greedy algorithm, a static fully polynomial time approximation scheme, which estimates the optimal solution to arbitrary accuracy, and (4) the predictive dynamic load balancing heuristic which uses information on the precedence relationships within the program and outperforms any of the static methods. It is also shown that the overhead incurred by the dynamic heuristic is reduced considerably if it is started off with a static assignment provided by either of the three strategies.

Iqbal, M. Ashraf↗
NASA NTRS1986

Performance tradeoffs in static and dynamic load balancing strategies

The problem of uniformly distributing the load of a parallel program over a multiprocessor system was considered. A program was analyzed whose structure permits the computation of the optimal static solution. Then four strategies for load balancing were described and their performance compared. The strategies are: (1) the optimal static assignment algorithm which is guaranteed to yield the best static solution, (2) the static binary dissection method which is very fast but sub-optimal, (3) the greedy algorithm, a static fully polynomial time approximation scheme, which estimates the optimal solution to arbitrary accuracy, and (4) the predictive dynamic load balancing heuristic which uses information on the precedence relationships within the program and outperforms any of the static methods. It is also shown that the overhead incurred by the dynamic heuristic is reduced considerably if it is started off with a static assignment provided by either of the other three strategies.

Iqbal, M. A.↗