Search NASA⌕ Search

SEARCH · Search NASA

Results for “approximate dynamic programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2010

Configuring Airspace Sectors with Approximate Dynamic Programming

In response to changing traffic and staffing conditions, supervisors dynamically configure airspace sectors by assigning them to control positions. A finite horizon airspace sector configuration problem models this supervisor decision. The problem is to select an airspace configuration at each time step while considering a workload cost, a reconfiguration cost, and a constraint on the number of control positions at each time step. Three algorithms for this problem are proposed and evaluated: a myopic heuristic, an exact dynamic programming algorithm, and a rollouts approximate dynamic programming algorithm. On problem instances from current operations with only dozens of possible configurations, an exact dynamic programming solution gives the optimal cost value. The rollouts algorithm achieves costs within 2% of optimal for these instances, on average. For larger problem instances that are representative of future operations and have thousands of possible configurations, excessive computation time prohibits the use of exact dynamic programming. On such problem instances, the rollouts algorithm reduces the cost achieved by the heuristic by more than 15% on average with an acceptable computation time.

Bloem, Michael↗
DOE OSTI2024

Approximate Dynamic Programming With Enhanced Off-Policy Learning for Coordinating Distributed Energy Resources

Herein this paper proposes an innovative approximate dynamic programming (ADP) method for distributed energy resource coordination with the loss of life of battery energy storage system (BESS) explicitly modeled. The dispatch policy is designed to account for both calendrical and cyclical aging effects on BESS, explicitly modeling the impacts of ambient temperature on BESS lifespan. The proposed ADP employs an adaptive critic method and enhanced off-policy deterministic policy gradient (DPG) strategy, addressing the limitations of the on-policy gradient-based ADP approaches, including inadequate exploration, low data usage, and computational complexity. In particular, a customized policy is proposed to guide the algorithm to explore some promising decisions and thereby improve exploration capability and learning efficiency compared to conventional DPG-based learning approaches, which may struggle to find a global optimum due to random noisy action-based exploration or require expert demonstration with extra effort. The proposed method is illustrated using the IEEE 123-node system and compared with the existing ADP methods to prove solution accuracy and demonstrate the effects of incorporating degradation models into control design. Case studies showed that the proposed ADP effectively coordinates DERs with a 10 times smaller optimization gap compared to existing methods, and the incorporation of the BESS life loss model ensures the expected lifespan and results in significant cost savings.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1983

A multi-item maintenance center inventory model for low-demand reparable items

In many military and commercial contexts, complex equipment undergoes scheduled maintenance overhauls at regular intervals during which all failed components are replaced. Failure to have replacements on hand for all failed parts requires emergency measures at premium cost. When reparable parts are highly reliable and expensive, both holding and shortage costs are high. This model determines the reparable parts inventory for a maintenance center under three alternative criteria: (1) maximizing job-completion rate subject to constraint on total holding costs, (2) minimizing total holding costs plus expected job noncompletion costs, and (3) minimizing total holding costs subject to a required minimum job-completion rate. Exact solutions may be obtained using dynamic programming. Approximate solutions, found easily by marginal analysis, have readily computed bounds on possible error. The solution methods for the three formulations are illustrated in a simple example.

Schaefer, M. K.↗
NASA NTRS2022

Hybrid data-driven and model-informed online tool wear detection in milling machines

Precision machining tool wear is responsible for low product throughput and quality. Monitoring the tool wear online is vital to prevent degradation in machining quality. However, direct real-time tool wear measurement is not practical. This paper presents residual-based anomaly detection models, combining a hybrid model comprised of a physics-based model and a data-driven model (a decision tree or a neural network) to predict signals of interest (e.g., power or forces) under nominal conditions, followed by Page’s cumulative sum test for detecting tool wear on-line using the computer numerical control machine measurements. The most informative features are ranked using dynamic programming and its approximation variants from real-time measurements and machine settings, such as the width of cut, depth of cut, feed rate and spindle speed, that serve as inputs to the predictive models. The baseline nominal model is incrementally updated with experimental data via a gradient boosted adaptation model to generate the residuals that account for discrepancies between the actual machine data under normal conditions and the baseline nominal model predictions. The hybrid model is validated against 20 Mazak milling machine experimental tests and one Haas run-to-failure experiment. The proposed anomaly detector is applied to synthetic data from simulations of the physics-based model at different operating conditions, measurement noise levels, and tool wear levels, and the methods were able to achieve an overall 92% accuracy in data with 1% noise. The anomaly detection methods based on hybrid model reduced the false alarms of either the data-driven or physical-based models alone, and are found to be capable of good online detection of tool wear.

Online anomaly detection↗
NASA NTRS1988

Dual adaptive control: Design principles and applications

The design of an actively adaptive dual controller based on an approximation of the stochastic dynamic programming equation for a multi-step horizon is presented. A dual controller that can enhance identification of the system while controlling it at the same time is derived for multi-dimensional problems. This dual controller uses sensitivity functions of the expected future cost with respect to the parameter uncertainties. A passively adaptive cautious controller and the actively adaptive dual controller are examined. In many instances, the cautious controller is seen to turn off while the latter avoids the turn-off of the control and the slow convergence of the parameter estimates, characteristic of the cautious controller. The algorithms have been applied to a multi-variable static model which represents a simplified linear version of the relationship between the vibration output and the higher harmonic control input for a helicopter. Monte Carlo comparisons based on parametric and nonparametric statistical analysis indicate the superiority of the dual controller over the baseline controller.

Mookerjee, Purusottam↗
NASA NTRS1994

Optimal active control for Burgers equations

A method for active fluid flow control based on control theory is discussed. Dynamic programming and fixed point successive approximations are used to accommodate the nonlinear control problem. The long-term goal of this project is to establish an effective method applicable to complex flows such as turbulence and jets. However, in this report, the method is applied to stochastic Burgers equation as an intermediate step towards this goal. Numerical results are compared with those obtained by gradient search methods.

Ikeda, Yutaka↗
NASA NTRS1986

Free vibration characteristics of multiple load path blades by the transfer matrix method

The determination of free vibrational characteristics is basic to any dynamic design, and these characteristics can form the basis for aeroelastic stability analyses. Conventional helicopter blades are typically idealized as single-load-path blades, and the transfer matrix method is well suited to analyze such blades. Several current helicopter dynamic programs employ transfer matrices to analyze the rotor blades. In this paper, however, the transfer matrix method is extended to treat multiple-load-path blades, without resorting to an equivalent single-load-path approximation. With such an extension, these current rotor dynamic programs which employ the transfer matrix method can be modified with relative ease to account for the multiple load paths. Unlike the conventional blades, the multiple-load-path blades require the introduction of the axial degree-of-freedom into the solution process to account for the differential axial displacements of the different load paths. The transfer matrix formulation is validated through comparison with the finite-element solutions.

Murthy, V. R.↗
NASA NTRS1988

A forward method for optimal stochastic nonlinear and adaptive control

A computational approach is taken to solve the optimal nonlinear stochastic control problem. The approach is to systematically solve the stochastic dynamic programming equations forward in time, using a nested stochastic approximation technique. Although computationally intensive, this provides a straightforward numerical solution for this class of problems and provides an alternative to the usual dimensionality problem associated with solving the dynamic programming equations backward in time. It is shown that the cost degrades monotonically as the complexity of the algorithm is reduced. This provides a strategy for suboptimal control with clear performance/computation tradeoffs. A numerical study focusing on a generic optimal stochastic adaptive control example is included to demonstrate the feasibility of the method.

Bayard, David S.↗
NASA NTRSDate not supplied

Adaptive Optimization for System Performance and Combined Bernstein Polynomial, Optimal Reciprocal Collision Avoidance, Differential Dynamic Programming for Trajectory Replanning and Collision Avoidance for UAM Vehicles

The emerging urban air mobility (UAM) sector in aerospace is driving development of unconventional multi-modal vehicle configurations and autonomous flight. The combination of multi-modal vehicle dynamics, complex environment, requirements to deal with flight contingencies in an efficient and safe manner, as well as necessity for precise trajectory following and performance, are the driving influence behind adaptive optimization for system performance. We are interested in trajectory optimization algorithm that would system parameter estimation and identifying the optimal switching time between modes of hybrid dynamical systems. This presentation discusses a parameterized optimal control trajectory optimization algorithm that is an extended and generalized version of Differential Dynamic Programming (DDP), titled Parameterized Differential Dynamic Programming (PDDP). DDP is an efficient trajectory optimization algorithm relying on second order approximations of a system’s dynamics and cost function and has recently been applied to optimize systems with time invariant parameters. Experiments are presented applying PDDP to solve model predictive control (MPC) and moving horizon estimation (MHE) tasks simultaneously. In particular, PDDP is used to determine the optimal transition point between flight regimes of a complex urban air mobility (UAM) class vehicle exhibiting multiple phases of flight and to identify and compensate for actuation faults.

optimization↗
NASA NTRS1986

Towards a general object-oriented software development methodology

An object is an abstract software model of a problem domain entity. Objects are packages of both data and operations of that data (Goldberg 83, Booch 83). The Ada (tm) package construct is representative of this general notion of an object. Object-oriented design is the technique of using objects as the basic unit of modularity in systems design. The Software Engineering Laboratory at the Goddard Space Flight Center is currently involved in a pilot program to develop a flight dynamics simulator in Ada (approximately 40,000 statements) using object-oriented methods. Several authors have applied object-oriented concepts to Ada (e.g., Booch 83, Cherry 85). It was found that these methodologies are limited. As a result a more general approach was synthesized with allows a designer to apply powerful object-oriented principles to a wide range of applications and at all stages of design. An overview is provided of this approach. Further, how object-oriented design fits into the overall software life-cycle is considered.

Seidewitz, ED↗
NASA NTRS1983

Method for compression testing of composite materials at high strain rates

A method is presented for testing composite materials in compression at strain rates up to approximately 500 per s. The method uses a thin ring specimen (4 in. in diameter, 1 in. wide, six-eight plies thick) loaded dynamically by an external pressure pulse applied explosively through a liquid. Strains in the specimen and in a steel calibration ring are recoorded with a digital processing oscilloscope. Results are plotted by an x-y plotter in the form of a dynamic stress-strain curve. Data analysis is based on a numerical solution of the equation of motion. A computer program is used which involves smoothing and approximation of the strain magnitude, strain rate, and strain acceleration. Dynamic stress-strain curves obtained for 0-deg and 90-deg specimens of two graphite/epoxy composites are presented.

Daniel, I. M.↗
NASA NTRS1968

Generalized Newton-Raphson trajectory optimization-generator 1

Computer program constructs a sequence of optimal solutions to dynamically-approximate linear equations. Specification of the number and type of subarcs in the optimal solution allows simultaneous satisfaction of all switching criteria.

Cope, D. D.↗
NASA NTRS1990

Forward Stochastic Nonlinear Adaptive Control Method

New method of computation for optimal stochastic nonlinear and adaptive control undergoing development. Solves systematically stochastic dynamic programming equations forward in time, using nested-stochastic-approximation technique. Main advantage, simplicity of programming and reduced complexity with clear performance/computation trade-offs.

Bayard, David S.↗
NASA NTRS1993

Autonomous vehicle motion control, approximate maps, and fuzzy logic

Progress on research on the control of actions of autonomous mobile agents using fuzzy logic is presented. The innovations described encompass theoretical and applied developments. At the theoretical level, results of research leading to the combined utilization of conventional artificial planning techniques with fuzzy logic approaches for the control of local motion and perception actions are presented. Also formulations of dynamic programming approaches to optimal control in the context of the analysis of approximate models of the real world are examined. Also a new approach to goal conflict resolution that does not require specification of numerical values representing relative goal importance is reviewed. Applied developments include the introduction of the notion of approximate map. A fuzzy relational database structure for the representation of vague and imprecise information about the robot's environment is proposed. Also the central notions of control point and control structure are discussed.

Ruspini, Enrique H.↗