A stochastic optimal control problem and its applications.
Stochastic optimal control problem solution by dynamic programming and relation to interplanetary guidance
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Stochastic optimal control problem solution by dynamic programming and relation to interplanetary guidance
In this paper we study an optimization problem arising from a sterilization process for packaged foods by using a microwave heating method. The goal of the optimal control is to find the optimal frequency function such that the temperature profile at the final stage has a relative uniform distribution in the food product. The underlying state variables are electric, magnetic fields and temperature which satisfy the coupled nonlinear Maxwell’s system and a nonlinear heat equation. The control variable for the system is chosen to be the electric frequency function. We show that there exists an optimal frequency which minimizes the cost functional. Moreover, an optimality condition is also derived.
This letter formulates and solves a periodic trajectory optimization problem for a tethered underwater kite. The goal is to maximize the average mechanical power harvested by the kite. The type of kite considered in this letter extracts electricity from ocean currents by moving cross-current as its reel away from its base station, and consumes electricity to reel back. The problem of optimizing this kite’s trajectory is challenging due to the high dimensionality and nonlinearity of its dynamics. To tackle this challenge, the literature often separates the problem into two subproblems focusing on optimizing the cross-current and the reel-in/reel-out components of the trajectory, respectively, which may be sub-optimal. In contrast, this letter solves for the combined cross-current and reel-in/reel-out trajectory by linearizing the dynamics of the kite around a zero-power reference equilibrium trajectory in spherical coordinates. This allows the trajectory optimization problem to be solved analytically for simple sinusoidal input perturbations from equilibrium. Here we use linear quadratic regulation to enable the nonlinear kite model to track the optimized trajectory. The result is a computationally efficient approach that achieves an attractive Loyd factor of 19.9%, while providing important insights into the nature of the optimal trajectory.
The effectiveness of viscous elements in introducing damping in a structure is a function of several variables, including their number, their location in the structure, and their physical properties. In this paper several optimization problems are posed to optimize these variables. The paper investigates various metrics to define the optimization problem, and compares the damping profiles that are obtained. Both discrete and continuous optimization problems are formulated and solved, corresponding, respectively, to the problems of placement of damping elements and to the tuning of their parameters. The paper particularly emphasizes techniques to make feasible the large scale problems resulting from the optimization formulations. Numerical results involving a lightly damped tested structure are presented.
We consider a non-quadratic optimal control problem associated to a nonlinear elliptic variational inequality, where the obstacle is the control itself. We show that, fixed a desired profile, there exists an optimal solution which is not far from it. Detailed characterizations of the optimal solution are given, also in terms of approximating problems.
A multiobjective optimization problem (MOOP) is an optimization problem in which multiple objectives are optimized simultaneously. The goal of a MOOP is to find solutions that describe the tradeoff between these (potentially conflicting) objectives. Such a tradeoff surface is called the Pareto front. Real-world MOOPs may also involve constraints – additional hard rules that every solution must adhere to. In a multiobjective simulation optimization problem, the objectives are derived from the outputs of one or more computationally expensive simulations. Such problems are ubiquitous in science and engineering.
Problems encountered by aerospace designers in attempting to optimize whole aircraft are discussed, along with possible solutions. Large scale optimization, as opposed to component-by-component optimization, is hindered by computational costs, software inflexibility, concentration on a single, rather than trade-off, design methodology and the incompatibility of large-scale optimization with single program, single computer methods. The software problem can be approached by placing the full analysis outside of the optimization loop. Full analysis is then performed only periodically. Problem-dependent software can be removed from the generic code using a systems programming technique, and then embody the definitions of design variables, objective function and design constraints. Trade-off algorithms can be used at the design points to obtain quantitative answers. Finally, decomposing the large-scale problem into independent subproblems allows systematic optimization of the problems by an organization of people and machines.
We study the existence of optimal solutions for a class of infinite horizon nonconvex autonomous discrete-time optimal control problems. This class contains optimal control problems without discounting arising in economic dynamics which describe a model with a nonconcave utility function.
The main idea of this report is to give an overview of the problems and difficulties that arise in solving optimal control problems with switching points. A brief discussion of existing optimality conditions is given and a numerical approach for solving the multipoint boundary value problems associated with the first-order necessary conditions of optimal control is presented. Two real-life aerospace optimization problems are treated explicitly. These are altitude maximization for a sounding rocket (Goddard Problem) in the presence of a dynamic pressure limit, and range maximization for a supersonic aircraft flying in the vertical, also in the presence of a dynamic pressure limit. In the second problem singular control appears along arcs with active dynamic pressure limit, which in the context of optimal control, represents a first-order state inequality constraint. An extension of the Generalized Legendre-Clebsch Condition to the case of singular control along state/control constrained arcs is presented and is applied to the aircraft range maximization problem stated above. A contribution to the field of Jacobi Necessary Conditions is made by giving a new proof for the non-optimality of conjugate paths in the Accessory Minimum Problem. Because of its simple and explicit character, the new proof may provide the basis for an extension of Jacobi's Necessary Condition to the case of the trajectories with interior point constraints. Finally, the result that touch points cannot occur for first-order state inequality constraints is extended to the case of vector valued control functions.
Quantum annealing (QA) has the potential to significantly improve solution quality and reduce time complexity in solving combinatorial optimization problems compared to classical optimization methods. However, due to the limited number of qubits and their connectivity, the QA hardware did not show such an advantage over classical methods in past benchmarking studies. Recent advancements in QA with more than 5000 qubits, enhanced qubit connectivity, and the hybrid architecture promise to realize the quantum advantage. Here, we use a quantum annealer with state-of-the-art techniques and benchmark its performance against classical solvers. To compare their performance, we solve over 50 optimization problem instances represented by large and dense Hamiltonian matrices using quantum and classical solvers. The results demonstrate that a state-of-the-art quantum solver has higher accuracy (~0.013%) and a significantly faster problem-solving time (~6561×) than the best classical solver. Our results highlight the advantages of leveraging QA over classical counterparts, particularly in hybrid configurations, for achieving high accuracy and substantially reduced problem solving time in large-scale real-world optimization problems.
A feed forward neural network based optimal control synthesis is presented for solving optimal control problems with control and state constraints. The paper extends adaptive critic neural network architecture proposed by [5] to the optimal control problems with control and state constraints. The optimal control problem is transcribed into a nonlinear programming problem which is implemented with adaptive critic neural network. The proposed simulation method is illustrated by the optimal control problem of nitrogen transformation cycle model. Results show that adaptive critic based systematic approach holds promise for obtaining the optimal control with control and state constraints.
For optimal power flow problems with chance constraints, a particularly effective method is based on a fixed point iteration applied to a sequence of deterministic power flow problems. However, a priori, the convergence of such an approach is not necessarily guaranteed. Here this article analyses the convergence conditions for this fixed point approach, and reports numerical experiments including for large IEEE networks.
Discontinuities occurring in closed loop optimal control problems when using maximum principle
In many applications, one can only access the inexact gradients and inexact hessian times vector products. Thus it is essential to consider algorithms that can handle such inexact quantities with a guaranteed convergence to solution. An inexact adaptive and provably convergent semismooth Newton method is considered to solve constrained optimization problems. In particular, dynamic optimization problems, which are known to be highly expensive, are the focus. A memory efficient semismooth Newton algorithm is introduced for these problems. The source of efficiency and inexactness is the randomized matrix sketching. Further, applications to optimization problems constrained by partial differential equations are also considered.
High-fidelity aeroelastic optimization capabilities are becoming more common in recent years. In this field, researchers typically present results on the design problems of greatest interest to them. The lack of consistency with respect to baseline configurations, design variables, objectives and constraints, etc., makes it difficult to compare approaches and creates an obstacle for the community to work together to advance the state of the art. The High-Fidelity Aeroelastic Optimization Benchmark Working Group has been established to create and study a series of benchmark aeroelastic optimization problems that the community can utilize to compare methods on the same optimization problem. This paper presents optimization results from NASA Langley Research Center for the first benchmark case of this series with analysis based on MPhys, a multiphysics library for OpenMDAO, and FUN3D, an unstructured computational fluid dynamics suite of tools. For the optimization case presented, it is demonstrated that a progression of increasing aerodynamic model fidelity reduces the wall time required to complete the optimization.
High-fidelity aeroelastic optimization capabilities are becoming more common in recent years. In this field, researchers typically present results on the design problems of greatest interest to them. The lack of consistency with respect to baseline configurations, design variables, objectives and constraints, etc., makes it difficult to compare approaches and creates an obstacle for the community to work together to advance the state of the art. The High-Fidelity Aeroelastic Optimization Benchmark Working Group has been established to create and study a series of benchmark aeroelastic optimization problems that the community can utilize to compare methods on the same optimization problem. This paper presents optimization results from NASA Langley Research Center for the first benchmark case of this series with analysis based on MPhys, a multiphysics library for OpenMDAO, and FUN3D, an unstructured computational fluid dynamics suite of tools. For the optimization case presented, it is demonstrated that a progression of increasing aerodynamic model fidelity reduces the wall time required to complete the optimization.
We present a robust optimization algorithm for the design of electromagnetic coils that generate vacuum magnetic fields with nested flux surfaces and precise quasi-symmetry. The method is based on a bilevel optimization problem, where the outer coil optimization is constrained by a set of inner least squares optimization problems whose solutions describe magnetic surfaces. The outer optimization objective targets coils that generate a field with nested magnetic surfaces and good quasi-symmetry. The inner optimization problems identify magnetic surfaces when they exist, and approximate surfaces in the presence of magnetic islands or chaos. We show that this formulation can be used to heal islands and chaos, thus producing coils that result in magnetic fields with precise quasi-symmetry. We show that the method can be initialized with coils from the traditional two-stage coil design process, as well as coils from a near-axis expansion optimization. We present a numerical example where island chains are healed and quasi-symmetry is optimized up to surfaces with aspect ratio 6. Another numerical example illustrates that the aspect ratio of nested flux surfaces with optimized quasi-symmetry can be decreased from 6 to approximately 4. In conclusion, the last example shows that our approach is robust and a cold-start using coils from a near-axis expansion optimization.
The trusted inertial terrain-aided navigation (TITAN) algorithm leverages an airborne vertical synthetic aperture radar to measure the range to the closest ground points along several prescribed iso-Doppler contours. These TITAN minimum-range, prescribed-Doppler measurements are the result of a constrained nonlinear optimization problem whose optimization function and constraints both depend on the radar position and velocity. Owing to the complexity of this measurement definition, analysis of the TITAN algorithm is lacking in prior work. This publication offers such an analysis, making the following three contributions: (1) an analytical solution to the TITAN constrained optimization measurement problem, (2) a derivation of the TITAN measurement function Jacobian, and (3) a derivation of the Cramér-Rao lower bound on the estimated position and velocity error covariance. These three contributions are verified via Monte Carlo simulations over synthetic terrain, which further reveal two remarkable properties of the TITAN algorithm: (1) the along-track positioning errors tend to be smaller than the cross-track positioning errors, and (2) the cross-track positioning errors are independent of the terrain roughness.