Normalization and convergence acceleration for indirect optimization methods
Indirect methods of trajectory optimization, improving convergence through normalization of state data and Lagrange multipliers
Williams, S. D.↗
SEARCH · Search NASA
Results for “optimization methods”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Nonlinear Optimization Methods for Large-Scale Learning
Future advances in large-scale machine learning depend heavily on the availability of new and more powerful nonlinear optimization methods. The goal of this project is to address this challenge by designing new optimization algorithms that can deal with the scale, nonlinearity and stochasticity arising in machine learning models. Our research will provide the theoretical foundations for the new methods, and will involve intensive numerical testing on state-of-the-art machine learning systems. The project is organized into three interrelated projects that constitute a broad research agenda in the area of large-scale stochastic optimization for data science.
97 MATHEMATICS AND COMPUTING↗
Do optimization methods in deep learning applications matter?
With advances in deep learning, exponential data growth and increasing model complexity, developing efficient optimization methods are attracting much research attention. Several implementations favor the use of Conjugate Gradient (CG) and Stochastic Gradient Descent (SGD) as being practical and elegant solutions to achieve quick convergence, however, these optimization processes also present many limitations in learning across deep learning applications. Recent research is exploring higher-order optimization functions as better approaches, but these present very complex computational challenges for practical use. Comparing first and higher-order optimization functions, in this paper, our experiments reveal that Levemberg-Marquardt (LM) significantly supersedes optimal convergence but suffers from very large processing time increasing the training complexity of both, classification and reinforcement learning problems. Our experiments compare off-the-shelf optimization functions(CG, SGD, LM and L-BFGS) in standard CIFAR, MNIST, CartPole and FlappyBird experiments.The paper presents arguments on which optimization functions to use and further, which functions would benefit from parallelization efforts to improve pretraining time and learning rate convergence.
Ozyildirim, Buse Melis↗
Structural optimization using optimality criteria methods
Optimality criteria methods take advantage of some concepts as those of statically determinate or indeterminate structures, and certain variational principles of structural dynamics, to develop efficient algorithms for the sizing of structures that are subjected to stiffness-related constraints. Some of the methods and iterative strategies developed over the last decade for calculations of the Lagrange multipliers in stressand displacement-limited problems, as well as for satisfying the appropriate optimality criterion, are discussed. The application of these methods are illustrated by solving problems with stress and displacement constraints.
Khot, N. S.↗
Yet another parameter-free shape optimization method
The use of node coordinates as design variables in shape optimization offers a larger design space than computer-aided design (CAD)-based shape parameterizations. It also allows for the optimization of legacy designs, i.e., a finite element mesh from an existing design can be readily optimized to meet new performance requirements without involving a CAD model. However, it is well known that the node coordinate parameterization method is fraught with numerical difficulties, which makes it impractical to use. This has led to several of “parameter-free” shape optimization methods that seek the advantages and avoid the pitfalls of the naïve node coordinate parameterization method. These methods come in two main varieties: sensitivity filtering (or gradient smoothing) and consistent filtering. The latter is analogous to the density filter method used in topology optimization (TO). In this work, we use the PDE filter from TO and energy-based filters to implement consistent shape optimization filtering schemes easily and efficiently. Numerical experiments demonstrate that consistent methods are more robust than sensitivity filtering methods.
42 ENGINEERING↗
A Hierarchical Optimization Method for Electric Vertical Takeoff and Landing Aircraft Network Design
Electric vertical takeoff and landing aircraft (eVTOLs) are expected to serve urban air mobility in a station-to-station configuration, which makes the optimal network design of eVTOL stations a critical question to explore. Existing approaches often face limitations, such as the inability to interact station locations with demand or difficulty in finding the optimal solution for large study regions. Here, this paper first proposes a mathematical model to generate optimal eVTOL station locations while considering associated potential eVTOL demand, and then proposes a heuristic algorithm, Hierarchical Optimization MEthod (HOME), to efficiently solve the model. With a case study of Southern California, HOME was compared to 1) directly solving the original integer linear programming-based network design problem, and 2) employing the widely used genetic algorithm. Results suggest that HOME can find optimal solutions with limited computational resources. The proposed framework powered by HOME provides a computationally efficient way to support urban air mobility planning.
97 MATHEMATICS AND COMPUTING↗
Optimization methods, flux conserving methods for steady state Navier-Stokes equation
Navier-Stokes equation as discretized by new flux conserving method proposed by Chang and Scott results in the system: vector F(vector x) = 0, where F is a vector valued function. The Optimization method we use is based on Quasi-Newton methods: given a nonlinear function vector F(vector x) = 0, we solve, Delta(vector x) = -BF(vector x), where Delta(vector x) is the correction term and B is the inverse Jacobian of F(x). Then, iteratively, vector(x(sub (i+1))) = vector(x (sub i)) + alpha.Delta(vector x(sub i)), where alpha is a line search correction term determined by a line search routine. We use the BFCG's update the Jacobian matrix B(sub k) at each iteration. It is well known that B(sub k) approaches B(*) at the solution X(*). This algorithm has several advantages over the Newton-Raphson method. For example, we do not need to calculate the Jacobian matrix at each iteration which is computationally very expensive.
Adeyeye, John↗
Active cooling design for scramjet engines using optimization methods
A methodology for using optimization in designing metallic cooling jackets for scramjet engines is presented. The optimal design minimizes the required coolant flow rate subject to temperature, mechanical-stress, and thermal-fatigue-life constraints on the cooling-jacket panels, and Mach-number and pressure contraints on the coolant exiting the panel. The analytical basis for the methodology is presented, and results for the optimal design of panels are shown to demonstrate its utility.
Scotti, Stephen J.↗
Active cooling design for scramjet engines using optimization methods
A methodology for using optimization in designing metallic cooling jackets for scramjet engines is presented. The optimal design minimizes the required coolant flow rate subject to temperature, mechanical-stress, and thermal-fatigue-life constraints on the cooling-jacket panels, and Mach-number and pressure constraints on the coolant exiting the panel. The analytical basis for the methodology is presented, and results for the optimal design of panels are shown to demonstrate its utility.
Scotti, Stephen J.↗
Application of multidisciplinary optimization methods to the design of a supersonic transport
An optimization design method is discussed. This method is based on integrating existing disciplinary analysis and sensitivity analysis techniques by means of generalized sensitivity equations. A generic design system implementing this method is described. The system is being used to design the configuration and internal structure of a supersonic transport wing for optimum performance. This problem combines the disciplines of linear aerodynamics, structures, and performance. Initial results which include the disciplines of aerodynamics and structures in a conventional minimum weight design under static aeroelastic constraints are presented.
Barthelemy, J.-F. M.↗
Calculation of free-fall trajectories using numerical optimization methods.
An important problem in space flight is the calculation of trajectories for nonthrusting vehicles between fixed points in a given time. A new procedure based on Hamilton's principle for solving such two-point boundary-value problems is presented. It employs numerical optimization methods to perform the extremization required by Hamilton's principle. This procedure is applied to the calculation of an Earth-Moon trajectory. The results show that the initial guesses required to obtain an iteration procedure which converges are not critical and that convergence can be obtained to any predetermined degree of accuracy.
Hull, D. G.↗
A multifidelity Bayesian optimization method for inertial confinement fusion design
Due to their cost, experiments for inertial confinement fusion (ICF) heavily rely on numerical simulations to guide design. As simulation technology progresses, so too can the fidelity of models used to plan for new experiments. However, these high-fidelity models are by themselves insufficient for optimal experimental design, because their computational cost remains too high to efficiently and effectively explore the numerous parameters required to describe a typical experiment. Therefore, traditionally, ICF design has relied on low-fidelity modeling to initially identify potentially interesting design regions, which are then subsequently explored via selected high-fidelity modeling. In this paper, we demonstrate that this two-step approach can be insufficient: even for simple design problems, a two-step optimization strategy can lead high-fidelity searching toward incorrect regions and consequently waste computational resources on parameter regimes far away from the true optimal solution. We reveal that a primary cause of this behavior in ICF design problems is the presence of low-fidelity optima in different regions of the parameter space far away from high-fidelity optima. To address this issue, we propose an iterative multifidelity Bayesian optimization method based on Gaussian Process Regression that leverages both low- and high-fidelity models simultaneously. We demonstrate, using both two- and eight-dimensional ICF test problems, that our algorithm can effectively utilize both low-fidelity and high-fidelity models to refine the designs. This approach proves to be more efficient than relying solely on high-fidelity modeling for optimization.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
ZEUS: An Efficient GPU Optimization Method Integrating PSO, BFGS, and Automatic Differentiation
We introduce a novel, efficient computational method, ZEUS, for numerical optimization, and provide an open-source implementation. It has four key ingredients: (1) particle swarm optimization (PSO), (2) the use of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, (3) automatic differentiation (AD), and (4) GPUs. Our approach addresses the computational challenges inherent in high-dimensional, non-convex optimization problems. In the first phase of the algorithm, we get a potentially good set of starting points using PSO. Thereafter, we run BFGS independently in parallel from these starting points. BFGS is one of the best-performing algorithms for numerical optimization. However, it requires the gradient of the function being optimized. ZEUS integrates automatic differentiation into BFGS thus avoiding the need for the user to calculate derivatives explicitly. The use of GPUs allows ZEUS to speed up the calculations substantially. We carry out systematic studies to explore the trade-offs between the number of PSO iterations taken, starting points, and BFGS iteration depth. We show that a handful of iterations of PSO can improve global convergence when combined with BFGS. We also present performance studies using common test functions. The source code can be found at https://github.com/fnal-numerics/global-optimizer-gpu.
Soos, Dominik [Old Dominion U.]↗
Solution of nonlinear finite difference ocean models by optimization methods with sensitivity and observational strategy analysis
The paper studies with finite difference nonlinear circulation models the uncertainties in interesting flow properties, such as western boundary current transport, potential and kinetic energy, owing to the uncertainty in the driving surface boundary condition. The procedure is based upon nonlinear optimization methods. The same calculations permit quantitative study of the importance of new information as a function of type, region of measurement and accuracy, providing a method to study various observing strategies. Uncertainty in a model parameter, the bottom friction coefficient, is studied in conjunction with uncertain measurements. The model is free to adjust the bottom friction coefficient such that an objective function is minimized while fitting a set of data to within prescribed bounds. The relative importance of the accuracy of the knowledge about the friction coefficient with respect to various kinds of observations is then quantified, and the possible range of the friction coefficients is calculated.
Schroeter, Jens↗
Graph Analytics and Optimization Methods for Insights from the Uber Movement Data
In this work we leverage the Uber movement dataset for the Los Angeles (LA) area where partial TAZ to TAZ (Traffic Analysis Zone) trip time data is available. We first create a TAZ-TAZ network based on nearest neighbors and propose a model that allows us to complete the $(O-D)$ (Origin-Destination) travel time matrix, using optimization methods such as non-negative least squares. We apply these algorithms for several communities in the TAZ-TAZ network and present insights in the form of completed $(O-D)$ matrices and associated temporal trends. We qualify the error performance and scalability of our flows. We conclude by pointing out the directions in our ongoing work to improve the quality and scale of travel time estimation.
Visweswara Sathanur, Arun↗
Efficient optimization method for finding minimum energy paths of magnetic transitions
Here, efficient algorithms for the calculation of minimum energy paths of magnetic transitions are implemented within the geodesic nudged elastic band (GNEB) approach. While an objective function is not available for GNEB and a traditional line search can, therefore, not be performed, the use of limited memory Broyden–Fletcher–Goldfarb–Shanno (LBFGS) and conjugate gradient algorithms in conjunction with orthogonal spin optimization (OSO) approach is shown to greatly outperform the previously used velocity projection and dissipative Landau–Lifschitz dynamics optimization methods. The implementation makes use of energy weighted springs for the distribution of the discretization points along the path and this is found to improve performance significantly. The various methods are applied to several test problems using a Heisenberg-type Hamiltonian, extended in some cases to include Dzyaloshinskii–Moriya and exchange interactions beyond nearest neighbours. Minimum energy paths are found for magnetization reversals in a nano-island, collapse of skyrmions in two-dimensional layers and annihilation of a chiral bobber near the surface of a three-dimensional magnet. The LBFGS-OSO method is found to outperform the dynamics based approaches by up to a factor of 8 in some cases.
75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
Applications of numerical optimization methods to helicopter design problems: A survey
A survey of applications of mathematical programming methods is used to improve the design of helicopters and their components. Applications of multivariable search techniques in the finite dimensional space are considered. Five categories of helicopter design problems are considered: (1) conceptual and preliminary design, (2) rotor-system design, (3) airframe structures design, (4) control system design, and (5) flight trajectory planning. Key technical progress in numerical optimization methods relevant to rotorcraft applications are summarized.
Miura, H.↗
Applications of numerical optimization methods to helicopter design problems - A survey
A survey of applications of mathematical programming methods is used to improve the design of helicopters and their components. Applications of multivariable search techniques in the finite dimensional space are considered. Five categories of helicopter design problems are considered: (1) conceptual and preliminary design, (2) rotor-system design, (3) airframe structures design, (4) control system design, and (5) flight trajectory planning. Key technical progress in numerical optimization methods relevant to rotorcraft applications are summarized.
Miura, H.↗