Results for “gradient based optimization” — Page 9

DOE OSTI2023

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

Fixed- and Free-Mode Flutter Derivatives during Aeroelastic Optimization

Aeroelastic flutter design derivatives may be approximated by ignoring the sensitivity of the structural mode shapes. This “fixed-mode” derivative is less-expensive to compute than the exact “free-mode” derivative (which accommodates the mode shape dependency), but also may provide inaccurate sensitivities for optimization. This work formulates both fixed- and free-mode flutter derivatives, and demonstrates the conditions under which they are equivalent. Two different types of fixed-mode derivatives are also developed in this work. Using a cantilevered-plate demonstration problem with both shape and sizing design parameters, the accuracy of the free-mode derivative and the two fixed-mode derivatives, are all demonstrated via gradient-based optimization.

Bret K. Stanford↗

NASA NTRSDate not supplied

Robust Trajectory Optimization Techniques Using a Sweeping Gradient Method and Linear Covariance Analysis

We present robust trajectory optimization techniques using a sweeping gradient method for ordinary differential equations with events (SGM) and linear covariance analysis (LinCov). SGM is a method for computing the gradient of trajectory analyses defined by performance indices over initial value problems with events with respect to static parameters. LinCov is an analytic technique for predicting stochastic behavior of dynamical systems. By combining SGM and LinCov, it is possible use efficient, off-the-shelf, gradient-based optimizers to solve robust optimal trajectory design problems. We describe the individual methods and some details on how they can be combined. Then we apply the combined techniques to a variety of orbital trajectory design problems to demonstrate its use, including minimum fuel transfer and mid-course correction burn scheduling.

Benjamin W L Margolis↗

DOE OSTI2025

Absence of Barren Plateaus and Scaling of Gradients in the Energy Optimization of Isometric Tensor Network States

Abstract Vanishing gradients can pose substantial obstacles for high-dimensional optimization problems. Here we consider energy minimization problems for quantum many-body systems with extensive Hamiltonians and finite-range interactions, which can be studied on classical computers or in the form of variational quantum eigensolvers on quantum computers. Barren plateaus correspond to scenarios where the average amplitude of the energy gradient decreases exponentially with increasing system size. This occurs, for example, for quantum neural networks and for brickwall quantum circuits when the depth increases polynomially in the system size. Here we prove that the variational optimization problems for matrix product states, tree tensor networks, and the multiscale entanglement renormalization ansatz are free of barren plateaus. The derived scaling properties for the gradient variance provide an analytical guarantee for the trainability of randomly initialized tensor network states (TNS) and motivate certain initialization schemes. In a suitable representation, unitary tensors that parametrize the TNS are sampled according to the uniform Haar measure. We employ a Riemannian formulation of the gradient based optimizations which simplifies the analytical evaluation.

Barthel, Thomas↗

NASA NTRS1995

Shape optimization of single- and two-element airfoils on multiblock grids

A multiblock, discrete sensitivity analysis method is used to couple a direct optimization method and a flow analysis method. The domain is divided into smaller subdomains for which the sensitivities are obtained separately. Then, an effective sensitivity equation is solved to complete the coupling of all the sensitivity information. The flow analysis is based on the thin-layer Navier-Stokes equations solved by an implicit, upwind-biased, finite-volume method. The method of feasible directions is used for the present gradient-based optimization approach. First, a transonic airfoil is optimized to investigate the behavior of the method in highly nonlinear flows as well as the effect of different blocking strategies on the procedure. A supercritical airfoil is produced from an initially symmetric airfoil with multiblocking affecting the path but not the final shape. Secondly, a two-element airfoil is shape optimized in subsonic flow to demonstrate the present method's capability of shaping aerodynamically interfering elements simultaneously. For a very low and a very high Reynolds number cases, the shape of the main airfoil and the flap are optimized to yield improved lift-to-drag ratios.

Lacasse, James M.↗

NASA NTRSDate not supplied

Sonic Boom Ground Noise Minimization Via the Adjoint Method

A multidisciplinary design optimization methodology to directly minimize the ground-level noise generated by the sonic boom of a high altitude supersonic body is presented. A Cartesian Euler flow solver is coupled with an atmospheric propagation tool to create a ground-noise analysis capability for supersonic bodies. Adjoint formulations for both the flow solver and propagation tool are also coupled to compute sensitivities of shape variations in the body in a highly efficient manner. A gradient-based optimizer is then introduced to forge a valuable design capability. Output-based mesh adaptation that is driven directly by ground-level noise is employed to increase accuracy and provide error estimation. The design method is demonstrated first on a simple axisymmetric body with few design variables to evaluate the efficacy of the optimization scheme. Guided by the results of this initial case, the problem is then repeated with somewhat different design variables to further demonstrate the capabilities of the design method. Finally, the method is applied to a real-world problem by optimizing control surface deflection settings of a low-boom aircraft to minimize ground noise while maintaining trimmed, level flight.

ARMD↗

NASA NTRS2000

Multidisciplinary Aerodynamic-Structural Shape Optimization Using Deformation (MASSOUD)

This paper presents a multidisciplinary shape parameterization approach. The approach consists of two basic concepts: (1) parameterizing the shape perturbations rather than the geometry itself and (2) performing the shape deformation by means of the soft object animation algorithms used in computer graphics. Because the formulation presented in this paper is independent of grid topology, we can treat computational fluid dynamics and finite element grids in a similar manner. The proposed approach is simple, compact, and efficient. Also, the analytical sensitivity derivatives are easily computed for use in a gradient-based optimization. This algorithm is suitable for low-fidelity (e.g., linear aerodynamics and equivalent laminated plate structures) and high-fidelity (e.g., nonlinear computational fluid dynamics and detailed finite element modeling analysis tools. This paper contains the implementation details of parameterizing for planform, twist, dihedral, thickness, camber, and free-form surface. Results are presented for a multidisciplinary design optimization application consisting of nonlinear computational fluid dynamics, detailed computational structural mechanics, and a simple performance module.

Samareh, Jamshid A.↗

NASA NTRS2000

Multidisciplinary Aerodynamic-Structural Shape Optimization Using Deformation (MASSOUD)

This paper presents a multidisciplinary shape parameterization approach. The approach consists of two basic concepts: (1) parameterizing the shape perturbations rather than the geometry itself and (2) performing the shape deformation by means of the soft object animation algorithms used in computer graphics. Because the formulation presented in this paper is independent of grid topology, we can treat computational fluid dynamics and finite element grids in the same manner. The proposed approach is simple, compact, and efficient. Also, the analytical sensitivity derivatives are easily computed for use in a gradient-based optimization. This algorithm is suitable for low-fidelity (e.g., linear aerodynamics and equivalent laminate plate structures) and high-fidelity (e.g., nonlinear computational fluid dynamics and detailed finite element modeling) analysis tools. This paper contains the implementation details of parameterizing for planform, twist, dihedral, thickness, camber, and free-form surface. Results are presented for a multidisciplinary application consisting of nonlinear computational fluid dynamics, detailed computational structural mechanics, and a simple performance module.

Samareh, Jamshid A.↗

NASA NTRSDate not supplied

Design Under Uncertainty with Design-Dependent Uncertain Variables

Uncertainty quantification (UQ) can provide a more robust understanding of a system, leading to better informed decisions earlier in the design process. The additional information that UQ provides can be leveraged during a design optimization process known as design under uncertainty that, when incorporated with multidisciplinary design and optimization, can become computationally infeasible due to the large number of responses required for meaningful results. Previous work addressed reducing the computational expense in design under uncertainty by incorporating analytic derivatives throughout polynomial chaos expansion. Although this allows design under uncertainty to be feasible for more systems, some multidisciplinary systems have design-dependent uncertain variables. This paper details an implementation of design dependent uncertain variables in a manner than preserves derivatives required for efficient gradient-based optimization throughout the process. Two analytic examples of design-dependent uncertain variables are given: the first transforms a uniform uncertain variable with one design variable and the second transforms a normal uncertain variable with two design variables. The polynomial chaos expansion (PCE) results are comparable to both the Monte Carlo (MC) results and the analytic results for the two examples. A case study that maximizes the lift-to-drag ratio with a design-dependence between the wing leading edge sweep angle and uncertain parameter percentage of laminar flow is compared to a MC and alternative optimization formulations. This paper demonstrates that design-dependent uncertain variables are valid and hold throughout PCE.

Joanna N Schmidt↗

NASA NTRSDate not supplied

Design Under Uncertainty with Design-Dependent Uncertain Variables

Uncertainty quantification (UQ) can provide a more robust understanding of a system, leading to better informed decisions earlier in the design process. The additional information that UQ provides can be leveraged during a design optimization process known as design under uncertainty that, when incorporated with multidisciplinary design and optimization, can become computationally infeasible due to the large number of responses required for meaningful results. Previous work addressed reducing the computational expense in design under uncertainty by incorporating analytic derivatives throughout polynomial chaos expansion. Although this allows design under uncertainty to be feasible for more systems, some multidisciplinary systems have design-dependent uncertain variables. This paper details an implementation of design dependent uncertain variables in a manner than preserves derivatives required for efficient gradient-based optimization throughout the process. Two analytic examples of design-dependent uncertain variables are given: the first transforms a uniform uncertain variable with one design variable and the second transforms a normal uncertain variable with two design variables. The polynomial chaos expansion (PCE) results are comparable to both the Monte Carlo (MC) results and the analytic results for the two examples. A case study that maximizes the lift-to-drag ratio with a design-dependence between the wing leading edge sweep angle and uncertain parameter percentage of laminar flow is compared to a MC and alternative optimization formulations. This paper demonstrates that design-dependent uncertain variables are valid and hold throughout PCE.

robust design↗

NASA NTRS2016

Parallel Aircraft Trajectory Optimization with Analytic Derivatives

Trajectory optimization is an integral component for the design of aerospace vehicles, but emerging aircraft technologies have introduced new demands on trajectory analysis that current tools are not well suited to address. Designing aircraft with technologies such as hybrid electric propulsion and morphing wings requires consideration of the operational behavior as well as the physical design characteristics of the aircraft. The addition of operational variables can dramatically increase the number of design variables which motivates the use of gradient based optimization with analytic derivatives to solve the larger optimization problems. In this work we develop an aircraft trajectory analysis tool using a Legendre-Gauss-Lobatto based collocation scheme, providing analytic derivatives via the OpenMDAO multidisciplinary optimization framework. This collocation method uses an implicit time integration scheme that provides a high degree of sparsity and thus several potential options for parallelization. The performance of the new implementation was investigated via a series of single and multi-trajectory optimizations using a combination of parallel computing and constraint aggregation. The computational performance results show that in order to take full advantage of the sparsity in the problem it is vital to parallelize both the non-linear analysis evaluations and the derivative computations themselves. The constraint aggregation results showed a significant numerical challenge due to difficulty in achieving tight convergence tolerances. Overall, the results demonstrate the value of applying analytic derivatives to trajectory optimization problems and lay the foundation for future application of this collocation based method to the design of aircraft with where operational scheduling of technologies is key to achieving good performance.

aircraft↗

NASA NTRSDate not supplied

Analysis and Optimization of Baseline Single Aisle Aircraft for Future Electrified Powertrain Flight Demonstrator Comparisons

The purpose of this study is to provide baseline single-aisle vehicles for future comparisons with NASA’s Electrified Powertrain Flight Demonstration (EPFD) turbofan-powered Vision Systems. State-of-the-art single-aisle transports with varying design capacities of 100 to 150 passengers are modeled using NASA Ames Research Center’s General Aviation Synthesis Program (GASP) as well as GASPy. GASPy is a modernized Python-based version of GASP built on the OpenMDAO framework to allow for future, efficient gradient-based optimization and coupled airframe-propulsion design. In order to meet projected NASA Aeronautics goals for 2035, advanced aircraft technologies must be incorporated into these vehicle systems. Methodology to parametrically infuse baseline aircraft models with advanced technologies simulating improvements in aerodynamics, structures, and propulsion systems is detailed, along with the results of technology sensitivity studies. Comparison of the baseline and advanced configurations will allow for future analysis of the benefits of future hybrid and fully electric aircraft concepts in the EPFD project, where fuel consumption and emissions will be modeled and assessed. This study has been conducted under the EPFD project to establish benchmark turbofan models and demonstrate System Analysis capabilities in multi-disciplinary aircraft design, analysis, and optimization for advanced turbofan concepts.

Carl J. Recine↗

NASA NTRS2001

Cryogenic Tank Structure Sizing With Structural Optimization Method

Structural optimization methods in MSC/NASTRAN are used to size substructures and to reduce the weight of a composite sandwich cryogenic tank for future launch vehicles. Because the feasible design space of this problem is non-convex, many local minima are found. This non-convex problem is investigated in detail by conducting a series of' analyses along a design line connecting two feasible designs. Strain constraint violations occur for some design points along the design line. Since MSC/NASTRAN uses gradient based optimization procedures, it does not guarantee that the lowest weight design can be found. In this study, a simple procedure is introduced to create a new starting point based on design variable values from previous optimization analyses. Optimization analysis using this new starting point can produce a lower weight design. Detailed inputs for setting up the MSC/NASTRAN optimization analysis and final tank design results are presented in this paper. Approaches for obtaining further weight reductions are also discussed.

Wang, J. T.↗

NASA NTRS1990

Multi-disciplinary optimization of aeroservoelastic systems

Efficient analytical and computational tools for simultaneous optimal design of the structural and control components of aeroservoelastic systems are presented. The optimization objective is to achieve aircraft performance requirements and sufficient flutter and control stability margins with a minimal weight penalty and without violating the design constraints. Analytical sensitivity derivatives facilitate an efficient optimization process which allows a relatively large number of design variables. Standard finite element and unsteady aerodynamic routines are used to construct a modal data base. Minimum State aerodynamic approximations and dynamic residualization methods are used to construct a high accuracy, low order aeroservoelastic model. Sensitivity derivatives of flutter dynamic pressure, control stability margins and control effectiveness with respect to structural and control design variables are presented. The performance requirements are utilized by equality constraints which affect the sensitivity derivatives. A gradient-based optimization algorithm is used to minimize an overall cost function. A realistic numerical example of a composite wing with four controls is used to demonstrate the modeling technique, the optimization process, and their accuracy and efficiency.

Karpel, Mordechay↗

NASA NTRS1992

Multidisciplinary optimization of aeroservoelastic systems using reduced-size models

Efficient analytical and computational tools for simultaneous optimal design of the structural and control components of aeroservoelastic systems are presented. The optimization objective is to achieve aircraft performance requirements and sufficient flutter and control stability margins with a minimal weight penalty and without violating the design constraints. Analytical sensitivity derivatives facilitate an efficient optimization process which allows a relatively large number of design variables. Standard finite element and unsteady aerodynamic routines are used to construct a modal data base. Minimum State aerodynamic approximations and dynamic residualization methods are used to construct a high accuracy, low order aeroservoelastic model. Sensitivity derivatives of flutter dynamic pressure, control stability margins and control effectiveness with respect to structural and control design variables are presented. The performance requirements are utilized by equality constraints which affect the sensitivity derivatives. A gradient-based optimization algorithm is used to minimize an overall cost function. A realistic numerical example of a composite wing with four controls is used to demonstrate the modeling technique, the optimization process, and their accuracy and efficiency.

Karpel, Mordechay↗

NASA NTRS2001

Cryogenic Tank Structure Sizing With Structural Optimization Method

Structural optimization methods in MSC /NASTRAN are used to size substructures and to reduce the weight of a composite sandwich cryogenic tank for future launch vehicles. Because the feasible design space of this problem is non-convex, many local minima are found. This non-convex problem is investigated in detail by conducting a series of analyses along a design line connecting two feasible designs. Strain constraint violations occur for some design points along the design line. Since MSC/NASTRAN uses gradient-based optimization procedures. it does not guarantee that the lowest weight design can be found. In this study, a simple procedure is introduced to create a new starting point based on design variable values from previous optimization analyses. Optimization analysis using this new starting point can produce a lower weight design. Detailed inputs for setting up the MSC/NASTRAN optimization analysis and final tank design results are presented in this paper. Approaches for obtaining further weight reductions are also discussed.

Wang, J. T.↗

NASA NTRS1999

A Novel Shape Parameterization Approach

This paper presents a novel parameterization approach for complex shapes suitable for a multidisciplinary design optimization application. The approach consists of two basic concepts: (1) parameterizing the shape perturbations rather than the geometry itself and (2) performing the shape deformation by means of the soft objects animation algorithms used in computer graphics. Because the formulation presented in this paper is independent of grid topology, we can treat computational fluid dynamics and finite element grids in a similar manner. The proposed approach is simple, compact, and efficient. Also, the analytical sensitivity derivatives are easily computed for use in a gradient-based optimization. This algorithm is suitable for low-fidelity (e.g., linear aerodynamics and equivalent laminated plate structures) and high-fidelity analysis tools (e.g., nonlinear computational fluid dynamics and detailed finite element modeling). This paper contains the implementation details of parameterizing for planform, twist, dihedral, thickness, and camber. The results are presented for a multidisciplinary design optimization application consisting of nonlinear computational fluid dynamics, detailed computational structural mechanics, performance, and a simple propulsion module.

Samareh, Jamshid A.↗

DOE OSTI2023

Training material models using gradient descent algorithms

High temperature design requires accurate constitutive models to describe material inelastic deformation and failure behavior. Oftentimes, calibrating accurate models devolves into the problem of fitting the model parameters against experimental test data. Here, we present the pyopmat package, an open source framework for calibrating constitutive models against experiment data subjected to various loading conditions using machine learning techniques. The package calculates the exact gradient of the model response with respect to the parameters using a combination of automatic differentiation and the adjoint method. Given this exact gradient, we compare the performance of several gradient-based optimization techniques in fitting realistic constitutive models against data. Here, we demonstrate the efficiency and accuracy of our package through example problems using both synthetic data, generated using known parameter sets, under monotonic and cyclic loading conditions and also with an example applying the techniques developed here to actual high temperature creep-fatigue test data.

36 MATERIALS SCIENCE↗