Fast Approximation of the Gauss--Newton Hessian Matrix for the Multilayer Perceptron
Not provided.
Mathematics↗
SEARCH · Search NASA
Results for “approximate Newton”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Newton's method applied to finite-difference approximations for the steady-state compressible Navier-Stokes equations
Finite-difference approximations for steady-state compressible Navier-Stokes equations, whose two spatial dimensions are written in generalized curvilinear coordinates and strong conservation-law form, are presently solved by means of Newton's method in order to obtain a lifting-airfoil flow field under subsonic and transonnic conditions. In addition to ascertaining the computational requirements of an initial guess ensuring convergence and the degree of computational efficiency obtainable via the approximate Newton method's freezing of the Jacobian matrices, attention is given to the need for auxiliary methods assessing the temporal stability of steady-state solutions. It is demonstrated that nonunique solutions of the finite-difference equations are obtainable by Newton's method in conjunction with a continuation method.
Bailey, Harry E.↗
Normalization and Implementation of Three Gravitational Acceleration Models
Unlike the uniform density spherical shell approximations of Newton, the consequence of spaceflight in the real universe is that gravitational fields are sensitive to the asphericity of their generating central bodies. The gravitational potential of an aspherical central body is typically resolved using spherical harmonic approximations. However, attempting to directly calculate the spherical harmonic approximations results in at least two singularities that must be removed to generalize the method and solve for any possible orbit, including polar orbits. Samuel Pines, Bill Lear, and Robert Gottlieb developed three unique algorithms to eliminate these singularities. This paper documents the methodical normalization of two of the three known formulations for singularity-free gravitational acceleration (namely, the Lear and Gottlieb algorithms) and formulates a general method for defining normalization parameters used to generate normalized Legendre polynomials and Associated Legendre Functions (ALFs) for any algorithm. A treatment of the conventional formulation of the gravitational potential and acceleration is also provided, in addition to a brief overview of the philosophical differences between the three known singularity-free algorithms.
Eckman, Randy A.↗
Normalization of Gravitational Acceleration Models
Unlike the uniform density spherical shell approximations of Newton, the con- sequence of spaceflight in the real universe is that gravitational fields are sensitive to the nonsphericity of their generating central bodies. The gravitational potential of a nonspherical central body is typically resolved using spherical harmonic approximations. However, attempting to directly calculate the spherical harmonic approximations results in at least two singularities which must be removed in order to generalize the method and solve for any possible orbit, including polar orbits. Three unique algorithms have been developed to eliminate these singularities by Samuel Pines [1], Bill Lear [2], and Robert Gottlieb [3]. This paper documents the methodical normalization of two1 of the three known formulations for singularity-free gravitational acceleration (namely, the Lear [2] and Gottlieb [3] algorithms) and formulates a general method for defining normalization parameters used to generate normalized Legendre Polynomials and ALFs for any algorithm. A treatment of the conventional formulation of the gravitational potential and acceleration is also provided, in addition to a brief overview of the philosophical differences between the three known singularity-free algorithms.
Eckman, Randy A.↗
A comparison of solar sail and ion drive trajectories for a Halley's comet rendezvous mission
According to the propulsion concept of solar sail spacecraft the thrust force is produced by the specular reflection of sunlight from a large, essentially flat, reflecting surface. The magnitude of this force is approximately 9 newtons for a perfectly reflecting sail with an area of 1 square kilometer oriented normal to the sunline at a distance of one astronomical unit from the sun. There exists a restriction in the types of orbit transfer trajectories which can be considered with this propulsion system. In the case of the second propulsion system being considered for the Halley's comet rendezvous mission, thrust is produced by the acceleration of ionized mercury atoms by an electric field. Power to the ion thrusters is supplied by lightweight solar arrays which can provide up to 100 kW of electrical power at a distance of 1 AU from the sun. Because of differing thrust constraints for the two propulsion systems, trajectories for a Halley's comet rendezvous mission are significantly different for the Ion Drive and Solar Sail spacecraft. Details concerning the trajectory characteristics are shown with the aid of a number of graphs.
Sauer, C. G., Jr.↗
Natural convection in steady solidification - Finite element analysis of a two-phase Rayleigh-Benard problem
Galerkin finite-element approximations and Newton's method for solving free boundary problems are combined with computer-implemented techniques from nonlinear perturbation analysis to study solidification problems with natural convection in the melt. The Newton method gives rapid convergence to steady state velocity, temperature and pressure fields and melt-solid interface shapes, and forms the basis for algebraic methods for detecting multiple steady flows and assessing their stability. The power of this combination is demonstrated for a two-phase Rayleigh-Benard problem composed of melt and solid in a veritical cylinder with the thermal boundary conditions arranged so that a static melt with a flat melt-solid interface is always a solution. Multiple cellular flows bifurcating from the static state are detected and followed as Rayleigh number is varied. Changing the boundary conditions to approach those appropriate for the vertical Bridgman solidification system causes imperfections that eliminate the static state. The flow structure in the Bridgman system is related to those for the Rayleigh-Benard system by a continuous evolution of the boundary conditions.
Chang, C. J.↗
A Preconditioning Method for Shape Optimization Governed by the Euler Equations
We consider a classical aerodynamic shape optimization problem subject to the compressible Euler flow equations. The gradient of the cost functional with respect to the shape variables is derived with the adjoint method at the continuous level. The Hessian (second order derivative of the cost functional with respect to the shape variables) is approximated also at the continuous level, as first introduced by Arian and Ta'asan (1996). The approximation of the Hessian is used to approximate the Newton step which is essential to accelerate the numerical solution of the optimization problem. The design space is discretized in the maximum dimension, i.e., the location of each point on the intersection of the computational mesh with the airfoil is taken to be an independent design variable. We give numerical examples for 86 design variables in two different flow speeds and achieve an order of magnitude reduction in the cost functional at a computational effort of a full solution of the analysis partial differential equation (PDE).
Arian, Eyal↗
A Fast Temporal Decomposition Procedure for Long-Horizon Nonlinear Dynamic Programming
We propose a fast temporal decomposition procedure for solving long-horizon nonlinear dynamic programs. The core of the procedure is sequential quadratic programming (SQP) that utilizes a differentiable exact augmented Lagrangian as the merit function. Within each SQP iteration, we approximately solve the Newton system using an overlapping temporal decomposition strategy. We show that the approximate search direction is still a descent direction of the augmented Lagrangian provided the overlap size and penalty parameters are suitably chosen, which allows us to establish the global convergence. Moreover, we show that a unit step size is accepted locally for the approximate search direction and further establish a uniform, local linear convergence over stages. This local convergence rate matches the rate of the recent Schwarz scheme (Na et al. 2022). However, the Schwarz scheme has to solve nonlinear subproblems to optimality in each iteration, whereas we only perform a single Newton step instead. Numerical experiments validate our theories and demonstrate the superiority of our method.
97 MATHEMATICS AND COMPUTING↗
Modified Interior Distance Functions (Theory and Methods)
In this paper we introduced and developed the theory of Modified Interior Distance Functions (MIDF's). The MIDF is a Classical Lagrangian (CL) for a constrained optimization problem which is equivalent to the initial one and can be obtained from the latter by monotone transformation both the objective function and constraints. In contrast to the Interior Distance Functions (IDF's), which played a fundamental role in Interior Point Methods (IPM's), the MIDF's are defined on an extended feasible set and along with center, have two extra tools, which control the computational process: the barrier parameter and the vector of Lagrange multipliers. The extra tools allow to attach to the MEDF's very important properties of Augmented Lagrangeans. One can consider the MIDFs as Interior Augmented Lagrangeans. It makes MIDF's similar in spirit to Modified Barrier Functions (MBF's), although there is a fundamental difference between them both in theory and methods. Based on MIDF's theory, Modified Center Methods (MCM's) have been developed and analyzed. The MCM's find an unconstrained minimizer in primal space and update the Lagrange multipliers, while both the center and the barrier parameter can be fixed or updated at each step. The MCM's convergence was investigated, and their rate of convergence was estimated. The extension of the feasible set and the special role of the Lagrange multipliers allow to develop MCM's, which produce, in case of nondegenerate constrained optimization, a primal and dual sequences that converge to the primal-dual solutions with linear rate, even when both the center and the barrier parameter are fixed. Moreover, every Lagrange multipliers update shrinks the distance to the primal dual solution by a factor 0 less than gamma less than 1 which can be made as small as one wants by choosing a fixed interior point as a 'center' and a fixed but large enough barrier parameter. The numericai realization of MCM leads to the Newton MCM (NMCM). The approximation for the primal minimizer one finds by Newton Method followed by the Lagrange multipliers update. Due to the MCM convergence, when both the center and the barrier parameter are fixed, the condition of the MDF Hessism and the neighborhood of the primal ninimizer where Newton method is 'well' defined remains stable. It contributes to both the complexity and the numerical stability of the NMCM.
Polyak, Roman A.↗
Advancing attenuation estimation through integration of the Hessian in multiparameter viscoacoustic full-waveform inversion
Accurate seismic attenuation models of subsurface structures not only enhance subsequent migration processes by improving fidelity, resolution, and facilitating amplitude-compliant angle gather generation but also provide valuable constraints on subsurface physical properties. Leveraging full-wavefield information, multiparameter viscoacoustic full-waveform inversion ( Q-FWI) simultaneously estimates seismic velocity and attenuation ( Q) models. However, a major challenge in Q-FWI is the contamination of crosstalk artifacts, where inaccuracies in the velocity model are mistakenly mapped to the inverted attenuation model. While incorporating the Hessian is expected to mitigate these artifacts, the explicit implementation is prohibitively expensive due to its formidable computational cost. In this study, we formulate and develop a Q-FWI algorithm via the Newton-conjugate gradient (CG) framework, where the search direction at each iteration is determined through an internal CG loop. In particular, the Hessian is integrated into each CG step in a matrix-free fashion using the second-order adjoint-state method. We find through synthetic experiments that our Newton-CG Q-FWI significantly mitigates crosstalk artifacts compared with the limited-memory Broyden-Fletcher-Goldfarb-Shanno method and the CG method, albeit with a notable computational cost. In the discussion of several key implementation details, we also determine the significance of the approximate Gauss-Newton Hessian, the second-order adjoint-state method, and the two-stage inversion strategy.
Geochemistry & Geophysics↗
A computationally-efficient method for flamelet calculations
A new open-source code for the simulation of the diffusion flamelet equations is proposed. Emphasis is placed on using an approximate Jacobian to reduce the computational cost of the matrix operations. Performance of the proposed solvers is tested by performing flamelet calculations with kinetic mechanisms of varying sizes. For the unity Lewis number equations, the present iterative Newton solver using an approximate Jacobian greatly outperforms direct Newton solvers using exact Jacobians. The computation cost scales linearly with the number of species, leading to a reduction in solution times by two orders of magnitude for mechanisms containing thousands of species. The applicability of the Jacobian approximations to the solution of the non-unity Lewis number flamelet equations is assessed. The approximations are generally inadequate to solve the full non-unity Lewis number equations but can be used in some applications depending on the balance of terms in the flamelet equations. As an example, the flamelet solver is applied to the study of sooting tendencies in laminar co-flow diffusion flames where modified non-unity Lewis number flamelet equations, previously shown to accurately reproduce experimentally-measured Yield Sooting Indices (YSI), are solved. Here, the accelerated flamelet solver is well suited for sensitivity analysis and uncertainty quantification with large detailed kinetic mechanisms, tasks for which the computational cost was previously prohibitive.
42 ENGINEERING↗
An optimization model for the US Air-Traffic System
A systematic approach for monitoring U.S. air traffic was developed in the context of system-wide planning and control. Towards this end, a network optimization model with nonlinear objectives was chosen as the central element in the planning/control system. The network representation was selected because: (1) it provides a comprehensive structure for depicting essential aspects of the air traffic system, (2) it can be solved efficiently for large scale problems, and (3) the design can be easily communicated to non-technical users through computer graphics. Briefly, the network planning models consider the flow of traffic through a graph as the basic structure. Nodes depict locations and time periods for either individual planes or for aggregated groups of airplanes. Arcs define variables as actual airplanes flying through space or as delays across time periods. As such, a special case of the network can be used to model the so called flow control problem. Due to the large number of interacting variables and the difficulty in subdividing the problem into relatively independent subproblems, an integrated model was designed which will depict the entire high level (above 29000 feet) jet route system for the 48 contiguous states in the U.S. As a first step in demonstrating the concept's feasibility a nonlinear risk/cost model was developed for the Indianapolis Airspace. The nonlinear network program --NLPNETG-- was employed in solving the resulting test cases. This optimization program uses the Truncated-Newton method (quadratic approximation) for determining the search direction at each iteration in the nonlinear algorithm. It was shown that aircraft could be re-routed in an optimal fashion whenever traffic congestion increased beyond an acceptable level, as measured by the nonlinear risk function.
Mulvey, J. M.↗
Error analysis of finite element solutions for postbuckled cylinders
A general method of error analysis and correction is investigated for the discrete finite-element results for cylindrical shell structures. The method for error analysis is an adaptation of the method of successive approximation. When applied to the equilibrium equations of shell theory, successive approximations derive an approximate continuous solution from the discrete finite-element results. The advantage of this continuous solution is that it contains continuous partial derivatives of an order higher than the basis functions of the finite-element solution. Preliminary numerical results are presented in this paper for the error analysis of finite-element results for a postbuckled stiffened cylindrical panel modeled by a general purpose shell code. Numerical results from the method have previously been reported for postbuckled stiffened plates. A procedure for correcting the continuous approximate solution by Newton's method is outlined.
Sistla, Rajaram↗
Rapid Design of Gravity Assist Trajectories
Several International Solar Terrestrial Physics (ISTP) missions require the design of complex gravity assisted trajectories in order to investigate the interaction of the solar wind with the Earth's magnetic field. These trajectories present a formidable trajectory design and optimization problem. The philosophy and methodology that enable an analyst to design and analyse such trajectories are discussed. The so called 'floating end point' targeting, which allows the inherently nonlinear multiple body problem to be solved with simple linear techniques, is described. The combination of floating end point targeting with analytic approximations with a Newton method targeter to achieve trajectory design goals quickly, even for the very sensitive double lunar swingby trajectories used by the ISTP missions, is demonstrated. A multiconic orbit integration scheme allows fast and accurate orbit propagation. A prototype software tool, Swingby, built for trajectory design and launch window analysis, is described.
Carrico, J.↗
Rapid design of gravity assist trajectories
Several International Solar-Terrestrial Physics (ISTP) missions require the design of complex gravity-assisted trajectories in order to investigate the interaction of the solar wind with the earth's magnetic field. These trajectories present a formidable trajectory design and optimization problem. This paper discusses the philosophy and methodology that enable an analyst to design and analyze such trajectories. The authors describe what is called 'floating end-point' targeting, which allows the inherently nonlinear multiple body problem to be solved with simple linear techniques. This paper demonstrates how floating end-point targeting combines analytic approximations with a Newton method targeter to achieve trajectory design goals quickly, even for the very sensitive double lunar swingby trajectories used by the ISTP missions. A Multiconic orbit integration scheme allows fast and accurate orbit propagation. This paper also describes a prototype software tool, Swingby, which has been built for trajectory design and launch window analysis.
Carrico, J.↗
Eigenvalue error analysis of viscously damped structures using a Ritz reduction method
The efficient solution of the eigenvalue problem that results from inserting passive dampers with variable stiffness and damping coefficients into a structure is addressed. Eigenanalysis of reduced models obtained by retaining a number of normal modes augmented with Ritz vectors corresponding to the static solutions resulting from the load patterns introduced by the dampers has been empirically shown to yield excellent approximations to the full eigenvalue problem. An analysis of this technique in the case of a single damper is presented. A priori and a posteriori error estimates are generated and tested on numerical examples. Comparison theorems with modally truncated models and a Markov parameter matching reduced-order model are derived. These theorems corroborate the heuristic that residual flexibility methods improve low-frequency approximation of the system. The analysis leads to other techniques for eigenvalue approximation. Approximate closed-form solutions are derived that include a refinement to eigenvalue derivative methods for approximation. An efficient Newton scheme is also developed. A numerical example is presented demonstrating the effectiveness of each of these methods.
Chu, Cheng-Chih↗
A Numerical Comparison of Barrier and Modified Barrier Methods for Large-Scale Bound-Constrained Optimization
When a classical barrier method is applied to the solution of a nonlinear programming problem with inequality constraints, the Hessian matrix of the barrier function becomes increasingly ill-conditioned as the solution is approached. As a result, it may be desirable to consider alternative numerical algorithms. We compare the performance of two methods motivated by barrier functions. The first is a stabilized form of the classical barrier method, where a numerically stable approximation to the Newton direction is used when the barrier parameter is small. The second is a modified barrier method where a barrier function is applied to a shifted form of the problem, and the resulting barrier terms are scaled by estimates of the optimal Lagrange multipliers. The condition number of the Hessian matrix of the resulting modified barrier function remains bounded as the solution to the constrained optimization problem is approached. Both of these techniques can be used in the context of a truncated-Newton method, and hence can be applied to large problems, as well as on parallel computers. In this paper, both techniques are applied to problems with bound constraints and we compare their practical behavior.
Nash, Stephen G.↗
Variability of a Stellar Corona on a Time Scale of Days: Evidence for Abundance Fractionation in an Emerging Coronal Active Region
Elemental abundance effects in active coronae have eluded our understanding for almost three decades, since the discovery of the first ionization potential (FIP) effect on the sun. The goal of this paper is to monitor the same coronal structures over a time interval of six days and resolve active regions on a stellar corona through rotational modulation. We report on four iso-phase X-ray spectroscopic observations of the RS CVn binary EI Eri with XMM-Newton, carried out approximately every two days, to match the rotation period of EI Eri. We present an analysis of the thermal and chemical structure of the EI Eri corona as it evolves over the six days. Although the corona is rather steady in its temperature distribution, the emission measure and FIP bias both vary and seem to be correlated. An active region, predating the beginning of the campaign, repeatedly enters into our view at the same phase as it rotates from beyond the stellar limb. As a result, the abundances tend slightly, but consistently, to increase for high FIP elements (an inverse FIP effect) with phase. We estimate the abundance increase of high FIP elements in the active region to be of about 75% over the coronal mean. This observed fractionation of elements in an active region on time scales of days provides circumstantial clues regarding the element enrichment mechanism of non-flaring stellar coronae.
Nordon, R.↗