Search NASA⌕ Search

SEARCH · Search NASA

Results for “Newton optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

A Stochastic Quasi-Newton Method in the Absence of Common Random Numbers

We present Q-SASS, a quasi-Newton method for unconstrained stochastic optimization that does not rely on common random numbers. Most existing quasi-Newton approaches leverage common random numbers to construct second-order updates. However, motivated by challenges in variational quantum algorithms—where such coordination is not possible—we consider the setting in which function values and gradients are accessible only through noisy probabilistic zeroth- and first-order oracles, and no common random numbers can be exploited. We derive high-probability tail bounds on the iteration complexity of our algorithm for nonconvex, convex, and strongly convex (more generally, those satisfying the PL condition) objective functions. Finally, we demonstrate the empirical benefits of our quasi-Newton updating scheme on both synthetic and quantum chemistry problems.

Complexity bound↗

Forward and inverse modeling of fault transmissibility in subsurface flows

Characterizing physical properties of faults, such as their transmissibility, is crucial for performing predictive numerical simulation of subsurface flows, such as those encountered in petroleum engineering and remediation of subsurface contamination. Here, this paper provides a complete investigation of the inverse problem for fault transmissibility in subsurface flow models, under appropriate assumptions on fault structure. In particular, the following aspects are considered: 1) fault modeling and well-posedness of the forward problem; 2) finite element (FEM) discretizations of the forward problem and their rigorous a priori convergence analysis; 3) Well-posedness of the Bayesian inverse problem, FEM discretization of the infinite dimensional Bayesian inverse formulation, and its rigorous a priori analysis. Moreover, computation of the maximum a posteriori (MAP) point via fast inexact Newton-conjugate gradient optimization and a Laplace approximation of the Bayesian posterior are also presented. Numerical results illustrate the use of the proposed fault model in forward and inverse problems for subsurface flows in two dimensional domains with multiple faults.

97 MATHEMATICS AND COMPUTING↗

Ensemble Quasi-Newton HMC

We present a modification of the Hybrid Monte Carlo algorithm for tackling the critical slowing down of generating Markov chains of lattice gauge configurations towards the continuum limit. We propose a new method to exchange information within an ensemble of Markov chains, and use it to construct an approximate inverse Hessian matrix of the action inspired from quasi-Newton algorithms for optimization. The kinetic term of the molecular dynamics evolution includes the approximate Hessian for long distance couplings among the momenta. We show the result of applying the new algorithm to the U(1) gauge theory in two dimensions, and discuss our future plans.

Jin, Xiao-Yong↗

The terminal area automated path generation problem

The automated terminal area path generation problem in the advanced Air Traffic Control System (ATC), has been studied. Definitions, input, output and the interrelationships with other ATC functions have been discussed. Alternatives in modeling the problem have been identified. Problem formulations and solution techniques are presented. In particular, the solution of a minimum effort path stretching problem (path generation on a given schedule) has been carried out using the Newton-Raphson trajectory optimization method. Discussions are presented on the effect of different delivery time, aircraft entry position, initial guess on the boundary conditions, etc. Recommendations are made on real-world implementations.

Hsin, C.-C.↗

First MEM task IRMEO in IRAF

The first task for image restoration using the Maximum Entropy Method (MEM) in IRAF, called IRMEO, is described. The underlining algorithm is the approximate Newton method for optimization. The basic input images and parameters for deconvolution are described in some detail. Results of preliminary tests, including the number of iterations, required CPU time on a variety of computers, and deconvolved images are reported and compared with those from other deconvolution methods. The merits and limitations of this task are pointed out. The possible development of better MEM tasks on the basis of IRMEO is also discussed.

Wu, Nailong↗

What Information Theory Says about Bounded Rational Best Response

Probability Collectives (PC) provides the information-theoretic extension of conventional full-rationality game theory to bounded rational games. Here an explicit solution to the equations giving the bounded rationality equilibrium of a game is presented. Then PC is used to investigate games in which the players use bounded rational best-response strategies. Next it is shown that in the continuum-time limit, bounded rational best response games result in a variant of the replicator dynamics of evolutionary game theory. It is then shown that for team (shared-payoff) games, this variant of replicator dynamics is identical to Newton-Raphson iterative optimization of the shared utility function.

Wolpert, David H.↗

The DESC stellarator code suite Part 3: Quasi-symmetry optimization

The DESC stellarator optimization code takes advantage of advanced numerical methods to search the full parameter space much faster than conventional tools. Only a single equilibrium solution is needed at each optimization step thanks to automatic differentiation, which efficiently provides exact derivative information. A Gauss–Newton trust-region optimization method uses second-order derivative information to take large steps in parameter space and converges rapidly. With just-in-time compilation and GPU portability, high-dimensional stellarator optimization runs take orders of magnitude less computation time with DESC compared to other approaches. This paper presents the theory of the DESC fixed-boundary local optimization algorithm along with demonstrations of how to easily implement it in the code. Example quasi-symmetry optimizations are shown and compared to results from conventional tools. Three different forms of quasi-symmetry objectives are available in DESC, and their relative advantages are discussed in detail. In the examples presented, the triple product formulation yields the best optimization results in terms of minimized computation time and particle transport. This paper concludes with an explanation of how the modular code suite can be extended to accommodate other types of optimization problems.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Neural Generalized Predictive Control: A Newton-Raphson Implementation

An efficient implementation of Generalized Predictive Control using a multi-layer feedforward neural network as the plant's nonlinear model is presented. In using Newton-Raphson as the optimization algorithm, the number of iterations needed for convergence is significantly reduced from other techniques. The main cost of the Newton-Raphson algorithm is in the calculation of the Hessian, but even with this overhead the low iteration numbers make Newton-Raphson faster than other techniques and a viable algorithm for real-time control. This paper presents a detailed derivation of the Neural Generalized Predictive Control algorithm with Newton-Raphson as the minimization algorithm. Simulation results show convergence to a good solution within two iterations and timing data show that real-time control is possible. Comments about the algorithm's implementation are also included.

Soloway, Donald↗

Accurate Holdup Calculations with Predictive Modeling & Data Integration

In facilities that process special nuclear material (SNM) it is important to account accurately for the fissile material that enters and leaves the plant. Although there are many stages and processes through which materials must be traced and measured, the focus of this project is material that is “held-up” in equipment, pipes, and ducts during normal operation and that can accumulate over time into significant quantities. Accurately estimating the holdup is essential for proper SNM accounting (vis-à-vis nuclear non-proliferation), criticality and radiation safety, waste management, and efficient plant operation. Usually it is not possible to directly measure the holdup quantity and location, so these must be inferred from measured radiation fields, primarily gamma and less frequently neutrons. Current methods to quantify holdup, i.e. Generalized Geometry Holdup (GGH), primarily rely on simple source configurations and crude radiation transport models aided by ad hoc correction factors. This project seeks an alternate method of performing measurement-based holdup calculations using a predictive model that employs state-of-the-art radiation transport codes capable of accurately simulating such situations. Inverse and data assimilation methods use the forward transport model to search for a source configuration that best matches the measured data and simultaneously provide an estimate of the level of confidence in the correctness of such configuration. In this work the holdup problem is re-interpreted as an inverse problem that is under-determined, hence may permit multiple solutions. A probabilistic approach is applied to solving the resulting inverse problem. This approach rates possible solutions according to their plausibility given the measurements and initial information. This is accomplished through the use of Bayes’ Theorem that resolves the issue of multiple solutions by giving an estimate of the probability of observing each possible solution. To use Bayes’ Theorem, one must have a model y(x) that maps the state variables x (the solution in this case) to the measurements y. In this case, the unknown state variables are the configuration and composition of the heldup SNM. The measurements are the detector readings. Thus, the natural model is neutral-particle radiation transport where a wealth of computational tools exists for performing these simulations accurately and efficiently. The combination of predictive model and Bayesian inference forms the Data Integration with Modeled Predictions (DIMP) method that serves as foundation for this project. The cost functional describing the model-to-data misfit is computed via a norm created by the inverse of the covariance matrix of the model parameters and responses. Since the model y(x) for the holdup problem is nonlinear, a nonlinear optimization on Q is conducted via Newton-type iterative methods to find the optimal values of the model parameters x. This project comprised a collaboration between NC State University (NCSU), the University of South Carolina (USC), and Oak Ridge National Laboratory (ORNL). The project was originally proposed in seven main tasks with an eighth contingency task to be performed if time and funding permitted; in fact time did not permit commencement of the contingency task and it was not performed. The remaining tasks involved holdup analysis with gamma detection strategies and separately with neutrons based on coincidence counting. Early in the project, and upon consultation with experts in coincidence counting it became evident that this approach is not viable for holdup applications and this task was replaced with an alternative, but valuable investigation that was carried out by the USC partner. Nevertheless, the experimental 4 measurements at ORNL of both gamma and neutron sources for the purpose of constructing Detector Response Functions (DRFs) with the associated uncertainties were indeed completed.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Distributed MPC for Efficient Coordination of Storage and Renewable Energy Sources Across Control Areas

In electric power systems, multiple entities are responsible for ensuring an economic and reliable way of delivering power from producers to consumers. With the increase of variable renewable generation it is becoming increasingly important to take advantage of the individual entities' (and their areas') capabilities for balancing variability. Hence, in this paper, we employ and extend the approximate Newton directions method to optimally coordinate control areas leveraging storage available in one area to balance variable resources in another area with only minimal information exchange among the areas. The problem to be decomposed is a model predictive control problem including generation constraints, energy storage constraints, and AC power flow constraints. Singularity issues encountered when formulating the respective Newton-Raphson steps due to intertemporal constraints are addressed and extensions to the original decomposition method are proposed to improve the convergence rate and required communication of the method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗

Robust, multidimensional mesh motion based on Monge-Kantorovich equidistribution

Mesh-motion (r-refinement) grid adaptivity schemes are attractive due to their potential to minimize the numerical error for a prescribed number of degrees of freedom. However, a key roadblock to a widespread deployment of this class of techniques has been the formulation of robust, reliable mesh-motion governing principles, which (1) guarantee a solution in multiple dimensions (2D and 3D), (2) avoid grid tangling (or folding of the mesh, whereby edges of a grid cell cross somewhere in the domain), and (3) can be solved effectively and efficiently. In this study, we formulate such a mesh-motion governing principle, based on volume equidistribution via Monge-Kantorovich optimization (MK). In earlier publications [1] and [2], the advantages of this approach with regard to these points have been demonstrated for the time-independent case. In this study, we demonstrate that Monge-Kantorovich equidistribution can in fact be used effectively in a time-stepping context, and delivers an elegant solution to the otherwise pervasive problem of grid tangling in mesh-motion approaches, without resorting to ad hoc time-dependent terms (as in moving-mesh PDEs, or MMPDEs [3] and [4]). We explore two distinct r-refinement implementations of MK: the direct method, where the current mesh relates to an initial, unchanging mesh, and the sequential method, where the current mesh is related to the previous one in time. We demonstrate that the direct approach is superior with regard to mesh distortion and robustness. The properties of the approach are illustrated with a hyperbolic PDE, the advection of a passive scalar, in 2D and 3D. Velocity flow fields with and without flow shear are considered. Three-dimensional grid, time-step, and nonlinear tolerance convergence studies are presented which demonstrate the optimality of the approach.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A new method for optimizing multiple flyby trajectories

A new procedure has been developed which minimizes total impulsive Delta V for multiple flyby trajectories with constraints on flyby parameters and maneuver times. The method involves solving a bounds-constrained parameter optimization problem with a Newton algorithm utilizing analytic first and second derivatives. Each trajectory segment connecting consecutive maneuver points is found by first targeting from the preceding maneuver point to the parameters of the upcoming flyby and then propagating the resulting trajectory to the next maneuver point. Multi-conic techniques are used for trajectory propagation and for computation of the state transition matrix. This procedure has successfully optimized Galileo satellite tours containing up to 11 flybys.

Damario, L. A.↗