Rayleigh-Gauss-Newton optimization with enhanced sampling for variational Monte Carlo
Not Available
Webber, Robert J. (ORCID:0000000182866315)↗
SEARCH · Search NASA
Results for “Newton optimization”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Economical quasi-Newton unitary optimization of electronic orbitals
State-of-the-art quasi-Newton unitary optimizer with trust-region (QUOTR) is nearly as efficient as the reference method for simple cases and is more robust for complex cases such as proteins with zero HOMO–LUMO gap in semilocal Kohn–Sham DFT.
Chemistry↗
Reducing measurement costs by recycling the Hessian in adaptive variational quantum algorithms
Abstract Adaptive protocols enable the construction of more efficient state preparation circuits in variational quantum algorithms (VQAs) by utilizing data obtained from the quantum processor during the execution of the algorithm. This idea originated with Adaptive Derivative-Assembled Problem-Tailored variational quantum eigensolver (ADAPT-VQE), an algorithm that iteratively grows the state preparation circuit operator by operator, with each new operator accompanied by a new variational parameter, and where all parameters acquired thus far are optimized in each iteration. In ADAPT-VQE and other adaptive VQAs that followed it, it has been shown that initializing parameters to their optimal values from the previous iteration speeds up convergence and avoids shallow local traps in the parameter landscape. However, no other data from the optimization performed at one iteration is carried over to the next. In this work, we propose an improved quasi-Newton optimization protocol specifically tailored to adaptive VQAs. The distinctive feature in our proposal is that approximate second derivatives of the cost function are recycled across iterations in addition to optimal parameter values. We implement a quasi-Newton optimizer where an approximation to the inverse Hessian matrix is continuously built and grown across the iterations of an adaptive VQA. The resulting algorithm has the flavor of a continuous optimization where the dimension of the search space is augmented when the gradient norm falls below a given threshold. We show that this inter-optimization exchange of second-order information leads the approximate Hessian in the state of the optimizer to be consistently closer to the exact Hessian. As a result, our method achieves a superlinear convergence rate even in situations where the typical implementation of a quasi-Newton optimizer converges only linearly. Our protocol decreases the measurement costs in implementing adaptive VQAs on quantum hardware as well as the runtime of their classical simulation.
Ramôa, Mafalda (ORCID:0000000302187801)↗
Stratigraphic Identification with Airborne Electromagnetic Methods at the Hanford Site, Washington
Stratigraphic units can influence the fate and transport of subsurface contaminants within groundwater. Units having coarse-grained sediments act as preferential flow pathways, and therefore can accelerate the transport of contaminants to reach human and ecological receptors. At legacy waste sites, detailed knowledge of subsurface stratigraphy can be used for effective monitoring and remediation planning to help minimize risk to human health and the environment. Airborne electromagnetic (AEM) methods can non-invasively provide information on kilometer-scale or larger subsurface stratigraphic features and fill informational gaps in directly sampled data from sparsely located boreholes. In this paper, we present inversion results of a 412 line-km frequency-domain AEM survey to delineate subsurface stratigraphic features at the Hanford Site, located in southeastern Washington State. The inversion was performed using a massively parallel 3D electromagnetic modeling and inversion code, where the modeling is based on solving frequency-domain Maxwell’s equations using an unstructured-mesh finite-element method and the inversion employs a Gauss-Newton optimization scheme. The results are compared to an underlying geologic framework model (GFM), built by interpolating contact depths of stratigraphic units interpreted from site borehole datasets. In areas with good borehole coverage, the inversion results show a good match with the GFM to a depth of about 60 m. Outside of these areas, the inversion results exhibit inconsistencies from the assumptions made to create the GFM, demonstrating that the AEM survey results can be used to improve the understanding of the geological conceptual model.
47 OTHER INSTRUMENTATION↗
Navigator function for the conformal bootstrap
Current numerical conformal bootstrap techniques carve out islands in theory space by repeatedly checking whether points are allowed or excluded. We propose a new method for searching theory space that replaces the binary information "allowed"/"excluded" with a continuous "navigator" function that is negative in the allowed region and positive in the excluded region. Such a navigator function allows one to efficiently explore high-dimensional parameter spaces and smoothly sail towards any islands they may contain. The specific functions we introduce have several attractive features: they are well-defined in large regions of parameter space, can be computed with standard methods, and evaluation of their gradient is immediate due to an SDP gradient formula that we provide. The latter property allows for the use of efficient quasi-Newton optimization methods, which we illustrate by navigating towards the 3d Ising island.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Massively parallel modeling and inversion of electrical resistivity tomography data using PFLOTRAN
Abstract. Electrical resistivity tomography (ERT) is a broadly accepted geophysical method for subsurface investigations. Interpretation of field ERT data usually requires the application of computationally intensive forward modeling and inversion algorithms. For large-scale ERT data, the efficiency of these algorithms depends on the robustness, accuracy, and scalability on high-performance computing resources. In this regard, we present a robust and highly scalable implementation of forward modeling and inversion algorithms for ERT data. The implementation is publicly available and developed within the framework of PFLOTRAN, an open-source, state-of-the-art massively parallel subsurface flow and transport simulation code. The forward modeling is based on a finite-volume discretization of the governing differential equations, and the inversion uses a Gauss–Newton optimization scheme. To evaluate the accuracy of the forward modeling, two examples are first presented by considering layered (1D) and 3D earth conductivity models. The computed numerical results show good agreement with the analytical solutions for the layered earth model and results from a well-established code for the 3D model. Inversion of ERT data, simulated for a 3D model, is then performed to demonstrate the inversion capability by recovering the conductivity of the model. To demonstrate the parallel performance of PFLOTRAN's ERT process model and inversion capabilities, large-scale scalability tests are performed by using up to 131 072 processes on a leadership class supercomputer. These tests are performed for the two most computationally intensive steps of the ERT inversion: forward modeling and Jacobian computation. For the forward modeling, we consider models with up to 122 ×106 degrees of freedom (DOFs) in the resulting system of linear equations and demonstrate that the code exhibits almost linear scalability on up to 10 000 DOFs per process. On the other hand, the code shows superlinear scalability for the Jacobian computation, mainly because all computations are fairly evenly distributed over each process with no parallel communication.
58 GEOSCIENCES↗
A Scalable Interior‐Point Gauss–Newton Method for PDE‐Constrained Optimization With Bound Constraints
Here, we present a scalable approach to solve a class of partial differential equation (PDE)‐constrained optimization problems with bound constraints. This approach utilizes a robust full‐space interior‐point (IP)‐Gauss–Newton optimization method. To cope with the poorly‐conditioned IP‐Gauss–Newton saddle‐point linear systems that need to be solved approximately, once per optimization step, we propose two spectrally related preconditioners. These preconditioners leverage the limited informativeness of data in regularized PDE‐constrained optimization problems. A block Gauss–Seidel preconditioner is proposed for the GMRES‐based solution of the IP‐Gauss–Newton linear systems. It is shown, for a large‐class of PDE‐ and bound‐constrained optimization problems, that the spectrum of the block Gauss–Seidel preconditioned IP‐Gauss–Newton matrix is asymptotically independent of discretization and is not impacted by the ill‐conditioning that notoriously plagues interior‐point methods. We exploit symmetry of the IP‐Gauss–Newton linear systems and propose a regularization and log‐barrier Hessian preconditioner for the preconditioned conjugate gradient (PCG)‐based solution of the equivalent IP‐Gauss–Newton–Schur complement linear systems. The eigenvalues of the block Gauss–Seidel preconditioned IP‐Gauss–Newton matrix, that are not equal to one, are identical to the eigenvalues of the regularization and log‐barrier Hessian preconditioned Schur complement matrix. The scalability of the approach is demonstrated on two example problems. The numerical solution of these optimization problems is shown to require a discretization independent number of IP‐Gauss–Newton linear solves. Furthermore, the linear systems are solved in a discretization and IP ill‐conditioning independent number of preconditioned Krylov subspace iterations. The parallel scalability of the preconditioner, achieved via algebraic multigrid component solvers when applicable, and the aforementioned algorithmic scalability permits a parallel scalable means to compute solutions of a large class of PDE‐ and bound‐constrained problems.
PDE-constrained optimization↗
Parameter Sensitivity Analysis of the SparTen High Performance Sparse Tensor Decomposition Software (Extended Analysis)
Tensor decomposition models play an increasingly important role in modern data science applications. One problem of particular interest is fitting a low-rank Canonical Polyadic (CP) tensor decomposition model when the tensor has sparse structure and the tensor elements are nonnegative count data. SparTen is a high-performance C++ library which computes a low-rank decomposition using different solvers: a first-order quasi-Newton or a second-order damped Newton method, along with the appropriate choice of runtime parameters. Since default parameters in SparTen are tuned to experimental results in prior published work on a single real-world dataset conducted using MATLAB implementations of these methods, it remains unclear if the parameter defaults in SparTen are appropriate for general tensor data. Furthermore, it is unknown how sensitive algorithm convergence is to changes in the input parameter values. This report addresses these unresolved issues with large-scale experimentation on three benchmark tensor data sets. Experiments were conducted on several different CPU architectures and replicated with many initial states to establish generalized profiles of algorithm convergence behavior.
97 MATHEMATICS AND COMPUTING↗
Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization
Here, we consider unconstrained stochastic optimization problems with no available gradient information. Such problems arise in settings from derivative-free simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients using finite differences of stochastic function evaluations within a common random number framework. We develop modified versions of a norm test and an inner product quasi-Newton test to control the sample sizes used in the stochastic approximations and provide global convergence results to the neighborhood of a locally optimal solution. We present numerical experiments on simulation optimization problems to illustrate the performance of the proposed algorithm. When compared with classical zeroth-order stochastic gradient methods, we observe that our strategies of adapting the sample sizes significantly improve performance in terms of the number of stochastic function evaluations required.
97 MATHEMATICS AND COMPUTING↗
Optimal design of acoustic metamaterial cloaks under uncertainty
In this work, we consider the problem of optimal design of an acoustic cloak under uncertainty and develop scalable approximation and optimization methods to solve this problem. The design variable is taken as an infinite-dimensional spatially-varying field that represents the material property, while an additive infinite-dimensional random field represents the variability of the material property or the manufacturing error. Discretization of this optimal design problem results in high-dimensional design variables and uncertain parameters. To solve this problem, we develop a computational approach based on a Taylor approximation and an approximate Newton method for optimization, which is based on a Hessian derived at the mean of the random field. We show our approach is scalable with respect to the dimension of both the design variables and uncertain parameters, in the sense that the necessary number of acoustic wave propagations is essentially independent of these dimensions, for numerical experiments with up to one million design variables and half a million uncertain parameters. Additionally, we demonstrate that, using our computational approach, an optimal design of the acoustic cloak that is robust to material uncertainty is achieved in a tractable manner. The optimal design under uncertainty problem is posed and solved for the classical circular obstacle surrounded by a ring-shaped cloaking region, subjected to both a single-direction single-frequency incident wave and multiple-direction multiple-frequency incident waves. Finally, we apply the method to a deterministic large-scale optimal cloaking problem with complex geometry, to demonstrate that the approximate Newton method’s Hessian computation is viable for large, complex problems.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
An inexact semismooth Newton method with application to adaptive randomized sketching for dynamic optimization
In many applications, one can only access the inexact gradients and inexact hessian times vector products. Thus it is essential to consider algorithms that can handle such inexact quantities with a guaranteed convergence to solution. An inexact adaptive and provably convergent semismooth Newton method is considered to solve constrained optimization problems. In particular, dynamic optimization problems, which are known to be highly expensive, are the focus. A memory efficient semismooth Newton algorithm is introduced for these problems. The source of efficiency and inexactness is the randomized matrix sketching. Further, applications to optimization problems constrained by partial differential equations are also considered.
97 MATHEMATICS AND COMPUTING↗
Improved stellarator permanent magnet designs through combined discrete and continuous optimizations
A common optimization problem in the areas of magnetized plasmas and fusion energy is the design of magnets to produce a given three-dimensional magnetic field distribution to high precision. When designing arrays of permanent magnets for stellarator plasma confinement, such problems have tens of thousands of degrees of freedom whose solutions, for practical reasons, should be constrained to discrete spaces. We perform a direct comparison between two algorithms that have been developed previously for this purpose, and demonstrate that composite procedures that apply both algorithms in sequence can produce substantially improved results. One approach uses a continuous, quasi-Newton procedure to optimize the dipole moments of a set of magnets and then projects the solution onto a discrete space. The second uses an inherently discrete greedy optimization procedure that has been enhanced and generalized for this work. Further, the approaches are both applied to design arrays cubic rare-Earth permanent magnets to confine a quasi-axisymmetric plasma with a magnetic field on axis of 0.5 T. The first approach tends to find solutions with higher field accuracy, whereas the second can find solutions with substantially (up to 30%) fewer magnets. When the approaches are combined, they can obtain solutions with magnet quantities comparable to the second approach while matching the field accuracy of the first.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
A Stochastic Quasi-Newton Method in the Absence of Common Random Numbers
We present Q-SASS, a quasi-Newton method for unconstrained stochastic optimization that does not rely on common random numbers. Most existing quasi-Newton approaches leverage common random numbers to construct second-order updates. However, motivated by challenges in variational quantum algorithms—where such coordination is not possible—we consider the setting in which function values and gradients are accessible only through noisy probabilistic zeroth- and first-order oracles, and no common random numbers can be exploited. We derive high-probability tail bounds on the iteration complexity of our algorithm for nonconvex, convex, and strongly convex (more generally, those satisfying the PL condition) objective functions. Finally, we demonstrate the empirical benefits of our quasi-Newton updating scheme on both synthetic and quantum chemistry problems.
Complexity bound↗
Forward and inverse modeling of fault transmissibility in subsurface flows
Characterizing physical properties of faults, such as their transmissibility, is crucial for performing predictive numerical simulation of subsurface flows, such as those encountered in petroleum engineering and remediation of subsurface contamination. Here, this paper provides a complete investigation of the inverse problem for fault transmissibility in subsurface flow models, under appropriate assumptions on fault structure. In particular, the following aspects are considered: 1) fault modeling and well-posedness of the forward problem; 2) finite element (FEM) discretizations of the forward problem and their rigorous a priori convergence analysis; 3) Well-posedness of the Bayesian inverse problem, FEM discretization of the infinite dimensional Bayesian inverse formulation, and its rigorous a priori analysis. Moreover, computation of the maximum a posteriori (MAP) point via fast inexact Newton-conjugate gradient optimization and a Laplace approximation of the Bayesian posterior are also presented. Numerical results illustrate the use of the proposed fault model in forward and inverse problems for subsurface flows in two dimensional domains with multiple faults.
97 MATHEMATICS AND COMPUTING↗
The DESC stellarator code suite Part 3: Quasi-symmetry optimization
The DESC stellarator optimization code takes advantage of advanced numerical methods to search the full parameter space much faster than conventional tools. Only a single equilibrium solution is needed at each optimization step thanks to automatic differentiation, which efficiently provides exact derivative information. A Gauss–Newton trust-region optimization method uses second-order derivative information to take large steps in parameter space and converges rapidly. With just-in-time compilation and GPU portability, high-dimensional stellarator optimization runs take orders of magnitude less computation time with DESC compared to other approaches. This paper presents the theory of the DESC fixed-boundary local optimization algorithm along with demonstrations of how to easily implement it in the code. Example quasi-symmetry optimizations are shown and compared to results from conventional tools. Three different forms of quasi-symmetry objectives are available in DESC, and their relative advantages are discussed in detail. In the examples presented, the triple product formulation yields the best optimization results in terms of minimized computation time and particle transport. This paper concludes with an explanation of how the modular code suite can be extended to accommodate other types of optimization problems.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
A Proximal Quasi-Newton Trust-Region Method for Nonsmooth Regularized Optimization
Not provided.
Mathematics↗
An Inexact Trust-Region Newton Method for Large-Scale Convex-Constrained Optimization.
Abstract not provided.
Kouri, Drew↗
Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks
Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.
97 MATHEMATICS AND COMPUTING↗