SEARCH · Search NASA
Results for “Accelerated convergence”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Convergence acceleration of Monte Carlo many-body perturbation methods by using many control variates
Herein, the use of many control variates is proposed as a method to accelerate the second- and third-order Monte Carlo (MC) many-body perturbation (MC-MP2 and MC-MP3) calculations. A control variate is an exactly integrable function that is strongly correlated or anti-correlated with the target function to be integrated by the MC method. Evaluating both integrals as well as their covariances in the same MC run, one can effect a mutual cancellation of the statistical uncertainties and biases in the MC integrations, thereby accelerating its convergence considerably. Six and thirty-six control variates, whose integrals are known a priori, are generated for MC-MP2 and MC-MP3, respectively, by systematically replacing one or more two-electron-integral vertexes of certain configurations by zero-valued overlap-integral vertexes in their Goldstone diagrams. The variances and co- variances of these control variates are computed at a marginal cost, enhancing the overall efficiency of the MC-MP2 and MC-MP3 calculations by a factor of up to 14 and 20, respectively.
Leveraging operator learning to accelerate convergence of the preconditioned conjugate gradient method
We propose a new deflation strategy to accelerate the convergence of the preconditioned conjugate gradient (PCG) method for solving parametric large-scale linear systems of equations. Unlike traditional deflation techniques that rely on eigenvector approximations or recycled Krylov subspaces, we generate the deflation subspaces using operator learning, specifically the Deep Operator Network (DeepONet). To this aim, we introduce two complementary approaches for assembling the deflation operators. The first approach approximates near-null space vectors of the discrete PDE operator using the basis functions learned by the DeepONet. The second approach directly leverages solutions predicted by the DeepONet. To further enhance convergence, we also propose several strategies for prescribing the sparsity pattern of the deflation operator. Here, a comprehensive set of numerical experiments encompassing steady-state, time-dependent, scalar, and vector-valued problems posed on both structured and unstructured geometries is presented and demonstrates the effectiveness of the proposed DeepONet-based deflated PCG method, as well as its generalization across a wide range of model parameters and problem resolutions.
Metric Learning to Accelerate Convergence of Operator Splitting Methods
Recent developments in machine learning have led to promising advances in accelerating the solution of constrained optimization problems. Increasing demand for real-time decision-making capabilities in applications such as artificial intelligence and optimal control has led to a variety of proposed strategies for learning to produce fast solutions to optimization problems. For example, recent works have shown that it is possible to accelerate the convergence of optimization algorithms by learning to select their parameters, such as gradient descent stepsizes. This work proposes a new approach, in which the underlying metric spaces of proximal operator splitting algorithms are learned to maximize convergence rate. While prior works in optimization theory have derived optimal metrics in simple cases, no such result exists for many practical problem forms including general Quadratic Programming (QP). This paper shows how differentiable optimization can enable the end-to-end learning of proximal metrics, enhancing the convergence of proximal algorithms for QP problems beyond what is possible based on known theory. Additionally, the results illustrate a strong connection between the learned proximal metrics and active constraints at the optima, leading to an interpretation in which the predicted proximal metrics can be viewed as a form of active set prediction.
Adaptive Activation Functions Accelerate Convergence in Deep and Physics-informed Neural Networks
We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima.
Accelerated Convergence of Contracted Quantum Eigensolvers through a Quasi-Second-Order, Locally Parameterized Optimization
Not Available
Convergence acceleration in machine learning potentials for atomistic simulations
Machine learning potentials (MLPs) for atomistic simulations have an enormous prospective impact on materials modeling, offering orders of magnitude speedup over density functional theory simulations without appreciably sacrificing accuracy of material property prediction.
Convergence acceleration of Monte Carlo many-body perturbation methods by direct sampling
In the Monte Carlo many-body perturbation (MC-MP) method, the conventional correlation-correction formula, which is a long sum of products of low-dimensional integrals, is first recast into a short sum of high-dimensional integrals over electron-pair and imaginary-time coordinates. These high-dimensional integrals are then evaluated by the Monte Carlo method with random coordinates generated by the Metropolis–Hasting algorithm according to a suitable distribution. The latter algorithm, while advantageous in its ability to sample nearly any distribution, introduces autocorrelation in sampled coordinates, which in turn increases the statistical uncertainty of the integrals and thus the computational cost. It also involves wasteful rejected moves and an initial “burn-in” step as well as displays hysteresis. Here, an algorithm is proposed that directly produces a random sequence of electron-pair coordinates for the same distribution used in the MC-MP method, which is free from autocorrelation, rejected moves, a burn-in step, or hysteresis. Furthermore, this direct-sampling algorithm is shown to accelerate second- (MC-MP2) and third-order Monte Carlo many-body perturbation (MC-MP3) calculations by up to 222% and 38%, respectively.
Accelerating the convergence of auxiliary-field quantum Monte Carlo in solids with optimized Gaussian basis sets
We investigate the use of optimized correlation-consistent Gaussian basis sets for the study of insulating solids with auxiliary-field quantum Monte Carlo (AFQMC). The exponents of the basis set are optimized through the minimization of the second-order Møller–Plesset perturbation theory (MP2) energy in a small unit cell of the solid. We compare against other alternative basis sets proposed in the literature, namely, calculations in the Kohn–Sham basis and in the natural orbitals of an MP2 calculation. We find that our optimized basis sets accelerate the convergence of the AFQMC correlation energy compared to a Kohn–Sham basis and offer similar convergence to MP2 natural orbitals at a fraction of the cost needed to generate them. We also suggest the use of an improved, method independent, MP2-based basis set correction that significantly reduces the required basis set sizes needed to converge the correlation energy. With these developments, we study the relative performance of these basis sets in LiH, Si, and MgO and determine that our optimized basis sets yield the most consistent results as a function of volume. Using these optimized basis sets, we systematically converge the AFQMC calculations to the complete basis set and thermodynamic limit and find excellent agreement with experiment for the systems studied. Although we focus on AFQMC, our basis set generation procedure is independent of the subsequent correlated wavefunction method used.
High-repetition-rate, multi-MeV deuteron acceleration from converging heavy water microjets at laser intensities of 10 21 W/cm 2
Herein we demonstrate high repetition-rate deuteron acceleration by irradiating a continuously flowing, ambient temperature liquid heavy water jet with the high-intensity ALEPH laser. The laser delivered up to 5.5 J (120 TW, 1.2 × 10 21 W/cm 2 ) laser energy on target at 0.5 Hz. A high repetition-rate Thomson parabola spectrometer measured the deuteron beam energy spectra on each shot for 60 sequential shots (two minutes). Peak fluxes of 5 x 10 10 deuterons/sr/pulse, corresponding to an average flux of 1.5 x 10 12 deuterons/sr/min, were demonstrated with deuteron energies reaching up to 4.4 MeV. High shot-to-shot stability is observed up to 40%–50% of the maximum deuteron energy. These deuteron beams are suited for fast neutron production through deuteron breakup in a converter yielding energies similar to deuteron–deuteron (D–D, 2.45 MeV) fusion reactions of importance for material damage studies.
Linear convergence of accelerated conditional gradient algorithms in spaces of measures
A class of generalized conditional gradient algorithms for the solution of optimization problem in spaces of Radon measures is presented. The method iteratively inserts additional Dirac-delta functions and optimizes the corresponding coefficients. Under general assumptions, a sub-linear [see formula in PDF] rate in the objective functional is obtained, which is sharp in most cases. To improve efficiency, one can fully resolve the finite-dimensional subproblems occurring in each iteration of the method. We provide an analysis for the resulting procedure: under a structural assumption on the optimal solution, a linear [see formula in PDF] convergence rate is obtained locally.
Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks
Here we propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep and physics-informed neural networks. The local adaptation of activation function is achieved by introducing a scalable parameter in each layer (layer-wise) and for every neuron (neuron-wise) separately, and then optimizing it using a variant of stochastic gradient descent algorithm. In order to further increase the training speed, an activation slope-based slope recovery term is added in the loss function, which further accelerates convergence, thereby reducing the training cost. On the theoretical side, we prove that in the proposed method, the gradient descent algorithms are not attracted to sub-optimal critical points or local minima under practical conditions on the initialization and learning rate, and that the gradient dynamics of the proposed method is not achievable by base methods with any (adaptive) learning rates. We further show that the adaptive activation methods accelerate the convergence by implicitly multiplying conditioning matrices to the gradient of the base method without any explicit computation of the conditioning matrix and the matrix–vector product. The different adaptive activation functions are shown to induce different implicit conditioning matrices. Furthermore, the proposed methods with the slope recovery are shown to accelerate the training process.
A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow
With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for alternating current optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvatureinformed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.
KKT Preconditioners for PDE-Constrained Optimization with the Helmholtz Equation
This paper considers preconditioners for the linear systems that arise from optimal control and inverse problems involving the Helmholtz equation. Specifically, we explore an all-at-once approach. The main contribution centers on the analysis of two block preconditioners. Variations of these preconditioners have been proposed and analyzed in prior works for optimal control problems where the underlying partial differential equation is a Laplace-like operator. In this paper, we extend some of the prior convergence results to Helmholtz-based optimization applications. Our analysis examines situations where control variables and observations are restricted to subregions of the computational domain. We prove that solver convergence rates do not deteriorate as the mesh is refined or as the wavenumber increases. More specifically, for one of the preconditioners we prove accelerated convergence as the wavenumber increases. Additionally, in situations where the control and observation subregions are disjoint, we observe that solver convergence rates have a weak dependence on the regularization parameter. We give a partial analysis of this behavior. We illustrate the performance of the preconditioners on control problems motivated by acoustic testing.
Estimating Eigenenergies from Quantum Dynamics: A Unified Noise-Resilient Measurement-Driven Approach
Ground state energy estimation in physical, chemical, and materials sciences is one of the most promising applications of quantum computing. In this work, we introduce a new hybrid approach that finds the eigenenergies by collecting real-time measurements and post-processing them using the machinery of dynamic mode decomposition (DMD). From the perspective of quantum dynamics, we establish that our approach can be formally understood as a stable variational method on the function space of observables available from a quantum many-body system. We also provide strong theoretical and numerical evidence that our method converges rapidly even in the presence of a large degree of perturbative noise, and show that the method bears an isomorphism to robust matrix factorization methods developed independently across various scientific communities. Our numerical benchmarks on spin and molecular systems demonstrate an accelerated convergence and a favorable resource reduction over state-of-the-art algorithms. The DMD-centric strategy can systematically mitigate noise and stands out as a leading hybrid quantum-classical eigensolver.
Graph-Based Representations and Applications to Process Simulation
Rapid and robust convergence of a process flowsheet is critical to enable large-scale simulations that address core scientific questions related to process design, optimization, and sustainability. However, due to the highly coupled and nonlinear nature of chemical processes, efficiently solving a flowsheet remains a challenge. In this work, we show that graph representations of the underlying physical phenomena in unit operations may help identify potential avenues to systematically reformulate the network of equations and enable more robust topology-based convergence of flowsheets. To this end, we developed graph abstractions of the governing equations of vapor-liquid and liquid-liquid equilibrium separation equipment. These graph abstractions consist of a mesh of interconnected variable nodes and equation nodes that are systematically generated through PhenomeNode, a new open-source library in Python developed in this study. We show that partitioning the graph into separate mass, energy, and equilibrium subgraphs can help decouple nonlinearities and guide decomposition algorithms. By employing the graph abstraction on an industrial separation process for separating glacial acetic acid from water, we implemented a new block decomposition scheme in BioSTEAM and demonstrated that this can accelerate convergence over a traditional sequential modular approach.