Search NASA⌕ Search

SEARCH · Search NASA

Results for “gradient methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Refinement for single-nanoparticle structure determination from low-quality single-shot coherent diffraction data

With the emergence of X-ray free-electron lasers, it is possible to investigate the structure of nanoscale samples by employing coherent diffractive imaging in the X-ray spectral regime. In this work, we developed a refinement method for structure reconstruction applicable to low-quality coherent diffraction data. The method is based on the gradient search method and considers the missing region of a diffraction pattern and the small number of detected photons. We introduced an initial estimate of the structure in the method to improve the convergence. The present method is applied to an experimental diffraction pattern of an Xe cluster obtained in an X-ray scattering experiment at the SPring-8 Angstrom Compact free-electron LAser (SACLA) facility. It is found that the electron density is successfully reconstructed from the diffraction pattern with a large missing region, with a good initial estimate of the structure. The diffraction pattern calculated from the reconstructed electron density reproduced the observed diffraction pattern well, including the characteristic intensity modulation in each ring. Our refinement method enables structure reconstruction from diffraction patterns under difficulties such as missing areas and low diffraction intensity, and it is potentially applicable to the structure determination of samples that have low scattering power.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

COYOTE

COYOTE is designed for the analysis of steady and transient, multi-dimensional, nonlinear heat conduction problems. The spatial discretization technique is based on the Galerkin finite element method. The code contains a library of linear and quadratic elements and incudes bars and shells. COYOTE Implicit and explicit time integration methods, with adaptive time step control, are available for transient problems and fixed-point methods are used for Conjugate gradient iterative methods are used for the matrix solution. Coupled enclosure radiation with efficient view factor computation is included. Simulation capabilities include material motion, dynamic radiation enclosures, contact boundary conditions, multipoint constraints, material (element) addition, and deletion, bulk nodes, chemically reactive materials, and coupling with other mechanics codes. The code is designed for single or multi-processor computing

Gartling, DavidK.↗

Design and convergence analysis of the conforming virtual element method for polyharmonic problems

In the recent years, there has been a tremendous interest to numerical methods that approximate partial differential equations (PDEs) on computational meshes with arbitrarily-shaped polygonal/polyhedral (polytopal, for short) elements. A nonexhaustive list of such methods include the Mimetic Finite Difference method [4, 9, 21, 24, 28, 33, 35, 62, 64–67], the Polygonal Finite Element Method [68, 73, 74], the polygonal Discontinuous Galerkin Finite Element Methods [5, 7, 8, 10, 13–15, 18, 36, 38, 39, 41, 50], the Hybridizable Discontinuous Galerkin and Hybrid High-Order Methods [46–48, 51], the Gradient Discretization method [49, 53, 54], the Finite Volume Method [52], and the BEM-based FEM [71]. An alternative approach that also proved to be very successful is provided by the Virtual Element method (VEM), which was originally proposed in [19] for the numerical treatment of second-order elliptic problems [43, 44], and readily extended to linear and nonlinear elasticity [22, 25, 56], plate bending problems [37], Cahn-Hilliard equation [3], Stokes equations [2], Laplace-Beltrami equation [55], Darcy-Brinkam equation [76], discrete topology optimization problems [6], fracture networks problems [29], eigenvalue problems [59, 75]. The mixed virtual element formulation was proposed in [20, 34]; the nonconforming Virtual element formulations was proposed for second-order elliptic problems in [16], and later extended to general advection-reaction-diffusion problems, Stokes equation, the biharmonic problems, the iegenvalue problem, and the Schrodinger equation in [11, 30, 42, 45, 58, 80]. The p- and hp-version of the VEM were developed in [26, 27] and efficient multigrid methods for the resulting linear system of equations in [12]. A posteriori error estimates can be found in [40].

97 MATHEMATICS AND COMPUTING↗

A frozen Gaussian approximation-based multi-level particle swarm optimization for seismic inversion

In this paper, we propose a frozen Gaussian approximation (FGA)-based multi-level particle swarm optimization (MLPSO) method for seismic inversion of high-frequency wave data. The method addresses two challenges in it: First, the optimization problem is highly non-convex, which makes hard for gradient-based methods to reach global minima. This is tackled by MLPSO which can escape from undesired local minima. Second, the character of high-frequency of seismic waves requires a large number of grid points in direct computational methods, and thus renders an extremely high computational demand on the simulation of each sample in MLPSO. We overcome this difficulty by three steps: First, we use FGA to compute high-frequency wave propagation based on asymptotic analysis on phase plane; Then we design a constrained full waveform inversion problem to prevent the optimization search getting into regions of velocity where FGA is not accurate; Last, we solve the constrained optimization problem by MLPSO that employs FGA solvers with different fidelity. The performance of the proposed method is demonstrated by a two-dimensional full-waveform inversion example of the smoothed Marmousi model.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Optimal control with accelerated convergence: Combining the Krotov and quasi-Newton methods

One of the most popular methods for solving numerical optimal control problems is the Krotov method, adapted for quantum control by Tannor and coworkers. The Krotov method has the following three appealing properties: (1) monotonic increase of the objective with iteration number, (2) no requirement for a line search, leading to a significant savings over gradient (first-order) methods, and (3) macrosteps at each iteration, resulting in significantly faster growth of the objective at early iterations than in gradient methods where small steps are required. The principal drawback of the Krotov method is slow convergence at later iterations, which is particularly problematic when high fidelity is desired. We show here that, near convergence, the Krotov method degenerates to a first-order gradient method. We then present a variation on the Krotov method that has all the advantages of the original Krotov method but with significantly enhanced convergence (second-order or quasi-Newton) as the optimal solution is approached. We illustrate the method by controlling the three-dimensional dynamics of the valence electron in the Na atom.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Hierarchical Optimal Power Flow with Improved Gradient Evaluation

Existing algorithms to solve alternating-current optimal power flow (AC-OPF) often exploit linear approximations to simplify system models and accelerate computations. In this paper, we improve a recent hierarchical OPF algorithm, which rested on primal-dual gradients evaluated in a linearized distribution power flow model. Specifically, we identify a risk of voltage violation arising from the model linearization, and propose a more accurate gradient evaluation method to eliminate that risk. We further develop a hierarchical primal-dual algorithm to solve OPF based on the proposed gradient evaluation method. Numerical results on IEEE networks show that our algorithm can enhance voltage safety with satisfactory computational efficiency.

distributed algorithm↗

Asynchronous and Distributed Tracking of Time-Varying Fixed Points

NREL researcher Andrey Bernstein discusses asynchronous and distributed tracking of time-varying fixed points as part of the Innovative Optimization and Control Methods for Highly Distributed Autonomous Systems workshop. NREL hosted the workshop to exchange ideas on state-of-the-art control and optimization strategies as well as get feedback on work being done at NREL.

97 MATHEMATICS AND COMPUTING↗

The liquidus temperature of nuclear waste glasses: an international Round-Robin Study

Ten institutions from five countries participated in a Round Robin study to contribute to the Precision and Bias section of an American Society for Testing and Materials standard procedure that Pacific Northwest National Laboratory (PNNL) is developing for measuring the liquidus temperature (TL) of radioactive and simulated waste glasses. In this study, three separate TL measurement methods were a gradient temperature (GT) method, a uniform temperature (UT) method, and a crystal fraction extrapolation (CF) method. Three different glasses were measured with a combination of these three methods. The TL values reported by different institutions are generally consistent and vary within a narrow range. The precision of a TL measurement was evaluated as ±10°C regardless of the method used for making the measurement. The Round Robin glasses were all previously studied at PNNL and included ARG-1 (Glass A), Zr-9 (Glass B), and AmCm2-19 (Glass C), with measured TL values spanning the temperature range ~960-1240°C. The three methods discussed here in more detail are the GT, UT, and CF methods. A best-case precision for TL has been obtained from the data, even though the data were not acquired for all three glasses using all three methods from each participating organization.

ASTM↗

Machine Learning Analysis of Temperature-Strain Relationships for Structural Health Monitoring of Pipes: Self-powered wireless sensor system for health monitoring of liquid-sodium cooled fast reactors

This report presents machine learning (ML) analysis of temperature-strain relationships for structural health monitoring of nuclear reactor stainless steel (SS) pipes with the strain gauge sensor directly printed on the pipe with a 3D conformal aerosol jet printer. We investigate correlations for two sensor pairs installed on the same SS304 pipe: commercial K-type thermocouple with a printed gold strain gauge (TC3-SG3), and commercial K-type thermocouple with commercial Kyowa strain gauge (TC0-SG0). The temperature ranges for the sensor pairs TC0-SG0 and TC3-SG3 are 20.00°C to 266.37°C and 39.95°C to 219.28°C respectively. ML algorithms in this study include Linear Regression (baseline method), Ridge Regression, Lasso Regression, and Gradient Boosting. Performance evaluation metrics include Root Mean Square Error (RMSE), Mean Square Error (MSE), Mean Absolute Error (MAE), R 2 Score, and Explained Variance. Using advanced feature engineering techniques, we extracted 27 temperature-based features and 30 strategic inclusion features. The best performance was obtained with the Gradient Boosting method, which achieves prediction accuracy of R 2 = 0.9999 and RMSE = 7.69 μStrain for TC0-SG0, and R 2 = 0.9998 and RMSE = 18.03 μStrain for TC3-SG3. While the temperature-strain correlations are weaker for the gauge directly printed on the pipe than for the commercial strain gauge, deployment-ready performance exceeding industry standards is achieved for both sensor pairs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Gradient-Informed Design Optimization of Select Nuclear Systems

In this work, we present a gradient-informed design optimization of nuclear reactor core components based on neutronics objectives with both continuous and discrete materials. The main argument in favor of using gradient-informed design optimization is that it scales well with increasing dimensionality of the design space. First, a challenge problem with 121 free parameters is solved with a gradient-informed method and then with a genetic algorithm. Then, a challenge problem to optimize the flux profile of a simplified assembly with eight axial zones is solved. Both challenge problems are solved using directly calculated derivatives from Tools for Sensitivity and Uncertainty Analysis Methodology Implementation (TSUNAMI) in the SCALE package. Furthermore, we demonstrate how a discrete optimization problem—selection of materials for 121 voxels—can be lifted into a continuous problem with mixed materials. In the continuous space, adjoint-based gradients are well-defined, and gradient descent is applicable. Then, a forcing function is introduced that with the selection of an appropriately sized hyperparameter can be used to guide the optimized continuous solution back into a discrete solution. This paper presents an account of the challenges that were faced when applying a gradient-informed optimization algorithm using a Monte Carlo calculation to estimate the gradient information and compares a gradient descent optimization method to a genetic algorithm optimization of the same geometry. Overall, this work demonstrates the potential use of adjoint-based gradient calculations in design optimization of nuclear systems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Efficient ptychographic phase retrieval via a matrix-free Levenberg-Marquardt algorithm

The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second order derivatives is even more laborious. Additionally, second order techniques often require the storage and inversion of large matrices of partial derivatives, with memory requirements that can be prohibitive for data-rich imaging modalities. We use a reverse-mode automatic differentiation (AD) framework to implement an efficient matrix-free version of the Levenberg-Marquardt (LM) algorithm, a longstanding method that finds popular use in nonlinear least-square minimization problems but which has seen little use in phase retrieval. Furthermore, we extend the basic LM algorithm so that it can be applied for more general constrained optimization problems (including phase retrieval problems) beyond just the least-square applications. Since we use AD, we only need to specify the physics-based forward model for a specific imaging application; the first and second-order derivative terms are calculated automatically through matrix-vector products, without explicitly forming the large Jacobian or Gauss-Newton matrices typically required for the LM method. We demonstrate that this algorithm can be used to solve both the unconstrained ptychographic object retrieval problem and the constrained “blind” ptychographic object and probe retrieval problems, under the popular Gaussian noise model as well as the Poisson noise model. We compare this algorithm to state-of-the-art first order ptychographic reconstruction methods to demonstrate empirically that this method outperforms best-in-class first-order methods: it provides excellent convergence guarantees with (in many cases) a superlinear rate of convergence, all with a computational cost comparable to, or lower than, the tested first-order algorithms.

36 MATERIALS SCIENCE↗

Analytical gradient-based optimization of CALPHAD model parameters

The calibration of CALPHAD (CALculation of PHAse Diagrams) models involves the solution of a very challenging high-dimensional multiobjective optimization problem. Traditional approaches to parameter fitting predominantly rely on gradient-free methods, which while robust, are computationally inefficient and often scale poorly with model complexity. In this work, we introduce and demonstrate a generalizable framework for analytic gradient-based optimization of the parameters of the CALPHAD model enabled by the recently formalized Jansson derivative technique. This method allows for efficient evaluation of gradients of thermodynamic properties at equilibrium with respect to model parameters, even in the presence of arbitrarily complex internal degrees of freedom. Leveraging these semi-analytic gradients, we employ the conjugate gradient (CG) method to optimize thermodynamic model parameters for four binary alloy systems: Cu-Mg, Fe-Ni, Cr-Ni, and Cr-Fe. Across all systems, CG achieves comparable or superior optimality relative to Bayesian ensemble Markov Chain Monte Carlo (MCMC) with improvements in computational efficiency ranging from one to three orders of magnitude. Furthermore, our results establish a new paradigm for CALPHAD assessments in which high fidelity data-rich model calibration becomes tractable using deterministic gradient-informed algorithms.

CALPHAD↗

Modeling error PDF optimization based wavelet neural network modeling of dynamic system and its application in blast furnace ironmaking

In general, the modeling errors of dynamic system model are a set of random variables. The traditional performance index of modeling such as means square error (MSE) and root means square error (RMSE) can not fully express the connotation of modeling errors with stochastic characteristics both in the dimension of time domain and space domain. Therefore, the probability density function (PDF) is introduced to completely describe the modeling errors in both time scales and space scales. Based on it, a novel wavelet neural network (WNN) modeling method is proposed by minimizing the two-dimensional (2D) PDF shaping of modeling errors. First, the modeling error PDF by the tradional WNN is estimated using data-driven kernel density estimation (KDE) technique. Then, the quadratic sum of 2D deviation between the modeling error PDF and the target PDF is utilized as performance index to optimize the WNN model parameters by gradient descent method. Since the WNN has strong nonlinear approximation and adaptive capability, and all the parameters are well optimized by the proposed method, the developed WNN model can make the modeling error PDF track the target PDF, eventually. Simulation example and application in a blast furnace ironmaking process show that the proposed method has a higher modeling precision and better generalization ability compared with the conventional WNN modeling based on MSE criteria. Furthermore, the proposed method has more desirable estimation for modeling error PDF that approximates to a Gaussian distribution whose shape is high and narrow.

Modelling↗

SwitchX : Gmin-Gmax Switching for Energy-efficient and Robust Implementation of Binarized Neural Networks on ReRAM Xbars

Memristive crossbars can efficiently implement Binarized Neural Networks (BNNs) wherein the weights are stored in high-resistance states (HRS) and low-resistance states (LRS) of the synapses. We propose SwitchX mapping of BNN weights onto ReRAM crossbars such that the impact of crossbar non-idealities, that lead to degradation in computational accuracy, are minimized. Essentially, SwitchX maps the binary weights in such a manner that a crossbar instance comprises of more HRS than LRS synapses. We find BNNs mapped onto crossbars with SwitchX to exhibit better robustness against adversarial attacks than the standard crossbar mapped BNNs, the baseline. Finally, we combine SwitchX with state-aware training (that further increases the feasibility of HRS states during weight mapping) to boost the robustness of a BNN on hardware. We find that this approach yields stronger defense against adversarial attacks than adversarial training, a state-of the-art software defense. We perform experiments on a VGG16 BNN with benchmark datasets (CIFAR-10, CIFAR-100 and TinyImagenet) and use Fast Gradient Sign Method (ϵ = 0.05 to 0.3) and Projected Gradient Descent (ϵ = $\frac{2}{255}$ to $\frac{32}{255}$, α = $\frac{2}{255}$) adversarial attacks. We show that SwitchX combined with state-aware training can yield upto ~35% improvements in clean accuracy and ~6–16% in adversarial accuracies against conventional BNNs. Furthermore, an important by-product of SwitchX mapping is increased crossbar power savings, owing to an increased proportion of HRS synapses, which is furthered with state-aware training. We obtain upto ~21–22% savings in crossbar power consumption for state-aware trained BNN mapped via SwitchX on 16 × 16 and 32 × 32 crossbars using the CIFAR-10 and CIFAR-100 datasets.

97 MATHEMATICS AND COMPUTING↗

Soft magnetism and microwave magnetic properties of Fe-Co-Hf films deposited by composition gradient sputtering

A composition gradient sputtering method was employed to deposit a series of (Fe{sub x}Co{sub 1-x}){sub 1-y}Hf{sub y} alloy films with different Fe/Co atomic ratios and Hf-doping. At Fe/Co = 70/30, the magnetically annealed (Fe{sub 0.7}Co{sub 0.3}){sub 1-y}Hf{sub y} films with y = 0.087-0.124 showed a very high uniaxial anisotropy and an ultrahigh ferromagnetic resonance frequency over 7 GHz after the films were annealed at 350 deg. C, while at Fe/Co 50/50, the as-deposited (Fe{sub 0.5}Co{sub 0.5}){sub 1-y}Hf{sub y} films with y 0.074-0.168 showed an excellent in-plane uniaxial anisotropy and high ferromagnetic resonance frequency over 3 GHz. These Fe-Co-Hf films deposited by the composition gradient sputtering method exhibited ahigh saturation magnetization of 1.8 - 2.2 T, a large uniaxial anisotropy field of 200 - 500 Oe, and a high ferromagnetic resonance frequency over 7 GHz, which provides great opportunities for integrated magnetic devices.

36 MATERIALS SCIENCE↗

Optimization Methods for Large-Scale Machine Learning

Here this paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.

97 MATHEMATICS AND COMPUTING↗

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

Here we propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep and physics-informed neural networks. The local adaptation of activation function is achieved by introducing a scalable parameter in each layer (layer-wise) and for every neuron (neuron-wise) separately, and then optimizing it using a variant of stochastic gradient descent algorithm. In order to further increase the training speed, an activation slope-based slope recovery term is added in the loss function, which further accelerates convergence, thereby reducing the training cost. On the theoretical side, we prove that in the proposed method, the gradient descent algorithms are not attracted to sub-optimal critical points or local minima under practical conditions on the initialization and learning rate, and that the gradient dynamics of the proposed method is not achievable by base methods with any (adaptive) learning rates. We further show that the adaptive activation methods accelerate the convergence by implicitly multiplying conditioning matrices to the gradient of the base method without any explicit computation of the conditioning matrix and the matrix–vector product. The different adaptive activation functions are shown to induce different implicit conditioning matrices. Furthermore, the proposed methods with the slope recovery are shown to accelerate the training process.

97 MATHEMATICS AND COMPUTING↗

Electronic structure and thermoelectric properties of CoAsSb with post-DFT approaches

Here, we study the electronic structure and thermoelectric properties of recently synthesized CoAsSb. The calculated bandgap becomes more accurate for increasingly-complex electronic structure methods: generalized gradient approximation, hybrid functionals, selfconsistent linearized quasiparticle GW method (LQSGW), and LQSGW with simplified vertex corrections. By equating the bandgaps of each method from a rigid shift of the bands, we evaluate the contributions made to thermoelectric properties beyond the bandgap. In doing so, we evaluate the efficacy of a commonpractice: a rigid shift applied to less-costly electronic structure methods in order to gain some accuracy of the more-costly methods. We find that while the shift made the Seebeck coefficients much closer to one another than from the original electronic structures, there remain differences in the Goldsmid-Sharp (thermoelectric) bandgap between the methods and from the intended electronic bandgap. Additionally, some lasting differences in temperature dependence remain between the methods.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗