Parallel and distributed asynchronous adaptive stochastic gradient methods
Not provided.
Computer Science↗
SEARCH · Search NASA
Results for “gradient methods”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Non-blocking Preconditioned Conjugate Gradient Methods for Extreme-scale Computing.
Abstract not provided.
Eller, Paul Ray↗
What Is the Best Way to Contour Lung Tumors on PET Scans? Multiobserver Validation of a Gradient-Based Method Using a NSCLC Digital PET Phantom
Purpose: To evaluate the accuracy and consistency of a gradient-based positron emission tomography (PET) segmentation method, GRADIENT, compared with manual (MANUAL) and constant threshold (THRESHOLD) methods. Methods and Materials: Contouring accuracy was evaluated with sphere phantoms and clinically realistic Monte Carlo PET phantoms of the thorax. The sphere phantoms were 10-37 mm in diameter and were acquired at five institutions emulating clinical conditions. One institution also acquired a sphere phantom with multiple source-to-background ratios of 2:1, 5:1, 10:1, 20:1, and 70:1. One observer segmented (contoured) each sphere with GRADIENT and THRESHOLD from 25% to 50% at 5% increments. Subsequently, seven physicians segmented 31 lesions (7-264 mL) from 25 digital thorax phantoms using GRADIENT, THRESHOLD, and MANUAL. Results: For spheres <20 mm in diameter, GRADIENT was the most accurate with a mean absolute % error in diameter of 8.15% (10.2% SD) compared with 49.2% (51.1% SD) for 45% THRESHOLD (p < 0.005). For larger spheres, the methods were statistically equivalent. For varying source-to-background ratios, GRADIENT was the most accurate for spheres >20 mm (p < 0.065) and <20 mm (p < 0.015). For digital thorax phantoms, GRADIENT was the most accurate (p < 0.01), with a mean absolute % error in volume of 10.99% (11.9% SD), followed by 25% THRESHOLD at 17.5% (29.4% SD), and MANUAL at 19.5% (17.2% SD). GRADIENT had the least systematic bias, with a mean % error in volume of -0.05% (16.2% SD) compared with 25% THRESHOLD at -2.1% (34.2% SD) and MANUAL at -16.3% (20.2% SD; p value <0.01). Interobserver variability was reduced using GRADIENT compared with both 25% THRESHOLD and MANUAL (p value <0.01, Levene's test). Conclusion: GRADIENT was the most accurate and consistent technique for target volume contouring. GRADIENT was also the most robust for varying imaging conditions. GRADIENT has the potential to play an important role for tumor delineation in radiation therapy planning and response assessment.
62 RADIOLOGY AND NUCLEAR MEDICINE↗
Surface Profile and Stress Field Evaluation using Digital Gradient Sensing Method
Shape and surface topography evaluation from measured orthogonal slope/gradient data is of considerable engineering significance since many full-field optical sensors and interferometers readily output accurate data of that kind. This has applications ranging from metrology of optical and electronic elements (lenses, silicon wafers, thin film coatings), surface profile estimation, wave front and shape reconstruction, to name a few. In this context, a new methodology for surface profile and stress field determination based on a recently introduced non-contact, full-field optical method called digital gradient sensing (DGS) capable of measuring small angular deflections of light rays coupled with a robust finite-difference-based least-squares integration (HFLI) scheme in the Southwell configuration is advanced here. The method is demonstrated by evaluating (a) surface profiles of mechanically warped silicon wafers and (b) stress gradients near growing cracks in planar phase objects.
42 ENGINEERING↗
Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing
We analyze the convergence of a nonlocal gradient descent method for minimizing a class of high-dimensional non-convex functions, where a directional Gaussian smoothing (DGS) is proposed to define the nonlocal gradient (also referred to as the DGS gradient). The method was first proposed in [Zhang et al., Enabling long-range exploration in minimization of multimodal functions, UAI 2021], in which multiple numerical experiments showed that replacing the traditional local gradient with the DGS gradient can help the optimizers escape local minima more easily and significantly improve their performance. However, a rigorous theory for the efficiency of the method on nonconvex landscape is lacking. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by deterministic oscillating noise. We provide a convergence theory under which the iterates exponentially converge to a tightened neighborhood of the solution, whose size is characterized by the noise wavelength. Here, we also establish a correlation between the optimal values of the Gaussian smoothing radius and the noise wavelength, thus justifying the advantage of using moderate or large smoothing radii with the method. Furthermore, if the noise level decays to zero when approaching the global minimum, we prove that DGS-based optimization converges to the exact global minimum with linear rates, similarly to standard gradient-based methods in optimizing convex functions. Several numerical experiments are provided to confirm our theory and illustrate the superiority of the approach over those based on the local gradient.
Tran, Hoang [Oak Ridge National Laboratory (ORNL),↗
First-principles calculation of excited states of diatomic molecules: a benchmark for the Gutzwiller conjugate gradient minimisation method
We recently proposed the Gutzwiller conjugate gradient minimisation (GCGM) method for efficient and accurate calculation of the ground state total energy of molecular and bulk systems. The GCGM method is developed under the framework of Gutzwiller wave function but goes beyond the commonly adopted Gutzwiller approximation to improve the accuracy and flexibility in treating the correlation effects. In this conference proceeding, we benchmark the GCGM method with the calculation of excited state potential energy curves of three diatomic molecules, namely H 2 , N 2 , and O 2 . Our calculations demonstrate the flexibility and reasonable accuracy of the method.
74 ATOMIC AND MOLECULAR PHYSICS↗
Real-Time Feedback-Based Optimization of Distribution Grids: A Unified Approach
This paper develops an algorithmic framework for real-time optimization of distribution-level distributed energy resources (DERs). The framework optimizes the operation of both DERs that are individually controllable and groups of DERs (i.e., aggregations) that are jointly controlled at an electrical point of connection. From an electrical standpoint, wye and delta single- and multi-phase connections are accounted for. The algorithm enables (groups of) DERs to pursue given performance objectives, while adjusting their (aggregate) powers to respond to services requested by grid operators and to maintain electrical quantities within engineering limits. The design of the algorithm leverages a time-varying bi-level problem formulation capturing various performance objectives and engineering constraints, and an online implementation of primal-dual projected-gradient methods. The gradient steps are suitably modified to accommodate appropriate measurements from the distribution network and the DERs. Stability and convergence claims are established in terms of tracking of the solution of the time-varying optimization problem. Finally, the method is tested in a realistic distribution.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Stochastic Spectral Descent for Discrete Graphical Models
Interest in deep probabilistic graphical models has in-creased in recent years, due to their state-of-the-art performance on many machine learning applications. Such models are typically trained with the stochastic gradient method, which can take a significant number of iterations to converge. Since the computational cost of gradient estimation is prohibitive even for modestly sized models, training becomes slow and practically usable models are kept small. In this paper we propose a new, largely tuning-free algorithm to address this problem. Our approach derives novel majorization bounds based on the Schatten- norm. Intriguingly, the minimizers of these bounds can be interpreted as gradient methods in a non-Euclidean space. We thus propose using a stochastic gradient method in non-Euclidean space. We both provide simple conditions under which our algorithm is guaranteed to converge, and demonstrate empirically that our algorithm leads to dramatically faster training and improved predictive ability compared to stochastic gradient descent for both directed and undirected graphical models.
97 MATHEMATICS AND COMPUTING↗
Gradient-Enhanced Polynomial Chaos Methods for Circuit Simulation.
Abstract not provided.
Keiter, Eric R.↗
Variational projection methods for gradient crystal plasticity using Lie algebras
Summary A computational method is developed for evaluating the plastic strain gradient hardening term within a crystal plasticity formulation. While such gradient terms reproduce the size effects exhibited in experiments, incorporating derivatives of the plastic strain yields a nonlocal constitutive model. Rather than applying mixed methods, we propose an alternative method whereby the plastic deformation gradient is variationally projected from the elemental integration points onto a smoothed nodal field. Crucially, the projection utilizes the mapping between Lie groups and algebras in order to preserve essential physical properties, such as orthogonality of the plastic rotation tensor. Following the projection, the plastic strain field is directly differentiated to yield the Nye tensor. Additionally, an augmentation scheme is introduced within the global Newton iteration loop such that the computed Nye tensor field is fed back into the stress update procedure. Effectively, this method results in a fully implicit evolution of the constitutive model within a traditional displacement‐based formulation. An elemental projection method with explicit time integration of the plastic rotation tensor is compared as a reference. A series of numerical tests are performed for several element types in order to assess the robustness of the method, with emphasis placed upon polycrystalline domains and multi‐axis loading. Copyright © 2016 John Wiley & Sons, Ltd.
Truster, Timothy J.↗
Enhancing ACPF Analysis: Integrating Newton-Raphson Method with Gradient Descent and Computational Graphs
This paper presents a new method for enhancing Alternating Current Power Flow (ACPF) analysis. The method integrates the Newton-Raphson (NR) method with Enhanced-Gradient Descent (GD) and computational graphs. The integration of renewable energy sources in power systems introduces variability and unpredictability, and this method addresses these challenges. It leverages the robustness of NR for accurate approximations and the flexibility of GD for handling variable conditions, all without requiring Jacobian matrix inversion. Furthermore, computational graphs provide a structured and visual framework that simplifies and systematizes the application of these methods. The goal of this fusion is to overcome the limitations of traditional ACPF methods and improve the resilience, adaptability, and efficiency of modern power grid analyses. We validate the effectiveness of our advanced algorithm through comprehensive testing on established IEEE benchmark systems. Furthermore, our findings demonstrate that our approach not only speeds up the convergence process but also ensures consistent performance across diverse system states, representing a significant advancement in power flow computation.
24 POWER TRANSMISSION AND DISTRIBUTION↗
The Gutzwiller conjugate gradient minimization method for correlated electron systems
In this report we review our recent work on the Gutzwiller conjugate gradient minimization method, an ab initio approach developed for correlated electron systems. The complete formalism has been outlined that allows for a systematic understanding of the method, followed by a discussion of benchmark studies of dimers, one- and two-dimensional single-band Hubbard models. In the end, we present some preliminary results of multi-band Hubbard models and large-basis calculations of F 2 to illustrate our efforts to further reduce the computational complexity.
75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
An adaptive Hessian approximated stochastic gradient MCMC method
Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.
97 MATHEMATICS AND COMPUTING↗
The molecular gradient using the divide-expand-consolidate resolution of the identity second-order Møller-Plesset perturbation theory: The DEC-RI-MP2 gradient
We report an implementation of the molecular gradient using the divide-expand-consolidate resolution of the identity second-order Møller-Plesset perturbation theory (DEC-RI-MP2). The new DEC-RI-MP2 gradient method combines the precision control as well as the linear-scaling and massively parallel features of the DEC scheme with efficient evaluations of the gradient contributions using the RI approximation. We further demonstrate that the DEC-RI-MP2 gradient method is capable of calculating molecular gradients for very large molecular systems. A test set of supramolecular complexes containing up to 158 atoms and 1960 contracted basis functions has been employed to demonstrate the general applicability of the DEC-RI-MP2 method and to analyze the errors of the DEC approximation. Moreover, the test set contains molecules of complicated electronic structures and is thus deliberately chosen to stress test the DEC-RI-MP2 gradient implementation. Additionally, as a showcase example the full molecular gradient for insulin (787 atoms and 7604 contracted basis functions) has been evaluated.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Discontinuous Skeletal Gradient Discretisation methods on polytopal meshes
Here, in this work we develop arbitrary-order Discontinuous Skeletal Gradient Discretisations (DSGD) on general polytopal meshes. Discontinuous Skeletal refers to the fact that the globally coupled unknowns are broken polynomials on the mesh skeleton. The key ingredient is a high-order gradient reconstruction composed of two terms: (i) a consistent contribution obtained mimicking an integration by parts formula inside each element and (ii) a stabilising term for which sufficient design conditions are provided. An example of stabilisation that satisfies the design conditions is proposed based on a local lifting of high-order residuals on a Raviart–Thomas–Nédélec subspace. We prove that the novel DSGDs satisfy coercivity, consistency, limit-conformity, and compactness requirements that ensure convergence for a variety of elliptic and parabolic problems. Lastly, links with Hybrid High-Order, non-conforming Mimetic Finite Difference and non-conforming Virtual Element methods are also studied. Numerical examples complete the exposition.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
A Nonlocal-Gradient Descent Method for Inverse Design in Nanophotonics
Local-gradient-based optimization approaches lack nonlocal exploration abilityrequired for escaping from local minima when searching non-convex landscapes.A directional Gaussian smoothing (DGS) approach was recently proposed in [29]and used to define a truly nonlocal gradient, referred to as the DGS gradient, inorder to enable nonlocal exploration in high-dimensional black-box optimization.Promising results show that replacing the traditional local gradient with the nonlocalDGS gradient can significantly improve the performance of gradient-based methodsin optimizing highly multi-modal loss functions. However, the current DGS methodis designed for unbounded and uncontrained optimization problems, making itinapplicable to real-world engineering optimization problems where the tuningparameters are often bounded and the loss function is usually constrained byphysical processes. In this work, we propose to extend to the DGS approachto the constrained inverse design framework in order to find better optima ofmulti-modal loss functions. A series of adaptive strategies for smoothing radiusand learning rate updating are developed to improve the computational efficiencyand robustness. Our methodology is demonstrated by an example of designing ananoscale wavelength demultiplexer, and shows superior performance compared tothe state-of-the-art approaches. By incorporating volume constraints, the optimizeddesign achieves an equivalently high performance but significantly reduces theamount of material usage.
Bi, Sirui↗
Computerized detection of noncalcified plaques in coronary CT angiography: Evaluation of topological soft gradient prescreening method and luminal analysis
Purpose: The buildup of noncalcified plaques (NCPs) that are vulnerable to rupture in coronary arteries is a risk for myocardial infarction. Interpretation of coronary CT angiography (cCTA) to search for NCP is a challenging task for radiologists due to the low CT number of NCP, the large number of coronary arteries, and multiple phase CT acquisition. The authors conducted a preliminary study to develop machine learning method for automated detection of NCPs in cCTA. Methods: With IRB approval, a data set of 83 ECG-gated contrast enhanced cCTA scans with 120 NCPs was collected retrospectively from patient files. A multiscale coronary artery response and rolling balloon region growing (MSCAR-RBG) method was applied to each cCTA volume to extract the coronary arterial trees. Each extracted vessel was reformatted to a straightened volume composed of cCTA slices perpendicular to the vessel centerline. A topological soft-gradient (TSG) detection method was developed to prescreen for NCP candidates by analyzing the 2D topological features of the radial gradient field surface along the vessel wall. The NCP candidates were then characterized by a luminal analysis that used 3D geometric features to quantify the shape information and gray-level features to evaluate the density of the NCP candidates. With machine learning techniques, useful features were identified and combined into an NCP score to differentiate true NCPs from false positives (FPs). To evaluate the effectiveness of the image analysis methods, the authors performed tenfold cross-validation with the available data set. Receiver operating characteristic (ROC) analysis was used to assess the classification performance of individual features and the NCP score. The overall detection performance was estimated by free response ROC (FROC) analysis. Results: With our TSG prescreening method, a prescreening sensitivity of 92.5% (111/120) was achieved with a total of 1181 FPs (14.2 FPs/scan). On average, six features were selected during the tenfold cross-validation training. The average area under the ROC curve (AUC) value for training was 0.87 ± 0.01 and the AUC value for validation was 0.85 ± 0.01. Using the NCP score, FROC analysis of the validation set showed that the FP rates were reduced to 3.16, 1.90, and 1.39 FPs/scan at sensitivities of 90%, 80%, and 70%, respectively. Conclusions: The topological soft-gradient prescreening method in combination with the luminal analysis for FP reduction was effective for detection of NCPs in cCTA, including NCPs causing positive or negative vessel remodeling. The accuracy of vessel segmentation, tracking, and centerline identification has a strong impact on NCP detection. Studies are underway to further improve these techniques and reduce the FPs of the CADe system.
62 RADIOLOGY AND NUCLEAR MEDICINE↗
Discontinuous Galerkin finite element methods for gradient plasticity.
In this report we apply discontinuous Galerkin finite element methods to the equations of an incompatibility based formulation of gradient plasticity. The presentation is motivated with a brief overview of the description of dislocations within a crystal lattice. A tensor representing a measure of the incompatibility with the lattice is used in the formulation of a gradient plasticity model. This model is cast in a variational formulation, and discontinuous Galerkin machinery is employed to implement the formulation into a finite element code. Finally numerical examples of the model are shown.
36 MATERIALS SCIENCE↗