Search NASA⌕ Search

SEARCH · Search NASA

Results for “Gradient method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Explaining machine-learning models for gamma-ray detection and identification

As more complex predictive models are used for gamma-ray spectral analysis, methods are needed to probe and understand their predictions and behavior. Recent work has begun to bring the latest techniques from the field of Explainable Artificial Intelligence (XAI) into the applications of gamma-ray spectroscopy, including the introduction of gradient-based methods like saliency mapping and Gradient-weighted Class Activation Mapping (Grad-CAM), and black box methods like Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). In addition, new sources of synthetic radiological data are becoming available, and these new data sets present opportunities to train models using more data than ever before. In this work, we use a neural network model trained on synthetic NaI(Tl) urban search data to compare some of these explanation methods and identify modifications that need to be applied to adapt the methods to gamma-ray spectral data. We find that the black box methods LIME and SHAP are especially accurate in their results, and recommend SHAP since it requires little hyperparameter tuning. We also propose and demonstrate a technique for generating counterfactual explanations using orthogonal projections of LIME and SHAP explanations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

The Effects of Compounded Model Size Reductions on Adversarial Robustness

Recent advances in Edge AI and Tiny Machine Learning (TinyML) have enabled the deployment of machine learning models on resource-constrained environments. However, deploying these models on edge devices, such as micro-controllers, requires significant model footprint reduction through a variety of techniques such as quantization, pruning, and clustering. While these optimization methods offer considerable advantages, they potentially introduce AI-related security vulnerabilities, particularly concerning model robustness with respect to adversarial AI attacks. Prior research has extensively examined the impact of quantization on adversarial robustness; however, the effects of alternative reduction techniques and their combinations remain understudied. This paper investigates the impact of model size reduction techniques on adversarial robustness, when applied individually and combined. We utilized Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks to generate adversarial perturbations for both training and testing data, and then evaluated the models' accuracy under adversarial training conditions. Our findings revealed that reduction techniques generally diminished robustness; although, combining techniques was not found to make robustness any worse than when applied individually. Moreover, specific techniques can potentially enhance resistance to small size perturbations. This research provides insights into the trade-offs between model size reduction and security, establishing a foundation for future investigations into improving adversarial training techniques and methodologies for maintaining robustness while preserving memory footprint benefits.

Austria, Phillipe [ORNL] (ORCID:0000000236223973)↗
DOE OSTI2024

Chemically Amplified, Dry-Develop Poly(aldehyde) Photoresist

The catalytic decomposition of poly(phthalaldehyde) with a photoacid generator can be used as dry-develop photoresist, where the exposed film depolymerizes into small molecules to allow the development of features via controlled vaporization. Higher temperatures enabled shorter dry-development times, but also promoted faster photoacid diffusion that compromised pattern fidelity. Trihexylamine was used as a base quencher to counteract acid diffusion in a phthalaldehyde-propanal co-polymer photoresist. The propanal co-monomer in the polymer improves the vaporization rate because it has a higher vapor pressure than phthalaldehyde. Addition of the base quencher was found to improve the contrast, pattern fidelity, and ease-of-handling of the dry-develop resist in a direct-write UV lithography tool. The dry-development of 4 μ m features was achieved with no appreciable residue. For large area features, a spatially variable exposure method was used to direct the residue away from the exposed area. The gradient exposure method was used to produce 100 μ m features. Plasma etching after dry-development was also used to achieve residue-free dry-developed patterns. These results show the benefits of incorporating base additives into a dry-develop depolymerizable resist system and highlight the need for addressing residue formation.

Materials Science↗
DOE OSTI2021

Preliminary Theoretical Analysis of Mixed Precision Krylov Subspace Methods (Q3 Report)

The third quarter of the project was spent performing theoretical finite precision analysis of Krylov subspace method variants that use mixed precision. Our focus here is on the Conjugate Gradient (CG) method and the Lanczos method. We have performed an analysis of maximum attainable accuracy for the classical CG method in which 3 precisions are used: a working precision ε, a precision ε IP for the inner product computations, and a precision ε MV for the matrix-vector products. Our results show that performing inner product computations in lower precision does not affect the attainable accuracy. Further, we have performed a complete error analysis of the s-step Lanczos algorithm. In this case, we show that the numerical behavior of the algorithm can be significantly improved by using extra precision in a small part of the computation. We summarize the main theorems in the remainder of the document. Other activities include attending biweekly xSDK meetings. The subsequent quarter will be spent finalizing these results into technical reports and/or manuscripts for submission to journals, as well as identifying opportunities for future work.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Significance of Low‐Velocity Zones on Solute Retention in Rough Fractures

Natural fractures are characterized by high internal heterogeneity. This internal variability is the cause of flow channeling, which in turn leads to contaminant transport taking place primarily along the high-velocity channels. Mass exchange between the high-velocity channels and the low-velocity zones has the potential to enhance contaminant retention, due to solute diffusion into the low-velocity zones and subsequent exposure to additional surface area for diffusion into the bordering rock matrix. Here, we derive a random walk particle tracking method for heterogeneous fractures, which includes an additional term to account for the aperture gradient. The method takes into account advection, diffusion in the fracture and matrix diffusion. The developed numerical framework is applied to assess the effect of low-velocity zones in rough self-affine fractures. The results show that diffusion into low-velocity zones has a visible but modest impact on contaminant retention. The magnitude of this impact does not change considerably, regardless of whether diffusion into the rock matrix is considered in the model, and increases for a decreasing average Péclet number of the fracture.

58 GEOSCIENCES↗
DOE OSTI2024

A Hierarchical OPF Algorithm with Improved Gradient Evaluation in Three-Phase Networks

Linear approximation commonly used in solving alternating-current optimal power flow (AC-OPF) simplifies the system models but incurs accumulated voltage errors in large power networks. Such errors will make the primal-dual type gradient algorithms converge to solutions with voltage violation. In this paper, we improve a recent hierarchical OPF algorithm that rested on primal-dual gradients evaluated with a linearized distribution power flow model. Specifically, we propose a more accurate gradient evaluation method based on an unbalanced three-phase nonlinear distribution power flow model to mitigate the errors arising from linearization. The resultant gradients feature a blocked structure that enables our development of an improved hierarchical primal-dual algorithm to solve the OPF problem. Numerical results on the IEEE 123-bus test feeder and a 4,518-node test feeder show that the proposed method can enhance voltage safety at comparable computational efficiency with the linearized algorithm.

approximation algorithms↗
DOE OSTI2025

Accelerating eigenvalue computation for nuclear structure calculations via perturbative corrections

Subspace projection methods utilizing perturbative corrections have been proposed for computing the lowest few eigenvalues and corresponding eigenvectors of large Hamiltonian matrices. In this paper, we build upon these methods and introduce the term Subspace Projection with Perturbative Corrections (SPPC) method to refer to this approach. We tailor the SPPC for nuclear many-body Hamiltonians represented in a truncated configuration interaction subspace, i.e., the no-core shell model (NCSM). We use the hierarchical structure of the NCSM Hamiltonian to partition the Hamiltonian as the sum of two matrices. The first matrix corresponds to the Hamiltonian represented in a small configuration space, whereas the second is viewed as the perturbation to the first matrix. Eigenvalues and eigenvectors of the first matrix can be computed efficiently. Because of the split, perturbative corrections to the eigenvectors of the first matrix can be obtained efficiently from the solutions of a sequence of linear systems of equations defined in the small configuration space. These correction vectors can be combined with the approximate eigenvectors of the first matrix to construct a subspace from which more accurate approximations of the desired eigenpairs can be obtained. We show by numerical examples that the SPPC method can be more efficient than conventional iterative methods for solving large-scale eigenvalue problems such as the Lanczos, block Lanczos and the locally optimal block preconditioned conjugate gradient (LOBPCG) method. The method can also be combined with other methods to avoid convergence stagnation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

DP-TwoLevel: two-stage gradient subspace learning for differentially private federated learning

Federated learning (FL) enables collaborative model training across distributed data sources without sharing raw data, but faces fundamental challenges in communication efficiency and privacy. Differentially private (DP) training mitigates information leakage but introduces noise that degrades model performance, especially in high-dimensional settings. We propose DP-TwoLevel, a hierarchical gradient projection method that improves utility under fixed DP constraints by exploiting low-dimensional structure in model updates. Our approach learns a two-level PCA-based representation of gradients and applies DP noise in a reduced-dimensional subspace, thereby lowering the effective noise magnitude while preserving dominant signal components. We evaluate the method across three datasets (MNIST, Fashion-MNIST, CIFAR-10) and three privacy regimes (ϵ∈0.5, 1.0, 2.0). Across nine experimental settings, DP-TwoLevel consistently outperforms DP-FedAvg, achieving an average accuracy improvement of 9.44%, with larger gains observed in lower ϵ(higher-noise) regimes (up to +22.31%). We further analyze scalability across models ranging from 100K to 1.49M parameters and identify a variance-based success criterion: performance remains strong when the projection preserves more than 75% of gradient variance, degrades in a marginal regime (65–75%), and fails below this threshold. Our results demonstrate that structure-aware dimensionality reduction can significantly improve the privacy–utility tradeoff in FL without modifying formal privacy guarantees. We also provide empirical evidence of scaling limitations for global projections and motivate per-layer extensions for larger models.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗
DOE OSTI2024

AI-Driven Detector Design for the EIC (Final Technical Report)

We developed an optimization workflow based on DNN-based fast-simulation and reconstruction algorithms. We used these methods to advance the design of calorimeter systems for the Electron-Ion Collider (EIC). This DNN-driven optimization provides a blueprint for integrating gradient-based methods into detector-design workflows. All software pipelines and methods have been released publicly and incorporated into the EIC collaboration’s physics studies, broadening their impact. Three journal articles detailing the methods developed here serve as a reference for the design and optimal use of next generation high-granularity calorimeter systems in nuclear and particle physics.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Classifying and analyzing small-angle scattering data using weighted k nearest neighbors machine learning techniques

A consistent challenge for both new and expert practitioners of small-angle scattering (SAS) lies in determining how to analyze the data, given the limited information content of said data and the large number of models that can be employed. Machine learning (ML) methods are powerful tools for classifying data that have found diverse applications in many fields of science. Here, ML methods are applied to the problem of classifying SAS data for the most appropriate model to use for data analysis. The approach employed is built around the method of weighted k nearest neighbors (wKNN), and utilizes a subset of the models implemented in the SasView package (https://www.sasview.org/) for generating a well defined set of training and testing data. The prediction rate of the wKNN method implemented here using a subset of SasView models is reasonably good for many of the models, but has difficulty with others, notably those based on spherical structures. A novel expansion of the wKNN method was also developed, which uses Gaussian processes to produce local surrogate models for the classification, and this significantly improves the classification accuracy. Further, by integrating a stochastic gradient descent method during post-processing, it is possible to leverage the local surrogate model both to classify the SAS data with high accuracy and to predict the structural parameters that best describe the data. The linking of data classification and model fitting has the potential to facilitate the translation of measured data into results for both novice and expert practitioners of SAS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Nonlinear Matrix Approximation with Radial Basis Function Components

We introduce and investigate matrix approximation by decomposition into a sum of radial basis function (RBF) components. An RBF component is a generalization of the outer product between a pair of vectors, where an RBF function replaces the scalar multiplication between individual vector elements. Even though the RBF functions are positive definite, the summation across components is not restricted to convex combinations and allows us to compute the decomposition for any real matrix that is not necessarily symmetric or positive definite. We formulate the problem of seeking such a decomposition as an optimization problem with a nonlinear and non-convex loss function. Several modern versions of the gradient descent method, including their scalable stochastic counterparts, are used to solve this problem. We provide extensive empirical evidence of the effectiveness of the RBF decomposition and that of the gradient-based fitting algorithm. While being conceptually motivated by singular value decomposition (SVD), our proposed nonlinear counterpart outperforms SVD by drastically reducing the memory required to approximate a data matrix with the same L2 error for a wide range of matrix types. For example, it leads to 2 to 6 times memory save for Gaussian noise, graph adjacency matrices, and kernel matrices. Moreover, this proximity-based decomposition can offer additional interpretability in applications that involve, e.g., capturing the inner low-dimensional structure of the data, retaining graph connectivity structure, and preserving the acutance of images.

Rebrova, Elizaveta↗
DOE OSTI2023

Securely Aggregated Coded Matrix Inversion

Coded computing is a method for mitigating straggling workers in a centralized computing network, by using erasure-coding techniques. Federated learning is a decentralized model for training data distributed across client devices. In this work we propose approximating the inverse of an aggregated data matrix, where the data is generated by clients; similar to the federated learning paradigm, while also being resilient to stragglers. To do so, we propose a coded computing method based on gradient coding. We modify this method so that the coordinator does not access the local data at any point; while the clients access the aggregated matrix in order to complete their tasks. Here, the network we consider is not centrally administrated, and the communications which take place are secure against potential eavesdroppers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Randomized Preconditioned Solvers for Strong Constraint 4D-Var Data Assimilation

The Strong Constraint 4D Variational (SC-4DVAR) data assimilation method is widely used in climate and weather applications. SC-4DVAR involves solving a minimization problem to compute the maximum a posteriori estimate, which we tackle using the Gauss-Newton method. The computation of the descent direction is expensive since it involves the solution of a large-scale and potentially ill-conditioned linear system, solved using the preconditioned conjugate gradient (PCG) method. Here, to address this cost, we efficiently construct scalable preconditioners using three different randomization techniques, which all rely on a certain low-rank structure involving the Gauss-Newton Hessian. The proposed techniques come with theoretical guarantees on the condition number, and at the same time, are amenable to parallelization. We also develop an adaptive approach to estimate the sketch size and choose between the reuse or recomputation of the preconditioner. We demonstrate the performance and effectiveness of our methodology on two representative model problems—the Burgers and barotropic vorticity equation—showing a drastic reduction in both the number of PCG iterations and the number of Gauss-Newton Hessian products after including the preconditioner construction cost.

Gauss-Newton↗
DOE OSTI2022

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs). PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. However, one disadvantage of the first generation of PINNs is that they usually have limited accuracy even with many training points. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs), for improving the accuracy of PINNs. gPINNs leverage gradient information of the PDE residual and embed the gradient into the loss function. We tested gPINNs extensively and demonstrated the effectiveness of gPINNs in both forward and inverse PDE problems. Our numerical results show that gPINN performs better than PINN with fewer training points. Additionally, we combined gPINN with the method of residual-based adaptive refinement (RAR), a method for improving the distribution of training points adaptively during training, to further improve the performance of gPINN, especially in PDEs with solutions that have steep gradients.

42 ENGINEERING↗
DOE OSTI2021

Large gradients via correlation in random parameterized quantum circuits

Scaling of variational quantum algorithms to large problem sizes requires efficient optimization of random parameterized quantum circuits. For such circuits with uncorrelated parameters, the presence of exponentially vanishing gradients in cost function landscapes is an obstacle to optimization by gradient descent methods. In this work, we prove that reducing the dimensionality of the parameter space by utilizing circuit modules containing spatially or temporally correlated gate layers can allow one to circumvent the vanishing gradient phenomenon. Here, examples are drawn from random separable circuits and asymptotically optimal variational versions of Grover's algorithm based on the quantum alternating operator ansatz. In the latter scenario, our bounds on cost function variation imply a transition between vanishing gradients and efficient trainability as the number of layers is increased toward $\mathcal{O}\left({2}^{n/2}\right)$, the optimal oracle complexity of quantum unstructured search.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Distributed-Memory Parallel JointNMF

Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

Eswar, Srinivas↗
DOE OSTI2024

A data-driven sensor placement approach for detecting voltage violations in distribution systems

Stochastic fluctuations in power injections from distributed energy resources (DERs) combined with load variability can cause constraint violations (e.g., exceeded voltage limits) in electric distribution systems. To monitor grid operations, sensors are placed to measure important quantities such as the voltage magnitudes. Here, in this paper, we consider a sensor placement problem which seeks to identify locations for installing sensors that can capture all possible violations of voltage magnitude limits. We formulate a bilevel optimization problem that minimizes the number of sensors and avoids false sensor alarms in the upper level while ensuring detection of any voltage violations in the lower level. This problem is challenging due to the nonlinearity of the power flow equations and the presence of binary variables. Accordingly, we employ recently developed conservative linear approximations of the power flow equations that overestimate or underestimate the voltage magnitudes. By replacing the nonlinear power flow equations with conservative linear approximations, we can ensure that the resulting sensor locations and thresholds are sufficient to identify any constraint violations. Additionally, we apply various problem reformulations to significantly improve computational tractability while simultaneously ensuring an appropriate placement of sensors. Lastly, we improve the quality of the results via an approximate gradient descent method that adjusts the sensor thresholds. We demonstrate the effectiveness of our proposed method for several test cases, including a system with multiple switching configurations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Scale setting the Möbius domain wall fermion on gradient-flowed HISQ action using the omega baryon mass and the gradient-flow scales 𝑡 0 and 𝑤 0

We report on a subpercent scale determination using the omega baryon mass and gradient-flow methods. The calculations are performed on 22 ensembles of 𝑁 𝑓 =2 +1 +1 highly improved, rooted staggered sea-quark configurations generated by the MILC and CalLat Collaborations. The valence quark action used is Möbius domain wall fermions solved on these configurations after a gradient-flow smearing is applied with a flowtime of 𝑡 gf = 1 in lattice units. The ensembles span four lattice spacings in the range 0.06 ≲ 𝑎 ≲0.15 fm, six pion masses in the range 130 ≲ 𝑚 𝜋 ≲ 400 MeV and multiple lattice volumes. On each ensemble, the gradient-flow scales 𝑡 0 /𝑎 2 and 𝑤 0 /𝑎 and the omega baryon mass 𝑎⁢𝑚 Ω are computed. The dimensionless product of these quantities is then extrapolated to the continuum and infinite volume limits and interpolated to the physical light, strange and charm quark mass point in the isospin limit, resulting in the determination of $\sqrt{t}_0$ = 0.1422⁢(14) fm and 𝑤 0 = 0.1709⁢(11) fm with all sources of statistical and systematic uncertainty accounted for. The dominant uncertainty in both results is the stochastic uncertainty, though for $\sqrt{t}_0$ there are comparable continuum extrapolation uncertainties. For 𝑤 0 , there is a clear path for a few-per-mille uncertainty just through improved stochastic precision, as recently obtained by the Budapest-Marseille-Wuppertal Collaboration.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗