Search NASA⌕ Search

SEARCH · Search NASA

Results for “Loss function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Towards accurate prediction of configurational disorder properties in materials using graph neural networks

Abstract The prediction of configurational disorder properties, such as configurational entropy and order-disorder phase transition temperature, of compound materials relies on efficient and accurate evaluations of configurational energies. Previous cluster expansion methods are not applicable to configurationally-complex material systems, including those with atomic distortions and long-range orders. In this work, we propose to leverage the versatile expressive capabilities of graph neural networks (GNNs) for efficient evaluations of configurational energies and present a workflow combining attention-based GNNs and Monte Carlo simulations to calculate the disorder properties. Using the dataset of face-centered tetragonal gold copper without and with local atomic distortions as an example, we demonstrate that the proposed data-driven framework enables the prediction of phase transition temperatures close to experimental values. We also elucidate that the variance of the energy deviations among configurations controls the prediction accuracy of disorder properties and can be used as the target loss function when training and selecting the GNN models. The work serves as a fundamental step toward a data-driven paradigm for the accelerated design of configurationally-complex functional material systems.

Chemistry↗
DOE OSTI2025

Accelerating charge estimation in molecular dynamics simulations using physics-informed neural networks: corrosion applications

Molecular Dynamics (MD) simulations are used to understand the effects of corrosion on metallic materials in salt brine. Reactive force fields in classical MD enable accurate modeling of bond formation and breakage in the aqueous medium and at the metal-electrolyte interface, while also facilitating dynamic partial charge equilibration. However, MD simulations are computationally intensive and unsuitable for modeling the long time scales characteristic of corrosive phenomena. To address this, we develop reduced-order machine learning models that provide accurate and efficient predictions of charge density in corrosive environments. Specifically, we use Long Short-Term Memory (LSTM) networks to forecast charge density evolution based on atomic environments represented by Smooth Overlap of Atomic Positions (SOAP) descriptors. A physics-informed loss function enforces charge neutrality and electronegativity equivalence. The atomic charges predicted by the deep learning model trained on this work were obtained two orders of magnitude faster than those from molecular dynamics (MD) simulations, with an error of less than 3% compared to the MD-obtained charges, even in extrapolative scenarios, while adhering to physical constraints. This demonstrates the excellent accuracy, computational efficiency, and validity of the developed model. Lastly, even though developed for corrosion, these protocols are formulated in a phenomenon-agnostic manner, allowing application to various variable-charge interatomic potentials and related fields.

Atomistic models↗
DOE OSTI2024

Quantifying local and global mass balance errors in physics-informed neural networks

Physics-informed neural networks (PINN) have recently become attractive for solving partial differential equations (PDEs) that describe physics laws. By including PDE-based loss functions, physics laws such as mass balance are enforced softly in PINN. This paper investigates how mass balance constraints are satisfied when PINN is used to solve the resulting PDEs. We investigate PINN’s ability to solve the 1D saturated groundwater flow equations (diffusion equations) for homogeneous and heterogeneous media and evaluate the local and global mass balance errors. We compare the obtained PINN’s solution and associated mass balance errors against a two-point finite volume numerical method and the corresponding analytical solution. We also evaluate the accuracy of PINN in solving the 1D saturated groundwater flow equation with and without incorporating hydraulic heads as training data. We demonstrate that PINN’s local and global mass balance errors are significant compared to the finite volume approach. Tuning the PINN’s hyperparameters, such as the number of collocation points, training data, hidden layers, nodes, epochs, and learning rate, did not improve the solution accuracy or the mass balance errors compared to the finite volume solution. Mass balance errors could considerably challenge the utility of PINN in applications where ensuring compliance with physical and mathematical properties is crucial.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Application-specific machine-learned interatomic potentials: exploring the trade-off between DFT convergence, MLIP expressivity, and computational cost

Machine-learned interatomic potentials (MLIPs) are revolutionizing computational materials science and chemistry by offering an efficient alternative to ab initio molecular dynamics (MD) simulations. However, fitting high-quality MLIPs remains a challenging, time-consuming, and computationally intensive task where numerous trade-offs have to be considered, e.g., How much and what kind of atomic configurations should be included in the training set? Which level of ab initio convergence should be used to generate the training set? Which loss function should be used for fitting the MLIP? Which machine learning architecture should be used to train the MLIP? The answers to these questions significantly impact both the computational cost of MLIP training and the accuracy and computational cost of subsequent MLIP MD simulations. In this study, we use a configurationally diverse beryllium dataset and quadratic spectral neighbor analysis potential. We demonstrate that joint optimization of energy versus force weights, training set selection strategies, and convergence settings of the ab initio reference simulations, as well as model complexity can lead to a significant reduction in the overall computational cost associated with training and evaluating MLIPs. This opens the door to computationally efficient generation of high-quality MLIPs for a range of applications which demand different accuracy versus training and evaluation cost trade-offs.

36 MATERIALS SCIENCE↗
DOE OSTI2025

FL‐ADS: Federated learning anomaly detection system for distributed energy resource networks

Abstract With the ongoing development of Distributed Energy Resources (DER) communication networks, the imperative for strong cybersecurity and data privacy safeguards is increasingly evident. DER networks, which rely on protocols such as Distributed Network Protocol 3 and Modbus, are susceptible to cyberattacks such as data integrity breaches and denial of service due to their inherent security vulnerabilities. This paper introduces an innovative Federated Learning (FL)‐based anomaly detection system designed to enhance the security of DER networks while preserving data privacy. Our models leverage Vertical and Horizontal Federated Learning to enable collaborative learning while preserving data privacy, exchanging only non‐sensitive information, such as model parameters, and maintaining the privacy of DER clients' raw data. The effectiveness of the models is demonstrated through its evaluation on datasets representative of real‐world DER scenarios, showcasing significant improvements in accuracy and F1‐score across all clients compared to the traditional baseline model. Additionally, this work demonstrates a consistent reduction in loss function over multiple FL rounds, further validating its efficacy and offering a robust solution that balances effective anomaly detection with stringent data privacy needs.

Purohit, Shaurya [Iowa State University Ames Iowa ↗
DOE OSTI2025

A physics-constrained neural ordinary differential equations approach for robust learning of stiff chemical kinetics

The high computational cost associated with solving for detailed chemistry poses a significant challenge for predictive computational fluid dynamics (CFD) simulations of turbulent reacting flows. While deep learning techniques have been explored to develop faster surrogate models, they often fail to integrate reliably with CFD solvers. This instability arises because traditional deep learning approaches optimize for training error without ensuring compatibility with ordinary differential equation (ODE) solvers, resulting in accumulation of errors over time. Recently, neuralODE (NODE) based approaches have been shown to be a promising technique to emulate and accelerate detailed chemistry computations. Here, in the present work, we extend this NODE framework for stiff chemical kinetics by incorporating mass conservation constraints directly into the loss function during training. This ensures that the total mass as well as the individual elemental species masses are conserved in an a-posteriori manner. Proof-of-concept studies are performed with the novel physics-constrained NODE (PC-NODE) approach for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. It is demonstrated that the PC-NODE framework not only improves the physical consistency of the resulting data-driven model with respect to mass conservation criteria, but also improves training efficiency. PC-NODE is shown to achieve 2–100× speedup relative to the hydrogen-air detailed chemical mechanism depending on the type of the ODE solver (implicit or explicit) used during autoregressive inference tests. Lastly, a-posteriori studies are performed wherein the trained PC-NODE model is coupled with a CFD solver. It is shown that higher accuracy is achieved with PC-NODE relative to the purely data-driven NODE approach. Moreover, PC-NODE also exhibits robustness and generalizability to unseen initial conditions from within (interpolative capability) as well as outside (extrapolative capability) the training regime.

computational combustion↗
DOE OSTI2026

The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameters

In scientific applications, predictive modeling is often of limited use without accurate uncertainty quantification (UQ) to indicate when a model may be extrapolating or when more data needs to be collected. Bayesian Neural Networks (BNNs) produce predictive uncertainty by propagating uncertainty in neural network (NN) weights and offer the promise of obtaining not only an accurate predictive model but also accurate UQ. However, in practice, obtaining accurate UQ with BNNs is difficult due in part to the approximations used for model training (such as those made in variational inference) and in part to the need to choose a suitable set of hyperparameters; these hyperparameters outnumber those needed for traditional NNs and often have opaque effects on the results. We aim to shed light on the effects of hyperparameter choices for variational BNNs by performing a global sensitivity analysis of variational BNN performance under varying hyperparameter settings. Our results indicate that many of the hyperparameters interact with each other to affect both predictive accuracy and UQ. For improved usage of variational BNNs in real-world applications, we suggest that thorough hyperparameter tuning, including tuning of prior hyperparameters and loss function parameters, is essential for accurate UQ in variational BNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Deep probabilistic direction prediction in 3D with applications to directional dark matter detectors

Abstract We present the first method to probabilistically predict 3D direction in a deep neural network model. The probabilistic predictions are modeled as a heteroscedastic von Mises-Fisher distribution on the sphere S 2 , giving a simple way to quantify aleatoric uncertainty. This approach generalizes the cosine distance loss which is a special case of our loss function when the uncertainty is assumed to be uniform across samples. We develop approximations required to make the likelihood function and gradient calculations stable. The method is applied to the task of predicting the 3D directions of electrons, the most complex signal in a class of experimental particle physics detectors designed to demonstrate the particle nature of dark matter and study solar neutrinos. Using simulated Monte Carlo data, the initial direction of recoiling electrons is inferred from their tortuous trajectories, as captured by the 3D detectors. For 40 keV electrons in a 70% He 30% CO 2 gas mixture at STP, the new approach achieves a mean cosine distance of 0.104 (26 ∘ ) compared to 0.556 (64 ∘ ) achieved by a non-machine learning algorithm. We show that the model is well-calibrated and accuracy can be increased further by removing samples with high predicted uncertainty. This advancement in probabilistic 3D directional learning could increase the sensitivity of directional dark matter detectors.

Computer Science↗
DOE OSTI2024

Simultaneous energy and mass calibration of large-radius jets with the ATLAS detector using a deep neural network

The energy and mass measurements of jets are crucial tasks for the Large Hadron Collider experiments. This paper presents a new calibration method to simultaneously calibrate these quantities for large-radius jets measured with the ATLAS detector using a deep neural network (DNN). To address the specificities of the calibration problem, special loss functions and training procedures are employed, and a complex network architecture, which includes feature annotation and residual connection layers, is used. The DNN-based calibration is compared to the standard numerical approach in an extensive series of tests. The DNN approach is found to perform significantly better in almost all of the tests and over most of the relevant kinematic phase space. In particular, it consistently improves the energy and mass resolutions, with a 30% better energy resolution obtained for transverse momenta $p$ T > $500$ GeV.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Active causal learning for decoding chemical complexities with targeted interventions

Abstract Predicting and enhancing inherent properties based on molecular structures is paramount to design tasks in medicine, materials science, and environmental management. Most of the current machine learning and deep learning approaches have become standard for predictions, but they face challenges when applied across different datasets due to reliance on correlations between molecular representation and target properties. These approaches typically depend on large datasets to capture the diversity within the chemical space, facilitating a more accurate approximation, interpolation, or extrapolation of the chemical behavior of molecules. In our research, we introduce an active learning approach that discerns underlying cause-effect relationships through strategic sampling with the use of a graph loss function. This method identifies the smallest subset of the dataset capable of encoding the most information representative of a much larger chemical space. The identified causal relations are then leveraged to conduct systematic interventions, optimizing the design task within a chemical space that the models have not encountered previously. While our implementation focused on the QM9 quantum-chemical dataset for a specific design task—finding molecules with a large dipole moment—our active causal learning approach, driven by intelligent sampling and interventions, holds potential for broader applications in molecular, materials design and discovery.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Physics-guided dual implicit neural representations for source separation

Significant challenges exist in efficient data analysis of most advanced experimental and observational techniques because the collected signals often include unwanted contributions, such as background and signal distortions, that can obscure the physically relevant information of interest. To address this, we have developed a self-supervised machine-learning approach for source separation using a dual implicit neural representation framework that jointly trains two neural networks: one for approximating distortions of the physical signal of interest and the other for learning the effective background contribution. Our method learns directly from the raw data by minimizing a reconstruction-based loss function without requiring labeled data or pre-defined dictionaries. We demonstrate the effectiveness of our framework by considering a challenging case study involving large-scale simulated, as well as experimental, momentum-energy-dependent inelastic neutron scattering data in a four-dimensional parameter space, characterized by heterogeneous background contributions and unknown distortions to the target signal. The method is found to successfully separate physically meaningful signals from a complex or structured background even when the signal characteristics vary across all four dimensions of the parameter space. An analytical approach that informs the choice of the regularization parameter is presented. Our method offers a versatile framework for addressing source separation problems across diverse domains, ranging from superimposed signals in astronomical measurements to structural features in biomedical image reconstructions.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Data imbalance in drug response prediction: multi-objective optimization approach in deep learning setting

Abstract Drug response prediction (DRP) methods tackle the complex task of associating the effectiveness of small molecules with the specific genetic makeup of the patient. Anti-cancer DRP is a particularly challenging task requiring costly experiments as underlying pathogenic mechanisms are broad and associated with multiple genomic pathways. The scientific community has exerted significant efforts to generate public drug screening datasets, giving a path to various machine learning models that attempt to reason over complex data space of small compounds and biological characteristics of tumors. However, the data depth is still lacking compared to application domains like computer vision or natural language processing domains, limiting current learning capabilities. To combat this issue and improves the generalizability of the DRP models, we are exploring strategies that explicitly address the imbalance in the DRP datasets. We reframe the problem as a multi-objective optimization across multiple drugs to maximize deep learning model performance. We implement this approach by constructing Multi-Objective Optimization Regularized by Loss Entropy loss function and plugging it into a Deep Learning model. We demonstrate the utility of proposed drug discovery methods and make suggestions for further potential application of the work to achieve desirable outcomes in the healthcare field.

Biochemistry & Molecular Biology↗
DOE OSTI2026

MAL33 drives natural variation in maltose metabolism in Saccharomyces eubayanus

Maltose is one of the most abundant sugars in brewer’s wort, and its efficient utilization is critical for successful fermentation. However, maltose consumption varies naturally among Saccharomyces eubayanus strains isolated from different host trees, such as Quercus and Nothofagus. To identify the genetic determinants underlying these phenotypic differences, we performed bulk segregant analysis (BSA) and quantitative trait loci (QTL) mapping using an F 2 offspring derived from QC18 (Quercus-associated) and CL467.1 (Nothofagus-associated) strains. QTL mapping identified two significant genomic regions on subtelomeric loci of chromosomes V-R and XVI-L, each containing complete MAL loci composed of MAL32 (encoding maltase), MAL31 (transporter), and MAL33 (transcriptional activator) genes. Comparative polymorphism analyses identified mutations in MAL32 and MAL33 of QC18, including frameshift mutations resulting in premature stop codons. Functional validation demonstrated that the heterologous expression of MAL33 ChrV from CL467.1 fully restored maltose utilization in QC18, indicating the functional presence of MAL33 cis-regulatory sequences and MAL32 and MAL31 genes in QC18. While structural protein predictions identified truncation and impaired functionality in the maltose-responsive activation domain of Mal33p from QC18, overexpression of QC18’s own MAL33 ChrV allele also improved maltose metabolism, suggesting dosage-dependent transcriptional limitations rather than complete functional loss. These results indicate that allelic variations in the maltose-responsive activation domain of Mal33p result in differences in maltose consumption between strains. Here, we hypothesized that reduced maltose metabolism in QC18 is an adaptive response to the distinct sugar composition in Quercus robur bark, contrasting with the starch-rich environment of Nothofagus pumilio. These findings highlight subtelomeric MAL gene diversity as a reservoir of genetic variation, representing a key evolutionary mechanism that influences maltose adaptation among natural Saccharomyces isolates.

evolutionary plasticity↗
DOE OSTI2026

Wilson loops with neural networks

Wilson loops are essential objects in QCD and have been pivotal in scale setting and demonstrating confinement. Various generalizations are crucial for computations needed in effective field theories. In lattice gauge theory, Wilson loop calculations face challenges, including excited-state contamination at short times and the signal-to-noise ratio issue at longer times. To address these problems, we develop a new method by using neural networks to parametrize interpolators for the static quark-antiquark pair. We construct gauge-equivariant layers for the network and train it to find the ground state of the system. The trained network itself is then treated as our new observable for the inference. Our results demonstrate a significant improvement in the signal compared to traditional Wilson loops, performing as well as Coulomb-gauge Wilson-line correlators while maintaining gauge invariance. Additionally, we present an example where the optimized ground state is used to measure the static force directly, as well as another example combining this method with the multilevel algorithm. Finally, we extend the formalism to find excited-state interpolators for static quark-antiquark systems. To our knowledge, this work is the first study of neural networks with a physically motivated loss function for Wilson loops.

Bellscheidt, Verena [Massachusetts Inst. of Techno↗
DOE OSTI2025

Introducing a multiscale feature integration network for inpainting with applications to enhanced CMB map reconstruction

We introduce a novel neural network, SkyReconNet, which combines the expanded receptive fields of dilated convolutional layers along with standard convolutions, to capture both the global and local features for reconstructing the missing information in an image. We implement our network to inpaint the masked regions in a full-sky cosmic microwave background (CMB) map. Inpainting CMB maps is a particularly formidable challenge when dealing with extensive and irregular masks, such as galactic masks which can obscure substantial fractions of the sky. The hybrid design of SkyReconNet leverages the strengths of standard and dilated convolutions to accurately predict CMB fluctuations in the masked regions by effectively utilizing the information from surrounding unmasked areas. During training, the network optimizes its weights by minimizing a composite loss function that combines the structural similarity index measure (SSIM) and mean squared error (MSE). SSIM preserves the essential structural features of the CMB, ensuring an accurate and coherent reconstruction of the missing CMB fluctuations, while MSE minimizes the pixelwise deviations, thus enhancing the overall accuracy of the predictions. The predicted CMB maps and their corresponding angular power spectra align closely with the targets, achieving the performance limited only by the fundamental uncertainty of cosmic variance. The network’s generic architecture enables application to other physics-based challenges involving data with missing or defective pixels, systematic artifacts, etc. In conclusion, our results demonstrate its effectiveness in addressing the challenges posed by large irregular masks, offering a significant inpainting tool not only for CMB analyses but also for image-based experiments across disciplines where such data imperfections are prevalent.

Cosmic microwave background↗
DOE OSTI2025

Optimizers for stabilizing likelihood-free inference

A growing number of applications in particle physics and beyond use neural networks as unbinned likelihood ratio estimators applied to real or simulated data. Precision requirements on the inference tasks demand a high-level of stability from these networks, which are affected by the stochastic nature of training. We show how physics concepts can be used to stabilize network training through a physics-inspired optimizer. In particular, the energy conserving descent (ECD) optimization framework uses classical Hamiltonian dynamics on the space of network parameters to reduce the dependence on the initial conditions while also stabilizing the result near the minimum of the loss function. We develop a version of this optimizer known as , which has few free hyperparameters with limited ranges guided by physical reasoning. We apply to representative likelihood-ratio estimation tasks in particle physics and find on average that it out-performs the widely used Adam optimizer. We expect that ECD will be a useful tool for wide array of data-limited problems, where it is computationally expensive to exhaustively optimize hyperparameters and mitigate fluctuations with ensembling.

Monte Carlo methods↗
DOE OSTI2024

Mixture-of-Experts for Multi-Domain Defect Identification in Non-Destructive Inspection

Composite materials are widely used in aircraft structures because of their superior mechanical properties. However, their complex failure modes require sophisticated inspection methods to ensure structural integrity. Ultrasonic testing (UT) is a common non-destructive inspection (NDI) technique for aircraft composites that can detect internal and external defects with high resolution and accuracy. Despite their effectiveness, traditional UT methods rely on the manual interpretation of ultrasonic signals, which is time-consuming, labor-intensive, and subjective. Furthermore, processing such large-scale data, particularly across materials of varying thicknesses, significantly increases the computational demands of deep learning model optimization. To overcome these challenges, we propose an efficient sparse mixture-of-experts (MoE) model with a multi-level loss function and introduce four novel training objectives to improve computational efficiency and accuracy in identifying surface defects in composite aircraft materials. Here, we evaluated our approach on material with multiple thicknesses or domains comprising various defects. Our experimental results demonstrate higher accuracy and F1-Score, with only 10% training epochs compared to baseline MoE.

composite materials↗
DOE OSTI2024

Deep Learning-Based Weather-Related Power Outage Prediction with Socio-Economic and Power Infrastructure Data

This paper presents a deep learning-based approach for hourly power outage probability prediction within census tracts encompassing a utility company's service territory. Two distinct deep learning models, conditional Multi-Layer Perceptron (MLP) and unconditional MLP, were developed to forecast power outage probabilities, leveraging a rich array of input features gathered from publicly available sources including weather data, weather station locations, power infrastructure maps, socio-economic and demographic statistics, and power outage records. Given a one-hour-ahead weather forecast, the models predict the power outage probability for each census tract, taking into account both the weather prediction and the location's characteristics. The deep learning models employed different loss functions to optimize prediction performance. Our experimental results underscore the significance of socio-economic factors in enhancing the accuracy of power outage predictions at the census tract level.

24 POWER TRANSMISSION AND DISTRIBUTION↗