Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep metric learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Explainable artificial intelligence relates perovskite luminescence images to current-voltage metrics

As the demand for low-cost, high-efficiency solar energy technologies grows, metal halide perovskite (MHP) solar cells have emerged as a promising candidate for next-generation photovoltaics due to their high power conversion efficiencies. However, their poor durability and issues with manufacturing consistency remain significant barriers to commercialization. In this work, we develop deep learning models to support materials characterization and provide insight into features and processes influencing performance. The models are trained using transfer learning of a pretrained model to predict relevant current-voltage (IV) metrics based on different combinations of input electroluminescence (EL) and photoluminescence (PL) images of MHP devices. We examine which image types are most informative in accurately predicting different IV metrics. Additionally, we use explainable artificial intelligence (XAI) techniques to provide insights into specific spatial features in the devices that drive differences in performance. We find that stabilized luminescence images (e.g. those collected after biasing the devices for at least 1 min) are better for predicting metrics of open-circuit voltage (by PL) and short-circuit current (by PL with EL), but that predicting fill factor and overall power output may use the time-evolution of EL images. Based on attribution masks generated by integrated gradients for each device performance metric, we further suggest different loss mechanisms associated with categories of large and small spatial defects. Overall, this case study highlights the potential applicability of XAI methodology for streamlining MHP device analysis and accelerating detailed understanding of the relationships between spatial defects and impacts on performance.

14 SOLAR ENERGY↗
DOE OSTI2022

Moving the needle: Employing deep reinforcement learning to push the boundaries of coarse-grained vaccine models

Highly mutable infectious disease pathogens (hm-IDPs) such as HIV and influenza evolve faster than the human immune system can contain them, allowing them to circumvent traditional vaccination approaches and causing over one million deaths annually. Agent-based models can be used to simulate the complex interactions that occur between immune cells and hm-IDP-like proteins (antigens) during affinity maturation—the process by which antibodies evolve. Compared to existing experimental approaches, agent-based models offer a safe, low-cost, and rapid route to study the immune response to vaccines spanning a wide range of design variables. However, the highly stochastic nature of affinity maturation and vast sequence space of hm-IDPs render brute force searches intractable for exploring all pertinent vaccine design variables and the subset of immunization protocols encompassed therein. To address this challenge, we employed deep reinforcement learning to drive a recently developed agent-based model of affinity maturation to focus sampling on immunization protocols with greater potential to improve the chosen metrics of protection, namely the broadly neutralizing antibody (bnAb) titers or fraction of bnAbs produced. Using this approach, we were able to coarse-grain a wide range of vaccine design variables and explore the relevant design space. Our work offers new testable insights into how vaccines should be formulated to maximize protective immune responses to hm-IDPs and how they can be minimally tailored to account for major sources of heterogeneity in human immune responses and various socioeconomic factors. Our results indicate that the first 3 to 5 immunizations, depending on the metric of protection, should be specially tailored to achieve a robust protective immune response, but that beyond this point further immunizations require only subtle changes in formulation to sustain a durable bnAb response.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Modeling Spatial Asymmetries in Teleconnected Extreme Temperatures

Abstract Combining strengths from deep learning and extreme value theory can help describe complex relationships between variables where extreme events have significant impacts (e.g., environmental or financial applications). Neural networks learn complicated nonlinear relationships from large datasets under limited parametric assumptions. By definition, the number of occurrences of extreme events is small, which limits the ability of the data-hungry, nonparametric neural network to describe rare events. Inspired by recent extreme cold winter weather events in North America caused by atmospheric blocking, we examine several probabilistic generative models for the entire multivariate probability distribution of daily boreal winter surface air temperature. We propose metrics to measure spatial asymmetries, such as long-range anticorrelated patterns that commonly appear in temperature fields during blocking events. Compared to vine copulas, the statistical standard for multivariate copula modeling, deep learning methods show improved ability to reproduce complicated asymmetries in the spatial distribution of ERA5 temperature reanalysis, including the spatial extent of in-sample extreme events.

Krock, Mitchell L.↗
DOE OSTI2024

LENS: Learning Enabled Network Synthesis

RTRC and UMD have developed novel machine learning based methods under the ARPA-E DIFFERENTIATE program for rapid acceleration of hypothesis generation in complex architecture design spaces involving both discrete choices of component inclusion and interconnection and continuous parametric decisions. The project named Learning Enabled Network Synthesis (LENS) further demonstrated the developed methods on challenging electrical power converter design problems by identifying the most suitable circuit topologies and simultaneously selecting the most appropriate components to achieve optimized design of power converter with improved performances. We demonstrated that LENS could enable exploration of very large design space of circuit topologies and components by addressing the limitations of conventional design process in non-linear, high switching speed, multi-dimensional power converter design and optimization. The key innovation developed in LENS is the seamless integration of statistical learning and logical reasoning techniques and building on the individual strengths of these techniques for rapid hypothesis discovery. The main component of LENS comprises of: 1) Graph Reasoning Engine (GRE) to enforce composition rules that rapidly reject all discrete architectures that are composed incorrectly and generates an adaptive database of feasible designs which can be used by ML modules, 2) Graph Generative Learning module which is a deep neural network based generative model for graph architectures which can enable design space exploration beyond the dataset generated by the GRE, 3) Graph Reduced Order Model (ROM) for graph domains for accelerating computation of output metrics, and 4) Active learning and Rule Discovery module for sample efficient learning and extracting logical rules from the learned ML models which will be integrated in the GRE to enhance the filtering effectiveness. LENS approach can be applied to any design domains where designs can be represented as multi-attribute graphs. The LENS team integrated the various technical innovations listed above into an optimization pipeline and exercised the optimization pipeline on the converter design problem. The LENS project demonstrated that the developed AI/ML technologies can be used to generate novel converter circuits >45x faster than experts on chosen use-cases. This can enable faster design space exploration and identification of new designs which are not considered by experts due to the increasing design space complexity. This has significant potential impact on the public and energy needs of the country. It is currently estimated that 30% of all electrical powers generated passes through power converters. The future estimate is that 80% of all power generated would be passing through converters. LENS fills a critical gap in this space since by accelerating the design process the designers would be able to generate more efficient converters which can lead to significant energy savings for the country.

42 ENGINEERING↗
DOE OSTI2023

Deep neural network uncertainty quantification for LArTPC reconstruction

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. Here we assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Dynamics-Based Peptide–MHC Binding Optimization by a Convolutional Variational Autoencoder: A Use-Case Model for CASTELO

An unsolved challenge in the development of antigen-specific immunotherapies is determining the optimal antigens to target. Comprehension of antigen–major histocompatibility complex (MHC) binding is paramount toward achieving this goal. Here, we apply CASTELO, a combined machine learning-molecular dynamics (ML-MD) approach, to identify per-residue antigen binding contributions and then design novel antigens of increased MHC-II binding affinity for a type 1 diabetes-implicated system. We build upon a small-molecule lead optimization algorithm by training a convolutional variational autoencoder (CVAE) on MD trajectories of 48 different systems across four antigens and four HLA serotypes. We develop several new machine learning metrics including a structure-based anchor residue classification model as well as cluster comparison scores. ML-MD predictions agree well with experimental binding results and free energy perturbation-predicted binding affinities. Moreover, ML-MD metrics are independent of traditional MD stability metrics such as contact area and root-mean-square fluctuations (RMSF), which do not reflect binding affinity data. Finally, our work supports the role of structure-based deep learning techniques in antigen-specific immunotherapy design.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Realistic galaxy image simulation via score-based generative models

ABSTRACT We show that a denoising diffusion probabilistic model (DDPM), a class of score-based generative model, can be used to produce realistic mock images that mimic observations of galaxies. Our method is tested with Dark Energy Spectroscopic Instrument (DESI) grz imaging of galaxies from the Photometry and Rotation curve OBservations from Extragalactic Surveys (PROBES) sample and galaxies selected from the Sloan Digital Sky Survey. Subjectively, the generated galaxies are highly realistic when compared with samples from the real data set. We quantify the similarity by borrowing from the deep generative learning literature, using the ‘Fréchet inception distance’ to test for subjective and morphological similarity. We also introduce the ‘synthetic galaxy distance’ metric to compare the emergent physical properties (such as total magnitude, colour, and half-light radius) of a ground truth parent and synthesized child data set. We argue that the DDPM approach produces sharper and more realistic images than other generative methods such as adversarial networks (with the downside of more costly inference), and could be used to produce large samples of synthetic observations tailored to a specific imaging survey. We demonstrate two potential uses of the DDPM: (1) accurate inpainting of occluded data, such as satellite trails, and (2) domain transfer, where new input images can be processed to mimic the properties of the DDPM training set. Here we ‘DESI-fy’ cartoon images as a proof of concept for domain transfer. Finally, we suggest potential applications for score-based approaches that could motivate further research on this topic within the astronomical community.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Geodesic-based relaxation for deep canonical correlation analysis

Deep canonical correlation analysis (DCCA) is often applied to paired data samples from diverse sources to extract meaningful common information. However, when the data sources are heterogeneous, some of the useful information may be complementary but not exactly common. In spite of this fact, existing techniques learn maximally correlated representations from multiple views and are formulated so that they aim to yield identical latent subspaces for each view. This approach is sub-optimal in estimating the true signal subspaces for heterogeneous data sources. We propose a residual relaxation for deep canonical correlation analysis (RDCCA) based on a subspace distance metric, which generalizes the existing problem formulation and extracts representations that are better estimates of the actual, non-identical subspaces. We demonstrate that when using such a relaxation, the learned representations are closer to the true ones and that RDCCA outperforms CCA and DCCA in scenarios with heterogeneous data.

Kuschel, Maurice↗
DOE OSTI2023

Data efficiency and extrapolation trends in neural network interatomic potentials

Abstract Recently, key architectural advances have been proposed for neural network interatomic potentials (NNIPs), such as incorporating message-passing networks, equivariance, or many-body expansion terms. Although modern NNIP models exhibit small differences in test accuracy, this metric is still considered the main target when developing new NNIP architectures. In this work, we show how architectural and optimization choices influence the generalization of NNIPs, revealing trends in molecular dynamics (MD) stability, data efficiency, and loss landscapes. Using the 3BPA dataset, we uncover trends in NNIP errors and robustness to noise, showing these metrics are insufficient to predict MD stability in the high-accuracy regime. With a large-scale study on NequIP, MACE, and their optimizers, we show that our metric of loss entropy predicts out-of-distribution error and data efficiency despite being computed only on the training set. This work provides a deep learning justification for probing extrapolation and can inform the development of next-generation NNIPs.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Counter Data Paucity through Adversarial Invariance Encoding: A Case Study on Modeling Battery Thermal Runaway

Lithium-ion batteries, widely used for their durability and high energy storage, face the risk of internal short circuits leading to catastrophic thermal runaway events. These events, triggered by external stimuli like mechanical loads, pose safety concerns in applications such as electric vehicles. Detecting and understanding thermal runaway events is crucial, but physics-driven models struggle to explain the non-linear evolution of battery temperature during these events, considering factors like material composition and state-of-charge. Due to the rarity of these events and the cost of data collection, we propose a deep learning (DL) model to predict battery temperature responses during thermal runaway. The challenge lies in the scarcity of data, making traditional DL models prone to overfitting and learning low-quality representations of the complex process.Our approach introduces a novel few-shot architecture that incorporates an adversarially governed invariant encoding process. This architecture aims to distill "invariant" relationships by addressing distributional shifts in data across various battery properties, facilitating the detection of thermal runaway events. Specifically, our results demonstrate that deep learning models conditioned on these "invariant" representations outperform state-of-the-art baselines, achieving a remarkable 96.8% performance improvement in terms of the popular metric MAPE. This framework presents a promising direction for enhancing battery safety modeling, particularly in the context of rare and complex events like thermal runaway. Our code and code and dataset used for the paper are public1.

Tabassum, Anika [ORNL] (ORCID:0000000254600955)↗
DOE OSTI2021

Characterization of Offshore Storage Resource Potential in the Central Planning Area of the Gulf of Mexico

This report contains a brief overview of the several studies that provide the basis for this work and details the reservoir characterization efforts that have been completed to date. Briefly, geological characterization data suggests that the deep-water reservoirs are the result of the interplay between turbidite depositional systems and salt tectonics. Project Partners have estimated the CO 2 storage capacity associated with saline reservoirs and that associated with CO 2 -Enhanced Oil Recovery (EOR) in the state waters of Louisiana. In addition, Project Partners have conducted a detailed analysis of CO 2 storage potential associated with CO 2 -EOR in the central planning area of the GOM. This work estimates that 3,140 million metric tonnes of CO 2 can be stored in optimal reservoirs of the shallow- and deep-water central GOM. A reduced order modeling and machine learning approach is introduced that will be used to evaluate CO 2 storage capacity and plume dynamics for down-selected, reservoir-specific, saline characterization. Last, a brief discussion establishes how the data presented in this report will support future work.

02 PETROLEUM↗
DOE OSTI2025

Dense Image Matching Uncertainty Estimation and Confidence Metrics

Dense stereo matching takes overlapping image pairs as input and outputs a disparity map which encodes pixel-by-pixel matches between the images. Recently, there has been an interest in ranking the quality, or even quantifying the accuracy, of disparity estimates. The proposed methods can be described as either uncertainty estimators or confidence metrics. Uncertainty estimators are a small minority of the research. However, they have the potential to be the most useful because they estimate disparity accuracy (in pixel units) that can be used to threshold matches or carried forward using error propagation. The majority of the research deals with confidence metrics which give an ordinal (or binary) ranking of a match’s quality relative to other matches. Confidence metrics do not have units and thus are useful primarily for thresholding matches from mismatches. The methods could also be described as handcrafted or deep-learning based. The majority of the research focused on outdoor driving scenes. Hence, our interest–application to a satellite semi-global matching pipeline–is a domain shift that may challenge deep-learning based methods. We conclude by recommending five handcrafted and two deep-learning based methods for evaluation in our pipeline.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Deep Reinforcement Learning-Based Edge Caching in Wireless Networks

With the purpose to offload data traffic in wireless networks, content caching techniques have recently been studied intensively. Using these techniques and caching a portion of the popular files at the local content servers, the users can be served with less delay. Most of the content replacement policies are based on the content popularity, that depends on the users' preferences. In practice, such information varies over time. Therefore, an approach to determine the file popularity patterns must be incorporated into caching policies. In this context, we study content caching at the wireless network edge using a deep reinforcement learning framework with Wolpertinger architecture. In particular, we propose deep actor-critic reinforcement learning based policies for both centralized and decentralized content caching. For centralized edge caching, we aim at maximizing the cache hit rate. In decentralized edge caching, we consider both the cache hit rate and transmission delay as performance metrics. These proposed frameworks are assumed to neither have any prior information on the file popularities nor know the potential variations in such information. Via simulation results, the superiority of the proposed frameworks is verified by comparing them with other policies, including least frequently used (LFU), least recently used (LRU), and first-in-first-out (FIFO) policies.

42 ENGINEERING↗
DOE OSTI2024

Machine Learning-Based Identification of the Interface Regions for Coupling Local and Nonlocal Models

Local-nonlocal coupling approaches provide a means to combine the computational efficiency of local models and the accuracy of nonlocal models. However, the coupling process can be challenging, requiring expertise to identify the interface between local and nonlocal regions. Here, this study introduces a machine learning-based approach to automatically detect the regions in which the local and nonlocal models should be used. The method uses loading functions evaluated at grid points to decide the model selection at those points. Training of the networks is based on datasets provided by classes of loading functions for which reference coupling configurations are computed using accurate coupled solutions, where accuracy is measured in terms of the relative error between the solution to the coupling approach and the solution to the nonlocal model. We study two approaches that vary in data structure. The first, the full-domain input data approach, uses the entire load vector and outputs a complete label vector, performing a global classification. The second, a window-based approach, processes loads into windows and addresses the problem as a node-wise classification where each window's central point is classified individually. The classification problems are solved via deep learning algorithms based on convolutional neural networks. The performance of these approaches is studied on one-dimensional numerical examples using F1-scores and accuracy metrics. Notably, the windowing approach achieves an accuracy of 0.96 and an F1-score of 0.97, highlighting its potential to automate coupling processes effectively and enhance computational efficiency in material science applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A Case For Intra-rack Resource Disaggregation in HPC

The expected halt of traditional technology scaling is motivating increased heterogeneity in high-performance computing (HPC) systems with the emergence of numerous specialized accelerators. As heterogeneity increases, so does the risk of underutilizing expensive hardware resources if we preserve today’s rigid node configuration and reservation strategies. This has sparked interest in resource disaggregation to enable finer-grain allocation of hardware resources to applications. However, there is currently no data-driven study of what range of disaggregation is appropriate in HPC. To that end, we perform a detailed analysis of key metrics sampled in NERSC’s Cori, a production HPC system that executes a diverse open-science HPC workload. In addition, we profile a variety of deep-learning applications to represent an emerging workload. We show that for a rack (cabinet) configuration and applications similar to Cori, a central processing unit with intra-rack disaggregation has a 99.5% probability to find all resources it requires inside its rack. In addition, ideal intra-rack resource disaggregation in Cori could reduce memory and NIC resources by 5.36% to 69.01% and still satisfy the worst-case average rack utilization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A novel statistical methodology for quantifying the spatial arrangements of axons in peripheral nerves

A thorough understanding of the neuroanatomy of peripheral nerves is required for a better insight into their function and the development of neuromodulation tools and strategies. In biophysical modeling, it is commonly assumed that the complex spatial arrangement of myelinated and unmyelinated axons in peripheral nerves is random, however, in reality the axonal organization is inhomogeneous and anisotropic. Present quantitative neuroanatomy methods analyze peripheral nerves in terms of the number of axons and the morphometric characteristics of the axons, such as area and diameter. In this study, we employed spatial statistics and point process models to describe the spatial arrangement of axons and Sinkhorn distances to compute the similarities between these arrangements (in terms of first- and second-order statistics) in various vagus and pelvic nerve cross-sections. We utilized high-resolution transmission electron microscopy (TEM) images that have been segmented using a custom-built high-throughput deep learning system based on a highly modified U-Net architecture. Our findings show a novel and innovative approach to quantifying similarities between spatial point patterns using metrics derived from the solution to the optimal transport problem. We also present a generalizable pipeline for quantitative analysis of peripheral nerve architecture. Our data demonstrate differences between male- and female-originating samples and similarities between the pelvic and abdominal vagus nerves.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Decode the Workload: Training Deep Learning Models for Efficient Compute Cluster Representation

Monitoring the status of a high throughput computing cluster running computationally intensive production jobs is a crucial yet challenging system administration task due to the complexity of such systems. To this end, we train autoencoders using the Linux kernel CPU metrics of the cluster. Additionally, we explore assisting these models with graph neural networks to share information across threads within a compute node. The models are compared in terms of their ability to: 1) Produce a compressed latent representation that captures the salient features of the input, 2) Detect anomalous activity, and 3) Make distinction between different kinds of jobs run at Jefferson Lab. The goal is to have a robust encoder whose compressed embeddings are used for several downstream tasks. We extend this study further by deploying these models in a human-in-the-loop production-based setting for the anomaly detection task and discuss the associated implementation aspects such as continual learning and the criterion to generate alarms. This study represents a first step in the endeavor towards building self-supervised large-scale foundation models for computing centers.

Mohammed, Ahmed↗
DOE OSTI2023

Denoising diffusion models with geometry adaptation for high fidelity calorimeter simulation

Simulation is crucial for all aspects of collider data analysis, but the available computing budget in the High Luminosity LHC era will be severely constrained. Generative machine learning models may act as surrogates to replace physics-based full simulation of particle detectors, and diffusion models have recently emerged as the state of the art for other generative tasks. Here, we introduce CaloDiffusion, a denoising diffusion model trained on the public CaloChallenge datasets to generate calorimeter showers. Our algorithm employs 3D cylindrical convolutions, which take advantage of symmetries of the underlying data representation. To handle irregular detector geometries, we augment the diffusion model with a new geometry latent mapping (GLaM) layer to learn forward and reverse transformations to a regular geometry that is suitable for cylindrical convolutions. The showers generated by our approach are nearly indistinguishable from the full simulation, as measured by several different metrics.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗