Search NASA⌕ Search

SEARCH · Search NASA

Results for “Contrastive learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Graph Metric Learning Quantifies Morphological Differences between Two Genotypes of Shoot Apical Meristem Cells in Arabidopsis

We present a method for learning “spectrally descriptive” edge weights for graphs. We generalize a previously known distance measure on graphs (Graph Diffusion Distance), thereby allowing it to be tuned to minimize an arbitrary loss function. Because all steps involved in calculating this modified GDD are differentiable, we demonstrate that it is possible for a small neural network model to learn edge weights which minimize loss. We apply this method to discriminate between graphs constructed from shoot apical meristem images of two genotypes of Arabidopsis thaliana specimens: wild-type and trm678 triple mutants with cell division phenotype. Training edge weights and kernel parameters with contrastive loss produces a learned distance metric with large margins between these graph categories. We demonstrate this by showing improved performance of a simple k-nearest-neighbors classifier on the learned distance matrix. We also demonstrate a further application of this method to biological image analysis. Once trained, we use our model to compute the distance between the biological graphs and a set of graphs output by a cell division simulator. Comparing simulated cell division graphs to biological ones allows us to identify simulation parameter regimes which characterize mutant vs. wild-type Arabidopsis cells. We find that trm678 mutant cells are characterized by increased randomness of division planes and decreased ability to avoid previous vertices between cell walls.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Low-energy Electron-track Imaging for a Liquid Argon Time-projection-chamber Telescope Concept Using Probabilistic Deep Learning

The GammaTPC is an MeV-scale single-phase liquid argon time-projection-chamber gamma-ray telescope concept with a novel dual-scale pixel-based charge-readout system. It promises to enable a significant improvement in sensitivity to MeV-scale gamma rays over previous telescopes. The novel pixel-based charge readout allows for imaging of the tracks of electrons scattered by Compton interactions of incident gamma rays. The two primary contributors to the accuracy of a Compton telescope in reconstructing an incident gamma-ray’s original direction are its energy and position resolution. In this work, we focus on using deep learning to optimize the reconstruction of the initial position and direction of electrons scattered in Compton interactions, including using probabilistic models to estimate predictive uncertainty. We show that the deep-learning models are able to predict locations of Compton scatters of MeV-scale gamma rays from simulated 500 μm pixel-based data to better than 1 mm rms error and are sensitive to the initial direction of the scattered electron. We compare and contrast different deep-learning uncertainty estimation algorithms for reconstruction applications. Additionally, we show that event-by-event estimates of the uncertainty of the locations of the Compton scatters can be used to select those events that were reconstructed most accurately, leading to improvement in locating the origin of gamma-ray sources on the sky.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Graph-based Reversible Evaluation and Tangents Library

GRETL is a C++ library for evaluation, re-evaluation and algorithmic differentiation of functional operations on an arbitrary computational graph with limited memory usage. Similar to popular machine learning frameworks in Python, like PyTorch and JAX, it tracks and stores both operations and output data as functions are evaluated. Once this composition of functions is built up, the entire chain of operations can be back propagated to compute sensitivities of the final result with respect to any number of inputs. In contrast to most machine learning applications, memory usage becomes the bottleneck for back propagation in many physics applications, especially for time-dependent PDEs. Dynamic check pointing becomes essential. An important distinguishing feature of GRETL is its ability to limit the maximum memory usage by automatically dynamic checkpointing the data output for each graph operation (see Wang, Moin, Iaccarino, 2009). During backpropagation, parts of the graph that are no longer in memory are automatically re-evaluated from upstream checkpointed states as needed for derivative sensitivity calculations (or more precisely, for vector-Jacobian products). GRETL is particularly beneficial for applications, such as coupled multi-physics, where deriving adjoint-based sensitivities and managing checkpoint memory across modules becomes onerous. Cases which can be readily handled by the GRETL library include: different time-integration algorithms per physics (e.g., coupled predictor-corrector algorithms, IMEX, etc.), sub-cycling, asynchronous integrators, state dependent timestep sizes, iterative solvers and coupling algorithms, controller algorithms, and more.

Tupek, MichaelR [Lawrence Livermore National Labor↗
DOE OSTI2024

Deep Learning and Likelihood Approaches for Viral Phylogeography Converge on the Same Answers Whether the Inference Model Is Right or Wrong

Abstract Analysis of phylogenetic trees has become an essential tool in epidemiology. Likelihood-based methods fit models to phylogenies to draw inferences about the phylodynamics and history of viral transmission. However, these methods are often computationally expensive, which limits the complexity and realism of phylodynamic models and makes them ill-suited for informing policy decisions in real-time during rapidly developing outbreaks. Likelihood-free methods using deep learning are pushing the boundaries of inference beyond these constraints. In this paper, we extend, compare, and contrast a recently developed deep learning method for likelihood-free inference from trees. We trained multiple deep neural networks using phylogenies from simulated outbreaks that spread among 5 locations and found they achieve close to the same levels of accuracy as Bayesian inference under the true simulation model. We compared robustness to model misspecification of a trained neural network to that of a Bayesian method. We found that both models had comparable performance, converging on similar biases. We also implemented a method of uncertainty quantification called conformalized quantile regression that we demonstrate has similar patterns of sensitivity to model misspecification as Bayesian highest posterior density (HPD) and greatly overlap with HPDs, but have lower precision (more conservative). Finally, we trained and tested a neural network against phylogeographic data from a recent study of the SARS-Cov-2 pandemic in Europe and obtained similar estimates of region-specific epidemiological parameters and the location of the common ancestor in Europe. Along with being as accurate and robust as likelihood-based methods, our trained neural networks are on average over 3 orders of magnitude faster after training. Our results support the notion that neural networks can be trained with simulated data to accurately mimic the good and bad statistical properties of the likelihood functions of generative phylogenetic models.

Evolutionary Biology↗
DOE OSTI2021

Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase-Contrast Transmission Electron Microscopy Images

Phase-contrast transmission electron microscopy (TEM) is a powerful tool for imaging the local atomic structure of materials. TEM has been used heavily in studies of defect structures of two-dimensional materials such as monolayer graphene due to its high dose efficiency. However, phase-contrast imaging can produce complex nonlinear contrast, even for weakly scattering samples. It is, therefore, difficult to develop fully automated analysis routines for phase-contrast TEM studies using conventional image processing tools. For automated analysis of large sample regions of graphene, one of the key problems is segmentation between the structure of interest and unwanted structures such as surface contaminant layers. In this study, we compare the performance of a conventional Bragg filtering method with a deep learning routine based on the U-Net architecture. We show that the deep learning method is more general, simpler to apply in practice, and produces more accurate and robust results than the conventional algorithm. We provide easily adaptable source code for all results in this paper and discuss potential applications for deep learning in fully automated TEM image analysis.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Online energy consumption forecast for battery electric buses using a learning-free algebraic method

Accurately predicting the energy consumption plays a vital role in battery electric buses (BEBs) route planning and deployment. Based on the algebraic derivative estimation, we present a novel method to forecast the energy consumption in real time. In contrast to the mainstream machine-learning-based methods, the proposed method does not require access to the historical energy consumption data. It eliminates the time-consuming and computationally expensive offline training. Consequently, its prediction performance is not constrained by the quantity and quality of the training data. Moreover, the method can swiftly adapt to new situations not included in the previous driving cycles, which makes it especially suitable for emerging transport modes, e.g., on-demand transit services. In addition, its online execution only involves algebraic calculations, yielding superior calculation efficiency. Using real-world data, we comprehensively compare the performance of the proposed learning-free algebraic method with multiple representative machine-learning-based methods. Finally, the advantages and limitations of the proposed method are discussed in detail.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Equivariant Graph Attention Network - 3D Conformers & Feature Fusion

EGAN-3F (Equivariant Graph Attention Network - 3D Conformers & Feature Fusion) presents an innovative approach for predicting binding affinity between small molecules and protein targets, a fundamental task in drug discovery. Traditional structure-based methods often depend on protein-ligand complex structures obtained from crystallography or molecular docking. In contrast, ligand-only machine learning models using 1D or 2D representations such as SMILES have been developed to predict binding affinity without structural information about the target; however, their accuracy is often limited due to the lack of 3D ligand information. EGAN-3F addresses this limitation by integrating spatially aware graph learning with traditional descriptor-based features. We systematically investigate how combining 2D and 3D molecular representations enhances binding affinity prediction from SMILES strings. This approach underscores the importance of modeling conformational diversity and incorporating chemically meaningful descriptors to improve predictive accuracy. The key innovation of EGAN-3F lies in its ability to achieve robust ligand-based binding affinity predictions without requiring protein-ligand complex structures, effectively bridging the gap between purely structural and ligand-only modeling paradigms.

Shim, Heesung [Lawrence Livermore National Laborat↗
DOE OSTI2024

Developing an Interactive OpenMP Book with Large Language Models

Abstract. This paper presents an approach to authoring a textbook titled Interactive OpenMP Programming with the assistance of Large Language Models (LLMs). The writing process utilized state-of-the-art LLMs, including Gemini Pro 1.5, Claude 3, and ChatGPT-4, to generate the initial structure and outline of the book, as well as the initial content for specific chapters. This content included detailed descriptions of individual OpenMP constructs and practical programming examples. The outline and content have then undergone extensive manual revisions to meet our book goals. In this paper, we report our findings about the capabilities and limitations of these LLMs. We address critical questions concerning the necessity of textbook resources and the effectiveness of LLMs in creating fundamental and practical programming content. Our findings suggest that while LLMs offer significant advantages in generating textbook content, they require careful integration with traditional educational methodologies to ensure depth, accuracy, and pedagogical effectiveness. The Interactive OpenMP Programming book is developed with the framework of Jupyter Book, enabling the execution of code within the book from the web browser, providing instant feedback and a dynamic learning experience that stands in contrast to traditional educational resources. The book represents a significant step towards modernizing programming education, offering insights into practical strategies for generating the textbook through advanced AI tools.

Large Language Model · OpenMP · Interactive Book ·↗
DOE OSTI2021

Training Restricted Boltzmann Machines With a D-Wave Quantum Annealer

Restricted Boltzmann Machine (RBM) is an energy-based, undirected graphical model. It is commonly used for unsupervised and supervised machine learning. Typically, RBM is trained using contrastive divergence (CD). However, training with CD is slow and does not estimate the exact gradient of the log-likelihood cost function. In this work, the model expectation of gradient learning for RBM has been calculated using a quantum annealer (D-Wave 2000Q), where obtaining samples is faster than Markov chain Monte Carlo (MCMC) used in CD. Training and classification results of RBM trained using quantum annealing are compared with the CD-based method. The performance of the two approaches is compared with respect to the classification accuracies, image reconstruction, and log-likelihood results. The classification accuracy results indicate comparable performances of the two methods. Image reconstruction and log-likelihood results show improved performance of the CD-based method. It is shown that the samples obtained from quantum annealer can be used to train an RBM on a 64-bit “bars and stripes” dataset with classification performance similar to an RBM trained with CD. Though training based on CD showed improved learning performance, training using a quantum annealer could be useful as it eliminates computationally expensive MCMC steps of CD.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Real-time X-ray phase-contrast imaging using SPINNet—a speckle-based phase-contrast imaging neural network

X-ray phase-contrast imaging has become indispensable for visualizing samples with low absorption contrast. In this regard, speckle-based techniques have shown significant advantages in spatial resolution, phase sensitivity, and implementation flexibility compared with traditional methods. However, the computational cost associated with data inversion has hindered their wider adoption. By exploiting the power of deep learning, we developed a speckle-based phase-contrast imaging neural network (SPINNet) that significantly improves the imaging quality and boosts the phase retrieval speed by at least 2 orders of magnitude compared to existing methods. To achieve this performance, we combined SPINNet with a coded-mask-based technique, an enhanced version of the speckle-based method. Using this scheme, we demonstrate the simultaneous reconstruction of absorption and phase images on the order of 100 ms, where a traditional correlation-based analysis would take several minutes even with a cluster. In addition to significant improvement in speed, our experimental results show that the imaging and phase retrieval quality of SPINNet outperform existing single-shot speckle-based methods. Furthermore, we successfully demonstrate SPINNet application in x-ray optics metrology and 3D x-ray phase-contrast tomography. Our result shows that SPINNet could enable many applications requiring high-resolution and fast data acquisition and processing, such as in situ and in operando 2D and 3D phase-contrast imaging and real-time at-wavelength metrology and wavefront sensing.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Provably efficient machine learning for quantum many-body problems

Classical machine learning (ML) provides a potentially powerful approach to solving challenging quantum many-body problems in physics and chemistry. However, the advantages of ML over traditional methods have not been firmly established. In this work, we prove that classical ML algorithms can efficiently predict ground-state properties of gapped Hamiltonians after learning from other Hamiltonians in the same quantum phase of matter. By contrast, under a widely accepted conjecture, classical algorithms that do not learn from data cannot achieve the same guarantee. We also prove that classical ML algorithms can efficiently classify a wide range of quantum phases. Extensive numerical experiments corroborate our theoretical results in a variety of scenarios, including Rydberg atom systems, two-dimensional random Heisenberg models, symmetry-protected topological phases, and topologically ordered phases.

Science & Technology - Other Topics↗
DOE OSTI2021

2D-to-3D image translation of complex nanoporous volumes using generative networks

Image-based characterization offers a powerful approach to studying geological porous media at the nanoscale and images are critical to understanding reactive transport mechanisms in reservoirs relevant to energy and sustainability technologies such as carbon sequestration, subsurface hydrogen storage, and natural gas recovery. Nanoimaging presents a trade off, however, between higher-contrast sample-destructive and lower-contrast sample-preserving imaging modalities. Furthermore, high-contrast imaging modalities often acquire only 2D images, while 3D volumes are needed to characterize fully a source rock sample. In this work, we present deep learning image translation models to predict high-contrast focused ion beam-scanning electron microscopy (FIB-SEM) image volumes from transmission X-ray microscopy (TXM) images when only 2D paired training data is available. We introduce a regularization method for improving 3D volume generation from 2D-to-2D deep learning image models and apply this approach to translate 3D TXM volumes to FIB-SEM fidelity. We then segment a predicted FIB-SEM volume into a flow simulation domain and calculate the sample apparent permeability using a lattice Boltzmann method (LBM) technique. Results show that our image translation approach produces simulation domains suitable for flow visualization and allows for accurate characterization of petrophysical properties from non-destructive imaging data.

58 GEOSCIENCES↗
DOE OSTI2020

The importance of temporal and spatial incoherence in quantitative interpretation of 4D-STEM

Recent developments in pixelated detectors, when combined with aberration correction of probe forming optics have greatly enhanced the field of scanning electron diffraction. Differential phase contrast is now routine and deep learning has been proposed as a method to extract maximum information from diffraction patterns. This study examines the effects of temporal and spatial incoherence on convergent beam electron diffraction patterns and demonstrates that simple center of mass measurements cannot be naively interpreted. The inclusion of incoherence in deep learning data sets is also discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Codiscovering graphical structure and functional relationships within data: A Gaussian Process framework for connecting the dots

Most problems within and beyond the scientific domain can be framed into one of the following three levels of complexity of function approximation. Type 1: Approximate an unknown function given input/output data. Type 2: Consider a collection of variables and functions, some of which are unknown, indexed by the nodes and hyperedges of a hypergraph (a generalized graph where edges can connect more than two vertices). Given partial observations of the variables of the hypergraph (satisfying the functional dependencies imposed by its structure), approximate all the unobserved variables and unknown functions. Type 3: Expanding on Type 2, if the hypergraph structure itself is unknown, use partial observations of the variables of the hypergraph to discover its structure and approximate its unknown functions. These hypergraphs offer a natural platform for organizing, communicating, and processing computational knowledge. While most scientific problems can be framed as the data-driven discovery of unknown functions in a computational hypergraph whose structure is known (Type 2), many require the data-driven discovery of the structure (connectivity) of the hypergraph itself (Type 3). We introduce an interpretable Gaussian Process (GP) framework for such (Type 3) problems that does not require randomization of the data, access to or control over its sampling, or sparsity of the unknown functions in a known or learned basis. Its polynomial complexity, which contrasts sharply with the super-exponential complexity of causal inference methods, is enabled by the nonlinear ANOVA capabilities of GPs used as a sensing mechanism.

Science & Technology - Other Topics↗
DOE OSTI2026

Contrasting Carbon–Water–Energy Dynamics in Perennial and Annual Bioenergy Agroecosystems Using Eddy Covariance and Interpretable Machine Learning

Understanding how agroecosystems respond to environmental variability is fundamental to predicting productivity and sustainability under a changing climate. We analyzed 55 site-years of high-frequency eddy covariance observations from five agroecosystems—two perennial grasses (miscanthus and switchgrass), two annual rotation systems (maize–soybean and sorghum–soybean), and a restored native prairie—to examine ecosystem-scale carbon, water, and energy fluxes. Using an interpretable machine-learning framework with regression tree ensembles, Shapley Additive Explanations, and Accumulated Local Effects, we quantified how environmental and temporal factors regulate gross primary productivity (GPP), evapotranspiration (ET), water-use efficiency, and the Bowen ratio. Perennials exhibited stronger physiological buffering and maintained fluxes across a broader range of temperature and moisture conditions, reflecting deeper rooting and persistent canopy cover. Annuals, in contrast, showed greater short-term variability and stronger coupling to atmospheric demand, with GPP and ET declining rapidly under low humidity or soil moisture. Differences in temperature sensitivity of Bowen ratio further revealed that perennials sustained proportionally greater sensible heat flux under cool conditions, whereas annuals exhibited constrained energy exchange when evaporative demand was low. Together, these results demonstrate that crop life cycle and canopy structure are fundamental determinants of ecosystem-scale carbon–water–energy coupling. By integrating long-term flux observations with interpretable machine learning, this study identifies the environmental drivers that shape agroecosystem function and highlights how conversion from annual to perennial feedstocks can enhance climatic resilience and alter land–atmosphere energy feedbacks. These findings provide a data-driven basis for improving crop and Earth-system models and for guiding bioenergy landscape design under future climate scenarios.

Accumulated Local Effects↗
DOE OSTI2025

Defect And Damage Characterization Of Additively Manufactured Titanium Alloy Ti-5553 Using Traditional Computed Tomography Volume Segmentation And Machine Learning Algorithms

The mechanical response of a component is affected by defects, such as porosity, arising from the laser powder bed fusion (LPBF) fabrication process. Thus, it is important to develop accurate and efficient inspection methods for identifying porosity. In this work, porosity identified in an X-ray computed tomography (XCT) volume of a Ti-5553 coupon was compared to pores identified in a serial sectioned volume that represented the ground truth. The porosity of the XCT scan was identified using contrast-based, ISO-based, and machine learning (ML) methods for segmentation. Large inherent porosity was easy to identify, but the ISO thresholding still struggled due to the intensity gradient resulting from both the beam hardening in XCT and the uneven lighting of the serial sectioning panels. Further, the results show that ML-based methods were better suited for identifying small pores and reducing the amount of false positives. Additionally, high strain-rate impact testing was done on some of the XCT samples as well as post-mortem XCT inspection, and the same suite of segmentation and quantification tools were used to identify the large spallation cavities. The comparison of porosity pre- and post-mortem provides insight on the influence of the LPBF porosity on the formation of spall cavities.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Preparing Precollege Students for the Second Quantum Revolution with Core Concepts in Quantum Information Science

After the passage of the U.S. National Quantum Initiative Act in December 2018, the National Science Foundation (NSF) and the Office of Science and Technology Policy (OSTP) recently assembled an interagency working group and conducted a workshop titled “Key Concepts for Future Quantum Information Science Learners” that focused on identifying core concepts for future curricular and educator activities to help precollege students engage with quantum information science (QIS). Helping precollege students learn these key concepts in QIS is an effective approach to introducing them to the second quantum revolution and inspiring them to become future contributors in the growing field of quantum information science and technology as leaders in areas related to quantum computing, communication, and sensing. This paper is a call to precollege educators to contemplate including QIS concepts into their existing courses at appropriate levels and get involved in the development of curricular materials suitable for their students. Also, research shows that compare-and-contrast activities can provide an effective approach to helping students learn. Therefore, we illustrate a pedagogical approach that contrasts the classical and quantum concepts so that educators can adapt them for their students in their lesson plans to help them learn the differences between key concepts in quantum and classical contexts.

Education & Educational Research↗