Search NASA⌕ Search

SEARCH · Search NASA

Results for “Machine Learning in Network Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

CuXASNet: Rapid and accurate prediction of copper L-edge x-ray absorption spectra using machine learning

In this work, we have developed CuXASNet, a dense neural network that predicts simulated Cu -edge x-ray absorption spectra (XAS) from atomic structures. Featurization of the Cu local environment is performed using a component of M3GNet, a graph neural network developed for predicting the potential energy surface. CuXASNet is trained on simulated spectra from FEFF9 at the multiple scattering level of theory, and can predict the and edges for Cu sites to quantitative accuracy. To validate our approach, we compare 14 experimental spectra extracted from the literature with the predictions of CuXASNet. The agreement of CuXASNet with experiments is shown by an average mean absolute error of 0.125 and an average Spearman's correlation coefficient of 0.891, which is comparable to FEFF9's values of 0.131 and 0.898 for the same metrics. As such, CuXASNet can rapidly predict a large number of -edge XAS spectra at the same accuracy as FEFF9 simulations. This can be used as a drop-in replacement for multiple scattering codes for fast screening of candidate atomic structure models of a measured system. This model establishes a general framework for Cu XAS prediction, and can be extended to more computationally expensive levels of theory and to other transition metal edges.

36 MATERIALS SCIENCE↗

Prediction of inter-chain distance maps of protein complexes with 2D attention-based deep neural networks

Residue-residue distance information is useful for predicting tertiary structures of protein monomers or quaternary structures of protein complexes. Many deep learning methods have been developed to predict intra-chain residue-residue distances of monomers accurately, but few methods can accurately predict inter-chain residue-residue distances of complexes. We develop a deep learning method CDPred (i.e., Complex Distance Prediction) based on the 2D attention-powered residual network to address the gap. Tested on two homodimer datasets, CDPred achieves the precision of 60.94% and 42.93% for top L/5 inter-chain contact predictions (L: length of the monomer in homodimer), respectively, substantially higher than DeepHomo’s 37.40% and 23.08% and GLINTER’s 48.09% and 36.74%. Tested on the two heterodimer datasets, the top Ls/5 inter-chain contact prediction precision (Ls: length of the shorter monomer in heterodimer) of CDPred is 47.59% and 22.87% respectively, surpassing GLINTER’s 23.24% and 13.49%. Moreover, the prediction of CDPred is complementary with that of AlphaFold2-multimer.

59 BASIC BIOLOGICAL SCIENCES↗

From sequence to protein structure and conformational dynamics with artificial intelligence/machine learning

The 2024 Nobel Prize in Chemistry was awarded in part for de novo protein structure prediction using AlphaFold2, an artificial intelligence/machine learning (AI/ML) model trained on vast amounts of sequence and three-dimensional structure data. AlphaFold2 and related models, including RoseTTAFold and ESMFold, employ specialized neural network architectures driven by attention mechanisms to infer relationships between sequence and structure. At a fundamental level, these AI/ML models operate on the long-standing hypothesis that the structure of a protein is determined by its amino acid sequence. More recently, AlphaFold2 has been adapted for the prediction of multiple protein conformations by subsampling multiple sequence alignments. Herein, we provide an overview of the deterministic relationship between sequence and structure, which was hypothesized over half a century ago with profound implications for the biological sciences ever since. We postulate that protein conformational dynamics are also determined, at least in part, by amino acid sequence and that this relationship may be leveraged for construction of AI/ML models dedicated to predicting protein conformational ensembles. Accordingly, we describe a conceptual model architecture, which may be trained on sequence data in combination with conformationally sensitive structural information, coming primarily from nuclear magnetic resonance (NMR) spectroscopy. Notwithstanding certain limitations in this context, NMR offers abundant structural heterogeneity conducive to conformational ensemble prediction. As NMR and other data continue to accumulate, sequence-informed prediction of protein structural dynamics with AI/ML has the potential to emerge as a transformative capability across the biological sciences.

Artificial intelligence↗

Neural network representation of electronic structure from ab initio molecular dynamics

Despite their rich information content, electronic structure data amassed at high volumes in ab initio molecular dynamics simulations are generally under-utilized. Here we introduce a transferable high-fidelity neural network representation of such data in the form of tight-binding Hamiltonians for crystalline materials. This predictive representation of ab initio electronic structure, combined with machine-learning boosted molecular dynamics, enables efficient and accurate electronic evolution and sampling. When it is applied to a one-dimension charge-density wave material, carbyne, we are able to compute the spectral function and optical conductivity in the canonical ensemble. The spectral functions evaluated during soliton-antisoliton pair annihilation process reveal significant renormalization of low-energy edge modes due to retarded electron-lattice coupling beyond the Born–Oppenheimer limit. The availability of an efficient and reusable surrogate model for the electronic structure dynamical system will enable calculating many interesting physical properties, paving the way to previously inaccessible or challenging avenues in materials modeling.

74 ATOMIC AND MOLECULAR PHYSICS↗

Virtual node graph neural network for full phonon prediction

Understanding the structure-property relationship is crucial for designing materials with desired properties. The past few years have witnessed remarkable progress in machine-learning methods for this connection. However, substantial challenges remain, including the generalizability of models and prediction of properties with materials-dependent output dimensions. Here we present the virtual node graph neural network to address the challenges. By developing three virtual node approaches, we achieve Γ-phonon spectra and full phonon dispersion prediction from atomic coordinates. We show that, compared with the machine-learning interatomic potentials, our approach achieves orders-of-magnitude-higher efficiency with comparable to better accuracy. This allows us to generate databases for Γ-phonon containing over 146,000 materials and phonon band structures of zeolites. Additionally, our work provides an avenue for rapid and high-quality prediction of phonon band structures enabling materials design with desired phonon properties. The virtual node method also provides a generic method for machine-learning design with a high level of flexibility. In this study, the authors present a virtual node graph neural network to enable the prediction of material properties with variable output dimensions. This method offers fast and accurate predictions of phonon band structures in complex solids.

36 MATERIALS SCIENCE↗

Beyond Fine Tuning: Adding capacity to leverage few labels

In this paper we present a technique to train neural network models on small amounts of data. Current methods for training neural networks on small amounts of rich data typically rely on strategies such as fine-tuning a pre-trained neural networks or the use of domain-specific hand-engineered features. Here we take the approach of treating network layers, or entire networks, as modules and combine pre-trained modules with untrained modules, to learn the shift in distributions between data sets. The central impact of using a modular approach comes from adding new representations to a network, as opposed to replacing representations via fine-tuning. Using this technique, we are able surpass results using standard fine-tuning transfer learning approaches, and we are also able to significantly increase performance over such approaches when using smaller amounts of data.

data science↗

Ensemble transfer learning for the prediction of anti-cancer drug response

Abstract Transfer learning, which transfers patterns learned on a source dataset to a related target dataset for constructing prediction models, has been shown effective in many applications. In this paper, we investigate whether transfer learning can be used to improve the performance of anti-cancer drug response prediction models. Previous transfer learning studies for drug response prediction focused on building models to predict the response of tumor cells to a specific drug treatment. We target the more challenging task of building general prediction models that can make predictions for both new tumor cells and new drugs. Uniquely, we investigate the power of transfer learning for three drug response prediction applications including drug repurposing, precision oncology, and new drug development, through different data partition schemes in cross-validation. We extend the classic transfer learning framework through ensemble and demonstrate its general utility with three representative prediction algorithms including a gradient boosting model and two deep neural networks. The ensemble transfer learning framework is tested on benchmark in vitro drug screening datasets. The results demonstrate that our framework broadly improves the prediction performance in all three drug response prediction applications with all three prediction algorithms.

60 APPLIED LIFE SCIENCES↗

A flexible event reconstruction based on machine learning and likelihood principles

Event reconstruction is a central step in many particle physics experiments, turning detector observables into parameter estimates; for example estimating the energy of an interaction given the sensor readout of a detector. A corresponding likelihood function is often intractable, and approximations need to be constructed. Here, in our work, we first show how the full likelihood for a many-sensor detector can be broken apart into smaller terms, and secondly how we can train neural networks to approximate all terms solely based on forward simulation. Our technique results in a fast, flexible, and close-to-optimal surrogate model proportional to the likelihood and can be used in conjunction with standard inference techniques allowing for a consistent treatment of uncertainties. We illustrate our technique for parameter inference in neutrino telescopes based on maximum likelihood and Bayesian posterior sampling. Given its great flexibility, we also showcase our method for geometry optimization enabling to learn optimal detector designs. Lastly, we apply our method to realistic simulation of a ton-scale water-based liquid scintillator detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Data reduction in deterministic neutron transport calculations using machine learning

Neutron cross section matrices for fission and scattering data are required for each material, temperature, and enrichment level to calculate the neutron transport equation accurately. Here, this information can be a limiting factor when using the multigroup discrete ordinates (S N ) method when the number of energy groups is large. Machine Learning (ML) can be used to replace the need for the cross section matrices by reproducing the function that maps the scalar flux to the scattering and fission sources. Through the use of autoencoders and Deep Jointly-Informed Neural Networks (DJINN), the data storage requirements are reduced by 94% of the original data for a 618 group problem. This is accomplished while preserving the scalar flux, maintaining generality, and decreasing wall clock times.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Multioutput Convolutional Neural Network for Improved Parameter Extraction in Time-Resolved Electrostatic Force Microscopy Data

Time-resolved scanning probe microscopy methods, like time-resolved electrostatic force microscopy (trEFM), enable imaging of dynamic processes ranging from ion motion in batteries to electronic dynamics in microstructured thin film semiconductors for solar cells. Reconstructing the underlying physical dynamics from these techniques can be challenging due to the interplay of cantilever physics with the actual transient kinetics of interest in the resulting signal. Previously, quantitative trEFM used empirical calibration of the cantilever or feed-forward neural networks trained on simulated data to extract the physical dynamics of interest. Both these approaches are limited by interpreting the underlying signal as a single exponential function, which serves as an approximation but does not adequately reflect many realistic systems. Here, we present a multi-branched, multi-output convolutional neural network (CNN) that uses the trEFM signal in addition to the physical cantilever parameters as input. The trained CNN accurately extracts parameters describing both single-exponential and bi-exponential underlying functions, and more accurately reconstructs real experimental data in the presence of noise. This article demonstrates an application of physics-informed machine learning to complex signal processing tasks, enabling more efficient and accurate analysis of trEFM.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Bayesian Deep Learning Approach to Near-Term Climate Prediction

Since model bias and associated initialization shock are serious shortcomings that reduce prediction skills in state-of-the-art decadal climate prediction efforts, we pursue a complementary machine-learning-based approach to climate prediction. The example problem setting we consider consists of predicting natural variability of the North Atlantic sea surface temperature on the interannual timescale in the pre-industrial control simulation of the Community Earth System Model. While previous works have considered the use of recurrent networks such as convolutional LSTMs and reservoir computing networks in this and other similar problem settings, we currently focus on the use of feedforward convolutional networks. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill. Next, we go on to consider a probabilistic formulation of the same network based on Stein variational gradient descent and find that in addition to providing useful measures of predictive uncertainty, the probabilistic (Bayesian) version improves on its deterministic counterpart in terms of predictive skill. Finally, we characterize the reliability of the ensemble of machine learning models obtained in the probabilistic setting by using analysis tools developed in the context of ensemble numerical weather prediction.

54 ENVIRONMENTAL SCIENCES↗

Computational insights into hydrogen adsorption energies on medium-entropy oxides

High entropy oxides (HEOs) have emerged as promising catalysts for several important chemical transformations including alkane activation. Hydrogen adsorption energy (HAE) has been used as a key descriptor for many reactions including methane C–H activation and hydrogen evolution reactions. Hence, understanding the relationship between HAEs and the surface chemistry of HEO surfaces could lay the foundation for meaningful correlations among methane C–H activation, HAE, and the complex, local environment of HEO surfaces. Here, we used a medium-entropy oxide as a prototypical system – Mg 0.25 Ni 0.25 Cu 0.25 Zn 0.25 O with a rock-salt structure – to interrogate these relationships. We sampled 2000 different surfaces of its (100) plane and calculated the HAEs at randomly chosen surface O sites using density functional theory (DFT). Our analysis of the 2000 data points reveals that the HAEs at the surface O sites are significantly influenced by the local environment around the adsorption sites, particularly the nature of the metal atom directly below the surface O site where H adsorbs. After comparing several popular graph-neural-network-based machine learning models, we found that the DimeNet++ model performed best achieving satisfactory accuracy in predicting HAEs for both Mg 0.25 Ni 0.25 Cu 0.25 Zn 0.25 O and slightly varied compositions. Our work underscores the promise of such models and the need for further refinement to address the complexity of HEOs.

Song, Haohong [Vanderbilt Univ., Nashville, TN (Un↗

Advanced Computing, Data Science, and Artificial Intelligence Research Opportunities for Energy-Focused Transportation Science

The Energy Efficient Mobility Systems (EEMS) technology landscape is complex and rapidly evolving, which provides both tremendous opportunities and formidable challenges. Significant alterations to the mobility landscape are underway due to the advent of vehicle and infrastructure connectivity, autonomous driving, and rapid passenger- and freight-vehicle electrification. Advanced computing will play an increasingly important role in enabling the EEMS program to understand and identify the most important levers to improve the energy productivity of future integrated mobility systems. It is also driving new approaches to mobility and the research to unlock an affordable, efficient, safe, and accessible transportation future. Driving much of this change is the collection, analysis, and strategic use of massive amounts of diverse, complex data from infrastructure and vehicles with on-board sensors and data storage and transmission capabilities. Diverse and representative data are key to implementing approaches to maximize mobility energy productivity. While high-fidelity modeling of integrated transportation networks has strengthened our understanding of dynamic movement and behavior patterns, existing tools must be expanded beyond their current focus. This work necessitates data infrastructure investments (e.g., secure-streaming data platforms driven by ubiquitous sensors and video analytics) as well as investments in critical capabilities for large-scale automated analysis and organization using modern machine learning, statistics, and artificial intelligence. Other chief needs include agile, large-scale storage that can be quickly searched and queried for relevant data to support validation and model development, data-sharing agreements, and formatting standards for key data types. The future of public transit must be explored in greater detail, research must inform design, and opportunities must be identified for improving the mobility productivity of public transit in both urban and rural America.

33 ADVANCED PROPULSION SYSTEMS↗

Deep-learning-guided high-resolution subsurface reflectivity imaging with application to ground-penetrating radar data

Subsurface reflectivity imaging is one of the most important geophysical characterization methods for revealing subsurface structures. In many cases, accurate subsurface reflectivity imaging is challenging because of, for example, random or coherent noise in the data and sparse source-receiver observation geometry. Here, we develop a deep-learning-guided iterative imaging method to improve subsurface structure imaging. Specifically, we train a supervised neural network to infer a noise-free, high-resolution image from a noisy, low-resolution image and use this estimated image as guidance to regularize least-squares imaging. We develop a systematic method to generate high-quality synthetic training data (data-label pairs) to train the guidance neural network. The trained neural network can provide high-fidelity predictions even for field-data images that are not in the training data. We validate our new imaging method using one synthetic and two field ground-penetrating radar data examples, and find that our method can produce clean, high-resolution subsurface reflectivity images where existing single-pass and least-squares imaging methods fail due to noise and insufficient data coverage.

58 GEOSCIENCES↗

Machine learning-assisted crystal engineering of a zeolite

Abstract It is shown that Machine Learning (ML) algorithms can usefully capture the effect of crystallization composition and conditions (inputs) on key microstructural characteristics (outputs) of faujasite type zeolites (structure types FAU, EMT, and their intergrowths), which are widely used zeolite catalysts and adsorbents. The utility of ML (in particular, Geometric Harmonics) toward learning input-output relationships of interest is demonstrated, and a comparison with Neural Networks and Gaussian Process Regression, as alternative approaches, is provided. Through ML, synthesis conditions were identified to enhance the Si/Al ratio of high purity FAU zeolite to the hitherto highest level (i.e., Si/Al = 3.5) achieved via direct (not seeded), and organic structure-directing-agent-free synthesis from sodium aluminosilicate sols. The analysis of the ML algorithms’ results offers the insight that reduced Na 2 O content is key to formulating FAU materials with high Si/Al ratio. An acid catalyst prepared by partial ion exchange of the high-Si/Al-ratio FAU (Si/Al = 3.5) exhibits improved proton reactivity (as well as specific activity, per unit mass of catalyst) in propane cracking and dehydrogenation compared to the catalyst prepared from the previously reported highest Si/Al ratio (Si/Al = 2.8).

36 MATERIALS SCIENCE↗

Crowdsourced reliable labeling of safety-rule violations on images of complex construction scenes for advanced vision-based workplace safety

Construction workplace hazard detection requires engineers to analyze scenes manually against many safety rules, which is time-consuming, labor-intensive, and error-prone. Computer vision algorithms are yet to achieve reliable discrimination of anomalous and benign object relations underpinning safety violation detections. Recently developed deep learning-based computer vision algorithms need tens of thousands of images, including labels of the safety rules violated, in order to train deep-learning networks for acquiring spatiotemporal reasoning capacity in complex workplaces. Such training processes need human experts to label images and indicate whether the relationship between the worker, resource, and equipment in the scenes violate spatiotemporal arrangement rules for safe and productive operations. False alarms in those manual labels (labeling no-violation images as having violations) can significantly mislead the machine learning process and result in computer vision models that produce inaccurate hazard detections. Compared with false alarms, another type of mislabels, false negatives (labeling images having violations as “no violations”), seem to have fewer impacts on the reliability of the trained computer vision models.

97 MATHEMATICS AND COMPUTING↗

GradDFT. A software library for machine learning enhanced density functional theory

Density functional theory (DFT) stands as a cornerstone method in computational quantum chemistry and materials science due to its remarkable versatility and scalability. Yet, it suffers from limitations in accuracy, particularly when dealing with strongly correlated systems. To address these shortcomings, recent work has begun to explore how machine learning can expand the capabilities of DFT: an endeavor with many open questions and technical challenges. In this work, we present GradDFT a fully differentiable JAX-based DFT library, enabling quick prototyping and experimentation with machine learning-enhanced exchange–correlation energy functionals. GradDFT employs a pioneering parametrization of exchange–correlation functionals constructed using a weighted sum of energy densities, where the weights are determined using neural networks. Moreover, GradDFT encompasses a comprehensive suite of auxiliary functions, notably featuring a just-in-time compilable and fully differentiable self-consistent iterative procedure. To support training and benchmarking efforts, we additionally compile a curated dataset of experimental dissociation energies of dimers, half of which contain transition metal atoms characterized by strong electronic correlations. The software library is tested against experimental results to study the generalization capabilities of a neural functional across potential energy surfaces and atomic species, as well as the effect of training data noise on the resulting model accuracy.

Chemistry↗

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗