Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,531 records · Page 85
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Hierarchical, rotation‐equivariant neural networks to select structural models of protein complexes

Abstract Predicting the structure of multi‐protein complexes is a grand challenge in biochemistry, with major implications for basic science and drug discovery. Computational structure prediction methods generally leverage predefined structural features to distinguish accurate structural models from less accurate ones. This raises the question of whether it is possible to learn characteristics of accurate models directly from atomic coordinates of protein complexes, with no prior assumptions. Here we introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes, without using any precomputed physics‐inspired or statistical terms. Our neural network architecture combines multiple ingredients that together enable end‐to‐end learning from molecular structures containing tens of thousands of atoms: a point‐based representation of atoms, equivariance with respect to rotation and translation, local convolutions, and hierarchical subsampling operations. When used in combination with previously developed scoring functions, our network substantially improves the identification of accurate structural models among a large set of possible models. Our network can also be used to predict the accuracy of a given structural model in absolute terms. The architecture we present is readily applicable to other tasks involving learning on 3D structures of large atomic systems.

Eismann, Stephan↗
DOE OSTI2024

RandONets: Shallow networks with random projections for learning linear and nonlinear operators

Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.

Interpretable machine learning↗
DOE OSTI2018

IL-Net: Using Expert Knowledge to Guide the Design of Furcated Neural Networks

Deep neural networks (DNN) excel at extracting patterns. Through representation learning and automated feature engineering on large datasets, such models have been highly successful in computer vision and natural language applications. Designing optimal network architectures from a principled or rational approach however has been less than successful, with the best successful approaches utilizing an additional machine learning algorithm to tune the network hyperparameters. However, in many technical fields, there exist established domain knowledge and understanding about the subject matter. In this work, we develop a novel furcated neural network architecture that utilizes domain knowledge as high-level design principles of the network. We demonstrate proof-of-concept by developing IL-Net, a furcated network for predicting the properties of ionic liquids, which is a class of complex multi-chemicals entities. Compared to existing state-of-the-art approaches, we show that furcated networks can improve model accuracy by approximately 20-35%, without using additional labeled data. Lastly, we distill two key design principles for furcated networks that can be adapted to other domains.

Sakloth, Khushmeen↗
DOE OSTI2022

Application of machine learning and artificial intelligence to extend EFIT equilibrium reconstruction

Recent progress in the application of machine learning (ML)/artificial intelligence (AI) algorithms to improve the Equilibrium Fitting (EFIT) code equilibrium reconstruction for fusion data analysis applications is presented. A device-independent portable core equilibrium solver capable of computing or reconstructing equilibrium for different tokamaks has been created to facilitate adaptation of ML/AI algorithms. A large EFIT database comprising of DIII-D magnetic, motional Stark effect, and kinetic reconstruction data has been generated for developments of EFIT model-order-reduction (MOR) surrogate models to reconstruct approximate equilibrium solutions. Furthermore, a neural-network MOR surrogate model has been successfully trained and tested using the magnetically reconstructed datasets with encouraging results. Other progress includes developments of a Gaussian process Bayesian framework that can adapt its many hyperparameters to improve processing of experimental input data and a 3D perturbed equilibrium database from toroidal full magnetohydrodynamic linear response modeling using the Magnetohydrodynamic Resistive Spectrum - Feedback (MARS-F) code for developments of 3D-MOR surrogate models.

Gaussian process↗
DOE OSTI2021

On the feasibility of using physics-informed machine learning for underground reservoir pressure management

In this work, we evaluate the feasibility of using physics-informed machine learning (PIML) for underground energy-related pressure management. To this end, we develop a PIML framework to manage underground reservoir pressures by training neural networks to determine fluid extraction rates for dedicated extraction wells during fluid injection operations given a range of reservoir conditions (e.g., transmissivity and storativity). We implement an automatically-differentiable analytical physics model of fluid flow in porous media within the PIML framework as a proxy for more complicated models. This allows us to execute a sufficient number of training scenarios to fully evaluate the feasibility of using PIML to support pressure management activities. We quantify the number of physics-model parameters required for automatic differentiation to become more efficient than finite-difference gradient calculations. We use a simple scenario with a single injector, extractor, and critical location for our feasibility analysis. We evaluate the effect of the size of the training dataset (i.e., the number of reservoir condition samples) on the accuracy and efficiency of the PIML framework. For an equivalent number of model evaluations, the larger training dataset took less time to train and produced a neural network that was able to more accurately manage reservoir pressures. We also evaluate the effect of the training dataset batch size (i.e., number of reservoir condition samples used to update the neural network coefficients during training; i.e., how the training dataset is partitioned). While training ran faster with larger batch sizes, they produced neural networks that managed pressures less accurately. We demonstrate the approach on a more complex scenario involving 10 injectors, 10 extractors, and 4 critical locations (a relatively high well density of 20 wells/km2). We provide the number of forward and adjoint model evaluations required in each case as an indication of the feasibility of using PIML for pressure management when more complicated physics models with longer execution times are used.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Neutrino interaction vertex reconstruction in DUNE with Pandora deep learning

The Pandora Software Development Kit and algorithm libraries perform reconstruction of neutrino interactions in liquid argon time projection chamber detectors. Pandora is the primary event reconstruction software used at the Deep Underground Neutrino Experiment, which will operate four large-scale liquid argon time projection chambers at the far detector site in South Dakota, producing high-resolution images of charged particles emerging from neutrino interactions. While these high-resolution images provide excellent opportunities for physics, the complex topologies require sophisticated pattern recognition capabilities to interpret signals from the detectors as physically meaningful objects that form the inputs to physics analyses. A critical component is the identification of the neutrino interaction vertex. Subsequent reconstruction algorithms use this location to identify the individual primary particles and ensure they each result in a separate reconstructed particle. A new vertex-finding procedure described in this article integrates a U-ResNet neural network performing hit-level classification into the multi-algorithm approach used by Pandora to identify the neutrino interaction vertex. The machine learning solution is seamlessly integrated into a chain of pattern-recognition algorithms. The technique substantially outperforms the previous BDT-based solution, with a more than 20% increase in the efficiency of sub-1 cm vertex reconstruction across all neutrino flavours.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Quantified uncertainties in fission yields from machine learning

As machine learning methods gain traction in the nuclear physics community, especially those methods that aim to propagate uncertainties to unmeasured quantities, it is important to understand how the uncertainty in the training data coming either from theory or experiment propagates to the uncertainty in the predicted values. Gaussian Processes and Bayesian Neural Networks are being more and more widely used, in particular to extrapolate beyond measured data. However, studies are typically not performed on the impact of the experimental errors on these extrapolated values. In this work, we focus on understanding how uncertainties propagate from input to prediction when using machine learning methods. We use a Mixture Density Network (MDN) to incorporate experimental error into the training of the network and construct uncertainties for the associated predicted quantities. Systematically, we study the effect of the size of the experimental error, both on the reproduced training data and extrapolated predictions for fission yields of actinides.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Machine learning to accelerate screening for Marcus reorganization energies

Understanding and predicting the charge transport properties of π-conjugated materials is an important challenge for designing new organic electronic devices, such as solar cells, plastic transistors, light-emitting devices, and chemical sensors. A key component of the hopping mechanism of charge transfer in these materials is the Marcus reorganization energy which serves as an activation barrier to hole or electron transfer. While modern density functional methods have proven to accurately predict trends in intramolecular reorganization energy, such calculations are computationally expensive. In this work, we outline active machine learning methods to predict computed intramolecular reorganization energies of a wide range of polythiophenes and their use toward screening new compounds with low internal reorganization energies. Our models have an overall root mean square error (RMSE) of ±0.113 eV, but a much smaller RMSE of only ±0.036 eV on the new screening set. Since the larger error derives from high-reorganization energy compounds, the new method is highly effective to screen for compounds with potentially efficient charge transport parameters.

14 SOLAR ENERGY↗
DOE OSTI2021

Machine‐learning‐based construction of barrier functions and models for safe model predictive control

Abstract In this paper, we propose a control Lyapunov‐barrier function‐based model predictive control method utilizing a feed‐forward neural network specified control barrier function (CBF) and a recurrent neural network (RNN) predictive model to stabilize nonlinear processes with input constraints, and to guarantee that safety requirements are met for all times. The nonlinear system is first modeled using RNN techniques, and a CBF is characterized by constructing a feed‐forward neural network (FNN) model with unique structures and properties. The FNN model for the CBF is trained based on data samples collected from safe and unsafe operating regions, and the resulting FNN model is verified to demonstrate that the safety properties of the CBF are satisfied. Given sufficiently small bounded modeling errors for both the FNN and the RNN models, the proposed control system is able to guarantee closed‐loop stability while preventing the closed‐loop states from entering unsafe regions in state‐space under sample‐and‐hold control action implementation. We provide the theoretical analysis for bounded unsafe sets in state‐space, and demonstrate the effectiveness of the proposed control strategy using a nonlinear chemical process example with a bounded unsafe region.

Chen, Scarlett↗
DOE OSTI2019

Benchmarking Parametric and Machine Learning Models for Genomic Prediction of Complex Traits

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e., ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e., feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Benchmarking parametric and machine learning models for genomic prediction of complex traits

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e. ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e. feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

..delta..-Learning of High-Fidelity Electronic Structure Using Graph Neural Networks with Modified Node-Level Features

In this work, we present a ..delta..-learning approach for predicting the eigenvalues calculated with the hybrid functional HSE06 (..epsilon..nkHSE) for a set of metal and nitrogen doped graphene catalysts (MNCs) from Perdew-Burke-Ernzerhof (PBE) inputs. The model presented here incorporates electronic scalar features along with structural information in a graph neural network (GNN). In particular, the PBE eigenvalues for different bands and k-points and orbital-resolved projectors are combined with the applied potential as node-level features along with structural information within the Atomistic Line Graph Neural Network (ALIGNN) architecture. These features enable flexibility for systems with electrified interfaces, such as in electrocatalysts and achieves mean absolute error (MAE) of less than 0.1 eV. The machine learning model reported here achieves a strong generalization to left-out adsorbates (MAE = 0.074 eV) and leave-one-chemical-space-out (MAE = 0.08 eV) and completely left-out metals (MAE = 0.072 eV), confirming the robustness of the machine learning (ML) model in predicting ..epsilon..nkHSE.

36 MATERIALS SCIENCE↗
DOE OSTI2026

PopGNN: Graph Neural Network-Based Flexible Future Population Forecasting Model

Accurate population forecasts is important to plan critical infrastructure and services, from housing and education to healthcare and transport. However, traditional population prediction studies have only employed traditional machine learning models limited to capture complex spatial interdependencies and patterns. Althogh recently computer vision-based framework was introduced with with promising accuracy, it has critical limitations for real-world planning applications: it function only at fixed spatial resolutions, restricting their use in diverse boundaries such as census tracts, neighborhoods, or administrative zones. Therefore, this study suggests a Graph Neural Network (GNN)-based population prediction framework, called PopGNN. This model recorded remarkable performance compared with state-of-the-art models and traditional baseline models in the grid and administrative boundaries. Furthermore, our framework achieved comparable predictive accuracy to a computer vision-based model in both the South Korea and Tennessee case studies. Consequently, this study is valuable in that a single model can provide accurate population forecasts that address diverse planning demands, ranging from granular grid-level estimates for precise service allocation and facility location planning to aggregate administrative-level forecasts for macro-scale regional policy and resource distribution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Python Codebase and Jupyter Notebooks - Applications of Machine Learning Techniques to Geothermal Play Fairway Analysis in the Great Basin Region, Nevada

Git archive containing Python modules and resources used to generate machine-learning models used in the "Applications of Machine Learning Techniques to Geothermal Play Fairway Analysis in the Great Basin Region, Nevada" project. This software is licensed as free to use, modify, and distribute with attribution. Full license details are included within the archive. See "documentation.zip" for setup instructions and file trees annotated with module descriptions.

Brown, Stephen↗
DOE OSTI2021

Class imbalance in out-of-distribution datasets: Improving the robustness of the TextCNN for the classification of rare cancer types

In the last decade, the widespread adoption of electronic health record documentation has created huge opportunities for information mining. Natural language processing (NLP) techniques using machine and deep learning are becoming increasingly widespread for information extraction tasks from unstructured clinical notes. Disparities in performance when deploying machine learning models in the real world have recently received considerable attention. In the clinical NLP domain, the robustness of convolutional neural networks (CNNs) for classifying cancer pathology reports under natural distribution shifts remains understudied. In this research, we aim to quantify and improve the performance of the CNN for text classification on out-of-distribution (OOD) datasets resulting from the natural evolution of clinical text in pathology reports. We identified class imbalance due to different prevalence of cancer types as one of the sources of performance drop and analyzed the impact of previous methods for addressing class imbalance when deploying models in real-world domains. Our results show that our novel class-specialized ensemble technique outperforms other methods for the classification of rare cancer types in terms of macro F1 scores. We also found that traditional ensemble methods perform better in top classes, leading to higher micro F1 scores. Based on our findings, we formulate a series of recommendations for other ML practitioners on how to build robust models with extremely imbalanced datasets in biomedical NLP applications.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Differentiable multiphase flow model for physics-informed machine learning in reservoir pressure management

Accurate subsurface reservoir pressure control is extremely challenging due to geological heterogeneity and multiphase fluid-flow dynamics. Predicting behavior in this setting relies on high-fidelity physics-based simulations that are computationally expensive. Yet, the uncertain, heterogeneous properties that control these flows make it necessary to perform many of these expensive simulations, which is often prohibitive. To address these challenges, we introduce a physics-informed machine learning workflow that couples a fully differentiable multiphase flow simulator, which is implemented in the DPFEHM framework with a convolutional neural network (CNN). The CNN learns to predict fluid extraction rates from heterogeneous permeability fields to enforce pressure limits at critical reservoir locations. By incorporating transient multiphase flow physics into the training process, our method enables more practical and accurate predictions for realistic injection-extraction scenarios compared to previous works. To speed up training, we pretrain the model on single-phase, steady-state simulations and then finetune it on full multiphase scenarios, which dramatically reduces the computational cost. We demonstrate that high-accuracy training can be achieved with fewer than three thousand full-physics multiphase flow simulations – compared to previous estimates requiring up to ten million. This drastic reduction in the number of simulations is achieved by leveraging transfer learning from much less expensive single phase simulations.

25 ENERGY STORAGE↗
DOE OSTI2024

Accurate numerical simulations of open quantum systems using spectral tensor trains

Decoherence between qubits is a major bottleneck in quantum computations. Decoherence results from intrinsic quantum and thermal fluctuations as well as noise in the external fields that perform the measurement and preparation processes. With prescribed colored noise spectra for intrinsic and extrinsic noise, we present a numerical method, Quantum Accelerated Stochastic Propagator Evaluation (Q-ASPEN), to solve the time-dependent noise-averaged reduced density matrix in the presence of intrinsic and extrinsic noise. Q-ASPEN is arbitrarily accurate and can be applied to provide estimates for the resources needed to error-correct quantum computations. We employ spectral tensor trains, which combine the advantages of tensor networks and pseudospectral methods, as a variational ansatz to the quantum relaxation problem and optimize the ansatz using methods typically used to train neural networks. Here, the spectral tensor trains in Q-ASPEN make accurate calculations with tens of quantum levels feasible. We present benchmarks for Q-ASPEN on the spin-boson model in the presence of intrinsic noise and on a quantum chain of up to 32 sites in the presence of extrinsic noise. In our benchmark, the memory cost of Q-ASPEN scales as a low-order polynomial in the size of the system once the number of system states surpasses the number of basis functions used in the spectral expansion.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Pre-training Vision Models for the Classification of Alerts from Wide-field Time-domain Surveys

Modern wide-field time-domain surveys facilitate the study of transient, variable and moving phenomena by conducting image differencing and relaying alerts to their communities. Machine learning tools have been used on data from these surveys and their precursors for more than a decade, and convolutional neural networks (CNNs), which make predictions directly from input images, saw particularly broad adoption through the 2010s. Since then, continually rapid advances in computer vision have transformed the standard practices around using such models. It is now commonplace to use standardized architectures pre-trained on large corpora of everyday images (e.g., ImageNet). In contrast, time-domain astronomy studies still typically design custom CNN architectures and train them from scratch. Here, we explore the effects of adopting various pre-training regimens and standardized model architectures on the performance of alert classification. We find that the resulting models match or outperform a custom, specialized CNN like what is typically used for filtering alerts. Moreover, our results show that pre-training on galaxy images from Galaxy Zoo tends to yield better performance than pre-training on ImageNet or training from scratch. We observe that the design of standardized architectures are much better optimized than the custom CNN baseline, requiring significantly less time and memory for inference despite having more trainable parameters. On the eve of the Legacy Survey of Space and Time and other image-differencing surveys, these findings advocate for a paradigm shift in the creation of vision models for alerts, demonstrating that greater performance and efficiency, in time and in data, can be achieved by adopting the latest practices from the computer vision field.

79 ASTRONOMY AND ASTROPHYSICS↗