Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep Convolutional Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Physics-Constrained Deep Learning Model for Simulating Multiphase Flow in 3D Heterogeneous Porous Media

Physics-based simulators for multiphase flow in porous media emulate nonlinear processes with coupled physics, and usually require extensive computational resources for software development, maintenance and simulation execution. As a result, a huge demand exists for fast modeling of coupled processes in a wide range of subsurface applications including geological sequestration, hydrocarbon recovery and geothermal energy extraction. In this work, an efficient physics-constrained deep learning model is developed for solving multiphase flow in 3-Dimensional (3D) heterogeneous porous media. The model fully leverages the spatial topology predictive capability of convolutional neural networks, specifically U-Net with successive contracting and expansive steps, and is coupled with an efficient continuity-based smoother to predict flow responses that need spatial continuity. Furthermore, the transient regions are penalized to steer the training process such that the model can accurately capture flow in these regions. The model takes inputs including properties of porous media, fluid properties and well controls, and predicts the temporal-spatial evolution of the state variables (pressure and saturation). While maintaining the continuity of fluid flow, the 3D spatial domain is decomposed into 2D images for reducing training cost, and the decomposition results in an increased number of training data samples and better training efficiency. Additionally, a surrogate model is separately constructed as a postprocessor to calculate well flow rate based on the predictions of state variables from the deep learning model. We use the example of CO 2 injection into saline aquifers, and apply the physics-constrained deep learning model that is trained from physics-based simulation data and emulates the physics process. The model performs prediction with a speedup of ~ 1400 times compared to physics-based simulations, and the average temporal errors of predicted pressure and saturation plumes are 0.27% and 0.099% respectively. Furthermore, water production rate is efficiently predicted by a surrogate model for well flow rate, with a mean error less than 5%. Therefore, with its unique scheme to cope with the fidelity in fluid flow in porous media, the physics-constrained deep learning model can become an efficient predictive model for computationally demanding inverse problems or other coupled processes.

58 GEOSCIENCES↗
DOE OSTI2024

Harnessing graph convolutional neural networks for identification of glassy states in metallic glasses

Graph Convolutional Neural Networks (GCNNs) have emerged as powerful tools for analyzing materials. In this study, we employ GCNNs to examine structural characteristics of CuZr metallic glasses (MGs) and identify their states. We use molecular dynamics to simulate the quenching process of CuZr, using cooling rates ranging from 10 9 to 10 15 K/s, to produce six unique glassy states. For each state, we create a dataset comprising 1,800 distinct samples. We evaluate the effectiveness of various GCNNs, including Graph Attention Neural Network (GANN), Graph Sample and AggreGatE (GraphSAGE), Graph Isomorphism Network (GIN), and Relational Graph Convolutional Neural Network (RGCN). GANN and GraphSAGE demonstrate comparable performance, achieving an overall accuracy of 81% in classifying the MG states. Furthermore, these results underscore the potential of GCNNs to detect subtle structural variances in disordered materials and point to broader application of deep learning in the analysis of MGs and other amorphous substances.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Noise signal identification in time projection chamber data using deep learning model

Deep learning has been employed in various scientific fields and has provided promising results. Here, in this study, a deep learning classifier was implemented to improve the quality of data obtained from a time projection chamber. Digital waveforms of the detected signals were classified into the following three categories: particles, noises, and particles piled up with noises. A simple 1-dimensional convolutional neural network was developed for the classification. The model demonstrated an excellent performance on the test dataset. Its practical performance was also examined using track images and particle identification plots by comparing the original and clean data without the noise signals. The comparison clearly showed that the deep learning model improved the quality of data. The current study presents an effective application of the deep learning model for the time projection chamber data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Graph neural networks for efficient learning of mechanical properties of polycrystals

Herein we present graph neural networks (GNNs) as an efficient and accurate machine learning approach to predict mechanical properties of polycrystalline materials. Here, a GNN was developed based on graph representation of polycrystals incorporating only fundamental features of grains including their crystallographic orientation, size, and grain neighbor connectivity information. We tested our method on modeling stiffness and yield strength of α -Ti microstructures, varying in their crystallographic texture. We find the GNN predicts both properties with high accuracy with mean relative errors of ~1% for unseen microstructures from a given set of textures and < 2% for microstructures of unseen texture, even when presented with limited training data. This accuracy is comparable to methods that require high-resolution three-dimensional (3D) microstructure data, such as 3D convolutional neural networks (3D-CNNs) and models that depend on the computation of spatial statistics. The present results show that graph-based deep learning is a promising framework for property prediction, especially considering the high cost associated with obtaining high-resolution 3D microstructure data and the general scarcity of experimental materials datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2020

A Hybrid Deep Learning Approach to Cosmological Constraints from Galaxy Redshift Surveys

We present a deep machine learning (ML)–based technique for accurately determining σ g and Ω m from mock 3D galaxy surveys. The mock surveys are built from the AbacusCosmos suite of N -body simulations, which comprises 40 cosmological volume simulations spanning a range of cosmological parameter values, and we account for uncertainties in galaxy formation scenarios through the use of generalized halo occupation distributions (HODs). We explore a trio of ML models: a 3D convolutional neural network (CNN), a power spectrum–based fully connected network, and a hybrid approach that merges the two to combine physically motivated summary statistics with flexible CNNs. We describe best practices for training a deep model on a suite of matched-phase simulations, and we test our model on a completely independent sample that uses previously unseen initial conditions, cosmological parameters, and HOD parameters. Despite the fact that the mock observations are quite small (~0.07 h -3 Gpc 3 ) and the training data span a large parameter space (six cosmological and six HOD parameters), the CNN and hybrid CNN can constrain estimates of σ g and Ω m to ~3% and ~4%, respectively.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Accurate and Data‐Efficient Micro X‐ray Diffraction Phase Identification Using Multitask Learning: Application to Hydrothermal Fluids

Traditional analysis of highly distorted micro X‐ray diffraction (μ‐XRD) patterns from hydrothermal fluid environments is a time‐consuming process, often requiring substantial data preprocessing and labeled experimental data. Herein, the potential of deep learning with a multitask learning (MTL) architecture to overcome these limitations is demonstrated. MTL models are trained to identify phase information in μ‐XRD patterns, minimizing the need for labeled experimental data and masking preprocessing steps. Notably, MTL models show superior accuracy compared to binary classification convolutional neural networks. Additionally, introducing a tailored cross‐entropy loss function improves MTL model performance. Most significantly, MTL models tuned to analyze raw and unmasked XRD patterns achieve close performance to models analyzing preprocessed data, with minimal accuracy differences. This work indicates that advanced deep learning architectures like MTL can automate arduous data handling tasks, streamline the analysis of distorted XRD patterns, and reduce the reliance on labor‐intensive experimental datasets.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2021

Microwave Radiometer RFI Detection Using Deep Learning

Radio frequency interference (RFI) is a risk for microwave radiometers due to their requirement of very high sensitivity. The Soil Moisture Active Passive (SMAP) mission has an aggressive approach to RFI detection and filtering using dedicated spaceflight hardware and ground processing software. As more sensors push to observe at larger bandwidths in unprotected or shared spectrum, RFI detection continues to be essential. This article presents a deep learning approach to RFI detection using SMAP spectrogram data as input images. The study utilizes the benefits of transfer learning to evaluate the viability of this method for RFI detection in microwave radiometers. The well-known pretrained convolutional neural networks, AlexNet, GoogleNet, and ResNet-101 were investigated. ResNet-101 provided the highest accuracy with respect to validation data (99%), while AlexNet exhibited the highest agreement with SMAP detection (92%).

Microwave radiometry↗
DOE OSTI2022

Optimal vocabulary selection approaches for privacy-preserving deep NLP model training for information extraction and cancer epidemiology

With the use of artificial intelligence and machine learning techniques for biomedical informatics, security and privacy concerns over the data and subject identities have also become an important issue and essential research topic. Without intentional safeguards, machine learning models may find patterns and features to improve task performance that are associated with private personal information. The privacy vulnerability of deep learning models for information extraction from medical textural contents needs to be quantified since the models are exposed to private health information and personally identifiable information. The objective of the study is to quantify the privacy vulnerability of the deep learning models for natural language processing and explore a proper way of securing patients’ information to mitigate confidentiality breaches. The target model is the multitask convolutional neural network for information extraction from cancer pathology reports, where the data for training the model are from multiple state population-based cancer registries. This study proposes the following schemes to collect vocabularies from the cancer pathology reports; (a) words appearing in multiple registries, and (b) words that have higher mutual information. We performed membership inference attacks on the models in high-performance computing environments. The comparison outcomes suggest that the proposed vocabulary selection methods resulted in lower privacy vulnerability while maintaining the same level of clinical task performance.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Few-view computed tomography reconstruction using deep neural network inference

A system for generating 2D slices of a 3D image of a target volume is provided. The system receives a target sinogram collected during a computed tomography scan of the target volume. The system inputs the target sinogram to a convolutional neural network (CNN) to generate predicted 2D slices of the 3D image. The CNN is trained using training 2D slices of training 3D images. The system initializes 2D slices to the predicted 2D slices. The system reconstructs 2D slices of the 3D image from the target sinogram and the initialized 2D slices.

Kim, Hyojin↗
DOE OSTI2023

Deep learning workflow for the inverse design of molecules with specific optoelectronic properties

The inverse design of novel molecules with a desirable optoelectronic property requires consideration of the vast chemical spaces associated with varying chemical composition and molecular size. First principles-based property predictions have become increasingly helpful for assisting the selection of promising candidate chemical species for subsequent experimental validation. However, a brute-force computational screening of the entire chemical space is decidedly impossible. To alleviate the computational burden and accelerate rational molecular design, we here present an iterative deep learning workflow that combines (i) the density-functional tight-binding method for dynamic generation of property training data, (ii) a graph convolutional neural network surrogate model for rapid and reliable predictions of chemical and physical properties, and (iii) a masked language model. As proof of principle, we employ our workflow in the iterative generation of novel molecules with a target energy gap between the highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital (LUMO).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak

Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. Here, in this study, we process high-speed camera data, at rates exceeding 100 kfps, on in situ field-programmable gate array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real time. Our system utilizes a convolutional neural network (CNN) model, which predicts the n = 1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6 μs and a throughput of up to 120 kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗
DOE OSTI2021

A Survey: Handling Irregularities in Neural Network Acceleration with FPGAs

In the last decade, Artificial Intelligence (AI) through Deep Neural Networks (DNNs) has penetrated virtually every aspect of science, technology, and business. Many types of DNNs have been and continue to be developed, including Convolutional Neural Networks (CNNs), Recurrent Neural Net- works (RNNs), and Graph Neural Networks (GNNs). The overall problem for all of these Neural Networks (NNs) is that their target applications generally pose stringent constraints on latency and throughput, while also having strict accuracy requirements. There have been many previous efforts in creating hardware to accelerate NNs. The problem designers face is that optimal NN models typically have significant irregularities, making them hardware-unfriendly. In this paper, we first define the problems in NN acceleration by characterizing common irregularities in NN processing into 4 types; then we summarize the existing works that handle the four types of irregularities efficiently using hardware, especially FPGAs; finally, we provide a new vision of next-generation FPGA-based NN acceleration: that the emerging heterogeneity in the next-generation FPGAs is the key to achieving higher performance.

Geng, Tong↗
DOE OSTI2021

MULTI-MODAL global surveillance methodology for predictive and on-demand characterization of localized processes using cube satellite platforms and deep learning techniques

This paper presents the work completed towards the development of a multi-modal global surveillance methodology using cube satellite (CubeSat) platforms and novel data analysis techniques. A CubeSat system equipped with adequate sensors and data analytics capabilities can autonomously characterize various phenomena of interest on the Earth’s surface. CubeSats are advantageous over conventional satellites in certain remote monitoring applications because of their reduced construction costs (due to the availability of commercially-off-the-shelf components) and are easier to launch. The CubeSat surveillance system developed in this paper focused on phenomena of interest surrounding the nuclear fuel cycle in support of nuclear non-proliferation and emergency response. To observe the phenomena, a constellation of 3U and 6U CubeSats deployed from the ISS with adequate components was chosen. Four different sensor configurations were identified for remote sensing: panchromatic/multispectral in the visible and near-infrared spectrum, multispectral in infrared spectrum, hyperspectral in infrared spectrum, and multispectral in ultraviolet spectrum. While a panchromatic/multispectral sensor configuration has CubeSat flight heritage at the required spatial resolutions, the other three sensor types need future 3 development to meet signature and system requirements. Once each sensor onboard the CubeSat system collects data on a target of interest, the onboard computers would then apply the deep learning-based characterization methodology developed in this paper to identify phenomena. Four surrogate datasets containing representative simplified “images” were created for each sensor type to train the characterization methodology. A convolutional neural network was applied to each dataset and produced recall rates for the phenomena between 89.7% - 99.3% and precision rates between 92.3% - 99.9%. Each phenomenon’s presence probability from each network is then combined into a final characterization solution for a target area. This paper covers multiple interdisciplinary areas to develop the foundation for a CubeSat surveillance system focused on phenomena surrounding the nuclear fuel cycle.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2021

A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Abstract This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Frame invariant neural network closures for Kraichnan turbulence

Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in developing parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In recent years, machine learning has been successful in extracting complex patterns from high-resolution spatio-temporal data, leading to improved parameterization models, and ultimately better coarse grid prediction. However, the inability to satisfy known physics and poor generalization hinders the application of these models for real-world problems. In this work, we put forth a frame invariant closure approach to improve the accuracy and generalizability of deep learning-based subgrid scale closure models by embedding physical symmetries directly into the structure of the neural network. Specifically, we utilized specialized layers within the convolutional neural network in such a way that desired constraints are theoretically guaranteed without the need for any regularization terms. We demonstrate our framework for a two-dimensional decaying turbulence test case mostly characterized by the forward enstrophy cascade. We show that our frame invariant SGS model (i) accurately predicts the subgrid scale source term, (ii) respects the physical symmetries such as translation, Galilean, and rotation invariance, and (iii) is numerically stable when implemented in coarse-grid simulation with generalization to different initial conditions and Reynolds number. This work opens up a possibility of connecting physics-based theories and data-driven modeling paradigms, and thus represents a promising step towards the development of physically consistent data-driven turbulence closure models.

42 ENGINEERING↗
DOE OSTI2022

PRODeepSyn: predicting anticancer synergistic drug combinations by embedding cell lines with protein–protein interaction network

Abstract Although drug combinations in cancer treatment appear to be a promising therapeutic strategy with respect to monotherapy, it is arduous to discover new synergistic drug combinations due to the combinatorial explosion. Deep learning technology holds immense promise for better prediction of in vitro synergistic drug combinations for certain cell lines. In methods applying such technology, omics data are widely adopted to construct cell line features. However, biological network data are rarely considered yet, which is worthy of in-depth study. In this study, we propose a novel deep learning method, termed PRODeepSyn, for predicting anticancer synergistic drug combinations. By leveraging the Graph Convolutional Network, PRODeepSyn integrates the protein–protein interaction (PPI) network with omics data to construct low-dimensional dense embeddings for cell lines. PRODeepSyn then builds a deep neural network with the Batch Normalization mechanism to predict synergy scores using the cell line embeddings and drug features. PRODeepSyn achieves the lowest root mean square error of 15.08 and the highest Pearson correlation coefficient of 0.75, outperforming two deep learning methods and four machine learning methods. On the classification task, PRODeepSyn achieves an area under the receiver operator characteristics curve of 0.90, an area under the precision–recall curve of 0.63 and a Cohen’s Kappa of 0.53. In the ablation study, we find that using the multi-omics data and the integrated PPI network’s information both can improve the prediction results. Additionally, the case study demonstrates the consistency between PRODeepSyn and previous studies.

Wang, Xiaowen↗
DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

MicroBooNE, a Liquid Argon Time Projection Chamber (LArTPC) located in the $\nu_{\mu}$-dominated Booster Neutrino Beam at Fermilab, has been studying $\nu_{e}$ charged-current (CC) interaction rates to shed light on the MiniBooNE low energy excess. The LArTPC technology employed by MicroBooNE provides the capability to image neutrino interactions with mm-scale precision. Computer vision and other machine learning techniques are promising tools for image processing that could boost efficiencies for selecting $\nu_{e}$-CC and other rare signals, reduce cosmic and beam-induced backgrounds, and improve the reconstruction of neutrino energies. The MicroBooNE experiment has been at the forefront of developing and testing such techniques for use in physics analyses. In this poster we overview deep-learning based reconstruction methods. We will showcase the use of a recurrent neural network to estimate neutrino energies and present a new reconstruction framework that uses convolutional neural networks to locate neutrino interaction vertices, tag pixels with track and shower labels, and perform particle identification on reconstructed clusters. We will present studies characterizing the performance of these new tools and demonstrate their effectiveness through their use in an inclusive $\nu_{e}$-CC event selection.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗