Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep Convolutional Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Dual-Biometric Human Identification Using Radar Deep Transfer Learning

Accurate human identification using radar has a variety of potential applications, such as surveillance, access control and security checkpoints. Nevertheless, radar-based human identification has been limited to a few motion-based biometrics that are solely reliant on micro-Doppler signatures. This paper proposes for the first time the use of combined radar-based heart sound and gait signals as biometrics for human identification. The proposed methodology starts by converting the extracted biometric signatures collected from 18 subjects to images, and then an image augmentation technique is applied and the deep transfer learning is used to classify each subject. A validation accuracy of 58.7% and 96% is reported for the heart sound and gait biometrics, respectively. Next, the identification results of the two biometrics are combined using the joint probability mass function (PMF) method to report a 98% identification accuracy. To the best of our knowledge, this is the highest reported in the literature to date. Lastly, the trained networks are tested in an actual scenario while being used in an office access control platform to identify different human subjects. We report an accuracy of 76.25%.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

FPDeep: Scalable Acceleration of CNN Training on Deeply-Pipelined FPGA Clusters

In this paper, we propose a framework called FPDeep, which uses a hybrid of model and layer paral- lelism to configure distributed reconfigurable clusters to train DNNs. This approach has numerous benefits. First, the design does not suffer from batch size growth. Second, novel workload and weight partitioning leads to balanced loads of both among nodes. And third, the entire system is fine-grained pipeline. This leads to high parallelism and utilization and also minimizes the time features need to be cached while waiting for back-propagation.

Wang, Tianqi↗
DOE OSTI2024

Uncertainty quantification and propagation in lithium-ion battery electrodes using bayesian convolutional neural networks

The complex nature of manufacturing processes stipulates electrodes to possess high variability with increased heterogeneity during production. X-ray computed tomography imaging has proved to be critical in visualizing the complicated stochastic particle distribution of as-manufactured electrodes in lithium-ion batteries. However, accurate prediction of their electrochemical performance necessitates precise evaluation of kinetic and transport properties from real electrodes. Image segmentation that characterizes voxels to particle/pore phase is often meticulous and fraught with subjectivity owing to a myriad of unconstrained choices and filter algorithms. Here we utilize a Bayesian convolutional neural network to tackle segmentation subjectivity and quantify its pertinent uncertainties. Otsu inter-variance and Blind/Referenceless Imaging Spatial Quality Evaluator are used to assess the relative image quality of grayscale tomograms, thus evaluating the uncertainty in the derived microstructural attributes. We analyze how image uncertainty is correlated with the uncertainties and magnitude of kinetic and transport properties of an electrode, further identifying pathways of uncertainty propagation within microstructural attributes. The coupled effect of spatial heterogeneity and microstructural anisotropy on the uncertainty quantification of transport parameters is also understood. This work demonstrates a novel methodology to extract microstructural descriptors from real electrode images through quantification of associated uncertainties and discerning the relative strength of their propagation, thus facilitating feedback to manufacturing processes from accurate image based electrochemical simulations.

25 ENERGY STORAGE↗
DOE OSTI2022

A Novel Method for Controlling Crud Deposition in Nuclear Reactors Using Optimization Algorithms and Deep Neural Network Based Surrogate Models

This work presents the use of a high-fidelity neural network surrogate model within a Modular Optimization Framework for treatment of crud deposition as a constraint within light-water reactor core loading pattern optimization. The neural network was utilized for the treatment of crud constraints within the context of an advanced genetic algorithm applied to the core design problem. This proof-of-concept study shows that loading pattern optimization aided by a neural network surrogate model can optimize the manner in which crud distributes within a nuclear reactor without impacting operational parameters such as enrichment or cycle length. Several analysis methods were investigated. Analysis found that the surrogate model and genetic algorithm successfully minimized the deviation from a uniform crud distribution against a population of solutions from a reference optimization in which the crud distribution was not optimized. Strong evidence is presented that shows boron deposition in crud can be optimized through the loading pattern. This proof-of-concept study shows that the methods employed provide a powerful tool for mitigating the effects of crud deposition in nuclear reactors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Fast and Accurate Predictions of Total Energy for Solid Solution Alloys with Graph Convolutional Neural Networks

We use graph convolutional neural networks (GCNNs) to produce fast and accurate predictions of the total energy of solid solution binary alloys. GCNNs allow us to abstract the lattice structure of a solid material as a graph, whereby atoms are modeled as nodes and metallic bonds as edges. This representation naturally incorporates information about the structure of the material, thereby eliminating the need for computationally expensive data pre-processing which would be required with standard neural network (NN) approaches. We train GCNNs on ab-initio density functional theory (DFT) for copper-gold (CuAu) and iron-platinum (FePt) data that has been generated by running the LSMS-3 code, which implements a locally self-consistent multiple scattering method, on OLCF supercomputers Titan and Summit. GCNN outperforms the ab-initio DFT simulation by orders of magnitude in terms of computational time to produce the estimate of the total energy for a given atomic configuration of the lattice structure. We compare the predictive performance of GCNN models against a standard NN such as dense feedforward multi-layer perceptron (MLP) by using the root-mean-squared errors to quantify the predictive quality of the deep learning (DL) models. We find that the attainable accuracy of GCNNs is at least an order of magnitude better than that of the MLP.

Lupo Pasini, Massimiliano↗
DOE OSTI2022

High–throughput measurement of plant fitness traits with an object detection method using Faster R–CNN

Revealing the contributions of genes to plant phenotype is frequently challenging because loss-of-function effects may be subtle or masked by varying degrees of genetic redundancy. Such effects can potentially be detected by measuring plant fitness, which reflects the cumulative effects of genetic changes over the lifetime of a plant. However, fitness is challenging to measure accurately, particularly in species with high fecundity and relatively small propagule sizes such as Arabidopsis thaliana. An image segmentation-based method using the software ImageJ and an object detection-based method using the Faster Region-based Convolutional Neural Network (R-CNN) algorithm were used for measuring two Arabidopsis fitness traits: seed and fruit counts. The segmentation-based method was error-prone (correlation between true and predicted seed counts, r 2 = 0.849) because seeds touching each other were undercounted. By contrast, the object detection-based algorithm yielded near perfect seed counts (r 2 = 0.9996) and highly accurate fruit counts (r 2 = 0.980). Comparing seed counts for wild-type and 12 mutant lines revealed fitness effects for three genes; fruit counts revealed the same effects for two genes. Our study provides analysis pipelines and models to facilitate the investigation of Arabidopsis fitness traits and demonstrates the importance of examining fitness traits when studying gene functions.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

From Simulation to Reality With Random Noise

The challenging environment of autonomous vehicle (AV) navigation necessitates certain functions be performed by deep neural networks. Optimizing these models involves collecting vast quantities of domain-specific training data and ensuring that the dataset is representative of expected conditions. High-fidelity simulation plays a vital role in making this process feasible, allowing a wide range of scenarios to be explored at low cost. However, learning from simulation introduces subtle biases into models, which can degrade real-world performance in unpredictable ways. This effect can be mitigated with learning schemes specialized to bridge distributional shifts (transfer learning). Given the complex nature of these methods, the underlying models, and their environments, meaningfully evaluating performance is notstraight forward. Many unrelated factors can effect an improvement in generalization accuracy, but a full ablation analysis is often difficult. To tease out signal from noise, it is necessary to understand how transfer learning performance is affected by noise itself. The goals of this paper are (i) to establish a domain randomization baseline for a simple classification transfer learning task and (ii) to validate the RRAV testbed as a platform for further research in sim-to-real learning. We generate imagery from a simulation of NASA Ames Research Center and train a small convolutional neural network (ConvNet) to classify position relative to a centerline. Further models are trained with different types of noise progressively added to the data. The models are deployed aboard the on-site test vehicle to test real-world performance. In our experiments, we find that such naive domain randomization raises sim-to-real accuracy from 64% to 79%, while training directly on real data yields an 89% accuracy ceiling. These results suggest that the isolated mechanism of domain randomization can significantly improve generalization.

simulation↗
DOE OSTI2022

Non-Blind Deblurring for Fluorescence: A Deformable Latent Space Approach with Kernel Parameterization

We report N\non-blind deblurring (NBD) is a modeling method of the image deblurring problem in computer vision, where the blurring kernel is known or can be externally estimated. In this paper, we attempt to solve a parametric NBD problem, inspired by the simultaneous acquisition of ptychography and fluorescent imaging (FI). Ptychography is an imaging method that favors larger probes, i.e. convolutional kernels, while FI relies on a small probe for high resolution. Also, the kernel can be solved during ptychographic reconstruction. With Ptycho-FI using the same larger kernel, we can perform NBD on the blurred fluorescent images to achieve high-resolution FI, and thus speed up the experiments. To this end, we design a deep latent space deformation network that is directly parameterized by the kernel. The network consists of three components: encoder, deformer, and decoder, where the deformer is specifically meant to rectify the latent space representations of blurred images to a standard latent space, regardless of the kernel. The deformation network is trained with a two-stage training scheme. We conduct extensive experiments to confirm that our parametric model can adapt to drastically different blurring kernels and perform robust deblurring.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2023

Graph neural networks predict energetic and mechanical properties for models of solid solution metal alloy phases

Here, we developed a PyTorch-based architecture called HydraGNN that implements graph convolutional neural networks (GCNNs) to predict the formation energy and the bulk modulus for models of solid solution alloys for various atomic crystal structures and relaxed volumes. We trained the GCNN surrogate model on a dataset for nickel–niobium (NiNb) generated by the embedded atom model (EAM) empirical interatomic potential for demonstration purposes. The dataset was generated by calculating the formation energy and the bulk modulus as a prototypical elastic property for optimized geometries starting from initial body-centered cubic (BCC), face-centered cubic (FCC), and hexagonal compact packed (HCP) crystal structures, with configurations spanning the possible compositional range for each of the three types of initial crystal structures. Numerical results show that the GCNN model effectively predicts both the formation energy and the bulk modulus as function of the optimized crystal structure, relaxed volume, and configurational entropy of the model structures for solid solution alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Neural Architecture Search Benchmarks: Insights and Survey

Neural Architecture Search (NAS), a promising and fast-moving research field, aims to automate the architectural design of Deep Neural Networks (DNNs) to achieve better performance on the given task and dataset. NAS methods have been very successful in discovering efficient models for various Computer Vision, Natural Language Processing, etc. The major obstacles to the advancement of NAS techniques are the demand for large computation resources and fair evaluation of various search methods. The differences in training pipeline and setting make it challenging to compare the efficiency of two NAS algorithms. A large number of NAS Benchmarks to simulate the architecture evaluation in seconds have been released over the last few years to ease the computation burden of training neural networks and can aid in the unbiased assessment of different search methods. This paper provides an extensive review of several publicly available NAS Benchmarks in the literature. We provide technical details and a deeper understanding of each benchmark and point out future directions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-Efficient ReRAM

The wide adoption of deep neural networks has been accompanied by ever-increasing energy and performance demands due to the expensive nature of training them. Additionally, numerous special-purpose architectures have been proposed to accelerate training: both digital and hybrid digital-analog using resistive RAM (ReRAM) crossbars. ReRAM-based accelerators have demonstrated the effectiveness of ReRAM crossbars at performing matrix-vector multiplication operations that are prevalent in training. However, they still suffer from inefficiency due to the use of serial reads and writes for performing the weight gradient and update step. A few works have demonstrated the possibility of performing outer products in crossbars, which can be used to realize the weight gradient and update step without the use of serial reads and writes. However, these works have been limited to low precision operations which are not sufficient for typical training workloads. Moreover, they have been confined to a limited set of training algorithms for fully-connected layers only. To address these limitations, we propose a bit-slicing technique for enhancing the precision of ReRAM-based outer products, which is substantially different from bit-slicing for matrix-vector multiplication only. We incorporate this technique into a crossbar architecture with three variants catered to different training algorithms. To evaluate our design on different types of layers in neural networks (fully-connected, convolutional, etc.) and training algorithms, we develop PANTHER, an ISA-programmable training accelerator with compiler support. Our design can also be integrated into other accelerators in the literature to enhance their efficiency. Our evaluation shows that PANTHER achieves up to 8.02×, 54.21×, and 103× energy reductions as well as 7.16×, 4.02×, and 16× execution time reductions compared to digital accelerators, ReRAM-based accelerators, and GPUs, respectively.

42 ENGINEERING↗
DOE OSTI2022

SAFE-OCC: A novelty detection framework for Convolutional Neural Network sensors and its application in process control

Herein we present a novelty detection framework for Convolutional Neural Network (CNN) sensors that we call Sensor-Activated Feature Extraction One-Class Classification (SAFE-OCC). We show that this framework enables the safe use of computer vision sensors in process control architectures. Emergent control applications use CNN models to map visual data to a state signal that can be interpreted by the controller. Incorporating such sensors introduces a significant system operation vulnerability because CNN sensors can exhibit high prediction errors when exposed to novel (abnormal) visual data. Unfortunately, identifying such novelties in real-time is nontrivial. To address this issue, the SAFE-OCC framework leverages the convolutional blocks of the CNN to create an effective feature space to conduct novelty detection using a desired one-class classification technique. This approach engenders a feature space that directly corresponds to that used by the CNN sensor and avoids the need to derive an independent latent space. We demonstrate the effectiveness of SAFE-OCC via simulated control environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Data-Efficient Dimensionality Reduction and Surrogate Modeling of High-Dimensional Stress Fields

Tensor datatypes representing field variables like stress, displacement, velocity, etc., have increasingly become a common occurrence in data-driven modeling and analysis of simulations. Numerous methods [such as convolutional neural networks (CNNs)] exist to address the meta-modeling of field data from simulations. As the complexity of the simulation increases, so does the cost of acquisition, leading to limited data scenarios. Modeling of tensor datatypes under limited data scenarios remains a hindrance for engineering applications. Here, in this article, we introduce a direct image-to-image modeling framework of convolutional autoencoders enhanced by information bottleneck loss function to tackle the tensor data types with limited data. The information bottleneck method penalizes the nuisance information in the latent space while maximizing relevant information making it robust for limited data scenarios. The entire neural network framework is further combined with robust hyperparameter optimization. We perform numerical studies to compare the predictive performance of the proposed method with a dimensionality reduction-based surrogate modeling framework on a representative linear elastic ellipsoidal void problem with uniaxial loading. The data structure focuses on the low-data regime (fewer than 100 data points) and includes the parameterized geometry of the ellipsoidal void as the input and the predicted stress field as the output. The results of the numerical studies show that the information bottleneck approach yields improved overall accuracy and more precise prediction of the extremes of the stress field. Additionally, an in-depth analysis is carried out to elucidate the information compression behavior of the proposed framework.

artificial intelligence↗
DOE OSTI2022

Accurate Prediction of Voltage of Battery Electrode Materials Using Attention-Based Graph Neural Networks

Performing first-principles calculations to discover electrodes’ properties in the large chemical space is a challenging task. While machine learning (ML) has been applied to effectively accelerate those discoveries, most of the applied methods ignore the materials’ spatial information and only use predefined features: based only on chemical compositions. Here, we propose two attention-based graph convolutional neural network techniques to learn the average voltage of electrodes. Our proposed methods, which combine both atomic composition and atomic coordinates in 3D-space, improve the accuracy in voltage prediction significantly when compared to composition-based ML models. The first model directly learns the chemical reaction of electrodes and metal ions to predict their average voltage, whereas the second model combines electrodes’ ML predicted formation energy (E form ) to compute their average voltage. Our E form -based model demonstrates improved accuracy in transferability from our subset of learned Li ions to Na ions. Moreover, we predicted the theoretical voltage of 10 Na x MPO 4 F (M = Ti, Cr, Fe, Cu, Mn, Co, and Ni) fluorophosphate battery frameworks, which are unavailable in the Material Project database. It could be shown that we can expect average voltages higher than 3.1 V from those Na battery frameworks except from the NaTiPO 4 F and TiPO 4 F pair of electrodes, which offer an average voltage of 1.32 V.

25 ENERGY STORAGE↗
DOE OSTI2025

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Reconstructing Ly$α$ Fields from Low-resolution Hydrodynamical Simulations with Deep Learning

Hydrodynamical cosmological simulations are a powerful tool for accurately predicting the properties of the intergalactic medium (IGM) and for producing mock skies that can be compared against observational data. However, the need to resolve density fluctuation in the IGM puts a stringent requirement on the resolution of such simulations, which in turn limits the volumes that can be modeled, even on the most powerful supercomputers. In this work, we present a novel modeling method that combines physics-driven simulations with data-driven generative neural networks to produce outputs that are qualitatively and statistically close to the outputs of hydrodynamical simulations employing eight times higher resolution. We show that the Ly$α$ flux field, as well as the underlying hydrodynamic fields, have greatly improved statistical fidelity over a low-resolution simulation. Importantly, the design of our neural network allows for sampling multiple realizations from a given input, enabling us to quantify the model uncertainty. Using test data, we demonstrate that this model uncertainty correlates well with the true error of the Ly$α$ flux prediction. Ultimately, our approach allows for training on small simulation volumes and applying it to much larger ones, opening the door to producing accurate Ly$α$ mock skies in volumes of Hubble size, as will be probed with DESI and future spectroscopic sky surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Passband Signal Detection at the Edge

Algorithms for radio frequency (RF) spectrum awareness need to be compatible with edge hardware to be practical for many applications. We developed a signal detection and classification model for the ZCU111 RF System-on-a-Chip (RFSoC) that operates on the fast Fourier transform of passband RF data. The system can detect and classify multiple signals of interest and display the predictions in real-time. The model consists of a modified ConvNeXt backbone and YOLOv3 head to operate on the Deep Learning Processing Unit on the RFSoC. We gathered datasets for training and testing by using a software defined radio to transmit example signals of Wi-Fi 802.11 b/g, Wi-Fi 802.11 n, FM Radio, LTE and LTE-M. By leveraging multiple inputs on the RFSoC frontend, the datasets span up to 4 GHz of bandwidth. The models showed high performance in classification accuracy, center frequency error, bandwidth error, and detection accuracy for both single and multi-signal datasets.

42 ENGINEERING↗