Graph Convolutional Neural Networks as Surrogate Models for Climate Simulation
Explore the source record for details and available documents.
Potter, Kevin Matthew↗
SEARCH · Search NASA
Results for “Convolutional neural network”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Facial Named Entity Recognition by Attention-Based Graph Convolutional Neural Network
In the realm of facial recognition and analysis, the ability to accurately cluster large datasets of facial images stands as a cornerstone for various applications, ranging from security surveillance to user biometric identification. This project evolves a novel approach to facial data clustering by embedding facial images into a high-dimensional vector space using an advanced embedding model trained on separate data and assumes a graph-like structure on the high-dimensional vectors. We find our method works significantly better than common shallow methods.
97 MATHEMATICS AND COMPUTING↗
Event Classifications on DNE2 Main Experiment Data using a Convolutional Neural Network Ensemble
The Dynamic Networks (DN) Experiment for FY24 (DNE2) is an experiment within DN with the goal of quantitatively evaluating the effectiveness of solutions developed so far by various researchers under the Low Yield Nuclear Monitoring (LYNM) program using a shared set of metrics and datasets. A key component of this experiment is the mimicking of a signature processing pipeline, and comparing currently accepted and standard-use processing methods to more state-of-the-art processes developed under DN. In this work, we focus specifically on the Event Characterization (EC) Focus Area (FA) of the pipeline, where a seismic event’s magnitude, yield and class are identified. We use Deep Learning (DL) to classify the type of events being processed as either earthquakes (EQs) or explosions (EXs) for three iterations of experiment datasets. The model is noticeably more confident and accurate in classifying explosions than earthquakes, reflecting a known shortcoming of the model, that being of a bias towards predicting explosions over earthquakes in the west coast due to training data biases.
97 MATHEMATICS AND COMPUTING↗
Accelerating kinetic plasma simulations with machine-learning-generated initial conditions
Computational models of plasma technologies often solve for the system operating conditions by time-stepping an initial value problem to a quasi-steady solution. However, the strongly nonlinear and multi-timescale nature of plasma dynamics often necessitate millions, or even hundreds of millions, of steps to reach convergence, reducing the effectiveness of these simulations for computer-aided engineering. We consider acceleration of kinetic plasma simulations via data-driven machine-learning-generated initial conditions, which initialize the simulations close to their final quasi-steady-state, thereby reducing the number of steps to reach convergence. Three machine-learning models are developed to predict the density and ion kinetic profiles of capacitively coupled plasma discharges relevant to the microelectronics industry. The models are trained on kinetic simulations over a range of device operating frequencies and pressures. Best performance was observed when simulations were initialized with ion kinetic profiles generated by a convolutional neural network, reducing the mean number of steps to reach convergence by 17.1× when compared to initialization with a zero-dimensional global model. We also outline a workflow for continuous data-driven model improvement and simulation speedup, with the aim of generating sufficient data for full device digital twins.
Artificial neural networks↗
Structure–Property Linkage in Alloys Using Graph Neural Network and Explainable Artificial Intelligence
Deep learning tools have recently shown significant potential for accelerating the prediction of microstructure–property linkage in materials. While deep neural networks like convolution neural networks (CNNs) can extract physics information from 3D microstructure images, they often require a large network architecture and substantial training time. In this research, we trained a graph neural network (GNN) using phase field generated microstructures of Ni-Al alloys to predict the evolution of mechanical properties. We found that a single GNN is capable of accurately predicting the strengthening of Ni-Al alloys with microstructures of varying sizes and dimensions, which cannot otherwise be done with a CNN. Additionally, GNN requires significantly less GPU utilization than CNN and offers more interpretable explanation of predictions using saliency analysis as features are manually defined in the graph. We also utilize explainable artificial intelligence tool Bayesian Inference to determine the coefficients in the power law equation that governs coarsening of precipitates. Overall, our work demonstrates the ability of the GNN to accurately and efficiently extract relevant information from material microstructures without having restrictions on microstructure size or dimension and offers an interpretable explanation.
Chemistry↗
Applying Machine Learning and Bayesian Inference to Identify and Locate Moving Anthropogenic Sources Using Distributed Acoustic Sensing Data
Distributed acoustic sensing (DAS) systems, which use existing telecommunication fibers, offer high‐resolution capabilities ideal for recording anthropogenic sources. However, the complexity of urban environments and the large amount of data recorded by DAS require automated methods to efficiently detect and categorize anthropogenic sources. Here, we evaluate how well three machine learning models (k‐nearest neighbor [k‐NN], convolutional neural networks, and recurrent‐convolutional neural networks) can identify various anthropogenic sources recorded by DAS. Our findings reveal that both k‐NN and neural network methods perform well in high signal‐to‐noise ratio (SNR) settings. However, their accuracy decreases at SNRs <4. We also use Kalman filtering, a form of Bayesian inference, on backprojected locations of these sources to recover locations that generally fall within standard smartphone Global Positioning System errors. By combining machine learning and Kalman filter results, we calculate a multidimensional model of moving anthropogenic sources. These results demonstrate the potential of DAS data in urban seismology for accurately identifying and locating such sources. Depending on the research objectives, these sources can be further studied or filtered out to improve the quality of seismic data for earthquake studies. Such methods provide a valuable tool for urban seismology and seismic hazard analysis.
Luckie, Thomas William [Sandia National Laboratori↗
ESM data downscaling: a comparison of super-resolution deep learning models
Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.
Pawar, Nikhil M. (ORCID:0000000211613289)↗
A deep learning approach to fast analysis of collective Thomson scattering spectra
Fast analysis of collective Thomson scattering ion acoustic wave features using a deep convolutional neural network model is presented. The network was trained from spectra to predict the plasma parameters, including ion velocities, population fractions, and ion and electron temperatures. A fully kinetic particle-in-cell simulation was used to model a laboratory astrophysics experiment and simulate a diagnostic image of the ion acoustic wave feature. Network predictions were compared with Bayesian inference of the plasma model parameters for both the simulated and experimentally measured images. Both approaches were fairly accurate predicting the simulated image and the network predictions matched a good portion of the Bayesian results for the experimentally measured image. The Bayesian approach is more robust to noise and motivates future work to train deep learning models with realistic noise. The advantage of the deep learning model is making thousands of predictions in a few hundred milliseconds, compared to a few seconds to minutes per prediction for the optimization and Bayesian approaches presented here. The results demonstrate promising capabilities of deep learning models to analyze Thomson data orders of magnitude faster than conventional methods when using the neural network for standalone analysis. If more rigorous analysis is needed, neural network predictions can be used to quickly initialize other optimization methods and increase chances of success. This is especially useful when the dataset becomes very large or highly dimensional and manually refining initial conditions for the entire dataset are no longer tractable.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Convolutional Neural Network-Based Protection-Zone Classification of Faults in Distribution Feeders with Photovoltaics.
Fault detection and isolation is critical for reliable operation of distribution systems. The ride-through requirements for the distributed energy resources (DER), mandated by the IEEE 1547-2018 standard, makes it challenging to use undervoltage (UV) conditions for fault detection. In addition, with low fault current contribution from these inverter-based DERs, the time-overcurrent relays are also less effective. Thus motivated, this paper presents a learning-based approach for fault detection and localization. A convolutional neural network (CNN)-based model is proposed which uses local voltage and current waveforms from DER locations and feeder substations, for training a zonal classifier. The classifier can be adopted into any relay-like device for discriminating between faults originating from different protection zones. The performance of the proposed approach was tested on publicly available test feeders with distributed photovoltaics (PVs).
Fault localization, convolutional neural networks,↗
Transfer learning for analysis of collective and non-collective Thomson scattering spectra
Thomson scattering (TS) diagnostics provide reliable, minimally perturbative measurements of fundamental plasma parameters, such as electron density (n e ) and electron temperature (T e ). Deep neural networks can provide accurate estimates of n e and T e when conventional fitting algorithms may fail, such as when TS spectra are dominated by noise, or when fast analysis is required for real-time operation. Although deep neural networks typically require large training sets, transfer learning can improve model performance on a target task with limited data by leveraging pre-trained models from related source tasks, where select hidden layers are further trained using target data. We present five architecturally diverse deep neural networks, pre-trained on synthetic TS data and adapted for experimentally measured TS data, to evaluate the efficacy of transfer learning in estimating n e and T e in both the collective and non-collective scattering regimes. We evaluate errors in n e and T e estimates as a function of training set size for models trained with and without transfer learning, and we observe decreases in model error from transfer learning when the training set contains ≲ 200 experimentally measured spectra.
Artificial neural networks↗
New Measurements of the Lyα Forest Continuum and Effective Optical Depth with LyCAN and DESI Y1 Data
Abstract We present the Ly α Continuum Analysis Network (LyCAN), a convolutional neural network that predicts the unabsorbed quasar continuum within the rest-frame wavelength range of 1040–1600 Å based on the red side of the Ly α emission line (1216–1600 Å). We developed synthetic spectra based on a Gaussian mixture model representation of nonnegative matrix factorization (NMF) coefficients. These coefficients were derived from high-resolution, low-redshift ( z < 0.2) Hubble Space Telescope/Cosmic Origins Spectrograph (COS) quasar spectra. We supplemented this COS-based synthetic sample with an equal number of DESI Year 5 mock spectra. LyCAN performs extremely well on testing sets, achieving a median error in the forest region of 1.5% on the DESI mock sample, 2.0% on the COS-based synthetic sample, and 4.1% on the original COS spectra. LyCAN outperforms principal component analysis (PCA) and NMF-based prediction methods using the same training set by 40% or more. We predict the intrinsic continua of 83,635 DESI Year 1 spectra in the redshift range of 2.1 ≤ z ≤ 4.2 and perform an absolute measurement of the evolution of the effective optical depth. This is the largest sample employed to measure the optical depth evolution to date. We fit a power law of the form τ ( z ) = τ 0 ( 1 + z ) γ to our measurements and find τ 0 = (2.46 ± 0.14) × 10 −3 and γ = 3.62 ± 0.04. Our results show particular agreement with high-resolution, ground-based observations around z = 2, indicating that LyCAN is able to predict the quasar continuum in the forest region with only spectral information outside the forest.
79 ASTRONOMY AND ASTROPHYSICS↗
Impurity gas detection for SNF canisters using probabilistic deep learning and acoustic sensing *
Abstract Monitoring impurity gases in spent nuclear fuel (SNF) canisters is a novel structural health monitoring approach for SNF in dry storage. The SNF canisters are sealed containers that do not facilitate visual access to the inside. Acoustic sensing can be deployed by taking advantage of the pathways unobstructed by internal hardware. Although the ultrasonic time-of-flight measurement can provide valuable information, it is limited in its ability to discern the concentration of only one impurity gas. As such, deep learning algorithms, particularly convolutional neural networks (CNNs), offer a promising solution. In this study, CNN-based probabilistic deep learning models were implemented to detect and quantify multiple impurity gases in helium. An experimental platform was established to simulate canister conditions, and ultrasonic test data were collected. The presence of argon and air in helium at concentrations ranging from 0% to 1.2% at increments of 0.05% was considered. The multi-layer perceptron, decision tree, and logistic regression classifiers achieved high accuracies when distinguishing pure helium from helium with impurities. CNN with dropout layers and CNN using maximum likelihood estimation showed a similar performance, indicating their ability to capture uncertainties. The ensemble CNN model exhibited improved predictions and the ability to balance individual gas concentration by integrating 1D- and 2D-CNN models. These findings contribute probabilistic deep learning solutions for impurity gas detection and analysis within SNF canisters, thus ensuring safe storage and management of SNFs.
47 OTHER INSTRUMENTATION↗
Neural network denoising of x-ray images from high-energy-density experiments
Noise is a consistent problem for x-ray transmission images of High-Energy-Density (HED) experiments because it can significantly affect the accuracy of inferring quantitative physical properties from these images. We consider experiments that use x-ray area backlighting to image a thin layer of opaque material within a physics package to observe its hydrodynamic evolution. The spatial variance of the x-ray transmission across the system due to changing opacity serves as an analog for measuring density in this evolving layer. The noise in these images adds nonphysical variations in measured intensity, which can significantly reduce the accuracy of our inferred densities, particularly at small spatial scales. Denoising these images is thus necessary to improve our quantitative analysis, but any denoising method also affects the underlying information in the image. In this paper, we present a method for denoising HED x-ray images via a deep convolutional neural network model with a modified DenseNet architecture. In our denoising framework, we estimate the noise present in the real (data) images of interest and apply the inferred noise distribution to a set of natural images. These synthetic noisy images are then used to train a neural network model to recognize and remove noise of that character. We show that our trained denoiser network significantly reduces the noise in our experimental images while retaining important physical features.
47 OTHER INSTRUMENTATION↗
Overview of RFID Applications Utilizing Neural Networks
As Radio Frequency Identification (RFID) methods continue to evolve to higher levels of complexity, one form of machine learning is making its appearance. The use of Neural Networks (NN) in the RFID field is steadily increasing, and in the fields of localization and activity recognition, promising results are being shown from a variety of research. RFID applications fall primarily under two types of problems including regression and classification. We analyze RIFD localization techniques which fall under regression, and activity recognition which falls under classification. Many works don’t classify themselves as activity recognition methods, but because they fall under the classification category, we still consider them as activity recognition techniques. This research overviews the Neural Network models in the localization field based on whether they can perform independently of the environment in which they were tested. For activity recognition and accessory fields, the major methods involve tag-based and tag-free approaches. In conclusion, after the models are surveyed, a comparison study is given to examine what may be the cause for increased accuracy between different Neural Network models.
42 ENGINEERING↗
Machine learning for single-ended event reconstruction in PROSPECT experiment
The Precision Reactor Oscillation and Spectrum Experiment, PROSPECT, was a segmented antineutrino detector that successfully operated at the High Flux Isotope Reactor in Oak Ridge, TN, during its 2018 run. Despite challenges with photomultiplier tube base failures affecting some segments, innovative machine learning approaches were employed to perform position and energy reconstruction, and particle classification. This work highlights the effectiveness of convolutional neural networks and graph convolutional networks in enhancing data analysis. By leveraging these techniques, a 3.3% increase in effective statistics was achieved compared to traditional methods, showcasing their potential to improve analysis performance. Furthermore, these machine learning methodologies offer promising applications for other segmented particle detectors, underscoring their versatility and impact.
47 OTHER INSTRUMENTATION↗
Neural Networks to Find the Optimal Forcing for Offsetting the Anthropogenic Climate Change Effects
Abstract Of great relevance to climate engineering is the systematic relationship between the radiative forcing to the climate system and the response of the system, a relationship often represented by the linear response function (LRF) of the system. However, estimating the LRF often becomes an ill-posed inverse problem due to high-dimensionality and nonunique relationships between the forcing and response. Recent advances in machine learning make it possible to address the ill-posed inverse problem through regularization and sparse system fitting. Here, we develop a convolutional neural network (CNN) for regularized inversion. The CNN is trained using the surface temperature responses from a set of Green’s function perturbation experiments as imagery input data together with data sample densification. The resulting CNN model can infer the forcing pattern responsible for the temperature response from out-of-sample forcing scenarios. This promising proof of concept suggests a possible strategy for estimating the optimal forcing to negate certain undesirable effects of climate change. The limited success of this effort underscores the challenges of solving an inverse problem for a climate system with inherent nonlinearity. Significance Statement Predicting the climate response for a given climate forcing is a direct problem, while inferring the forcing for a given desired climate response is often an inverse, ill-posed, problem, posing a new challenge to the climate community. This study makes the first attempt to infer the radiative forcing for a given target pattern of global surface temperature response using a deep learning approach. The resulting deeply trained convolutional neural network inversion model shows promise in capturing the forcing pattern corresponding to a given surface temperature response, with a significant implication on the design of an optimal solar radiation management strategy for curbing global warming. This study also highlights the technical challenges that future research should prioritize in seeking feasible solutions to the inverse climate problem.
Ren, Huiying↗
Automated Fire Detection for Industrial Settings with Pretrained Convolutional Networks
Early fire detection in industrial environments is critical to preventing equipment damage, personal injury, and operational disruptions. Traditional smoke detectors, while effective, often experience delays due to the time required for smoke to reach sensors, allowing fires to spread. Manual fire watch operations and human surveillance of camera feeds are resource-intensive and prone to human error. To address these challenges, this paper explores the application of convolutional neural networks for automated fire detection, specifically in industrial settings. By leveraging 11 different pre-trained machine vision models from TensorFlow and enhancing them with transfer learning on a custom-built industrial fire dataset, we optimized fire detection performance. Here, we analyzed each machine vision model architecture in terms of its depth, width, and input image resolution, considering both resource requirements and detection accuracy. We further explored the option of combining multiple models into an ensemble classifier to evaluate whether the performance improvements could justify the much greater computational complexity and other practical impacts. A cost-benefit analysis is presented to evaluate the trade-offs between performance and computational expense. Our findings identify that EfficientNetV2L, specifically tailored for industrial applications, provides the optimal balance between costs involved in training and using the model versus the overall fire detection performance. Additionally, we present a qualitative analysis of model performance using the technique of gradient-based class activation mapping to provide explainability by visualizing model decisions.
artificial intelligence↗