Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,531 records · Page 85

Accelerator Real-time Edge AI for Distributed Systems (READS) (Proposal)

Over the last decade, Machine Learning (ML) technologies have slowly made their way into the accelerator community. Rapid advances in recent years in deep learning, particularly reinforcement learning for control system applications and the accessibility of deep learning in embedded hardware, have generated renewed interest and spawned a number of applications. The Fermilab Accelerator Complex, shown in Fig. 1, has provided High Energy Physics (HEP) experiments with proton beams for nearly fifty years. The current focus of the laboratory is its world-class experimental program at the intensity frontier. While increasing beam intensity certainly presents its own challenges, preserving beam size while minimizing beam losses – particles lost through interactions with the beam vacuum pipe – turns out to be, in many ways, the main challenge. The accelerator is controlled via a complex system of hundreds of thousands of devices. Enabling fine tuning and real-time optimization of their parameters using ML methods and stepping beyond experience-based reasoning of human operators are key to the success of future intensity upgrades. Our objective will be to integrate ML into accelerator operations and furthermore, provide an accessible framework, which can also be used by a broad range of other accelerator systems with dynamic tuning needs.

43 PARTICLE ACCELERATORS↗

Classification of Sunyaev-Zel’dovich Galaxy Clusters using Deep Learning

Galaxy clusters are the largest gravitationally bound objects in the universe. One way to find and characterize them is through the Sunyaev-Zel'dovich (SZ) effect in millimeter wave observations of the cosmic microwave background (CMB). From cluster abundance across redshifts and cluster masses, we can infer the growth rate of the universe and thereby extract information about dark energy and cosmic structure evolution. This work represents a first step in applying convolutional neural networks (CNNs) to extract this information. We train a CNN on realistic simulations of the millimeter-wave sky to find galaxy clusters through classification and then to measure their gas mass through regression. We employ relatively simple CNN architectures on a suite of simulated CMB maps, each of which has a different combination of signals and noise: SZ effect, CMB signal, instrument noise, and different foregrounds. In our initial experiments we find varying classification succe ss --- worsening as more of the aforementioned contaminants are included. For regression, the correlation between the predicted and input SZ signal is clear for the case of maps with SZ, CMB, and instrument noise. While the performance does not beat current methods, the CNN is easier to set up and is less susceptible to the influence of highly structured noise terms, such as galactic dust.

79 ASTRONOMY AND ASTROPHYSICS↗

STSR-INR: Spatiotemporal super-resolution for multivariate time-varying volumetric data via implicit neural representation

Implicit neural representation (INR) has surfaced as a promising direction for solving different scientific visualization tasks due to its continuous representation and flexible input and output settings. We present STSR-INR, an INR solution for generating simultaneous spatiotemporal super-resolution for multivariate time-varying volumetric data. Inheriting the benefits of the INR-based approach, STSR-INR supports unsupervised learning and permits data upscaling with arbitrary spatial and temporal scale factors. Unlike existing GAN- or INR-based super-resolution methods, STSR-INR focuses on tackling variables or ensembles and enabling joint training across datasets of various spatiotemporal resolutions. Here we achieve this capability via a variable embedding scheme that learns latent vectors for different variables. In conjunction with a modulated structure in the network design, we employ a variational auto-decoder to optimize the learnable latent vectors to enable latent-space interpolation. To combat the slow training of INR, we leverage a multi-head strategy to improve training and inference speed with significant speedup. We demonstrate the effectiveness of STSR-INR with multiple scalar field datasets and compare it with conventional tricubic+linear interpolation and state-of-the-art deep-learning-based solutions (STNet and CoordNet).

97 MATHEMATICS AND COMPUTING↗

Deep learning of interface structures from simulated 4D STEM data: cation intermixing vs. roughening ∗

Abstract Interface structures in complex oxides remain an active area of condensed matter physics research, largely enabled by recent advances in scanning transmission electron microscopy (STEM). Yet the nature of the STEM contrast in which the structure is projected along the given direction precludes separation of possible structural models. Here, we utilize deep convolutional neural networks (DCNN) trained on simulated 4D STEM datasets to predict structural descriptors of interfaces. We focus on the widely studied interface between LaAlO 3 and SrTiO 3 , using dynamical diffraction theory and leveraging high performance computing to simulate thousands of possible 4D STEM datasets to train the DCNN to learn properties of the underlying structures on which the simulations are based. We test the DCNN on simulated data and show that it is possible (with >95% accuracy) to identify a physically rough from a chemically diffuse interface and create a DCNN regression model to predict step positions. We quantify the applicability of the model to different thicknesses and the transferability of the approach. The method shown here is general and can be applied for any inverse imaging problem where forward models are present.

42 ENGINEERING↗

Cluster, classify, regress: A general method for learning discontinuous functions

This paper presents a method for solving the supervised learning problem in which the output is highly nonlinear and discontinuous. Here, it is proposed to solve this problem in three stages: (i) cluster the pairs of input-output data points, resulting in a label for each point; (ii) classify the data, where the corresponding label is the output; and finally (iii) perform one separate regression for each class, where the training data corresponds to the subset of the original input-output pairs which have that label according to the classifier. It has not yet been proposed to combine these 3 fundamental building blocks of machine learning in this simple and powerful fashion. This can be viewed as a form of deep learning, where any of the intermediate layers can itself be deep. The utility and robustness of the methodology is illustrated on some toy problems, including one example problem arising from simulation of plasma fusion in a tokamak.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

A Review of Physics-Informed Machine Learning in Fluid Mechanics

Physics-informed machine-learning (PIML) enables the integration of domain knowledge with machine learning (ML) algorithms, which results in higher data efficiency and more stable predictions. This provides opportunities for augmenting—and even replacing—high-fidelity numerical simulations of complex turbulent flows, which are often expensive due to the requirement of high temporal and spatial resolution. In this review, we (i) provide an introduction and historical perspective of ML methods, in particular neural networks (NN), (ii) examine existing PIML applications to fluid mechanics problems, especially in complex high Reynolds number flows, (iii) demonstrate the utility of PIML techniques through a case study, and (iv) discuss the challenges and opportunities of developing PIML for fluid mechanics.

42 ENGINEERING↗

Utilization of Data Augmentation Techniques in Automated Inspection Systems for Defect Detection in Metals With Limited Data

Accurate identification of defects on metal surfaces is of great interest to many industry sectors, such as the automotive and aerospace industries. In contrast to conventional manual inspection techniques, recent automated inspection systems employ deep learning models trained to detect defects rapidly and precisely. The development of these models often requires a substantial image dataset to acquire adequate knowledge of defect features and enhance their predictive accuracy. When data is limited, augmentation techniques are often used to improve the precision and accuracy of defect detection systems. This study examined the prediction performance of two object detection models, namely Faster Region‐based Convolutional Neural Network (Faster R‐CNN) and You Only Look Once version 8 (YOLOv8), to identify dent defects in limited images of cast iron cylinder head surfaces. The original image set contains 46 images with 563 dents. To overcome limited data availability, common image augmentation techniques along with a copy‐paste method were applied. Results show that standard augmentation improved YOLOv8 accuracy by 8.00% and average precision (AP) by 3.00%. On the other hand, the copy‐paste technique achieved a 20.00% increase in accuracy and a 1% increase in AP with just 200 synthetic dents. Furthermore, these results provide support for using the copy‐paste augmentation strategy to enhance defect detection performance, with a limited dataset, contributing to more accurate defect identification in remanufacturing processes.

36 MATERIALS SCIENCE↗

Learning generative neural networks with physics knowledge

Deep generative neural networks have enabled modeling complex distributions, but incorporating physics knowledge into the neural networks is still challenging and is at the core of current physics-based machine learning research. To this end, we propose a physics generative neural network (PhysGNN), a new class of generative neural networks for learning unknown distributions in a physical system described by partial differential equations (PDE). PhysGNN couples PDE systems with generative neural networks. It is a fully differentiable model that allows back-propagation of gradients through both numerical PDE solvers and generative neural networks, and is trained by minimizing the discrete Wasserstein distance between generated and observed probability distributions of the PDE outputs using the stochastic gradient descent method. Moreover, PhysGNN does not require adversarial training like standard generative neural networks, which offers better stability than adversarial training. We show that PhysGNN can learn complex distributions in stochastic inverse problems, where conventional methods such as maximum likelihood estimation and momentum matching methods may be inapplicable when little knowledge is known about the form of unknown distributions or the physical model is too complex. Furthermore, our method allows physics-based generative neural network training for learning complex distributions in the context of differential equations.

97 MATHEMATICS AND COMPUTING↗

Solving inverse problems in stochastic models using deep neural networks and adversarial training

Inverse problems associated with stochastic models constitute a significant portion of scientific and engineering applications. In such cases the unknown quantities are distributions. The applicability of traditional methods is limited because of their demanding assumptions or prohibitive computational consumption; for example, maximum likelihood methods require closed-form density functions, and Markov Chain Monte Carlo needs a large number of simulations. We propose a new method that estimates the unknown distribution by matching the statistical properties between observed and simulated random processes. We leverage the expressive power of neural networks to approximate the unknown distribution and use a discriminative neural network for computing the statistical discrepancies between the observed and simulated random processes. Here we demonstrated numerically that the proposed methods can estimate both the model parameters and learn complicated unknown distributions.

42 ENGINEERING↗

Investigating Aerosol and Meteorological Influences on Convective Clouds in Houston, Texas, during the TRACER/ESCAPE Field Campaigns

Aerosols serve as cloud condensation nuclei, shaping the microphysical properties of cloud droplets. Aerosol effects on convective clouds are complex and remain controversial. The debate centers around the process of aerosol-induced invigoration of deep convection, a phenomenon that could significantly affect convective cloud properties but lacks robust evidence due to methodological limitations in observational approaches and questions about the robustness of modeling studies. Resolving these discrepancies is crucial for understanding how aerosols affect the atmosphere. Here, this study examines the effects of meteorological and aerosol parameters in a weakly synoptic-driven convective environment, where the influence of aerosols may be more pronounced and observable. Daily atmospheric soundings and aerosol concentrations from several ground instruments collected during the summer of 2022 in Houston, Texas, as part of the Tracking Aerosol Convection interactions Experiment (TRACER) and Experiment of Sea Breeze Convection, Aerosols, Precipitation, and Environment (ESCAPE) field campaigns are analyzed. Statistical learning methods are applied to uncover the complex relationships between aerosols, meteorology, and convective cloud characteristics, such as cell area and echo-top height. The findings reveal that higher aerosol concentrations are associated with narrower convective cells, which we argue contradicts the idea of stronger convection with increased aerosol loading. However, once the data are clustered by the synoptic environment, the relationship between aerosol loading and convective cell area diminishes, indicating that the covariablity between synoptic-scale weather patterns, local thermodynamics, and aerosol loading makes it challenging to draw definitive conclusions about the specific impacts of aerosols on convective cloud properties.

54 ENVIRONMENTAL SCIENCES↗

Safeguards-Informed Hybrid Imagery Dataset [Poster]

Deep Learning computer vision models require many thousands of properly labelled images for training, which is especially challenging for safeguards and nonproliferation, given that safeguards-relevant images are typically rare due to the sensitivity and limited availability of the technologies. Creating relevant images through real-world staging is costly and limiting in scope. Expert-labeling is expensive, time consuming, and error prone. We aim to develop a data set of both realworld and synthetic images that are relevant to the nuclear safeguards domain that can be used to support multiple data science research questions. In the process of developing this data, we aim to develop a novel workflow to validate synthetic images using machine learning explainability methods, testing among multiple computer vision algorithms, and iterative synthetic data rendering. We will deliver one million images – both real-world and synthetically rendered – of two types uranium storage and transportation containers with labelled ground truth and associated adversarial examples.

97 MATHEMATICS AND COMPUTING↗

Measuring photometric redshifts for high-redshift radio source surveys

With the advent of deep, all-sky radio surveys, the need for ancillary data to make the most of the new, high-quality radio data from surveys like the Evolutionary Map of the Universe (EMU), GaLactic and Extragalactic All-sky Murchison Widefield Array survey eXtended, Very Large Array Sky Survey, and LOFAR Two-metre Sky Survey is growing rapidly. Radio surveys produce significant numbers of Active Galactic Nuclei (AGNs) and have a significantly higher average redshift when compared with optical and infrared all-sky surveys. Thus, traditional methods of estimating redshift are challenged, with spectroscopic surveys not reaching the redshift depth of radio surveys, and AGNs making it difficult for template fitting methods to accurately model the source. Machine Learning (ML) methods have been used, but efforts have typically been directed towards optically selected samples, or samples at significantly lower redshift than expected from upcoming radio surveys. This work compiles and homogenises a radio-selected dataset from both the northern hemisphere (making use of Sloan Digital Sky Survey optical photometry) and southern hemisphere (making use of Dark Energy Survey optical photometry). We then test commonly used ML algorithms such as k-Nearest Neighbours (kNN), Random Forest, ANNz, and GPz on this monolithic radio-selected sample. We show that kNN has the lowest percentage of catastrophic outliers, providing the best match for the majority of science cases in the EMU survey. We note that the wider redshift range of the combined dataset used allows for estimation of sources up to z = 3 before random scatter begins to dominate. When binning the data into redshift bins and treating the problem as a classification problem, we are able to correctly identify ≈ 76% of the highest redshift sources—sources at redshift z > 2.51 —as being in either the highest bin (z > 2.51) or second highest (z = 2.25).

79 ASTRONOMY AND ASTROPHYSICS↗

DeepCMB: Lensing Reconstruction of the CMB with Deep Neural Networks

Next-generation cosmic microwave background (CMB) experiments will have lower noise and therefore increased sensitivity, enabling improved constraints on fundamental physics parameters such as the sum of neutrino masses and the tensor-to-scalar ratio r. Achieving competitive constraints on these parameters requires high signal-to-noise extraction of the projected gravitational potential from the CMB maps. Standard methods for reconstructing the lensing potential employ the quadratic estimator (QE). However, the QE performs suboptimally at the low noise levels expected in upcoming experiments. Other methods, like maximum likelihood estimators (MLE), are under active development. In this work, we demonstrate reconstruction of the CMB lensing potential with deep convolutional neural networks (CNN) - ie, a ResUNet. The network is trained and tested on simulated data, and otherwise has no physical parametrization related to the physical processes of the CMB and gravitati onal lensing. We show that, over a wide range of angular scales, ResUNets recover the input gravitational potential with a higher signal-to-noise ratio than the QE method, reaching levels comparable to analytic approximations of MLE methods. We demonstrate that the network outputs quantifiably different lensing maps when given input CMB maps generated with different cosmologies. We also show we can use the reconstructed lensing map for cosmological parameter estimation. This application of CNN provides a few innovations at the intersection of cosmology and machine learning. First, while training and regressing on images, we predict a continuous-variable field rather than discrete classes. Second, we are able to establish uncertainty measures for the network output that are analogous to standard methods. We expect this approach to excel in capturing hard-to-model non-Gaussian astrophysical foreground and noise contributions.

79 ASTRONOMY AND ASTROPHYSICS↗

Probing for Artifacts: Detecting Imagenet Model Evasions

While deep learning models have made incredible progress across a variety of machine learning tasks, they remain vulnerable to adversarial examples crafted to fool otherwise trustworthy models. In this work we approach this problem through the lens of a detection framework. We propose a classification network that uses the hidden layer activations of a trained model as inputs to detect adversarial artifacts in an input. We train this classification network simultaneously against multiple adversarial algorithms to create a more robust detector and show higher detection rates than several alternatives. The novelty of our approach is in the scale and scope of probing Imagenet models for adversarial artifacts. In addition, we propose an improvement to feature squeezing, another common adversarial example detection method.

Rounds, Jeremiah↗

Improving scalability of parallel CNN training by adaptively adjusting parameter update frequency

Synchronous SGD with data parallelism, the most popular parallelization strategy for CNN training, suffers from the expensive communication cost of averaging gradients among all workers. The iterative parameter updates of SGD cause frequent communications and it becomes the performance bottleneck. In this paper, we propose a lazy parameter update algorithm that adaptively adjusts the parameter update frequency to address the expensive communication cost issue. Our algorithm accumulates the gradients if the difference of the accumulated gradients and the latest gradients is sufficiently small. Here, the less frequent parameter updates reduce the per-iteration communication cost while maintaining the model accuracy. Our experimental results demonstrate that the lazy update method remarkably improves the scalability while maintaining the model accuracy. For ResNet50 training on ImageNet, the proposed algorithm achieves a significantly higher speedup (739.6 on 2048 Cori KNL nodes) as compared to the vanilla synchronous SGD (276.6) while the model accuracy is almost not affected (<0.2% difference).

97 MATHEMATICS AND COMPUTING↗

Evolution of Induced Fracture Networks Using Machine Learning Correlation Image Analysis

Enhanced geothermal systems (EGS) can enrich the Nation’s energy portfolio by increasing the amount of renewable baseload power available to the electrical grid. Microseismic activity, due to fluid resource injection and production, provides critical insight into the processes that occur within EGS subsurface fracture networks. To more precisely characterize these fracture networks, we leverage recent advances in deep neural network (DNN) image analysis techniques. Using seismic spectrogram data collected during a meso scale EGS field experiment as image inputs, we developed a new approach to correlate microseismic events. Our results indicate that our chosen DNN is able to estimate the correlation value between highly correlated events (i.e., cross-correlation coefficients greater than approximately 0.7), but has difficulty estimating correlation values between poorly correlated events. In physical systems however, the highly correlated microseismic events are the events of interest. Therefore, this data-driven method has shown potential for improving domestic Energy and Resource Security, advancing the current methods available to image subsurface EGS thermal reservoirs to enhance the availability of clean and reliable energy to the national electric grid.

58 GEOSCIENCES↗

Deep Learning Scene Classification Experiments in Automatic Detection of Slums on Planetscope Imagery

Population growth is increasingly happening in slum settlements of the large urban centers in the Global South. The term "slum" encompasses a wide range of communities, located mostly in underserved areas, and often exhibiting distinct structural and functional informalities with a relatively high concentration of marginalized populations. To address the issues confronting slums for effective planning and development, including the realistic estimation of the resident population, identifying them accurately is fundamental. Given the disagreements over a universal definition, diverse characteristic features, and socio-political limitations, global detection of slums is a veritable challenge. In this paper, we present experiments in slum detection using a scene classification algorithm and 3-meter spatial resolution satellite imagery. We train and evaluate the model for slum detection in Mumbai, India for the year 2023 and test the temporal generalization of the trained model on Mumbai in 2020 and 2018. In addition, we explore the pathways toward geographic generalization to Kolkata and Delhi (India). We discuss several limitations in the workflow and model, situate our findings in the existing literature, and suggest improvements and alternatives. With this, we establish baseline methods and experiments as a first step towards developing an image-based global slum detection framework and algorithm. This work adds to the community discussion on methods, data challenges, and open questions related to the detection of slums globally. With this research, we hope to improve our understanding of human settlements, especially in critical areas, improve population estimates, and help measure progress towards the sustainable development goals.

Arndt, Jacob↗

Accelerating template generation in resonant anomaly detection searches with optimal transport

We introduce Resonant Anomaly Detection with Optimal Transport (RAD-OT), a method for generating signal templates in resonant anomaly detection searches. RAD-OT leverages the fact that the samples from the conditional probability density of the target features vary approximately linearly along the optimal transport path connecting the resonant feature. This does not assume that the conditional density itself is linear with the resonant feature, allowing RAD-OT to efficiently capture multimodal relationships, changes in resolution, etc. By solving the optimal transport problem, RAD-OT can quickly build a template by interpolating between the background distributions in two sideband regions. We demonstrate the performance of RAD-OT using the LHC Olympics R&D dataset, where we find comparable sensitivity and improved stability with respect to deep learning-based approaches.

Automation↗