Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,567 records · Page 87

Learning the structure of wind: A data-driven nonlocal turbulence model for the atmospheric boundary layer

In this work, we develop a novel data-driven approach to modeling the atmospheric boundary layer. This approach leads to a nonlocal, anisotropic synthetic turbulence model which we refer to as the deep rapid distortion (DRD) model. Our approach relies on an operator regression problem that characterizes the best fitting candidate in a general family of nonlocal covariance kernels parameterized in part by a neural network. This family of covariance kernels is expressed in Fourier space and is obtained from approximate solutions to the Navier–Stokes equations at very high Reynolds numbers. Each member of the family incorporates important physical properties such as mass conservation and a realistic energy cascade. The DRD model can be calibrated with noisy data from field experiments. After calibration, the model can be used to generate synthetic turbulent velocity fields. To this end, we provide a new numerical method based on domain decomposition which delivers scalable, memory-efficient turbulence generation with the DRD model as well as others. We demonstrate the robustness of our approach with both filtered and noisy data coming from the 1968 Air Force Cambridge Research Laboratory Kansas experiments. Using these data, we witness exceptional accuracy with the DRD model, especially when compared to the International Electrotechnical Commission standard.

17 WIND ENERGY↗

Exploration with Scalable Gaussian Process Reinforcement Learning

Exploration is a challenging problem in reinforcement learning (RL), especially in environments with sparse rewards. Quantifying and utilizing the parametric uncertainty has been shown to be paramount for successful exploration [Osband et al., 2018]. Bayesian, or approximately Bayesian, methods present a principled means of estimating the parametric uncertainty in RL problems. Gaussian processes, nonparametric Bayesian models, are often impractical due to poor scalability and computational bottlenecks. We introduce a scalable Gaussian process RL (GPRL) method which directly induces sparsity in the covariance matrix to facilitate faster computation. This is a departure from previous GPRL methods which instead rely on data reduction and subsampling. We compare various covariance-based exploration techniques (Thompson sampling, upper confidence bound, and probabilistic maximum variance) which leverage our scalable GP framework in sparse reward environments. Finally, we show favorable comparison against the bootstrapped deep Q-Network.

97 MATHEMATICS AND COMPUTING↗

Computer-aided Abnormality Detection in Chest Radiographs in a Clinical Setting via Domain-adaptation

Deep learning (DL) models are being deployed at medical centers to aid radiologists for diagnosis of lung conditions from chest radiographs. Such models are often trained on a large volume of publicly available labeled radiographs. These pre-trained DL models’ ability to generalize in clinical settings is poor because of the changes in data distributions between publicly available and privately held radiographs. In chest radiographs, the heterogeneity in distributions arises from the diverse conditions in X-ray equipment and their configurations used for generating the images. In the machine learning community, the challenges posed by the heterogeneity in the data generation source is known as domain shift, which is a mode shift in the generative model. In this work, we introduce a domain-shift detection and removal method to overcome this problem. Our experimental results show the proposed method’s effectiveness in deploying a pre-trained DL model for abnormality detection in chest radiographs in a clinical setting.

Dubey, Abhishek↗

Anderson Acceleration for Distributed Training of Deep Learning Models

Anderson acceleration (AA) is an extrapolation technique that has recently gained interest in the deep learning (DL) community to speed-up the sequential training of DL models. However, when performed at large scale, the DL training is exposed to a higher risk of getting trapped into steep local minima of the training loss function, and standard AA does not provide sufficient acceleration to escape from these steep local minima. This results in poor generalizability and makes AA ineffective. To restore AA’s advantage to speed-up the training of DL models on large scale computing platforms, we combine AA with an adaptive moving average procedure that boosts the training to escape from steep local minima. By monitoring the relative standard deviation between consecutive iterations, we also introduce a criterion to automatically assess whether the moving average is needed. We applied the method to the following DL instantiations for image classification: (i) ResNet50 trained on the open-source CIFAR100 dataset and (ii) ResNet50 trained on the open-source ImageNet1k dataset. Numerical results obtained using up to 1,536 NVIDIA V100 GPUs on the OLCF supercomputer Summit showed the stabilizing effect of the moving average on AA for all the problems above.

Lupo Pasini, Massimiliano↗

The deep-DRT: A deep neural network approach to deconvolve the distribution of relaxation times from multidimensional electrochemical impedance spectroscopy data

Electrochemical impedance spectroscopy (EIS) is an experimental technique ubiquitously used to study electrochemical systems. However, conventional EIS data interpretation through physical and equivalent circuit models is challenging because physical models are problem-specific, and equivalent circuits are often just lumped-element analogs lacking physical meaning. The distribution of relaxation times (DRT) has emerged as a complementary approach to resolve these issues. One drawback of conventional DRT deconvolution is that the EIS data is understood to be (only) a function of frequency (i.e. 1D data) and deconvolved accordingly. This work proposes a novel deconvolution method based on deep neural networks (DNNs), allowing the analysis of multidimensional EIS spectra to bridge data dependency on both frequencies and experimental conditions. Two particularly appealing traits of the deep-DRT method developed in this article are that neither regularization nor specific spacing on the state variables defining the experiment are required. Finally, leveraging DNN to examine complex EIS spectra and their dependence on experimental conditions, this work opens a new research direction in the area of EIS analysis and DRT deconvolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DCTRGAN: improving the precision of generative models with reweighting

Significant advances in deep learning have led to more widely used and precise neural network-based generative models such as Generative Adversarial Networks (Gans). We introduce a post-hoc correction to deep generative models to further improve their fidelity, based on the Deep neural networks using the Classification for Tuning and Reweighting (Dctr) protocol. The correction takes the form of a reweighting function that can be applied to generated examples when making predictions from the simulation. We illustrate this approach using Gans trained on standard multimodal probability densities as well as calorimeter simulations from high energy physics. We show that the weighted Gan examples significantly improve the accuracy of the generated samples without a large loss in statistical power. This approach could be applied to any generative model and is a promising refinement method for high energy physics applications and beyond.

47 OTHER INSTRUMENTATION↗

Machine Learning Assisted HPC Workload Trace Generation for Leadership Scale Storage Systems

Monitoring and analyzing a wide range of I/O activities in an HPC cluster is important in maintaining mission-critical performance in a large-scale, multi-user, parallel storage system. Center-wide I/O traces can provide high-level information and fine-grained activities per application or per user running in the system. Studying such large-scale traces can provide helpful insights into the system. It can be used to develop predictive methods for making predictive decisions, adjusting scheduling policies, or providing decisions for the design of next-generation systems. However, sharing real-world I/O traces to expedite such research efforts leaves a few concerns; i) the cost of sharing the large traces is expensive due to this large size, and ii) privacy concern is an issue.We address such issues by building an end-to-end machine learn- ing (ML) workflow that can generate I/O traces for large-scale HPC applications. We leverage ML based feature selection and gener- ative models for I/O trace generation. The generative models are trained on I/O traces collected by the darshan I/O characterization tool over a period of one year. We present a two-step generation process consisting of two deep-learning models, called the feature generator and the trace generator. The combination of two-step generative models provides robustness by reducing the bias of the model and accounting for the stochastic nature of the I/O traces across different runs of an application. We evaluate the performance of the generative models and show that the two-step model can generate time-series I/O traces with less than 20% root mean square error.

Paul, Arnab↗

Coupling Visualization, Simulation, and Deep Learning for Ensemble Steering of Complex Energy Models: Preprint

We have developed a framework for the exploration, design, and planning of energy systems that combines interactive visualization with machine-learning based approximations of simulations through a general purpose dataflow API. Our system provides a visual inter- face allowing users to explore an ensemble of energy simulations representing a subset of the complex input parameter space, and spawn new simulations to 'fill in' input regions corresponding to new enegery system scenarios. Unfortunately, many energy simula- tions are far too slow to provide interactive responses. To support interactive feedback, we are developing reduced-form models via machine learning techniques, which provide statistically sound esti- mates of the full simulations at a fraction of the computational cost and which are used as proxies for the full-form models. Fast com- putation and an agile dataflow enhance the engagement with energy simulations, and allow researchers to better allocate computational resources to capture informative relationships within the system and provide a low-cost method for validating and quality-checking large-scale modeling efforts.

97 MATHEMATICS AND COMPUTING↗

Deep learning of structural morphology imaged by scanning X-ray diffraction microscopy

Scanning X-ray nanodiffraction microscopy is a powerful technique for spatially resolving nanoscale structural morphologies by diffraction contrast. One of the critical challenges in experimental nanodiffraction data analysis is posed by the convergence angle of nanoscale focusing optics which creates simultaneous dependency of the far-field scattering data on three independent components of the local strain tensor-corresponding to dilation and two potential rigid body rotations of the unit cell. All three components are in principle resolvable through a spatially mapped sample tilt series; however, traditional data analysis is computationally expensive and prone to artifacts. In this study, we implement NanobeamNN, a convolutional neural network specifically tailored to the analysis of scanning probe X-ray microscopy data. NanobeamNN learns lattice strain and rotation angles from simulated diffraction of a focused X-ray nanobeam by an epitaxial thin film and can directly make reasonable predictions on experimental data without the need for additional fine-tuning. We demonstrate that this approach represents a significant advancement in computational speed over conventional methods, as well as a potential improvement in accuracy over the current standard.

Luo, Aileen [Cornell Univ., Ithaca, NY (United Sta↗

Advanced Seismic While Drilling System

A breakthrough has been discovered for controlling seismic sources to generate selectable low frequencies. Conventional seismic sources, including sparkers, rotary mechanical, hydraulic, air guns, and explosives, by their very nature produce high-frequencies. This is counter to the need for long signal transmission through rock. The patent pending SeismicPULSER{trademark} methodology has been developed for controlling otherwise high-frequency seismic sources to generate selectable low-frequency peak spectra applicable to many seismic applications. Specifically, we have demonstrated the application of a low-frequency sparker source which can be incorporated into a drill bit for Drill Bit Seismic While Drilling (SWD). To create the methodology of a controllable low-frequency sparker seismic source, it was necessary to learn how to maximize sparker efficiencies to couple to, and transmit through, rock with the study of sparker designs and mechanisms for (a) coupling the sparker-generated gas bubble expansion and contraction to the rock, (b) the effects of fluid properties and dynamics, (c) linear and non-linear acoustics, and (d) imparted force directionality. After extensive seismic modeling, the design of high-efficiency sparkers, laboratory high frequency sparker testing, and field tests were performed at the University of Texas Devine seismic test site. The conclusion of the field test was that extremely high power levels would be required to have the range required for deep, 15,000+ ft, high-temperature, high-pressure (HTHP) wells. Thereafter, more modeling and laboratory testing led to the discovery of a method to control a sparker that could generate low frequencies required for deep wells. The low frequency sparker was successfully tested at the Department of Energy Rocky Mountain Oilfield Test Center (DOE RMOTC) field test site in Casper, Wyoming. An 8-in diameter by 26-ft long SeismicPULSER{trademark} drill string tool was designed and manufactured by TII. An APS Turbine Alternator powered the SeismicPULSER{trademark} to produce two Hz frequency peak signals repeated every 20 seconds. Since the ION Geophysical, Inc. (ION) seismic survey surface recording system was designed to detect a minimum downhole signal of three Hz, successful performance was confirmed with a 5.3 Hz recording with the pumps running. The two Hz signal generated by the sparker was modulated with the 3.3 Hz signal produced by the mud pumps to create an intense 5.3 Hz peak frequency signal. The low frequency sparker source is ultimately capable of generating selectable peak frequencies of 1 to 40 Hz with high-frequency spectra content to 10 kHz. The lower frequencies and, perhaps, low-frequency sweeps, are needed to achieve sufficient range and resolution for realtime imaging in deep (15,000 ft+), high-temperature (150 C) wells for (a) geosteering, (b) accurate seismic hole depth, (c) accurate pore pressure determinations ahead of the bit, (d) near wellbore diagnostics with a downhole receiver and wired drill pipe, and (e) reservoir model verification. Furthermore, the pressure of the sparker bubble will disintegrate rock resulting in an increased overall rates of penetration. Other applications for the SeismicPULSER{trademark} technology are to deploy a low-frequency source for greater range on a wireline for Reverse Vertical Seismic Profiling (RVSP) and Cross-Well Tomography. Commercialization of the technology is being undertaken by first contacting stakeholders to define the value proposition for rig site services utilizing SeismicPULSER{trademark} technologies. Stakeholders include national oil companies, independent oil companies, independents, service companies, and commercial investors. Service companies will introduce a new Drill Bit SWD service for deep HTHP wells. Collaboration will be encouraged between stakeholders in the form of joint industry projects to develop prototype tools and initial field trials. No barriers have been identified for developing, utilizing, and exploiting the low-frequency SeismicPULSER{trademark} source in a variety of applications. Risks will be minimized since Drill Bit SWD will not interfere with the drilling operation, and can be performed in a relatively quiet environment when the pumps are turned off. The new source must be integrated with other Measurement While Drilling (MWD) tools. To date, each of the oil companies and service companies contacted have shown interest in participating in the commercialization of the low-frequency SeismicPULSER{trademark} source. A technical paper has been accepted for presentation at the 2009 Offshore Technology Conference (OTC) in a Society of Exploration Geologists/American Association of Petroleum Geophysicists (SEG/AAPG) technical session.

02 PETROLEUM↗

A Hybrid Reinforcement Learning-MPC Approach for Distribution System Critical Load Restoration: Preprint

This paper proposes a hybrid control approach for distribution system critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while MPC models grid operations incorporating RL policy actions, i.e., the reserve requirement, renewable (wind and solar) power predictions, and load demand forecasts. We formulate the reserve requirement determination problem as a sequential decision making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The RL algorithm is trained off-line using historical forecast of renewable generation and load demand. The method is tested using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine and battery. Case studies demonstrated that the proposed method outperforms other operating reserve determination methods.

distribution system↗

A Hybrid Reinforcement Learning-MPC Approach for Distribution System Critical Load Restoration

This paper proposes a hybrid control approach for distribution system critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while MPC models grid operations incorporating RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. We formulate the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The RL algorithm is trained offline using a historical forecast of renewable generation and load demand. The method is tested using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery. Case studies demonstrated that the proposed method outperforms other policies with static operating reserves.

distribution system↗

Deep Data Analytics in Structural and Functional Imaging of Nanoscale Materials

Recent advances in scanning probe microscopy and scanning transmission electron microscopy have opened unprecedented opportunities in probing the materials structural parameters and electronic properties in real space on a picometre-scale. At the same time, the ability of modern day microscopes to quickly produce large, high-resolution datasets has created a challenge for rapid physics-guided analysis of data that typically contain several hundreds to several thousand atomic or molecular units per image. Here it is demonstrated how the advanced statistical analysis and machine learning techniques can be used for extracting relevant physical and chemical information from microscope data on multiple functional materials. Specifically, the following three case studies are discussed (i) application of a combination of convolutional neural network and Markov model for analyzing positional and orientational order in molecular self-assembly; (ii) a combination of sliding window fast Fourier transform, Pearson correlation matrix and canonical correlation analysis methods to study the relationships between lattice distortions and electron scattering patterns in graphene; (iii) application of a non-negative matrix factorization with physics-based constraints and Moran’s analysis of spatial associations to extracting electronic responses linked to different types of structural domains from multi-modal imaging datasets on iron-based superconductors. The approaches demonstrated here are universal in nature and can be applied to a variety of microscopic measurements on different materials.

Ziatdinov, Maxim↗

Scalar bounded-from-below conditions from Bayesian active learning

We present a procedure leveraging Bayesian deep active learning to rapidly produce highly accurate approximate bounded-from-below conditions for arbitrary renormalizable scalar potentials, in the form of a neural network which may be saved and exported for use in arbitrary parameter space scans. We explore the performance of our procedure on three different scalar potentials with either highly nontrivial or unknown symbolic bounded-from-below conditions (the most general two-Higgs doublet model, the three-Higgs doublet model, and a version of the Georgi-Machacek model without custodial symmetry). We find that we can produce fast and highly accurate binary classifiers for all three potentials. Furthermore, for the potentials for which no known symbolic necessary and sufficient conditions on boundedness-from-below exist, our classifiers substantially outperform some common approximate analytical methods, such as producing tractable sufficient but not necessary conditions or evaluating boundedness-from-below conditions for scenarios in which only a subset of the theory’s fields achieve vacuum expectation values. Our methodology can be readily adapted to any renormalizable scalar field theory. For the community’s use, we have developed a package, BFBrain, which allows for the rapid implementation of our analysis procedure on user-specified scalar potentials with a high degree of customizability. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

Real-time inference and extrapolation with Time-Conditioned UNet: Applications in hypersonic flows, incompressible flows, and global temperature forecasting

Neural Operators are fast and accurate surrogates for nonlinear mappings between functional spaces within training domains. Extrapolation beyond the training domain remains a grand challenge across all application areas. We present Time-Conditioned UNet (TC-UNet) as an operator learning method to solve time-dependent PDEs continuously in time without any temporal discretization, including in extrapolation scenarios. TC-UNet incorporates the temporal evolution of the PDE into its architecture by combining a parameter conditioning approach with the attention mechanism from the Transformer architecture. After training, TC-UNet makes real-time inferences on an arbitrary temporal grid. We demonstrate its extrapolation capability on a climate problem by estimating the global temperature for several years and also for inviscid hypersonic flow around a double cone. We propose different training strategies involving temporal bundling and sub-sampling. We demonstrate performance improvements for several benchmarks, performing extrapolation for long time intervals and zero-shot super-resolution time.

Deep learning↗

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Variational Neural-Network Ansatz for Continuum Quantum Field Theory

Physicists dating back to Feynman have lamented the difficulties of applying the variational principle to quantum field theories. In nonrelativistic quantum field theories, the challenge is to parametrize and optimize over the infinitely many n-particle wave functions comprising the state’s Fock-space representation. Here we approach this problem by introducing neural-network quantum field states, a deep learning ansatz that enables application of the variational principle to nonrelativistic quantum field theories in the continuum. Our ansatz uses the Deep Sets neural network architecture to simultaneously parametrize all of the n-particle wave functions comprising a quantum field state. We employ our ansatz to approximate ground states of various field theories, including an inhomogeneous system and a system with long-range interactions, thus demonstrating a powerful new tool for probing quantum field theories.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Beam Hardening Artifact Reduction in X-Ray CT Reconstruction of 3D Printed Metal Parts Leveraging Deep Learning and CAD Models

Nondestructive evaluation (NDE) of additively manufactured (AM) parts is important for understanding the impacts of various process parameters and qualifying the built part. X-ray computed tomography (XCT) has played a critical role in rapid NDE and characterization of AM parts. However, XCT of metal AM parts can be challenging because of artifacts produced by standard reconstruction algorithms as a result of a confounding effect called “beam hardening.” Beam hardening artifacts complicate the analysis of XCT images and adversely impact the process of detecting defects, such as pores and cracks, which is key to ensuring the quality of the parts being printed. In this work, we propose a novel framework based on using available computer-aided design (CAD) models for parts to be manufactured, accurate XCT simulations, and a deep-neural network to produce high-quality XCT reconstructions from data that are affected by noise and beam hardening. Using extensive experiments with simulated data sets, we demonstrate that our method can significantly improve the reconstruction quality, thereby enabling better detection of defects compared with the state of the art. We also present promising preliminary results of applying the deep networks trained using CAD models to experimental data obtained from XCT of an AM jet-engine turbine blade.

Ziabari, Amir↗