Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,711 records · Page 95

Scalar bounded-from-below conditions from Bayesian active learning

We present a procedure leveraging Bayesian deep active learning to rapidly produce highly accurate approximate bounded-from-below conditions for arbitrary renormalizable scalar potentials, in the form of a neural network which may be saved and exported for use in arbitrary parameter space scans. We explore the performance of our procedure on three different scalar potentials with either highly nontrivial or unknown symbolic bounded-from-below conditions (the most general two-Higgs doublet model, the three-Higgs doublet model, and a version of the Georgi-Machacek model without custodial symmetry). We find that we can produce fast and highly accurate binary classifiers for all three potentials. Furthermore, for the potentials for which no known symbolic necessary and sufficient conditions on boundedness-from-below exist, our classifiers substantially outperform some common approximate analytical methods, such as producing tractable sufficient but not necessary conditions or evaluating boundedness-from-below conditions for scenarios in which only a subset of the theory’s fields achieve vacuum expectation values. Our methodology can be readily adapted to any renormalizable scalar field theory. For the community’s use, we have developed a package, BFBrain, which allows for the rapid implementation of our analysis procedure on user-specified scalar potentials with a high degree of customizability. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

Real-time inference and extrapolation with Time-Conditioned UNet: Applications in hypersonic flows, incompressible flows, and global temperature forecasting

Neural Operators are fast and accurate surrogates for nonlinear mappings between functional spaces within training domains. Extrapolation beyond the training domain remains a grand challenge across all application areas. We present Time-Conditioned UNet (TC-UNet) as an operator learning method to solve time-dependent PDEs continuously in time without any temporal discretization, including in extrapolation scenarios. TC-UNet incorporates the temporal evolution of the PDE into its architecture by combining a parameter conditioning approach with the attention mechanism from the Transformer architecture. After training, TC-UNet makes real-time inferences on an arbitrary temporal grid. We demonstrate its extrapolation capability on a climate problem by estimating the global temperature for several years and also for inviscid hypersonic flow around a double cone. We propose different training strategies involving temporal bundling and sub-sampling. We demonstrate performance improvements for several benchmarks, performing extrapolation for long time intervals and zero-shot super-resolution time.

Deep learning↗

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Variational Neural-Network Ansatz for Continuum Quantum Field Theory

Physicists dating back to Feynman have lamented the difficulties of applying the variational principle to quantum field theories. In nonrelativistic quantum field theories, the challenge is to parametrize and optimize over the infinitely many n-particle wave functions comprising the state’s Fock-space representation. Here we approach this problem by introducing neural-network quantum field states, a deep learning ansatz that enables application of the variational principle to nonrelativistic quantum field theories in the continuum. Our ansatz uses the Deep Sets neural network architecture to simultaneously parametrize all of the n-particle wave functions comprising a quantum field state. We employ our ansatz to approximate ground states of various field theories, including an inhomogeneous system and a system with long-range interactions, thus demonstrating a powerful new tool for probing quantum field theories.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Beam Hardening Artifact Reduction in X-Ray CT Reconstruction of 3D Printed Metal Parts Leveraging Deep Learning and CAD Models

Nondestructive evaluation (NDE) of additively manufactured (AM) parts is important for understanding the impacts of various process parameters and qualifying the built part. X-ray computed tomography (XCT) has played a critical role in rapid NDE and characterization of AM parts. However, XCT of metal AM parts can be challenging because of artifacts produced by standard reconstruction algorithms as a result of a confounding effect called “beam hardening.” Beam hardening artifacts complicate the analysis of XCT images and adversely impact the process of detecting defects, such as pores and cracks, which is key to ensuring the quality of the parts being printed. In this work, we propose a novel framework based on using available computer-aided design (CAD) models for parts to be manufactured, accurate XCT simulations, and a deep-neural network to produce high-quality XCT reconstructions from data that are affected by noise and beam hardening. Using extensive experiments with simulated data sets, we demonstrate that our method can significantly improve the reconstruction quality, thereby enabling better detection of defects compared with the state of the art. We also present promising preliminary results of applying the deep networks trained using CAD models to experimental data obtained from XCT of an AM jet-engine turbine blade.

Ziabari, Amir↗

Volumetric Segmentation via Neural Networks Improves Neutron Crystallography Data Analysis

Crystallography is the powerhouse technique for molecular structure determination, with applications in fields ranging from energy storage to drug design. Accurate structure determination, however, relies partly on determining the precise locations and integrated intensities of Bragg peaks in the resulting data. Here, we describe a method for Bragg peak integration that is accomplished using neural networks. The network is based on a U-Net and identifies peaks in three-dimensional reciprocal space through segmentation, allowing prediction of the full 3D peak shape from noisy data that is commonly difficult to process. The procedure for generating appropriate training sets is detailed. Trained networks achieve Dice coefficients of 0.82 and mean IoUs of 0.69. Carrying out integration over entire datasets, it is demonstrated that integrating neural network-predicted peaks results in improved intensity statistics. Furthermore, using a second dataset, the possibility of transfer learning between datasets is shown. Given the ubiquity and growing complexity of crystallography, we anticipate integration by machine learning to play an increasingly important role across the physical sciences. These early results demonstrate the applicability of deep learning techniques for integrating crystallography data and suggest a possible role in the next generation of crystallography experiments.

Sullivan, Brendan T.↗

Deep Generative Models in Energy System Applications: Review, Challenges, and Future Directions

In recent years, with the advent of mature machine learning products like ChatGPT, Stable Diffusion, and Sora, the world has witnessed tremendous changes driven by the rapid development of generative artificial intelligence (GAI). Beyond applications in text, speech, image, and video creation, deep generative models (DGMs) underpinning these cutting-edge technologies have also been employed by domain researchers to address scientific and engineering challenges. This paper aims to fill a gap in the research community by providing a systematic review of how DGMs have been utilized in energy system applications. After introducing four most popular DGMs, we review and categorize 196 research articles into five focus areas: data generation, forecasting, situational awareness, modeling, and optimal decision-making. Through this classification, we uncover trends in how DGMs are employed for each type of problem, highlighting GAI techniques that contribute to breakthroughs over traditional methods. We discuss limitations in existing literature, engineering challenges, and propose future directions, all tailored to the unique nature of problems in energy system engineering. Our goal is to offer insights for energy system domain researchers, providing a comprehensive view of existing studies and potential future opportunities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Scalable Deep-Learning-Accelerated Topology Optimization for Additively Manufactured Materials

Topology optimization (TO) is a popular and powerful computational approach for designing novel structures, materials, and devices. Two computational challenges have limited the applicability of TO to a variety of industrial applications. First, a TO problem often involves a large number of design variables to guarantee sufficient expressive power. Second, many TO problems require a large number of expensive physical model simulations, and those simulations cannot be parallelized. To address these issues, we propose a general scalable deep-learning (DL) based TO framework, referred to as SDL-TO, which utilizes parallel schemes in high performance computing (HPC) to accelerate the TO process for designing additively manufactured (AM) materials. Unlike the existing studies of DL for TO, our framework accelerates TO by learning the iterative history data and simultaneously training on the mapping between the given design and its gradient. The surrogate gradient is learned by utilizing parallel computing on multiple CPUs incorporated with a distributed DL training on multiple GPUs. The learned TO gradient enables a fast online update scheme instead of an expensive update based on the physical simulator or solver. Using a local sampling strategy, we achieve to reduce the intrinsic high dimensionality of the design space and improve the training accuracy and the scalability of the SDL-TO framework. The method is demonstrated by benchmark examples and AM materials design for heat conduction. The proposed SDL-TO framework shows competitive performance compared to the baseline methods but significantly reduces the computational cost by a speed up of around 8.6x over the standard TO implementation.

Bi, Sirui↗

Mark 4A project training evaluation

A participant evaluation of a Deep Space Network (DSN) is described. The Mark IVA project is an implementation to upgrade the tracking and data acquisition systems of the dSN. Approximately six hundred DSN operations and engineering maintenance personnel were surveyed. The survey obtained a convenience sample including trained people within the population in order to learn what training had taken place and to what effect. The survey questionnaire used modifications of standard rating scales to evaluate over one hundred items in four training dimensions. The scope of the evaluation included Mark IVA vendor training, a systems familiarization training seminar, engineering training classes, a on-the-job training. Measures of central tendency were made from participant rating responses. Chi square tests of statistical significance were performed on the data. The evaluation results indicated that the effects of different Mark INA training methods could be measured according to certain ratings of technical training effectiveness, and that the Mark IVA technical training has exhibited positive effects on the abilities of DSN personnel to operate and maintain new Mark IVA equipment systems.

Stephenson, S. N.↗

DDxNet: a deep learning model for automatic interpretation of electronic health records, electrocardiograms and electroencephalograms

Effective patient care mandates rapid, yet accurate, diagnosis. With the abundance of non-invasive diagnostic measurements and electronic health records (EHR), manual interpretation for differential diagnosis has become time-consuming and challenging. This has led to wide-spread adoption of AI-powered tools, in pursuit of improving accuracy and efficiency of this process. While the unique challenges presented by each modality and clinical task demand customized tools, the cumbersome process of making problem-specific choices has triggered the critical need for a generic solution to enable rapid development of models in practice. In this spirit, we develop DDxNet, a deep architecture for time-varying clinical data, which we demonstrate to be well-suited for diagnostic tasks involving different modalities (ECG/EEG/EHR), required level of characterization (abnormality detection/phenotyping) and data fidelity (single-lead ECG/22-channel EEG). Using multiple benchmark problems, we show that DDxNet produces high-fidelity predictive models, and sometimes even provides significant performance gains over problem-specific solutions.

60 APPLIED LIFE SCIENCES↗

Uncertainty Aware Deep Learning for Fault Prediction Using Multivariate Time Series Signals

The superconducting radio-frequency cavities are a crucial component of the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. When a cavity faults, beam delivery to experimental end users is disrupted. Prediction of cavity faults prior to onset is essential to reduce operation and maintenance costs. In this work, a parallel long short-term memory (LSTM)-convolution neural network (CNN)-based deep learning (DL) model is proposed to predict impending faults using pre-fault signals. Further, we introduce an uncertainty quantification approach using Monte Carlo dropout with the LSTM-CNN model to ascertain confidence in the prediction. The model was tested using multivariate time series signals from stable cavity operations and before faults. Initial results show that on the test dataset, the model can identify impending faults before their onset with an average 10-fold cross validation accuracy of 97.39% and a standard deviation of 0.12% using a 100-ms time window. It is also observed that the model performs better as the prediction time moves closer to the fault onset. For additional context, we compare the performance of the model with three machine-learning-based (ML) fault prediction models. Our proposed parallel LSTM-CNN-based DL method shows better performance than the ML-based methods.

Rahman, Md Monibor↗

Learning Optimal Aerodynamic Designs

This project created a framework for efficient, accurate, and scalable deep neural network representations of design optimization problem solutions. The inputs to these DNN representations are the vector of design requirement parameters, the outputs are the optimal design variables, and the goal is to learn the map from inputs to outputs (i.e., inverse design). The team addressed the problem of the optimal shape design of aerodynamic lifting surfaces—in particular aircraft wings—using a Reynolds-Average Navier Stokes model to govern the CFD-based aerodynamic shape optimization. The inverse design map for such problems is very complex and high-dimensional, involving inputs and outputs on the order of 1000s. To approximate this inverse design map, the team developed algorithms to construct parsimonious DNN architectures, which automatically identify low-dimensional manifolds in which design requirements affect optimal shape parameters, and trained these architectures with multifidelity optimization methods. The resulting methodology accurately and automatically designs optimal aerodynamic lifting surfaces with very high accuracy (99%) at interactive speeds, of the order of milliseconds, resulting in factors of one million or more speedup relative to CFD-based design optimization.

97 MATHEMATICS AND COMPUTING↗

The role of international collaboration in the United States geologic disposal research program

Abstract. More than a decade ago, the United States disposal program discontinued all research activities focused on the unsaturated fractured tuff formation at Yucca Mountain as the geologic disposal site for spent fuel and high-level radioactive waste. A new research and development (R&D) program was initiated to provide a sound technical basis for alternative disposal options across clay, crystalline, and salt rocks. The goals of this broad program were (and still are) to (1) increase confidence in the robustness of generic disposal concepts, (2) develop the science and engineering tools needed to support disposal concept implementation, and (3) conduct R&D on the direct disposal of existing dual-purpose (storage and transportation) canisters. Recognizing the benefits of international collaboration toward the common goal of safely and efficiently managing the back end of the nuclear fuel cycle, the program emphasized international cooperation as an effective strategy for sharing information and knowledge. In a multi-laboratory effort coordinated by Lawrence Berkeley National Laboratory, the United States Department of Energy (DOE) program established formal and informal cooperation partnerships with several international initiatives and institutions and developed a number of collaborative R&D activities in important research areas, such as engineered barrier integrity, near-field perturbations, radionuclide transport, performance assessment, and methods for characterization and monitoring of engineered and natural barriers. This presentation gives an overview of these R&D activities, with a specific focus on activities that improve our current understanding of the coupled thermal–hydrological–mechanical–chemical (THMC) processes occurring in engineered and natural barriers. We start with a brief review of selected international collaboration initiatives and then describe a few specific collaboration projects. We focus specifically on such studies that use experimental data sets provided by international research cooperation for joint modeling work to increase confidence in performance-relevant predictions of coupled processes. Overall, the focus on international collaboration has allowed deep engagement of US researchers with the international waste management R&D community in terms of best practices, new scientific advances, state-of-the-art simulation tools, new monitoring and performance confirmation approaches, and lessons learned. The joint R&D with international researchers, worldwide sharing of knowledge and experience, and access to relevant data and experiments from a variety of host rocks have helped our researchers to significantly improve their understanding of the current technical basis for disposal in a range of potential host rock environments. International collaboration also provides ample opportunity for training and educating junior staff that are well suited to move the United States disposal research program forward into the next decades, a promising avenue for developing a next-generation workforce of disposal scientists.

Birkholzer, Jens T.↗

Modeling the Solvation and Acidity of Carboxylic Acids Using an Ab Initio Deep Neural Network Potential

Formic and acetic acid constitute the simplest of carboxylic acids, yet they exhibit fascinating chemistry in the condensed phase such as proton transfer and dimerization. The go-to method of choice for modeling these rare events have been accurate but expensive ab-initio molecular dynamics simulations. Here, we present a deep neural network potential trained using accurate ab-initio data that can be used in tandem with enhanced-sampling methods to perform an efficient exploration of the free-energy surface of aqueous solutions of weak carboxylic acids. In particular, we show that our model captures proton dissociation and provides a good estimate of the pK a , as well as the dimerization of formic and acetic acid. This provides a suitable starting point for applications in different research areas where computational efficiency coupled with the accuracy of ab-initio methods is required.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Automatic detection of cataclysmic variables from SDSS images

Abstract Investigating rare and new objects have always been an important direction in astronomy. Cataclysmic variables (CVs) are ideal and natural celestial bodies for studying the accretion process of semi-detached binaries with accretion processes. However, the sample size of CVs must increase because a lager gap exists between the observational and the theoretical expanding CVs. Astronomy has entered the big data era and can provide massive images containing CV candidates. CVs as a type of faint celestial objects, are highly challenging to be identified directly from images using automatic manners. Deep learning has rapidly developed in intelligent image processing and has been widely applied in some astronomical fields with excellent detection results. YOLOX, as the latest YOLO framework, is advantageous in detecting small and dark targets. This work proposes an improved YOLOX-based framework according to the characteristics of CVs and Sloan Digital Sky Survey (SDSS) photometric images to train and verify the model to realise CV detection. We use the Convolutional Block Attention Module to increase the number of output features with the feature extraction network and adjust the feature fusion network to obtain fused features. Accordingly, the loss function is modified. Experimental results demonstrate that the improved model produces satisfactory results, with average accuracy (mean average Precision at 0.5) of 92.0%, Precision of 92.9%, Recall of 94.3%, and $F1-score$ of 93.6% on the test set. The proposed method can efficiently achieve the identification of CVs in test samples and search for CV candidates in unlabeled images. The image data vastly outnumber the spectra in the SDSS-released data. With supplementary follow-up observations or spectra, the proposed model can help astronomers in seeking and detecting CVs in a new manner to ensure that a more extensive CV catalog can be built. The proposed model may also be applied to the detection of other kinds of celestial objects.

Astronomy & Astrophysics↗

MDLoader: A Hybrid Model-Driven Data Loader for Distributed Graph Neural Network Training

Scalable data management is essential for processing large scientific dataset on HPC platforms for distributed deep learning. In-memory distributed storage is preferred for its speed, enabling rapid, random, and frequent data access required by stochastic optimizers. Processes use one-sided or collective communication to fetch remote data, with optimal performance depending on (i) dataset characteristics, (ii) training scale, and (iii) interconnection network. Empirical analysis shows collective communication excels with larger mini-batch sizes and/or fewer processes, whereas one-sided communication outperforms at larger scales. We propose MDLoader, a hybrid in-memory data loader for distributed graph neural network training. MDLoader features a model-driven performance estimator that dynamically selects between one-sided and collective communication at the beginning of training using Tree of Parzen Estimators (TPE). Evaluations on NERSC Perlmutter and OLCF Summit show MDLoader outperforms single-backend loaders by up to 2.83 × and predicts the suitable communication method with 96.3% (Perlmutter) and 94.3% (Summit) success rate.

Bae, Jonghyun↗

$\mathrm{SageNet}$: Fast Neural Network Emulation of the Stiff-amplified Gravitational Waves from Inflation

Accurate modeling of the inflationary gravitational waves (GWs) requires time-consuming, iterative numerical integrations of differential equations to take into account their backreaction on the expansion history. To improve computational efficiency while preserving accuracy, we present the Stiff-amplified Gravitational-wave Emulator Network (SageNet), a deep learning framework designed to replace conventional numerical solvers (code available at https://github.com/YifangLuo/SageNet). SageNet employs a long short-term memory architecture to emulate the present-day energy density spectrum of the inflationary GWs with possible stiff amplification, Ω GW (f). Trained on a data set of 25,689 numerically generated solutions, SageNet allows accurate reconstructions of Ω GW (f) and generalizes well to a wide range of cosmological parameters; 90.9% of the test emulations with randomly distributed parameters exhibit errors of under 4%. In addition, SageNet demonstrates its ability to learn and reproduce the artificial, adaptive sampling patterns in numerical calculations, which implement denser sampling of frequencies around changes in spectral indices in Ω GW (f). The dual capability of learning both physical and artificial features of the numerical GW spectra establishes SageNet as a robust alternative to exact numerical methods. Finally, our benchmark tests show that SageNet reduces the computation time from tens of seconds to milliseconds, achieving a speedup of ∼10 4 times over standard CPU-based numerical solvers with the potential for further acceleration on GPU hardware. These capabilities make SageNet a powerful tool for accelerating Bayesian inference procedures for extended cosmological models. In a broad sense, the SageNet framework offers a fast, accurate, and generalizable solution to modeling cosmological observables whose theoretical predictions demand costly differential equation solvers.

Astronomy data modeling↗

Pile-up mitigation using attention

Abstract Particle production from secondary proton-proton collisions, commonly referred to as pile-up, impair the sensitivity of both new physics searches and precision measurements at large hadron collider (LHC) experiments. We propose a novel algorithm, Puma , for modeling pile-up with the help of deep neural networks based on sparse transformers. These attention mechanisms were developed for natural language processing but have become popular in other applications. In a realistic detector simulation, our method outperforms classical benchmark algorithms for pile-up mitigation in key observables. It provides a perspective for mitigating the effects of pile-up in the high luminosity era of the LHC, where up to 200 proton-proton collisions are expected to occur simultaneously.

47 OTHER INSTRUMENTATION↗