Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep neural network (DNN)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Freely scalable and reconfigurable optical hardware for deep learning

Abstract As deep neural network (DNN) models grow ever-larger, they can achieve higher accuracy and solve more complex problems. This trend has been enabled by an increase in available compute power; however, efforts to continue to scale electronic processors are impeded by the costs of communication, thermal management, power delivery and clocking. To improve scalability, we propose a digital optical neural network (DONN) with intralayer optical interconnects and reconfigurable input values. The path-length-independence of optical energy consumption enables information locality between a transmitter and a large number of arbitrarily arranged receivers, which allows greater flexibility in architecture design to circumvent scaling limitations. In a proof-of-concept experiment, we demonstrate optical multicast in the classification of 500 MNIST images with a 3-layer, fully-connected network. We also analyze the energy consumption of the DONN and find that digital optical data transfer is beneficial over electronics when the spacing of computational units is on the order of $$>10\,\upmu $$ > 10 μ m.

42 ENGINEERING↗
DOE OSTI2020

Development of interatomic potential for Al–Tb alloys using a deep neural network learning method

An interatomic potential for the Al–Tb alloy around the composition of Al 90 Tb 10 is developed using the deep neural network (DNN) learning method. The atomic configurations and the corresponding total potential energies and forces on each atom obtained from ab initio molecular dynamics (AIMD) simulations are collected to train a DNN model to construct the interatomic potential for the Al–Tb alloy. Here we show that the obtained DNN model can well reproduce the energies and forces calculated by AIMD simulations. Molecular dynamics (MD) simulations using the DNN interatomic potential also accurately describe the structural properties of the Al 90 Tb 10 liquid, such as partial pair correlation functions (PPCFs) and bond angle distributions, in comparison with the results from AIMD simulations. Furthermore, the developed DNN interatomic potential predicts the formation energies of the crystalline phases of the Al–Tb system with an accuracy comparable to ab initio calculations. The structure factors of the Al 90 Tb 10 metallic liquid and glass obtained by MD simulations using the developed DNN interatomic potential are also in good agreement with the experimental X-ray diffraction data. The development of short-range order (SRO) in the Al 90 Tb 10 liquid and the undercooled liquid is also analyzed and three dominant SROs, i.e., Al-centered distorted icosahedron (DISICO) and Tb-centered ‘3661’ and ‘15551’ clusters, respectively, are identified.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Machine learning for design principles for single atom catalysts towards electrochemical reactions

Machine learning (ML) integrated density functional theory (DFT) calculations have recently been used to accelerate the design and discovery of heterogeneous catalysts such as single atom catalysts (SACs) through the establishment of deep structure–activity relationships. Here, this review provides recent progress in the ML-aided rational design of heterogeneous catalysts with the focus on SACs in terms of structure–activity relationships, feature importance analysis, high-throughput screening, stability, and metal–support interactions for electrochemistry. Support vector machine (SVM), random forest regression (RFR), and deep neural networks (DNN) along with atomic properties are mainly used for the design of SACs. The ML results have shown that the number of electrons in the d orbital, oxide formation enthalpy, ionization energy, Bader charge, d-band center, and enthalpy of vaporization are mainly the most important parameters for the defining of the structure–activity relationships for electrochemistry. However, the black-box nature of ML techniques occasionally makes a physical interpretation of descriptors, such as the Bader charge, d-band center, and enthalpy of vaporization, non-trivial. At the current stage, ML application is limited by the lack of a large and high-quality database. Future prospects for the development of a large database and a generalized ML algorithm for SAC design are discussed to give insights for further studies in this field.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

From lattice QCD to in-medium heavy-quark interactions via deep learning

Bottomonium states are key probes for experimental studies of the quark-gluon plasma (QGP) created in high-energy nuclear collisions. Theoretical models of bottomonium productions in high-energy nuclear collisions rely on the in-medium interactions between the bottom and antibottom quarks, which can be characterized by real (V R (T, r)) and imaginary (V I (T, r)) potentials, as functions of temperature and spatial separation. Recently, the masses and thermal widths of up to 3S and 2P bottomonium states in QGP were calculated using lattice quantum chromodynamics (LQCD). Starting from these LQCD results and through a novel application of deep neural network (DNN), here, we obtain model-independent results for V R (T, r) and V I (T, r). The temperature dependence of V R (T, r) was found to be very mild between T ≈ 0 - 330 MeV. Meanwhile, VI(T, r) shows rapid increase with T and r, which is much larger than the perturbation theory based expectations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Probing the self-ionization of liquid water with ab initio deep potential molecular dynamics

The chemical equilibrium between self-ionized and molecular water dictates the acid–base chemistry in aqueous solutions, yet understanding the microscopic mechanisms of water self-ionization remains experimentally and computationally challenging. Herein, Density Functional Theory (DFT)–based deep neural network (DNN) potentials are combined with enhanced sampling techniques and a global acid–base collective variable to perform extensive atomistic simulations of water self-ionization for model systems of increasing size. The explicit inclusion of long-range electrostatic interactions in the DNN potential is found to be crucial to accurately reproduce the DFT free energy profile of solvated water ion pairs in small (64 and 128 H 2 O) cells. The reversible work to separate the hydroxide and hydronium to a distance S is found to converge for simulation cells containing more than 500 H 2 O, and a distance of ∼ 8 Å is the threshold beyond which the work to further separate the two ions becomes approximately zero. The slow convergence of the potential of mean force with system size is related to a restructuring of water and an increase of the local order around the water ions. Calculation of the dissociation equilibrium constant illustrates the key role of long-range electrostatics and entropic effects in the water autoionization process.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Measurement of a radial flow profile with eddy current flow meters and deep neural networks

Eddy current flow meters (ECFMs) measure flows of conductive fluids. Recent interest in ECFMs has increased due to applications in advanced nuclear reactors. ECFMs are well suited for such applications, as they can provide non-invasive measurements of flow in fluids that are often difficult to measure. Traditionally, ECFMs are operated using an alternating current at a single frequency, limiting ECFMs to measure average fluid velocities, blockages, or voids. Here, we expand the capabilities of ECFMs by measuring the fluid radial velocity profile of liquid mercury. To accomplish this, we made several ECFM sensitivity measurements at a range of frequencies. Different frequencies vary the electromagnetic skin depth of the device. By adjusting frequencies, we probed the fluid velocity at various radial locations and constructed a flow-velocity profile. The relationship between the ECFM measurements and velocity profile is nonlinear and requires solving an inverse problem. Using electromagnetic finite-element simulations to train a deep neural network (DNN), we created a model that provides a stable general relationship between the sensitivity measurements of an ECFM and the fluid velocity profile. Using ECFM measurements of liquid mercury, our DNN model calculates a flow profile that agrees well with computational fluid dynamics (CFD) simulations. This technique has potential to improve flow monitoring for optimization, safe operation of conductive fluid loops, and/or validating complex CFD models.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Deep-Learning based Reconstruction of the Shower Maximum $X_{\mathrm{max}}$ using the Water-Cherenkov Detectors of the Pierre Auger Observatory

The atmospheric depth of the air shower maximum X max is an observable commonly used for the determination of the nuclear mass composition of ultra-high energy cosmic rays. Direct measurements of X max are performed using observations of the longitudinal shower development with fluorescence telescopes. At the same time, several methods have been proposed for an indirect estimation of X max from the characteristics of the shower particles registered with surface detector arrays. In this paper, we present a deep neural network (DNN) for the estimation of X max . The reconstruction relies on the signals induced by shower particles in the ground based water-Cherenkov detectors of the Pierre Auger Observatory. The network architecture features recurrent long short-term memory layers to process the temporal structure of signals and hexagonal convolutions to exploit the symmetry of the surface detector array. We evaluate the performance of the network using air showers simulated with three different hadronic interaction models. Thereafter, we account for long-term detector effects and calibrate the reconstructed X max using fluorescence measurements. Finally, we show that the event-by-event resolution in the reconstruction of the shower maximum improves with increasing shower energy and reaches less than 25 g/cm 2 at energies above 2 × 10 19 eV.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Distilling particle knowledge for fast reconstruction at high-energy physics experiments

Knowledge distillation is a form of model compression that allows artificial neural networks of different sizes to learn from one another. Its main application is the compactification of large deep neural networks to free up computational resources, in particular on edge devices. In this article, we consider proton-proton collisions at the High-Luminosity Large Hadron Collider (HL-LHC) and demonstrate a successful knowledge transfer from an event-level graph neural network (GNN) to a particle-level small deep neural network (DNN). Our algorithm, DistillNet, is a DNN that is trained to learn about the provenance of particles, as provided by the soft labels that are the GNN outputs, to predict whether or not a particle originates from the primary interaction vertex. The results indicate that for this problem, which is one of the main challenges at the HL-LHC, there is minimal loss during the transfer of knowledge to the small student network, while improving significantly the computational resource needs compared to the teacher. This is demonstrated for the distilled student network on a CPU, as well as for a quantized and pruned student network deployed on a field programmable gate array. Our study proves that knowledge transfer between networks of different complexity can be used for fast artificial intelligence (AI) in high-energy physics that improves the expressiveness of observables over non-AI-based reconstruction algorithms. Such an approach can become essential at the HL-LHC experiments, e.g. to comply with the resource budget of their trigger stages.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Simultaneous energy and mass calibration of large-radius jets with the ATLAS detector using a deep neural network

The energy and mass measurements of jets are crucial tasks for the Large Hadron Collider experiments. This paper presents a new calibration method to simultaneously calibrate these quantities for large-radius jets measured with the ATLAS detector using a deep neural network (DNN). To address the specificities of the calibration problem, special loss functions and training procedures are employed, and a complex network architecture, which includes feature annotation and residual connection layers, is used. The DNN-based calibration is compared to the standard numerical approach in an extensive series of tests. The DNN approach is found to perform significantly better in almost all of the tests and over most of the relevant kinematic phase space. In particular, it consistently improves the energy and mass resolutions, with a 30% better energy resolution obtained for transverse momenta $p$ T > $500$ GeV.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Heat transport in liquid water from first-principles and deep neural network simulations

In this work, we compute the thermal conductivity of water within linear response theory from equilibrium molecular dynamics simulations, by adopting two different approaches. In one, the potential energy surface (PES) is derived on the fly from the electronic ground state of density functional theory (DFT) and the corresponding analytical expression is used for the energy flux. In the other, the PES is represented by a deep neural network (DNN) trained on DFT data, whereby the PES has an explicit local decomposition and the energy flux takes a particularly simple expression. By virtue of a gauge invariance principle, established by Marcolongo, Umari, and Baroni, the two approaches should be equivalent if the PES were reproduced accurately by the DNN model. We test this hypothesis by calculating the thermal conductivity, at the GGA (PBE) level of theory, using the direct formulation and its DNN proxy, finding that both approaches yield the same conductivity, in excess of the experimental value by approximately 60%. Besides being numerically much more efficient than its direct DFT counterpart, the DNN scheme has the advantage of being easily applicable to more sophisticated DFT approximations, such as meta-GGA and hybrid functionals, for which it would be hard to derive analytically the expression of the energy flux. We find in this way that a DNN model, trained on meta-GGA (SCAN) data, reduces the deviation from experiment of the predicted thermal conductivity by about 50%, leaving the question open as to whether the residual error is due to deficiencies of the functional, to a neglect of nuclear quantum effects in the atomic dynamics, or, likely, to a combination of the two.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Deconvoluting experimental decay energy spectra: The O 26 case

In nuclear reaction experiments, the measured decay energy spectra can give insights into the shell structure of decaying systems. However, extracting the underlying physics from the measurements is challenging due to detector resolution and acceptance effects. The Richardson-Lucy (RL) algorithm, a deblurring method that is commonly used in optics and has proven to be a successful technique for restoring images, was applied to our experimental nuclear physics data. The only inputs to the method are the observed energy spectrum and the detector's response matrix also known as the transfer matrix. We demonstrate that the technique can help access information about the shell structure of particle-unbound systems from the measured decay energy spectrum that is not immediately accessible via traditional approaches such as χ-square fitting. For a similar purpose, we developed a machine learning model that uses a deep neural network (DNN) classifier to identify resonance states from the measured decay energy spectrum. We tested the performance of both methods on simulated data and experimental measurements. Then, we applied both algorithms to the decay energy spectrum of 26 O → 24 O + n + n measured via invariant mass spectroscopy. Here, the resonance states restored using the RL algorithm to deblur the measured decay energy spectrum agree with those found by the DNN classifier. Both deblurring and DNN approaches suggest that the raw decay energy spectrum of 26 O exhibits three peaks at approximately 0.15 MeV, 1.50 MeV, and 5.00 MeV, with half-widths of 0.29 MeV, 0.80 MeV, and 1.85 MeV, respectively.

Spectrometers & spectroscopic techniques↗
DOE OSTI2024

Neural network emulation of flow in heavy-ion collisions at intermediate energies

Applications of new techniques in machine learning are speeding up progress in research in various fields. In this work, we construct and evaluate a deep neural network (DNN) to be used within a Bayesian statistical framework as a faster and more reliable alternative to the Gaussian process (GP) emulator of an isospin-dependent Boltzmann-Uehling-Uhlenbeck (IBUU) transport model simulator of heavy-ion reactions at intermediate beam energies. We found strong evidence of the DNN being able to emulate the IBUU simulator's prediction on the strengths of protons' directed and elliptical flow very efficiently even with small training datasets and with accuracy about ten times higher than the GP. Here, limitations of our present work and future improvements are also discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Group-equivariant autoencoder for identifying spontaneously broken symmetries

We introduce the group-equivariant autoencoder (GE autoencoder), a deep neural network (DNN) method that locates phase boundaries by determining which symmetries of the Hamiltonian have spontaneously broken at each temperature. We use group theory to deduce which symmetries of the system remain intact in all phases, and then use this information to constrain the parameters of the GE autoencoder such that the encoder learns an order parameter invariant to these “never-broken” symmetries. This procedure produces a dramatic reduction in the number of free parameters such that the GE-autoencoder size is independent of the system size. We include symmetry regularization terms in the loss function of the GE autoencoder so that the learned order parameter is also equivariant to the remaining symmetries of the system. By examining the group representation by which the learned order parameter transforms, we are then able to extract information about the associated spontaneous symmetry breaking. We test the GE autoencoder on the 2D classical ferromagnetic and antiferromagnetic Ising models, finding that the GE autoencoder (1) accurately determines which symmetries have spontaneously broken at each temperature; (2) estimates the critical temperature in the thermodynamic limit with greater accuracy, robustness, and time efficiency than a symmetry-agnostic baseline autoencoder; and (3) detects the presence of an external symmetry-breaking magnetic field with greater sensitivity than the baseline method. Lastly, we describe various key implementation details, including a quadratic-programming-based method for extracting the critical temperature estimate from trained autoencoders and calculations of the DNN initialization and learning rate settings required for fair model comparisons.

42 ENGINEERING↗
DOE OSTI2023

Why Dissolving Salt in Water Decreases Its Dielectric Permittivity

The dielectric permittivity of salt water decreases on dissolving more salt. For nearly a century, this phenomenon has been explained by invoking saturation in the dielectric response of the solvent water molecules. Herein, we employ an advanced deep neural network (DNN), built using data from density functional theory, to study the dielectric permittivity of sodium chloride solutions. Notably, the decrease in the dielectric permittivity as a function of concentration, computed using the DNN approach, agrees well with experiments. Detailed analysis of the computations reveals that the dominant effect, caused by the intrusion of ionic hydration shells into the solvent hydrogen-bond network, is the disruption of dipolar correlations among water molecules. Accordingly, the observed decrease in the dielectric permittivity is mostly due to increasing suppression of the collective response of solvent waters.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2022

Designing Monte Carlo Simulation and an Optimal Machine Learning to Optimize and Model Space Missions

This paper investigates applying artificial intelligence (AI) algorithms to attitude control system of satellites to optimally tune the controller using high performance computing. This methodology is applied to the Virtual Telescope for X-ray Observation mission, which is a precise formation of two separate spacecraft observing multiple objects in the space in the X-ray domain. The mission is divided into phases based on the instrumentation and the mission goal. To reach an stable precise formation robust to stochastic slew and slew rate (i.e., Euler angles and angular velocities) in a minimal constrained time T , consumed energy of the attitude control system, denoted as E, and root-mean-square state error of attitude control system, denoted as e, are minimized. Monte-Carlo simulation is used for the sensitivity analysis of optimization and designing a controller. Deep neural networks (DNN), Gaussian processes (GP), and support vector regression (SVR) learn this optimization as a surrogate model, while their hyperparameters are optimized in a novel approach. THETA supercomputer at Argonne Leadership Computing Facility (ALCF) is used for optimizing the hyperparameters of DNN. The surrogate model meets the requirements of the mission, and it shows a better performance over the optimization and Monte-Carlo. The optimal DNN can satisfy the mission requirements e and T while reducing E for 90% compared to the other given methods.

42 ENGINEERING↗
DOE OSTI2025

FedCSpc: A Cross-Silo Federated Learning System With Error-Bounded Lossy Parameter Compression

Cross-Silo federated learning is widely used for scaling deep neural network (DNN) training over data silos from different locations worldwide while guaranteeing data privacy. Communication has been identified as the main bottleneck when training large-scale models due to large-volume model parameters and gradient transmission across public networks with limited bandwidth. Most previous works focus on gradient compression, while limited work tries to compress parameters that can not be ignored and extremely affect communication performance during the training. Here, to bridge this gap, we propose FedCSpc: an efficient cross-silo federated learning system with an XAI-driven adaptive parameter compression strategy for large-scale model training. Our work substantially differs from existing gradient compression techniques due to the distinct data features of gradient and parameter. The key contributions of this paper are fourfold. (1) Our designed FedCSpc proposes to compress the parameter during the training using the state-of-the-art error-bounded lossy compressor – SZ3. (2) We develop an adaptive compression error bound adjustment algorithm to guarantee the model accuracy effectively. (3) We exploit an efficient approach to utilize the idle CPU resources of clients to compress the parameters. (4) We perform a comprehensive evaluation with a wide range of models and benchmarks on a GPU cluster with 65 GPUs. Results show that FedCSpc can achieve the same model accuracy as FedAvg while reducing the data volume of parameters and gradients in communication by up to 7.39× and 288×, respectively. With 32 clients on a 4 Gb size model, FedCSpc significantly outperforms FedAvg in wall-clock time in the emulated WAN environment (at the bandwidth of 1 Gbps or lower without loss of generality).

SZ3↗
DOE OSTI2024

Scalable Risk Assessment of Rare Events in Power Systems With Uncertain Wind Generation and Loads

Risk assessment of rare events has become increasingly important in power system planning and operation with the increasing integration of renewable energy and the presence of system uncertainties. However, quantifying the risk posed by rare events via the traditional method, i.e., Monte Carlo sampling (MCS), incurs substantial computational expense stemming from the vast ensemble of power flow simulations. To accelerate the assessment, this paper proposes a Deep Neural Network (DNN)-kernelized vector-valued Gaussian Process (VVGP) approach with excellent computational efficiency while maintaining high accuracy. Consequently, serving as a surrogate model for the power flow solver, the DNN-kernelized VVGP enables significantly faster but accurate risk assessment compared to the power flow solver. The developed surrogate model evaluates low-order N - k events that contain more than 90% instances by adeptly capturing the topological features while the high-order N - k events are assessed via a power flow solver, thereby striking a balance between computational efficiency and uncertainty quantification accuracy. Moreover, the model incorporates a Support Vector Machine (SVM) classifier to resample concerning low-probability tail events to counteract the biases potentially introduced during the DNN-kernelized VVGP evaluations. Simulations conducted on the modified IEEE 24-bus, 118-bus, and European 1354-bus systems demonstrate that the proposed method maintains the accuracy benchmark set by MCS while significantly reducing computational demands in large-scale power systems as compared to other state-of-the-art methods.

17 WIND ENERGY↗