Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

DNCON2_Inter: predicting interchain contacts for homodimeric and homomultimeric protein complexes using multiple sequence alignments of monomers and deep learning

Deep learning methods that achieved great success in predicting intrachain residue-residue contacts have been applied to predict interchain contacts between proteins. However, these methods require multiple sequence alignments (MSAs) of a pair of interacting proteins (dimers) as input, which are often difficult to obtain because there are not many known protein complexes available to generate MSAs of sufficient depth for a pair of proteins. In recognizing that multiple sequence alignments of a monomer that forms homomultimers contain the co-evolutionary signals of both intrachain and interchain residue pairs in contact, we applied DNCON2 (a deep learning-based protein intrachain residue-residue contact predictor) to predict both intrachain and interchain contacts for homomultimers using multiple sequence alignment (MSA) and other co-evolutionary features of a single monomer followed by discrimination of interchain and intrachain contacts according to the tertiary structure of the monomer. We name this tool DNCON2_Inter. Allowing true-positive predictions within two residue shifts, the best average precision was obtained for the Top-L/10 predictions of 22.9% for homodimers and 17.0% for higher-order homomultimers. In some instances, especially where interchain contact densities are high, DNCON2_Inter predicted interchain contacts with 100% precision. We also developed Con_Complex, a complex structure reconstruction tool that uses predicted contacts to produce the structure of the complex. Using Con_Complex, we show that the predicted contacts can be used to accurately construct the structure of some complexes. Our experiment demonstrates that monomeric multiple sequence alignments can be used with deep learning to predict interchain contacts of homomeric proteins.

59 BASIC BIOLOGICAL SCIENCES↗

Exact enforcement of temporal continuity in sequential physics-informed neural networks

The use of deep learning methods in scientific computing represents a potential paradigm shift in engineering problem solving. One of the most prominent developments is Physics-Informed Neural Networks (PINNs), in which neural networks are trained to satisfy partial differential equations (PDEs). While this method shows promise, the standard version has been shown to struggle in accurately predicting the dynamic behavior of time-dependent problems. To address this challenge, methods have been proposed that decompose the time domain into multiple segments, employing a distinct neural network in each segment and directly incorporating continuity between them in the loss function of the minimization problem. In this work we introduce a method to exactly enforce continuity between successive time segments via a solution ansatz. This hard constrained sequential PINN (HCS-PINN) method is simple to implement and eliminates the need for any loss terms associated with temporal continuity. The method is tested for a number of benchmark problems involving both linear and non-linear PDEs. Examples include various first order time dependent problems in which traditional PINNs struggle, namely advection, Allen–Cahn, and Korteweg–de Vries equations. Furthermore, second and third order time-dependent problems are demonstrated via wave and Jerky dynamics examples, respectively. Notably, the Jerky dynamics problem is chaotic, making the problem especially sensitive to temporal accuracy. Finally, the numerical experiments conducted with the proposed method demonstrated superior convergence and accuracy over both traditional PINNs and the soft-constrained counterparts.

42 ENGINEERING↗

Probabilistic neural networks for improved analyses with phenomenological R -matrix

Here we present a method for measurement analyses based on probabilistic deep neural networks that provide several advantages over conventional analyses with phenomenological models. These include predicting physical quantities directly from data, the rapid generation of statistically robust uncertainties, and the ability to bypass some parameters that may induce ambiguities and complications in data analysis. As deep learning methods make predictions through “black boxes,” the uncertainty quantification is typically challenging. We use a probabilistic framework that provides thorough uncertainty quantification and is straightforward to follow in practice. With the network architecture based on the Transformer, we demonstrate the current method for predicting nuclear resonance parameters from scattering data using the phenomenological R-matrix model.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

HDBind: encoding of molecular structure with hyperdimensional binary representations

Traditional methods for identifying “hit” molecules from a large collection of potential drug-like candidates rely on biophysical theory to compute approximations to the Gibbs free energy of the binding interaction between the drug and its protein target. These approaches have a significant limitation in that they require exceptional computing capabilities for even relatively small collections of molecules. Increasingly large and complex state-of-the-art deep learning approaches have gained popularity with the promise to improve the productivity of drug design, notorious for its numerous failures. However, as deep learning models increase in their size and complexity, their acceleration at the hardware level becomes more challenging. Hyperdimensional Computing (HDC) has recently gained attention in the computer hardware community due to its algorithmic simplicity relative to deep learning approaches. The HDC learning paradigm, which represents data with high-dimension binary vectors, allows the use of low-precision binary vector arithmetic to create models of the data that can be learned without the need for the gradient-based optimization required in many conventional machine learning and deep learning methods. This algorithmic simplicity allows for acceleration in hardware that has been previously demonstrated in a range of application areas (computer vision, bioinformatics, mass spectrometery, remote sensing, edge devices, etc.). To the best of our knowledge, our work is the first to consider HDC for the task of fast and efficient screening of modern drug-like compound libraries. We also propose the first HDC graph-based encoding methods for molecular data, demonstrating consistent and substantial improvement over previous work. We compare our approaches to alternative approaches on the well-studied MoleculeNet dataset and the recently proposed LIT-PCBA dataset derived from high quality PubChem assays. We demonstrate our methods on multiple target hardware platforms, including Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs), showing at least an order of magnitude improvement in energy efficiency versus even our smallest neural network baseline model with a single hidden layer. Our work thus motivates further investigation into molecular representation learning to develop ultra-efficient pre-screening tools. We make our code publicly available at https://github.com/LLNL/hdbind.

59 BASIC BIOLOGICAL SCIENCES↗

Regional Medium-Term Hourly Electricity Demand Forecasting Based on LSTM

This paper aims to forecast high-resolution (hourly) aggregated load for a certain region in the medium term (a few days to over a year). One region is defined as some places with similar climate characteristics because the climate influences people's daily lifestyles and hence the electric usage. We decom- pose the electric usage records into two parts: base load and seasonal load. Considering both temperature and time factors, different deep learning methods are adopted to characterize them. The first goal of our approach is to predict the peak load which is critical for power system planning. Furthermore, our proposed forecast method can provide the depiction of the hourly load profile to provide customized load curves for high- level real-time applications. The proposed method is tested on real-world historical data collected by CAISO, BPA, and PACW. The experimental results show that trained by three years of data, our method could reduce the prediction error for one-year lead hourly load below 5% MAPE, and predict the occurrence of the peak load for next year in CAISO with an error within three days. Furthermore, as a byproduct, an interesting observation on the impact of COVID-19 on human life was made and discussed based on these case studies.

deep learning↗

Regional Medium-Term Hourly Electricity Demand Forecasting Based on LSTM

This paper aims to forecast high-resolution (hourly) aggregated load for a certain region in the medium term (a few days to over a year). One region is defined as some places with similar climate characteristics because the climate influences people's daily lifestyles and hence the electric usage. We decompose the electric usage records into two parts: base load and seasonal load. Considering both temperature and time factors, different deep-learning methods are adopted to characterize them. The first goal of our approach is to predict the peak load which is critical for power system planning. Furthermore, our proposed forecast method can provide the depiction of the hourly load profile to provide customized load curves for high-level real-time applications. The proposed method is tested on real-world historical data collected by CAISO, BPA, and PACW. The experimental results show that trained by three years of data, our method could reduce the prediction error for a one-year lead hourly load below $5\%$ MAPE, and predict the occurrence of the peak load for next year in CAISO with an error within three days. Furthermore, as a byproduct, an interesting observation on the impact of COVID-19 on human life was made and discussed based on these case studies.

deep learning↗

Regional Medium-Term Hourly Electricity Demand Forecasting Based on LSTM: Preprint

This paper aims to forecast high-resolution (hourly) aggregated load for a certain region in the medium term (a few days to over a year). One region is defined as some places with similar climate characteristics because the climate influences people's daily lifestyles and hence the electric usage. We decompose the electric usage records into two parts: base load and seasonal load. Considering both temperature and time factors, different deep-learning methods are adopted to characterize them. The first goal of our approach is to predict the peak load which is critical for power system planning. Furthermore, our proposed forecast method can provide the depiction of the hourly load profile to provide customized load curves for high-level real-time applications. The proposed method is tested on real-world historical data collected by CAISO, BPA, and PACW. The experimental results show that trained by three years of data, our method could reduce the prediction error for a one-year lead hourly load below 5% MAPE, and predict the occurrence of the peak load for next year in CAISO with an error within three days. Furthermore, as a byproduct, an interesting observation on the impact of COVID-19 on human life was made and discussed based on these case studies.

deep learning↗

Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

In this paper, a day-ahead electricity market bidding problem with multiple strategic generation company (GEN-CO) bidders is studied. The problem is formulated as a Markov game model, where GENCO bidders interact with each other todevelop their optimal day-ahead bidding strategies. Considering unobservable information in the problem, a model-free and data-driven approach, known as multi-agent deep deterministic policy gradient (MADDPG), is applied for approximating the Nash equilibrium (NE) in the above Markov game. The MADDPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks. The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested caseand a congested case. Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient (DDPG) demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains. In addition, the comparison with a conventional model-based method shows that the MADDPG algorithm has higher computational efficiency, which is feasible for real-world applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Inversion of Time-Lapse Seismic Reservoir Monitoring Data Using CycleGAN: A Deep Learning-Based Approach for Estimating Dynamic Reservoir Property Changes

Carbon capture and storage is being pursued globally as a geoengineering measure for reducing the emission of anthropogenic CO 2 the atmosphere. Comprehensive monitoring, verification, and accounting programs must be established for demonstrating the safe storage of injected CO 2 . One of the most commonly deployed monitoring techniques is time-lapse seismic reservoir monitoring (also known as 4-D seismic), which involves comparing 3-D seismic survey data taken at the same study site but over different times. Analyses of 4-D seismic data volumes can help improve the quality of storage reservoir characterization, track the movement of injected CO 2 plume, and identify potential CO 2 spillover/leakage from the storage reservoirblue. However, the derivation of high-resolution CO 2 saturation maps from 4-D seismic data is a highly nonlinear and ill-posed inverse problem, often requiring significant computational effort. In this research, we apply a physics-based deep learning method to facilitate the solution of both the forward and inverse problems in seismic inversion while honoring physical constraints. A cycle generative adversarial neural network (CycleGAN) model is trained to learn the bidirectional functional mappings between the reservoir dynamic property changes and seismic attribute changes, such that both forward and inverse solutions can be obtained efficiently from the trained model. We show that our CycleGAN-based approach not only improves the reliability of 4-D seismic inversion but also expedites the quantitative interpretation. Our deep learning-based workflow is generic and can be readily used for reservoir characterization and reservoir model updates involving the use of 4-D seismic data.

58 GEOSCIENCES↗

Three-Dimensional Analysis of Particle Distribution on Filter Layers inside N95 Respirators by Deep Learning

The global COVID-19 pandemic has changed many aspects of daily lives. Wearing personal protective equipment, especially respirators (face masks), has become common for both the public and medical professionals, proving to be effective in preventing spread of the virus. Nevertheless, a detailed understanding of respirator filtration-layer internal structures and their physical configurations is lacking. Here, we report three-dimensional (3D) internal analysis of N95 filtration layers via X-ray tomography. Using deep learning methods, we uncover how the distribution and diameters of fibers within these layers directly affect contaminant particle filtration. The average porosity of the filter layers is found to be 89.1%. Contaminants are more efficiently captured by denser fiber regions, with fibers <1.8 μm in diameter being particularly effective, presumably because of the stronger electric field gradient on smaller diameter fibers. This study provides critical information for further development of N95-type respirators that combine high efficiency with good breathability.

42 ENGINEERING↗

Precision Polishing of Ablator Capsules via in situ Process Monitoring and Machine Learning–Based Optimization

In inertial confinement fusion (ICF) experiments seeking output gains of unity and beyond, the quality of the ablator capsule is paramount for minimizing the hydrodynamic mix that quenches the central hot spot. Defects in the form of foreign particles or missing mass on the surface and within the wall of the capsule are primary offenders. High-density carbon capsules made for ICF experiments at the National Ignition Facility are precision polished to achieve surface smoothness on the order of a few nanometers as well as to minimize isolated defects in the form of pits. Given the critical role of this process, we are developing smart manufacturing techniques with the goal of elevating the efficiency of this process. Our approach is to use MEMS (micro-electromechanical systems)–based sensors to capture the fine vibration signals generated during the polishing process and combine them with synchronized visual feedback as needed. Beyond using these sensors for process monitoring, we use specific deep learning methods to analyze the data and extract correlations with both the process parameters and the final performance of the polishing run. Here, in this work, we describe the multiple fronts we have explored in this regard and the results we have gotten so far. This approach promises to have the potential to ultimately provide real-time feedback that can be used to ensure the progress of the run as well as a means for faster optimization.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Biologically inspired approaches for biosurveillance anomaly detection and data fusion

This study developed and tested biologically inspired computational methods to detect anomalous signals in data streams that could indicate a pending outbreak or bio-weapon attack. Current large-scale biosurveillance systems are plagued by two principal deficiencies: (1) timely detection of disease-indicating signals in noisy data and (2) anomaly detection across multiple channels. Anomaly detectors and data fusion components modeled after human immune system processes were tested against a variety of natural and synthetic surveillance datasets. A pilot scale immune-system-based biosurveillance system performed at least as well as traditional statistical anomaly detection data fusion approaches. Machine learning approaches leveraging Deep Learning recurrent neural networks were developed and applied to challenging unstructured and multimodal health surveillance data. Within the limits imposed of data availability, both immune systems and deep learning methods were found to improve anomaly detection and data fusion performance for particularly challenging data subsets.

59 BASIC BIOLOGICAL SCIENCES↗

Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation

Non-intrusive reduced-order models (ROMs) have recently generated considerable interest for constructing computationally efficient counterparts of nonlinear dynamical systems emerging from various domain sciences. They provide a low-dimensional emulation framework for systems that may be intrinsically high-dimensional. This is accomplished by utilizing a construction algorithm that is purely data-driven. It is no surprise, therefore, that the algorithmic advances of machine learning have led to non-intrusive ROMs with greater accuracy and computational gains. However, in bypassing the utilization of an equation-based evolution, it is often seen that the interpretability of the ROM framework suffers. This becomes more problematic when black-box deep learning methods are used which are notorious for lacking robustness outside the physical regime of the observed data. In this article, we propose the use of a novel latent-space interpolation algorithm based on Gaussian process regression. Notably, this reduced-order evolution of the system is parameterized by control parameters to allow for interpolation in space. The use of this procedure also allows for a continuous interpretation of time which allows for temporal interpolation. The latter aspect provides information, with quantified uncertainty, about full-state evolution at a finer resolution than that utilized for training the ROMs. This research assesses the viability of this algorithm for an advection-dominated system given by the inviscid shallow water equations.

97 MATHEMATICS AND COMPUTING↗

LSTM-Based Data Integration to Improve Snow Water Equivalent Prediction and Diagnose Error Sources

Accurate prediction of snow water equivalent (SWE) can be valuable for water resource managers. Recently, deep learning methods such as long short-term memory (LSTM) have exhibited high accuracy in simulating hydrologic variables and can integrate lagged observations to improve prediction, but their benefits were not clear for SWE simulations. Here we tested an LSTM network with data integration (DI) for SWE in the western United States to integrate 30-day-lagged or 7-day-lagged observations of either SWE or satellite-observed snow cover fraction (SCF) to improve future predictions. SCF proved beneficial only for shallow-snow sites during snowmelt, while lagged SWE integration significantly improved prediction accuracy for both shallow- and deep-snow sites. The median Nash–Sutcliffe model efficiency coefficient (NSE) in temporal testing improved from 0.92 to 0.97 with 30-day-lagged SWE integration, and root-mean-square error (RMSE) and the difference between estimated and observed peak SWE values d max were reduced by 41% and 57%, respectively. DI effectively mitigated accumulated model and forcing errors that would otherwise be persistent. Moreover, by applying DI to different observations (30-day-lagged, 7-day-lagged), we revealed the spatial distribution of errors with different persistent lengths. For example, integrating 30-day-lagged SWE was ineffective for ephemeral snow sites in the southwestern United States, but significantly reduced monthly-scale biases for regions with stable seasonal snowpack such as high-elevation sites in California. These biases are likely attributable to large interannual variability in snowfall or site-specific snow redistribution patterns that can accumulate to impactful levels over time for nonephemeral sites. These results set up benchmark levels and provide guidance for future model improvement strategies.

54 ENVIRONMENTAL SCIENCES↗

Reduced volume of diabetic pancreatic islets in rodents detected by synchrotron X-ray phase-contrast microtomography and deep learning network

The pancreatic islet is a highly structured micro-organ that produces insulin in response to rising blood glucose. Here we develop a label-free and automatic imaging approach to visualize the islets in situ in diabetic rodents by the synchrotron radiation X-ray phase-contrast microtomography (SRμCT) at the ID17 station of the European Synchrotron Radiation Facility. The large-size images (3.2 mm × 15.97 mm) were acquired in the pancreas in STZ-treated mice and diabetic GK rats. Each pancreas was dissected by 3000 reconstructed images. The image datasets were further analysed by a self-developed deep learning method, AA-Net. All islets in the pancreas were segmented and visualized by the three-dimension (3D) reconstruction. After quantifying the volumes of the islets, we found that the number of larger islets (=>1500 μm 3 ) was reduced by 2-fold (wt 1004 ± 94 vs GK 419 ± 122, P < 0.001) in chronically developed diabetic GK rat, while in STZ-treated diabetic mouse the large islets were decreased by half (189 ± 33 vs 90 ± 29, P < 0.001) compared to the untreated mice. Our study provides a label-free tool for detecting and quantifying pancreatic islets in situ. It implies the possibility of monitoring the state of pancreatic islets in vivo diabetes without labelling.

59 BASIC BIOLOGICAL SCIENCES↗

HD-Bind: Encoding of Molecular Structure with Low Precision, Hyperdimensional Binary Representations

Publicly available collections of drug-like molecules have grown to comprise tens of billions of compounds due to advances in combinatorial chemistry. Traditional methods for identifying "hit" molecules from a large collection of potential drug-like candidates have relied on biophysical theory to compute approximations to the Gibbs free energy of the binding interaction between the drug and its protein target. These approaches have the major drawback that they require exceptional computing capabilities for even relatively small collections of molecules. Hyperdimensional Computing (HDC) is a recently-proposed learning paradigm that represents data with high-dimension binary vectors; this allows the use of low-precision binary vector arithmetic to create models of the data that can be learned without the need for the gradient-based optimization required in many conventional machine learning and deep learning methods. This algorithmic simplicity allows for acceleration in hardware that has been previously demonstrated in a range of application areas. We consider existing HDC approaches for molecular property classification and introduce two novel encodings of a commonly-used molecular representation, the extended connectivity fingerprint (ECFP). We show that HDC-based inference methods are as much as 91 times more efficient than traditional machine learning methods, and achieve an acceleration of nearly nine orders of magnitude compared to molecular docking. Our results show that HDC accelerated methods retain competitive accuracy on a number of well-studied tasks such as molecular property predictions using the MoleculeNet dataset, and bind/no-bind activity classification using the DUD-E and LIT-PCBA datasets. Our work thus motivates further investigation into molecular representation learning to develop ultraefficient pre-screening tools.

Jones, William↗

An end-to-end deep learning solution for automated LiDAR tree detection in the urban environment

Cataloging and classifying trees in the urban environment is a crucial step in urban and environmental planning; however, manual collection and maintenance of this data is expensive and time-consuming. Although algorithmic approaches that rely on remote sensing data have been developed for tree detection in forests, they generally struggle in the more varied urban environment. This work proposes a novel end-to-end deep learning method for the detection of trees in the urban environment from remote sensing data. Specifically, we develop and train a novel PointNet-based neural network architecture to predict tree locations directly from LiDAR data augmented with multi-spectral imagery. We compare this model to a number of high-performing baselines on a large and varied dataset in the Southern California region, and find that our method outperforms all baselines in terms of tree detection ability (75.5% F-score) and positional accuracy (2.28 meter root mean squared error), while being highly efficient. We then analyze and compare the sources of errors, and how these reveal the strengths and weaknesses of each approach. Our results highlight the importance of fusing spectral and structural information for remote sensing tasks in complex urban environments.

54 ENVIRONMENTAL SCIENCES↗

A machine learning approach to thermal conductivity modeling: A case study on irradiated uranium-molybdenum nuclear fuels

A deep neural network was developed for the purpose of predicting thermal conductivity with a case study performed on irradiated nuclear fuel. Traditional thermal conductivity modeling approaches rely on existing theoretical frameworks that describe known, relevant phenomena that govern the microstructural evolution processes during neutron irradiation (such as recrystallization, and pore size, distribution and morphology). Current empirical modeling approaches, however, do not represent all irradiation test data well. We develop a machine learning approach to thermal conductivity modeling that does not require a priori knowledge of a specific material microstructure and system of interest. Our approach allows re-searchers to probe dependency of thermal conductivity on a variety of reactor operating and material conditions. The purpose of building such a model is to allow for improved predictive capabilities linking structure-property-processing-performance relationships in the system of interest (here, irradiated nuclear fuel), which could lead to improved experimental test planning and characterization. The uranium-molybdenum system is the fuel system studied in this work, and historic irradiation test data is leveraged for model development. Our model achieved a mean absolute percent error of approximately 10%. Results indicate our model generalizes well to never before seen data, and thus use of deep learning methods for material property predictions from limited, historic irradiation data is a viable approach. This work challenges the current paradigm in materials science, where material property models are based on limited experimental data fitted to low-dimensionality phenomenological models, or by simulations in which fundamental equations are explicitly solved. The work presented here aims to demonstrate the potential and limitations of machine learning in the field of materials science and material property modeling.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗