Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

The LSST AGN Data Challenge: Selection Methods

Abstract Development of the Rubin Observatory Legacy Survey of Space and Time (LSST) includes a series of Data Challenges (DCs) arranged by various LSST Scientific Collaborations that are taking place during the project's preoperational phase. The AGN Science Collaboration Data Challenge (AGNSC-DC) is a partial prototype of the expected LSST data on active galactic nuclei (AGNs), aimed at validating machine learning approaches for AGN selection and characterization in large surveys like LSST. The AGNSC-DC took place in 2021, focusing on accuracy, robustness, and scalability. The training and the blinded data sets were constructed to mimic the future LSST release catalogs using the data from the Sloan Digital Sky Survey Stripe 82 region and the XMM-Newton Large Scale Structure Survey region. Data features were divided into astrometry, photometry, color, morphology, redshift, and class label with the addition of variability features and images. We present the results of four submitted solutions to DCs using both classical and machine learning methods. We systematically test the performance of supervised models (support vector machine, random forest, extreme gradient boosting, artificial neural network, convolutional neural network) and unsupervised ones (deep embedding clustering) when applied to the problem of classifying/clustering sources as stars, galaxies, or AGNs. We obtained classification accuracy of 97.5% for supervised models and clustering accuracy of 96.0% for unsupervised ones and 95.0% with a classic approach for a blinded data set. We find that variability features significantly improve the accuracy of the trained models, and correlation analysis among different bands enables a fast and inexpensive first-order selection of quasar candidates.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Uncertainties in Parameters Estimated with Neural Networks: Application to Strong Gravitational Lensing

In Hezaveh et al. (2017) we showed that deep learning can be used for model parameter estimation and trained convolutional neural networks to determine the parameters of strong gravitational lensing systems. Here we demonstrate a method for obtaining the uncertainties of these parameters. We review the framework of variational inference to obtain approximate posteriors of Bayesian neural networks and apply it to a network trained to estimate the parameters of the Singular Isothermal Ellipsoid plus external shear and total flux magnification. We show that the method can capture the uncertainties due to different levels of noise in the input data, as well as training and architecture-related errors made by the network. To evaluate the accuracy of the resulting uncertainties, we calculate the coverage probabilities of marginalized distributions for each lensing parameter. By tuning a single hyperparameter, the dropout rate, we obtain coverage probabilities approximately equal to the confidence levels for which they were calculated, resulting in accurate and precise uncertainty estimates. Our results suggest that neural networks can be a fast alternative to Monte Carlo Markov Chains for parameter uncertainty estimation in many practical applications, allowing more than seven orders of magnitude improvement in speed.

Perreault Levasseur, Laurence↗

Towards Efficient Uncertainty estimation in deep learning for robust energy prediction in crystal materials

In material science, recent studies have started to explore the potential of using deep learning to improve property prediction from high-fidelity simulations, e.g, density functional theory (DFT). However, the design spaces are sometimes too large and intractable to sample completely. This results in a critical question that is how to evaluate the confidence and robustness of the prediction. In this paper, we propose an efficient approach to estimate uncertainty in deep learning using a single forward pass and then apply it for robust prediction of the total energy in crystal lattice structures. Our approach is built upon the deep kernel learning (DKL) that originally introduces to leverage the expressiveness of deep neural networks as input with a probabilistic prediction of Gaussian processes (GPs) as output. Existing DKL methods have difficulties in the accuracy of predictive uncertainty, training stability, and scaling to large datasets, which lead to significant barriers in real-world applications. We propose to address these challenges by using an inducing point approximate GP in feature space combined with spectral normalization as a regularization. We finally demonstrate our robust performance on an artificial example and a real-world application from materials chemistry.

Bi, Sirui↗

Sensitive detection of structural dynamics using a statistical framework for comparative crystallography

Chemical and conformational changes are crucial to protein function and its pharmacological control. X-ray crystallography can reveal these changes in atomic detail, but standard analysis methods, which refine separate datasets, often overlook differences that are subtle or arise in only a subset of molecules. Direct comparison of crystallographic datasets is, in principle, more powerful, but systematic errors (“scales”) often mask changes in the crystallographic observables (“structure factors”). Machine learning algorithms that jointly estimate scales and structure factors can address this limitation. Here, we augment this approach with multivariate, structured priors derived from crystallographic theory, implemented in the variational deep learning framework Careless. Doing so strongly improves the detection of protein dynamics, element-specific anomalous signals, and the binding of drug candidates, offering a robust approach to comparative crystallography and, potentially, to detection of protein dynamics by other structure determination methods.

Hekstra, Doeke R. [Harvard Univ., Cambridge, MA (U↗

A Bayesian Learning Approach to Wireless Outdoor Heatmap Construction using Deep Gaussian Process

We present a novel Bayesian learning approach to outdoor radio heatmap construction utilizing deep Gaussian process (GP). The proposed approach employs a two-layer hierarchy which consists of two cascaded Gaussian processes that are capable of modeling more complex input-output relations than standard single-layer Gaussian processes. Since deriving the exact model likelihood is challenging, a lower bound is optimized instead so that gradient descent-based methods can be performed to find out the optimal model parameters. Typically, inducing points are used in GPs to facilitate low-rank approximation of covariance (kernel) matrices for computation speedup. However, the inaccuracy induced by inducing points can accumulate when stacking multiple layers of GP which may hinder the performance of deep GP. Moreover, since inducing points need to be learned, having them at all layers of deep GP also incurs computational burden. To overcome the above challenges, in contrast to the canonical deep GP model, we use a modified architecture where a full standard GP resides in the first layer and inducing points are only introduced for the second layer. This modified architecture strikes a balance between model accuracy and training complexity. In the proposed model, the noise parameter of the first GP layer is also eliminated to improve the training efficiency as the noise parameter at the output of the second layer suffices to model the uncertainty in the output. The proposed approach is evaluated on real-world datasets, in the form of location-Received Signal Strength (RSS) pairs, collected from the Platform for Open Wireless Data-driven Experimental Research (POWDER) located at the campus of the University of Utah. Experiment results show that the proposed approach can achieve smaller prediction errors on various training and testing data configurations than DNN-based and GP-based methods.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Object detection with deep learning for rare event search in the GADGET II TPC

In the pursuit of identifying rare two-particle events within the GADGET II Time Projection Chamber (TPC), this paper presents a comprehensive approach for leveraging Convolutional Neural Networks (CNNs) and various data processing methods. To address the inherent complexities of 3D TPC track reconstructions, the data is expressed in 2D projections and 1D quantities. This approach capitalizes on the diverse data modalities of the TPC, allowing for the efficient representation of the distinct features of the 3D events, with no loss in topology uniqueness. Additionally, it leverages the computational efficiency of 2D CNNs and benefits from the extensive availability of pre-trained models. Given the scarcity of real training data for the rare events of interest, simulated events are used to train the models to detect real events. To account for potential distribution shifts when predominantly depending on simulations, significant perturbations are embedded within the simulations. This produces a broad parameter space that works to account for potential physics parameter and detector response variations and uncertainties. These parameter-varied simulations are used to train sensitive 2D CNN object detectors. When combined with 1D histogram peak detection algorithms, this multi-modal detection framework is highly adept at identifying rare, two-particle events in data taken during experiment 21072 at the Facility for Rare Isotope Beams (FRIB), demonstrating a 100% recall for events of interest. Here, we present the methods and outcomes of our investigation and discuss the potential future applications of these techniques.

Convolutional neural network↗

Deep learning classification of lipid droplets in quantitative phase images

We report the application of supervised machine learning to the automated classification of lipid droplets in label-free, quantitative-phase images. By comparing various machine learning methods commonly used in biomedical imaging and remote sensing, we found convolutional neural networks to outperform others, both quantitatively and qualitatively. We describe our imaging approach, all implemented machine learning methods, and their performance with respect to computational efficiency, required training resources, and relative method performance measured across multiple metrics. Overall, our results indicate that quantitative-phase imaging coupled to machine learning enables accurate lipid droplet classification in single living cells. As such, the present paradigm presents an excellent alternative of the more common fluorescent and Raman imaging modalities by enabling label-free, ultra-low phototoxicity, and deeper insight into the thermodynamics of metabolism of single cells.

59 BASIC BIOLOGICAL SCIENCES↗

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

De novo Nanopore read quality improvement using deep learning

BACKGROUND: Long read sequencing technologies such as Oxford Nanopore can greatly decrease the complexity of de novo genome assembly and large structural variation identification. Currently Nanopore reads have high error rates, and the errors often cluster into low-quality segments within the reads. The limited sensitivity of existing read-based error correction methods can cause large-scale mis-assemblies in the assembled genomes, motivating further innovation in this area. RESULTS: Here we developed a Convolutional Neural Network (CNN) based method, called MiniScrub, for identification and subsequent "scrubbing" (removal) of low-quality Nanopore read segments to minimize their interference in downstream assembly process. MiniScrub first generates read-to-read overlaps via MiniMap2, then encodes the overlaps into images, and finally builds CNN models to predict low-quality segments. Applying MiniScrub to real world control datasets under several different parameters, we show that it robustly improves read quality, and improves read error correction in the metagenome setting. Compared to raw reads, de novo genome assembly with scrubbed reads produces many fewer mis-assemblies and large indel errors. CONCLUSIONS: MiniScrub is able to robustly improve read quality of Oxford Nanopore reads, especially in the metagenome setting, making it useful for downstream applications such as de novo assembly. We propose MiniScrub as a tool for preprocessing Nanopore reads for downstream analyses. MiniScrub is open-source software and is available at https://bitbucket.org/berkeleylab/jgi-miniscrub .

59 BASIC BIOLOGICAL SCIENCES↗

Invertible Neural Networks for Airfoil Design

We report the airfoil design problem, in which an engineer seeks a shape with desired performance characteristics, is fundamental to aerodynamics. Design workflows traditionally rely on iterative optimization methods using low-fidelity integral boundary-layer methods as higher-fidelity adjoint-based computational fluid dynamics methods are computationally expensive. Surrogate-based approaches can accelerate the design process but still rely on some iterative inverse design procedure. In this work, we leverage emerging invertible neural network (INN) tools to enable the rapid inverse design of airfoil shapes for wind turbines. INNs are specialized deep-learning models with well-defined inverse mappings. When trained appropriately, INN surrogate models are capable of forward prediction of aerodynamic and structural quantities for a given airfoil shape as well as inverse recovery of airfoil shapes with specified aerodynamic and structural characteristics. The INN approach offers a roughly 100 times speed-up compared to adjoint-based methods for inverse design. We demonstrate the INN tool for inverse design on three test cases of 100 airfoils each that satisfy the performance characteristics close to those of airfoils used in wind-turbine blades. All generated shapes satisfy the desired aerodynamic characteristics, demonstrating the success of the INN approach for inverse design of airfoils.

17 WIND ENERGY↗

Intra-hour Solar Irradiance Forecast in Multiple Locations using Deep Transfer Learning

In recent years, solar power system installation imposes several challenges on the operations of local and regional power grids due to the inherent variability of ground-level solar irradiance. This work proposes a novel real-time solar forecast methodology for intra-hour solar irradiance based on deep transfer learning from ground-based sky imager for time horizons ranging from 5-15 min. There are three unique aspects of the proposed methodology: (1) a Deep Learning based algorithm development which is modeled as a classification approach rather than a traditional regression approach; (2) the use of the Transfer Learning technique to show generalization capability, robustness, and portability of baseline model in the newly deployed location where availability of enough data for training is typically scarce, and (3) redefinition of point-based irradiation forecast error estimation technique with a window-based one that is more intuitive and user-friendly. The system is developed using multiple years of irradiance and sky image recording in New Jersey and one-year data from Colorado, USA. The method is validated against ground telemetry from these two locations of diverse geographic and climatic conditions. Results show that the forecasting method proposed in this work is robust and highly accurate (8% MAPE error) for multiple locations deployment.

Deep Learning, Convolution Neural Networks, transf↗

Deep Learning Accelerated Determination of Hydride Locations in Metal Nanoclusters

Abstract Although the coordinates of the metal atoms can be accurately determined by X‐ray crystallography, locations of hydrides in metal nanoclusters are challenging to determine. In principle, neutron crystallography can be employed to pinpoint the hydride positions, but it requires a large crystal and a neutron source, which prevents its routine use. Here, we present a deep‐learning approach that can accelerate determination of hydride locations in single‐crystal X‐ray structure of metal nanoclusters of different sizes. We demonstrate the efficiency of our method in predicting the most probable hydride sites and their combinations to determine the total structure for two recently reported copper nanoclusters, [Cu 25 H 10 (SPhCl 2 ) 18 ] 3− and [Cu 61 (S t Bu) 26 S 6 Cl 6 H 14 ] + whose hydride locations have not been determined by neutron diffraction. Our method can be generalized and applied to other metal systems, thereby eliminating a bottleneck in atomically precise metal hydride nanochemistry.

Wang, Song↗

Advancing spatiotemporal forecasts of CO 2 plume migration using deep learning networks with transfer learning and interpretation analysis

Accurate and timely forecasts of CO 2 plume distribution throughout the injection and post-injection phases are crucial for detecting plume migration, assessing leakage risks, and supporting operational decisions in geologic carbon storage (GCS). Current convolutional neural network-based approaches primarily focus on spatial information and overlook temporal dependencies in plume distributions, thus limiting their ability to capture dynamic movement effects and provide accurate predictions of plume migration. In this work, we propose two deep learning models, Auto-Encoder (AE)-LSTM and Encoder-Decoder (ED)-ConvLSTM, each uniquely designed to capture both spatial and temporal features. We apply the proposed methods to forecast the dynamic distribution of CO 2 plumes based on 108 reservoir simulations over a 30-year injection and a 30-year post-injection period. The results indicate that the ED-ConvLSTM model outperforms the AE-LSTM model in accurately predicting the spatiotemporal dynamics of CO 2 plume migration, achieving R 2 values above 0.99. To provide a deeper understanding of these model predictions, we employ a gradient-based explanation method on the trained models. This approach provides insights into the influence of input variables on plume migration forecasts and uncovers the underlying prediction mechanisms of the proposed models. Furthermore, we introduce a transfer learning technique, enabling fast and accurate plume migration forecasting in the post-injection phase by leveraging the trained model during the injection phase. This reduces the necessity for extensive data collection or re-training. In conclusion, the methods proposed in our work enhances the performance and interpretability of CO 2 plume migration forecasts, thereby facilitating informed decision-making throughout the entire lifecycle of GCS applications.

58 GEOSCIENCES↗

Deep learning analysis of defect and phase evolution during electron beam-induced transformations in WS2

Abstract Recent advances in scanning transmission electron microscopy (STEM) allow the real-time visualization of solid-state transformations in materials, including those induced by an electron beam and temperature, with atomic resolution. However, despite the ever-expanding capabilities for high-resolution data acquisition, the inferred information about kinetics and thermodynamics of the process, and single defect dynamics and interactions is minimal. This is due to the inherent limitations of manual ex situ analysis of the collected volumes of data. To circumvent this problem, we developed a deep-learning framework for dynamic STEM imaging that is trained to find the lattice defects and apply it for mapping solid state reactions and transformations in layered WS 2 . The trained deep-learning model allows extracting thousands of lattice defects from raw STEM data in a matter of seconds, which are then classified into different categories using unsupervised clustering methods. We further expanded our framework to extract parameters of diffusion for sulfur vacancies and analyzed transition probabilities associated with switching between different configurations of defect complexes consisting of Mo dopant and sulfur vacancy, providing insight into point-defect dynamics and reactions. This approach is universal and its application to beam-induced reactions allows mapping chemical transformation pathways in solids at the atomic level.

36 MATERIALS SCIENCE↗

Astronomical image time series classification using CONVolutional attENTION (ConvEntion)

Aims: The treatment of astronomical image time series has won increasing attention in recent years. Indeed, numerous surveys following up on transient objects are in progress or under construction, such as the Vera C. Rubin Observatory Legacy Survey for Space and Time (LSST), which is poised to produce huge amounts of these time series. The associated scientific topics are extensive, ranging from the study of objects in our galaxy to the observation of the most distant supernovae for measuring the expansion of the universe. With such a large amount of data available, the need for robust automatic tools to detect and classify celestial objects is growing steadily. Methods: This study is based on the assumption that astronomical images contain more information than light curves. In this paper, we propose a novel approach based on deep learning for classifying different types of space objects directly using images. We named our approach ConvEntion, which stands for CONVolutional attENTION. It is based on convolutions and transformers, which are new approaches for the treatment of astronomical image time series. Our solution integrates spatio-temporal features and can be applied to various types of image datasets with any number of bands. Results: In this work, we solved various problems the datasets tend to suffer from and we present new results for classifications using astronomical image time series with an increase in accuracy of 13%, compared to state-of-the-art approaches that use image time series, and a 12% increase, compared to approaches that use light curves.

79 ASTRONOMY AND ASTROPHYSICS↗

Atlas of putative minima and low-lying energy networks of water clusters n = 3 - 25

We report a database consisting of the putative minima and ~3.2 million local minima lying within 5 kcal/mol from the putative minima for water clusters of sizes n = 3 – 25 using an improved version of the Monte Carlo Temperature Basin Paving (MCTBP) global optimization procedure in conjunction with the ab-initio based, flexible, polarizable Thole-Type Model (TTM2.1-F, version 2.1) interaction potential for water. Several of the low-lying structures, as well as lowlying penta-coordinated water networks obtained with the TTM2.1-F potential, were further refined at the Møller-Plesset second order perturbation MP2/aug-cc-pVTZ level of theory. In total, we have identified 3,138,303 networks corresponding to local minima of the clusters n = 3 – 25, whose Cartesian coordinates and relative energies can be obtained from the webpage https://sites.uw.edu/wdbase/. Networks containing penta-coordinated water molecules start to appear at n = 11 and, quite surprisingly, are energetically close (within 1 – 3 kcal/mol) to the putative minima, a fact that has been confirmed from the MP2 calculations. This large database of water cluster minima spanning quite dissimilar hydrogen bonding networks is expected to influence the development and assessment of the accuracy of interaction potentials for water as well as lower scaling electronic structure methods (such as different density functionals). Furthermore, it can also be used in conjunction with data science approaches (including but no limited to neural networks, machine and deep learning) to understand the properties of water, nature’s most important substance.

Rakshit, Avijit↗

Evaluating algorithmic bias on biomarker classification of breast cancer pathology reports

Objectives: This work evaluated algorithmic bias in biomarkers classification using electronic pathology reports from female breast cancer cases. Bias was assessed across 5 subgroups: cancer registry, race, Hispanic ethnicity, age at diagnosis, and socioeconomic status. Materials and Methods: We utilized 594 875 electronic pathology reports from 178 121 tumors diagnosed in Kentucky, Louisiana, New Jersey, New Mexico, Seattle, and Utah to train 2 deep-learning algorithms to classify breast cancer patients using their biomarkers test results. We used balanced error rate (BER), demographic parity (DP), equalized odds (EOD), and equal opportunity (EOP) to assess bias. Results: We found differences in predictive accuracy between registries, with the highest accuracy in the registry that contributed the most data (Seattle Registry, BER ratios for all registries >1.25). BER showed no significant algorithmic bias in extracting biomarkers (estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2) for race, Hispanic ethnicity, age at diagnosis, or socioeconomic subgroups (BER ratio <1.25). DP, EOD, and EOP all showed insignificant results. Discussion: We observed significant differences in BER by registry, but no significant bias using the DP, EOD, and EOP metrics for socio-demographic or racial categories. This highlights the importance of employing a diverse set of metrics for a comprehensive evaluation of model fairness. Conclusion: A thorough evaluation of algorithmic biases that may affect equality in clinical care is a critical step before deploying algorithms in the real world. We found little evidence of algorithmic bias in our biomarker classification tool. Artificial intelligence tools to expedite information extraction from clinical records could accelerate clinical trial matching and improve care.

60 APPLIED LIFE SCIENCES↗

Improving galaxy clustering measurements with deep learning: analysis of the DECaLS DR7 data

ABSTRACT Robust measurements of cosmological parameters from galaxy surveys rely on our understanding of systematic effects that impact the observed galaxy density field. In this paper, we present, validate, and implement the idea of adopting the systematics mitigation method of artificial neural networks for modelling the relationship between the target galaxy density field and various observational realities including but not limited to Galactic extinction, seeing, and stellar density. Our method by construction allows a wide class of models and alleviates overtraining by performing k-fold cross-validation and dimensionality reduction via backward feature elimination. By permuting the choice of the training, validation, and test sets, we construct a selection mask for the entire footprint. We apply our method on the extended Baryon Oscillation Spectroscopic Survey (eBOSS) Emission Line Galaxies (ELGs) selection from the Dark Energy Camera Legacy Survey (DECaLS) Data Release 7 and show that the spurious large-scale contamination due to imaging systematics can be significantly reduced by up-weighting the observed galaxy density using the selection mask from the neural network and that our method is more effective than the conventional linear and quadratic polynomial functions. We perform extensive analyses on simulated mock data sets with and without systematic effects. Our analyses indicate that our methodology is more robust to overfitting compared to the conventional methods. This method can be utilized in the catalogue generation of future spectroscopic galaxy surveys such as eBOSS and Dark Energy Spectroscopic Instrument (DESI) to better mitigate observational systematics.

79 ASTRONOMY AND ASTROPHYSICS↗