Search NASA⌕ Search

SEARCH · Search NASA

Results for “sparse learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

KamNet: An integrated spatiotemporal deep neural network for rare event searches in KamLAND-Zen

Rare event searches allow us to search for new physics at energy scales inaccessible with other means by leveraging specialized large-mass detectors. Machine learning provides a new tool to maximize the information provided by these detectors. The information is sparse, which forces these algorithms to start from the lowest level data and exploit all symmetries in the detector to produce results. In this work we present KamNet, which harnesses breakthroughs in geometric deep learning and spatiotemporal data analysis to maximize the physics reach of KamLAND-Zen, a kiloton scale spherical liquid scintillator detector searching for 0⁢νβ⁢β. Using a simplified background model for KamLAND, we show that KamNet outperforms a conventional convolutional neural network (CNN) on benchmarking Monte Carlo simulations with an increasing level of robustness. Using simulated data, we then demonstrate KamNet's ability to increase KamLAND-Zen's sensitivity to 0ν⁢ββ and 2⁢ν⁢β⁢β decay to excited states. A key component of this work is the addition of an attention mechanism to elucidate the underlying physics KamNet is using for the background rejection.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

MLtool: Universal Supervised Machine Learning Tool to Model Tabulated Data

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine learning↗

MLtool Python Code

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine Learning↗

Physics-informed graphical neural network for power system state estimation

State estimation is highly critical for accurately observing the dynamic behavior of the power grids and minimizing risks from cyber threats. However, existing state estimation methods encounter challenges in accurately capturing power system dynamics, primarily because of limitations in encoding the grid topology and sparse measurements. Here, this paper proposes a physics-informed graphical learning state estimation method to address these limitations by leveraging both domain physical knowledge and a graph neural network (GNN). We employ a GNN architecture that can handle the graph-structured data of power systems more effectively than traditional data-driven methods. The physics-based knowledge is constructed from the branch current formulation, making the approach adaptable to both transmission and distribution systems. The validation results of three IEEE test systems show that the proposed method can achieve lower mean square error more than 20% than the conventional methods.

43 PARTICLE ACCELERATORS↗

Exact Gaussian processes for massive datasets via non-stationary sparsity-discovering kernels

Abstract A Gaussian Process (GP) is a prominent mathematical framework for stochastic function approximation in science and engineering applications. Its success is largely attributed to the GP’s analytical tractability, robustness, and natural inclusion of uncertainty quantification. Unfortunately, the use of exact GPs is prohibitively expensive for large datasets due to their unfavorable numerical complexity of $$O(N^3)$$ O ( N 3 ) in computation and $$O(N^2)$$ O ( N 2 ) in storage. All existing methods addressing this issue utilize some form of approximation—usually considering subsets of the full dataset or finding representative pseudo-points that render the covariance matrix well-structured and sparse. These approximate methods can lead to inaccuracies in function approximations and often limit the user’s flexibility in designing expressive kernels. Instead of inducing sparsity via data-point geometry and structure, we propose to take advantage of naturally-occurring sparsity by allowing the kernel to discover—instead of induce—sparse structure. The premise of this paper is that the data sets and physical processes modeled by GPs often exhibit natural or implicit sparsities, but commonly-used kernels do not allow us to exploit such sparsity. The core concept of exact, and at the same time sparse GPs relies on kernel definitions that provide enough flexibility to learn and encode not only non-zero but also zero covariances. This principle of ultra-flexible, compactly-supported, and non-stationary kernels, combined with HPC and constrained optimization, lets us scale exact GPs well beyond 5 million data points.

97 MATHEMATICS AND COMPUTING↗

Climate change-resilient snowpack estimation in the Western United States

Abstract In the 21st century, warmer temperatures and changing atmospheric circulation will likely produce unprecedented changes in Western United States snowfall 1–3 , with impacts on the timing, amount, and spatial patterns of snowpack 4–7 . The ~900 snow pillow stations are indispensable to water resource management by measuring snow-water equivalent (SWE) 8,9 in strategic but fixed locations 10,11 . However, this network may not be impacted by climate change in the same way as the surrounding area 12 and thus fail to accurately represent unmeasured locations; climate change thereby threatens our ability to measure the effects of climate change on snow. In this work, we show that maintaining the current peak SWE estimation skill is nonetheless possible. We find that explicitly including spatial correlations—either from gridded observations or learned by the model—improves skill at predicting distributed snowpack from sparse observations by 184%. Existing artificial intelligence methods can be useful tools to harness the many available sources of snowpack information to estimate snowpack in a nonstationary climate.

54 ENVIRONMENTAL SCIENCES↗

The Value of Information From Horizontal Distributed Acoustic Sensing Compared to Multicomponent Geophones Via Machine Learning

Abstract Faults play an important role in recharging many geothermal reservoirs, and seismic information can image the locations of these faults. The value of information (VOI) metric is used to objectively quantify and compare the value of two types of seismic receiver data via a machine learning approach. The demonstrated VOI methodology is novel by including spatial models from seismic data and obtaining the information statistics from machine learning. Our two-dimensional numerical experiments compare images created from sparsely spaced (80 m), two-component geophone sampling to high spatial resolution (1 m), single-component DAS. We used a three-fold cross validation of a U-Net convolutional neural networks to achieve average classification statistics. The results suggest that when horizontal sources are utilized, geophones and DAS identify reflectors and non-reflectors at roughly the same rate. The average F1 score for horizontal DAS is 0.939 and 0.931 for geophones. For images created from a vertical source, DAS performed marginally better (F1 = 0.919) than geophones (F1 = 0.877). Our transferrable methodology can provide guidance on which acquisition scenarios can improve images of important structures in the subsurface and present an efficient method for obtaining reliability statistics from high-dimensional, spatial data.

Energy & Fuels↗

Development of Thermal Tomography Imaging System for In-Service Nondestructive Evaluation of Additively Manufactured Components

Additive manufacturing (AM) is an emerging method for cost-efficient fabrication of nuclear reactor parts. AM of metallic structures for nuclear energy applications is currently based on laser powder bed fusion (LPBF) process, which can introduce internal material flaws, such as pores and anisotropy. Integrity of AM structures needs to be evaluated nondestructively because material flaws could lead to premature failures due to exposure to high temperature, radiation and corrosive environment in a nuclear reactor. Thermal tomography (TT) provides a capability for non-destructive evaluation of sub-surface defects in arbitrary size structures. We investigate TT of AM stainless steel 316L specimen with imprinted internal porosity defects using relatively low-cost, small form factor infrared (IR) camera based on uncooled micro bolometer detector. Sparse coding related K-means singular value decomposition (SVD) machine learning, image processing algorithms are developed to improve quality of TT images through removal of Additive white Gaussian noise without blurring the images. Following initial qualification of an AM component for deployment in a nuclear reactor, a compact TT can also be used for in-service nondestructive evaluation (NDE). With capability to perform in-service NDE of the AM component lifecycle, TT data can be used for development of a component digital twin. MIT research reactor (MITR) has been identified as a representative environment to perform investigation of feasibility of in-service NDE during shutdown. Preliminary study of radiation level in the MITR viewport, with geometric parameters potentially compatible with compact TT system, has been conducted.

36 MATERIALS SCIENCE↗

Tensor Extraction of Latent Features (TELF)

Tensor ELF is a user-friendly parallel tensor decomposition Python toolbox that includes a suite of machine learning algorithms for CPU and GPU architectures for the analysis of sparse and dense data including utility tools for pre-processing and post-processing.

Eren, Maksim↗

Machine-Learning of Nonlocal Kernels for Anomalous Subsurface Transport from Breakthrough Curves

Anomalous behavior is ubiquitous in subsurface solute transport due to the presence of high degrees of heterogeneity at different scales in the media. Although fractional models have been extensively used to describe the anomalous transport in various subsurface applications, their application is hindered by computational challenges. Simpler nonlocal models characterized by integrable kernels and finite interaction length represent a computationally feasible alternative to fractional models; yet, the informed choice of their kernel functions still remains an open problem. We propose a general data-driven framework for the discovery of optimal kernels on the basis of very small and sparse data sets in the context of anomalous subsurface transport. Using spatially sparse breakthrough curves recovered from fine-scale particle-density simulations, we learn the best coarse-scale nonlocal model using a nonlocal operator regression technique. Predictions of the breakthrough curves obtained using the optimal nonlocal model show good agreement with fine-scale simulation results even at locations and time intervals different from the ones used to train the kernel, confirming the excellent generalization properties of the proposed algorithm. A comparison with trained classical models and with black-box deep neural networks confirms the superiority of the predictive capability of the proposed model.

97 MATHEMATICS AND COMPUTING↗

Data-Driven Modeling and Correction of Vehicle Dynamics

We develop a data-driven framework for learning and correcting nonautonomous vehicle dynamics. Physics-based vehicle models are often simplified for tractability and therefore exhibit inherent model-form uncertainty, motivating the need for data-driven correction. Moreover, nonautonomous dynamics are governed by time-dependent control inputs, which pose challenges in learning predictive models directly from temporal snapshot data. To address these, we reformulate the vehicle dynamics via a local parameterization of the time-dependent inputs, yielding a modified system composed ofa sequence of local parametric dynamical systems. Here, we approximate these parametric systems using two complementary approaches. First, we employ the dimension reduction and interpolation in parameter space (DRIPS) methodology to construct efficient linear surrogate models, equipped with lifted observable spaces and manifold-based operator interpolation. This enables data-efficient learning of vehicle models whose dynamics admit accurate linear representations in the lifted spaces. Second, for more strongly nonlinear systems, we employ flow map learning (FML), a deep neural network (DNN) approach that approximates the parametric evolution map without requiring special treatment of nonlinearities. We further extend FML with a transfer-learning-based model correction procedure, enabling the correction of misspecified prior models using only a sparse set of high-fidelity or experimental measurements, without assuming a prescribed form for the correction term. Through a suite of numerical experiments on unicycle, simplified bicycle, and slip-based bicycle models, we demonstrate that DRIPS offers robust and highly data-efficient learning of nonautonomous vehicle dynamics, while FML provides expressive nonlinear modeling and effective correction of model-form errors under severe data scarcity.

data-driven modeling↗

SU-E-J-212: Identifying Bones From MRI: A Dictionary Learnign and Sparse Regression Approach

Purpose: To develop an efficient and robust scheme to identify bony anatomy based on MRI-only simulation images. Methods: MRI offers important soft tissue contrast and functional information, yet its lack of correlation to electron-density has placed it as an auxiliary modality to CT in radiotherapy simulation and adaptation. An effective scheme to identify bony anatomy is an important first step towards MR-only simulation/treatment paradigm and would satisfy most practical purposes. We utilize a UTE acquisition sequence to achieve visibility of the bone. By contrast to manual + bulk or registration-to identify bones, we propose a novel learning-based approach for improved robustness to MR artefacts and environmental changes. Specifically, local information is encoded with MR image patch, and the corresponding label is extracted (during training) from simulation CT aligned to the UTE. Within each class (bone vs. nonbone), an overcomplete dictionary is learned so that typical patches within the proper class can be represented as a sparse combination of the dictionary entries. For testing, an acquired UTE-MRI is divided to patches using a sliding scheme, where each patch is sparsely regressed against both bone and nonbone dictionaries, and subsequently claimed to be associated with the class with the smaller residual. Results: The proposed method has been applied to the pilot site of brain imaging and it has showed general good performance, with dice similarity coefficient of greater than 0.9 in a crossvalidation study using 4 datasets. Importantly, it is robust towards consistent foreign objects (e.g., headset) and the artefacts relates to Gibbs and field heterogeneity. Conclusion: A learning perspective has been developed for inferring bone structures based on UTE MRI. The imaging setting is subject to minimal motion effects and the post-processing is efficient. The improved efficiency and robustness enables a first translation to MR-only routine. The scheme generalizes to multiple tissue classes.

60 APPLIED LIFE SCIENCES↗

Transfer-Learnt Energy Models for Predicting Electricity Consumption in Buildings with Limited and Sparse Field Data

Modeling energy consumption is critical for energy-efficient utilization of the electric appliances in a building, smart grid programs (like demand-response), and many other smart home applications. State-of-the-art energy modeling techniques either rely on theoretical models, or extensive instrumentation of the building envelope to gather ``big" data to train a deep neural network. While theoretical models are often limited by their estimation accuracy, it is not always feasible to gather a significant amount of field data. In this paper, we explore transfer learning-based strategies to train much more accurate model for energy estimation when using a sparse field data. We transferred knowledge, in the form of data and parameters, from the simulation framework to the field data. We evaluated the efficacy of our approach on field data collected from six commercial buildings and our results indicate that transfer learning-based models trained over one month data can perform comparative (and in some cases better) than the state-of-the-art machine learning and deep learning solutions.

Jain, Milan↗

Modular, Hierarchical Learning By Artificial Neural Networks

Modular and hierarchical approach to supervised learning by artificial neural networks leads to neural networks more structured than neural networks in which all neurons fully interconnected. These networks utilize general feedforward flow of information and sparse recurrent connections to achieve dynamical effects. The modular organization, sparsity of modular units and connections, and fact that learning is much more circumscribed are all attractive features for designing neural-network hardware. Learning streamlined by imitating some aspects of biological neural networks.

Baldi, Pierre F.↗

Machine Learning Inference of Random Medium Properties

Earth materials are heterogeneous across a range of spatial scales, but the resolvability of small structures is limited by sparse data coverage, noise, bandlimitedness, and other difficulties. In practice, heterogeneities below a certain size cannot be recovered from seismic data except through statistical medium descriptions, which even then can be difficult to uniquely determine. To improve the characterization of such heterogeneities, we develop a novel supervised machine learning (ML) model that provides insight about the recoverability of statistical medium properties from elastic waveform data and succeeds despite cycle-skipping and other challenges well known from elastic waveform inversion. We demonstrate the approach using random media generated by superimposing self-affine random variations on homogeneous and layered background structures. After training on sparsely-recorded, high-frequency waveforms from hundreds of different random medium realizations, we show the ability of our ML model to recover correlation lengths and other statistical properties of interest to near-surface and crustal seismology, among other fields. For frequency passbands and spatial offsets encountered in seismology, Gaussian correlation lengths and the amplitude of the random variations relative to the background model are recovered even in challenging scenarios involving unknown medium parameters, complex crustal structures, and low signal-to-noise ratio. In comparison, von Kármán correlation lengths, which are related to larger-wavelength variations of the medium than Gaussian correlation lengths, are not as well recovered. These results provide one of the first and most systematic investigations of the recoverability of statistical properties of heterogeneities below the resolution limit of deterministic seismic tomography, and suggest practical ML strategies for high-frequency waveform seismology.

58 GEOSCIENCES↗

Scaling the training of particle classification on simulated MicroBooNE events to multiple GPUs

Measurements in Liquid Argon Time Projection Chamber (LArTPC) neutrino detectors, such as the MicroBooNE detector at Fermilab, feature large, high fidelity event images. Deep learning techniques have been extremely successful in classification tasks of photographs, but their application to LArTPC event images is challenging, due to the large size of the events. Events in these detectors are typically two orders of magnitude larger than images found in classical challenges, like recognition of handwritten digits contained in the MNIST database or object recognition in the ImageNet database. Ideally, training would occur on many instances of the entire event data, instead of many instances of cropped regions of interest from the event data. However, such efforts lead to extremely long training cycles, which slow down the exploration of new network architectures and hyperparameter scans to improve the classification performance. We present studies of scaling a LArTPC classification problem on multiple architectures, spanning multiple nodes. The studies are carried out on simulated events in the MicroBooNE detector. We emphasize that it is beyond the scope of this study to optimize networks or extract the physics from any results here. Institutional computing at Pacific Northwest National Laboratory and the SummitDev machine at Oak Ridge National Laboratory’s Leadership Computing Facility have been used. To our knowledge, this is the first use of state-of-the-art Convolutional Neural Networks for particle physics and their attendant compute techniques onto the DOE Leadership Class Facilities. We expect benefits to accrue particularly to the Deep Underground Neutrino Experiment (DUNE) LArTPC program, the flagship US High Energy Physics (HEP) program for the coming decades.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Weak-Form Latent Space Dynamics Identification

This software showcases the enhanced capabilities of the Latent Space Dynamics Identification (LaSDI) algorithm through the application of the weak form, resulting in WLaSDI. WLaSDI first compresses the data, then projects it onto test functions, and subsequently learns the local latent space models. Notably, WLaSDI demonstrates significantly improved robustness to noise. Using weak-form equation learning techniques, WLaSDI achieves local latent space modeling. Compared to the standard sparse identification of nonlinear dynamics (SINDy) used in LaSDI, the variance reduction of the weak form ensures robust and precise latent space recovery, enabling fast, robust, and accurate simulations. We demonstrate the efficacy of WLaSDI against LaSDI using several common benchmark examples, including viscid and inviscid Burgers', radial advection, and heat conduction. For instance, in 1D inviscid Burgers' simulations with up to 100% Gaussian white noise, WLaSDI maintains relative errors consistently below 6%, whereas LaSDI errors can exceed 10,000%. Similarly, in radial advection simulations, WLaSDI keeps relative errors below 16%, compared to potential errors of up to 10,000% with LaSDI. Additionally, WLaSDI achieves significant speedups, such as a 140X speedup in 1D Burgers' simulations compared to the corresponding full order model.

Choi, Youngsoo↗