Search NASA⌕ Search

SEARCH · Search NASA

Results for “sparse data representation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗

A dictionary learning algorithm for compression and reconstruction of streaming data in preset order

There has been an emerging interest in developing and applying dictionary learning (DL) to process massive datasets in the last decade. Many of these efforts, however, focus on employing DL to compress and extract a set of important features from data, while considering restoring the original data from this set a secondary goal. On the other hand, although several methods are able to process streaming data by updating the dictionary incrementally as new snapshots pass by, most of those algorithms are designed for the setting where the snapshots are randomly drawn from a probability distribution. In this paper, we present a new DL approach to compress and denoise massive dataset in real time, in which the data are streamed through in a preset order (instances are videos and temporal experimental data), so at any time, we can only observe a biased sample set of the whole data. Here, our approach incrementally builds up the dictionary in a relatively simple manner: if the new snapshot is adequately explained by the current dictionary, we perform a sparse coding to find its sparse representation; otherwise, we add the new snapshot to the dictionary, with a Gram-Schmidt process to maintain the orthogonality. To compress and denoise noisy datasets, we apply the denoising to the snapshot directly before sparse coding, which deviates from traditional dictionary learning approach that achieves denoising via sparse coding. Compared to full-batch matrix decomposition methods, where the whole data is kept in memory, and other mini-batch approaches, where unbiased sampling is often assumed, our approach has minimal requirement in data sampling and storage: i) each snapshot is only seen once then discarded, and ii) the snapshots are drawn in a preset order, so can be highly biased. Through experiments on climate simulations and scanning transmission electron microscopy (STEM) data, we demonstrate that the proposed approach performs competitively to those methods in data reconstruction and denoising.

97 MATHEMATICS AND COMPUTING↗

Monthly Mean In Situ Surface Flux Observations Paired with Satellite-Derived and Reanalysis-Based Flux Data for the Great Lakes Region, 2001–2020

Surface radiative and turbulent heat fluxes over the Great Lakes strongly influence regional hydrological and meteorological processes, and their accurate representation is critical for numerical weather prediction and coupled atmosphere–lake modeling. However, direct flux observations are spatially sparse across the region, so gridded reanalysis and satellite-derived products are often used for climatological analyses and model evaluation despite differences in their flux representations. This dataset provides processed, quality-controlled, monthly mean surface flux observations from the Great Lakes Evaporation Network (GLEN), AmeriFlux, and the National Data Buoy Center, paired with spatiotemporally matched flux estimates from two reanalysis products, the fifth generation European Centre for Medium-Range Weather Forecasts (ECMWF) reanalysis dataset (ERA5) and the Modern Era Reanalysis for Research and Applications, version 2 (MERRA-2), and two satellite-derived products, the Clouds and Earth's Radiant Energy Systems Energy Balanced and Filled (CERES-EBAF) and the Cloud, Albedo and Surface Radiation dataset from AVHRR data - Edition 3 (CLARA-A3). The dataset includes sixteen observational stations with variable temporal coverage within 2001–2020. For each station, a CSV file contains monthly time series of available flux variables, including surface downwelling shortwave radiation (SW), surface downwelling longwave radiation (LW), sensible heat (SH) flux, and latent heat flux (LH), alongside matched gridded product values where available. Columns in the CSV file correspond to different variables sourced from each dataset, with column titles structured as "{dataset}_{variable}". Columns with relevant metadata are also provided in each CSV file, including station latitude and longitude, monthly timestamps, and the name of the sourced observational data. These files are structured for direct use in common analysis tools, including Microsoft Excel, Python pandas, and Python matplotlib. This dataset supports climatological analysis of the Great Lakes regional surface energy budget, evaluation of satellite-derived and reanalysis-based flux products, and development or validation of flux representations in numerical weather prediction and coupled atmosphere–lake models.

Great Lakes↗

Exploiting voxel-sparsity for bone imaging with sparse-view cone-beam computed tomography

An optimization-based image reconstruction frame work is developed specifically for bone imaging. This framework exploits voxel-sparsity by use of ℓ 1 -norm image regularization and it enables image reconstruction from sparse-view cone-beam computed tomography (CBCT) acquisition. The effectiveness of the voxel-sparsity regularization is enhanced by using a blurred image representation. Ramp-filtering is included in the data discrepancy term and it has the effect of acting as a preconditioner, reducing the necessary number of iterations. The bone image reconstruction framework is demonstrated on CBCT data taken from an equine metacarpal condyle specimen.

Bone imaging↗

Anomaly Detection in Moving-Camera Video Sequences Using Principal Subspace Analysis

This study presents a family of algorithms based on sparse decompositions that detect anomalies in video sequences obtained from slow moving cameras. These algorithms start by computing the union of subspaces that best represents all the frames from a reference (anomaly free) video as a low-rank projection plus a sparse residue. Then, they perform a low-rank representation of a target (possibly anomalous) video by taking advantage of both the union of subspaces and the sparse residue computed from the reference video. Such algorithms provide good detection results while at the same time obviating the need for previous video synchronization. However, this is obtained at the cost of a large computational complexity, which hinders their applicability. Another contribution of this paper approaches this problem by using intrinsic properties of the obtained data representation in order to restrict the search space to the most relevant subspaces, providing computational complexity gains of up to two orders of magnitude. The developed algorithms are shown to cope well with videos acquired in challenging scenarios, as verified by the analysis of 59 videos from the VDAO database that comprises videos with abandoned objects in a cluttered industrial scenario.

47 OTHER INSTRUMENTATION↗

Predictive Modeling for Differential Diagnosis and Mortality Risk Assessment

The prevalence of electronic health record (EHR) systems has brought prodigious biomedical informatics opportunity. Automated machine learning methods can effectively utilize such data and have become common tools for healthcare predictive modeling. Researches in medical informatics have explored the potential of deep learning and classical models in emergent care scenarios. In particular, predicting differential diagnoses for admissions have proven useful in decreasing unnecessary lab tests and improving inpatient triage decision-making. Moreover, identification of high-risk patients for in-hospital mortality is vitally important to maximize allocation of medical resources.The Medical Information Mart for Intensive Care (MIMIC-III) database, containing de-identified critical care inpatient was used in our study. This data set captures hospital patient laboratory measurements, pharmacologic prescriptions, diagnostic data and procedure event recordings. When considering adult patients and discounting admissions with ICU length of stay less than 24 hours, there were 37,787 unique admissions and 30,414 total patients. We examined the top 25 most prevalent ICD-9 group-level disease specificities in MIMIC-III using a multi-label classification model. In-hospital mortality was modeled as binary classification with 4,155 (13%) adult patients that expired, of which 3,138 (75.5%) were in the ICU setting. The metrics AUC, F1 score, sensitivity and specificity values calculated for each disease label measured prediction performance.The usage of ICD-9 group codes reduced feature dimension from 14,567 to 942 and greatly improved distribution of patient diagnostic categories. Disease temporal patterns were captured by considering the most frequently sampled 6 vital signs and 13 laboratory values. Missing data were imputed at each time-stamp. Time-series raw hourly average values were converted into 5 summary features (mean, standard deviation, number of observations, min & max values). Patient demographic variables such as age, gender, marital status and ethnicity were also factored into the modeling. Choi et al showed that contextual embedding of medical data, diagnostic and procedural codes alone can predict future diagnoses with sensitivity as high as 0.79. We utilized an embedding technique called word2vec which allowed sparse representations of medical history to be transformed into dense word vectors. The mappings captured contextual information by treating each admission as a sentence and learning the most likely neighboring words in a sliding window fashion. Binary and multi-label classification was achieved via collapse models, which do not consider temporal information, as well as recurrent neural networks with regularization, Softmax output layer activation together with categorical cross-entropy as the loss function.

US Army collaboration↗

Dynamic graph system for a semantic database

A method and system in a computer system for dynamically providing a graphical representation of a data store of entries via a matrix interface is disclosed. A dynamic graph system provides a matrix interface that exposes to an application program a graphical representation of data stored in a data store such as a semantic database storing triples. To the application program, the matrix interface represents the graph as a sparse adjacency matrix that is stored in compressed form. Each entry of the data store is considered to represent a link between nodes of the graph. Each entry has a first field and a second field identifying the nodes connected by the link and a third field with a value for the link that connects the identified nodes. The first, second, and third fields represent the rows, column, and elements of the adjacency matrix.

97 MATHEMATICS AND COMPUTING↗

Dynamic graph system for a semantic database

A method and system in a computer system for dynamically providing a graphical representation of a data store of entries via a matrix interface is disclosed. A dynamic graph system provides a matrix interface that exposes to an application program a graphical representation of data stored in a data store such as a semantic database storing triples. To the application program, the matrix interface represents the graph as a sparse adjacency matrix that is stored in compressed form. Each entry of the data store is considered to represent a link between nodes of the graph. Each entry has a first field and a second field identifying the nodes connected by the link and a third field with a value for the link that connects the identified nodes. The first, second, and third fields represent the rows, column, and elements of the adjacency matrix.

97 MATHEMATICS AND COMPUTING↗

MetaStrider: Architectures for Scalable Memory-centric Reduction of Sparse Data Streams

Reduction is an operation performed on the values of two or more key-value pairs that share the same key. Reduction of sparse data streams finds application in a wide variety of domains such as data and graph analytics, cybersecurity, machine learning, and HPC applications. However, these applications exhibit low locality of reference, rendering traditional architectures and data representations inefficient. This article presents MetaStrider, a significant algorithmic and architectural enhancement to the state-of-the-art, SuperStrider. Furthermore, these enhancements enable a variety of parallel, memory-centric architectures that we propose, resulting in demonstrated performance that scales near-linearly with available memory-level parallelism.

97 MATHEMATICS AND COMPUTING↗

Efficient Uncertainty Quantification in Stochastic Economic Dispatch

Stochastic economic dispatch models usually address uncertainties in forecasts of renewable generation output by considering a finite number of realizations drawn from a stochastic process model, typically via Monte Carlo sampling. Accurate evaluations of expectations or higher order moments for quantities of interest, e.g., generating cost, can require a prohibitively large number of samples. We propose an alternative to Monte Carlo sampling based on polynomial chaos expansions. These representations enable efficient and accurate propagation of uncertainties in model parameters, using sparse quadrature methods. Furthermore, we use Karhunen-Loève expansions for efficient representation of uncertain renewable energy generation that follows geographical and temporal correlations derived from historical data at each wind farm. Considering expected production cost, we show that the proposed approach can yield several orders of magnitude reduction in computational cost for solving stochastic economic dispatch relative to Monte Carlo sampling, for a given target error threshold.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Scale-Dependent Fracture-Matrix Interactions and Their Impact on Radionuclide Transport: Development of efficient particle-tracking methods

Matrix Diffusion and Adsorption within a rock matrix are important mechanisms for retarding transport of radionuclides in fractured rock. Due to computational limitations and difficulties in characterizing complex subsurface systems, diffusive exchange between a fracture network and surrounding rock matrix is often modeled using simplified conceptual representations. There is significant uncertainty in “effective” parameters used in these models, such as the “effective matrix diffusivity”. Often, these parameters are estimated by fitting sparse breakthrough data, and estimated values fall outside meaningful ranges, because simplified interpretive models do not consider complex three-dimensional flow. There is limited understanding of the relationship between the effective parameters and rock mass characteristics including network structure and matrix properties. There is also evidence for an apparent scale-dependence in “effective matrix diffusion” coefficients. These observations raise questions on whether fracture-matrix interaction parameters estimated from small-scale tracer tests can be used for predicting radionuclide fate and transport at the scale of DOE field sites. High-resolution three-dimensional Discrete-Fracture-Network-Matrix (DFNM) models based on well-defined local scale transport equations can help to address some of these questions. Due to tremendous advances in computational technology over the last 10 years, DFNM modeling in relatively large domains is now feasible. The overarching objective of our research is to use DFNM modeling to improve fundamental understanding of how effective parameters in conceptual models are related to fracture network structure and matrix properties. An advanced three-dimensional DFNM model is being developed, which combines upscaled particle-tracking algorithms for fracture-matrix interaction and a parallel fracture-network flow simulator. The particle-tracking algorithms allow complexity in flow fields at different scales, and track transport across fracture-matrix interfaces based on rigorous local approximations to the transport equations. This modeling approach can incorporate aperture variability, multi-scale preferential flow and matrix heterogeneity. We developed efficient particle-tracking methods for handling matrix diffusion and adsorption on fracture walls and demonstrated their efficiency for use within the context of large-scale complex fracture network models with variability in apertures across a network of fractures and within individual fractures.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Compressive Classification for TEM-EELS

Acquiring atomic-resolution images of beam sensitive materials or under dynamic conditions (e.g. operando STEM) is very difficult and requires precise dose fractioning to minimize the dose delivered to the sample and acquire the data more rapidly than dynamics present. Typical methods of low-dose STEM imaging include adjusting the beam current, dwell-time, and probe Scanning transmission electron microscopy (STEM) is used routinely to obtain atomic-resolution images of a small selection of radiation tolerant materials under stable conditions.size. All three of these determine the dose over a spatial area in STEM. Recently, compressive sensing [1] has been applied to electron microscopy. The computational imaging method of inpainting [2] has been proposed as a method to reduce dose and increase acquisition speed for STEM. Inpainting is a technique to recover missing pixels in an image. By skipping pixels (randomly) during acquisition, a fourth dose-control mechanism is realized [3]. A few implementations of randomly-subsampled STEM exist (e.g., [4]). To make the concepts of compressive sensing accessible to the microscopy community we begin with an example using atomic resolution images of a GaAs crystal (see Figure 1). Because the crystal is periodic it has a sparse representation in reciprocal space. In fact, a sparse representation is a key component of compressive sensing. If an image is sub-sampled randomly we find that the Fourier transform remains sparse, but becomes increasing noisy as less and less data is acquired. Yet, by finding the peaks in the reciprocal image and removing the noise it is possible to recover atomic resolution images with only 1% of the pixels. Figure 1 shows the reconstruction process for GaAs sub-sampled at 50%, 10%, 5%, and 1%. Periodic images form a special class for which the optimal sampling approach (according to CS theory) is a random selection of pixels. For non-periodic images, such as those arising in in-situ STEM, methods such as dictionary learning are necessary [2,5]. The data can be compressed in time for TEM [5] or pixels can be skipped randomly for STEM. There are several issues that arise in developing a random scan for STEM [4] and a coded aperture for TEM. The remainder of the talk will address these issues. References: [1] E Candes and M Wakin. Signal Processing Magazine, IEEE. 2(25) (2008), pp. 21. [2] A Stevens, Pu Y, Sun Y, et al. Artificial Intelligence & Statistics (AISTATS), 2017. [3] A Stevens, H Yang, L Carin et al. Microscopy 63(1), (2014), pp. 41. [4] L Kovarik, A Stevens, A Liyu et al. Appl. Phys. Lett. 109, 164102 (2016). [5] A Stevens, L Kovarik, P Abellan et al. Adv. Structural and Chemical Imaging 1(10), (2015), pp. 1. [6] Supported by the Chemical Imaging, Signature Discovery, and Analytics in Motion Initiatives at PNNL. PNNL is operated by Battelle Memorial Inst. for the US DOE; contract DE-AC05-76RL01830.

Hao, Weituo↗

Inference of Causal Networks from Time-Varying Transcriptome Data via Sparse Coding

Temporal analysis of genome-wide data can provide insights into the underlying mechanism of the biological processes in two ways. First, grouping the temporal data provides a richer, more robust representation of the underlying processes that are co-regulated. The net result is a significant dimensional reduction of the genome-wide array data into a smaller set of vocabularies for bioinformatics analysis. Second, the computed set of time-course vocabularies can be interrogated for a potential causal network that can shed light on the underlying interactions. The method is coupled with an experiment for investigating responses to high doses of ionizing radiation with and without a small priming dose. From a computational perspective, inference of a causal network can rapidly become computationally intractable with the increasing number of variables. Additionally, from a bioinformatics perspective, larger networks always hinder interpretation. Therefore, our method focuses on inferring the simplest network that is computationally tractable and interpretable. The method first reduces the number of temporal variables through consensus clustering to reveal a small set of temporal templates. It then enforces simplicity in the network configuration through the sparsity constraint, which is further regularized by requiring continuity between consecutive time points. We present intermediate results for each computational step, and apply our method to a time-course transcriptome dataset for a cell line receiving a challenge dose of ionizing radiation with and without a prior priming dose. Our analyses indicate that (i) the priming dose increases the diversity of the computed templates (e.g., diversity of transcriptome signatures); thus, increasing the network complexity; (ii) as a result of the priming dose, there are a number of unique templates with delayed and oscillatory profiles; and (iii) radiation-induced stress responses are enriched through pathway and subnetwork studies.

59 BASIC BIOLOGICAL SCIENCES↗

Beyond union of subspaces: Subspace pursuit on Grassmann manifold for data representation

Discovering the underlying structure of a high-dimensional signal or big data has always been a challenging topic, and has become harder to tackle especially when the observations are exposed to arbitrary sparse perturbations. Here in this paper, built on the model of a union of subspaces (UoS) with sparse outliers and inspired by a basis pursuit strategy, we exploit the fundamental structure of a Grassmann manifold, and propose a new technique of pursuing the subspaces systematically by solving a non-convex optimization problem using the alternating direction method of multipliers. This problem as noted is further complicated by non-convex constraints on the Grassmann manifold, as well as the bilinearity in the penalty caused by the subspace bases and coefficients. Nevertheless, numerical experiments verify that the proposed algorithm, which provides elegant solutions to the sub-problems in each step, is able to de-couple the subspaces and pursue each of them under time-efficient parallel computation.

97 MATHEMATICS AND COMPUTING↗

Understanding the Design Space of Sparse/Dense Multiphase Dataflows for Mapping Graph Neural Networks on Spatial Accelerators

Graph Neural Networks (GNNs) have garnered a lot of recent interest because of their success in learning representations from graph-structured data across several critical applications in cloud and HPC. Owing to their unique compute and memory characteristics that come from an interplay between dense and sparse phases of computations, the emergence of reconfigurable dataflow (aka spatial) accelerators offers promise for acceleration by mapping optimized dataflows (i.e., computation order and parallelism) for both phases. The goal of this work is to characterize and understand the design-space of dataflow choices for running GNNs on spatial accelerators in order for the compilers to optimize the dataflow based on the workload. Specifically, we propose a taxonomy to describe all possible choices for mapping the dense and sparse phases of GNNs spatially and temporally over a spatial accelerator, capturing both the intra-phase dataflow and the inter-phase (pipelined) dataflow. Using this taxonomy, we do deep-dives into the cost and benefits of several dataflows and perform case studies on implications of hardware parameters for dataflows and value of flexibility to support pipelined execution.

97 MATHEMATICS AND COMPUTING↗

Disentangling multidimensional spatio-temporal data into their common and aberrant responses

With the advent of high-throughput measurement techniques, scientists and engineers are starting to grapple with massive data sets and encountering challenges with how to organize, process and extract information into meaningful structures. Multidimensional spatio-temporal biological data sets such as time series gene expression with various perturbations over different cell lines, or neural spike trains across many experimental trials, have the potential to acquire insight about the dynamic behavior of the system. For this potential to be realized, we need a suitable representation to understand the data. A general question is how to organize the observed data into meaningful structures and how to find an appropriate similarity measure. A natural way of viewing these complex high dimensional data sets is to examine and analyze the large-scale features and then to focus on the interesting details. Since the wide range of experiments and unknown complexity of the underlying system contribute to the heterogeneity of biological data, we develop a new method by proposing an extension of Robust Principal Component Analysis (RPCA), which models common variations across multiple experiments as the lowrank component and anomalies across these experiments as the sparse component. We show that the proposed method is able to find distinct subtypes and classify data sets in a robust way without any prior knowledge by separating these common responses and abnormal responses. Thus, the proposed method provides us a new representation of these data sets which has the potential to help users acquire new insight from data.

59 BASIC BIOLOGICAL SCIENCES↗

A multiresolution adaptive wavelet method for nonlinear partial differential equations

We report the multiscale complexity of modern problems in computational science and engineering can prohibit the use of traditional numerical methods in multi-dimensional simulations. Therefore, novel algorithms are required in these situations to solve partial differential equations (PDEs) with features evolving on a wide range of spatial and temporal scales. To meet these challenges, we present a multiresolution wavelet algorithm to solve PDEs with significant data compression and explicit error control. We discretize in space by projecting fields and spatial derivative operators onto wavelet basis functions. We provide error estimates for the wavelet representation of fields and their derivatives. Then, our estimates are used to construct a sparse multiresolution discretization which guarantees the prescribed accuracy. Additionally, we embed a predictor-corrector procedure within the temporal integration to dynamically adapt the computational grid and maintain the accuracy of the solution of the PDE as it evolves. We present examples to highlight the accuracy and adaptivity of our approach.

97 MATHEMATICS AND COMPUTING↗

Regularizing INR with Diffusion Prior for Self-Supervised 3D Reconstruction OF Neutron Computed Tomography Data

Recently, generative diffusion priors have made huge strides as inverse problem solvers, including the ability to be adapted for inference on out-of-distribution data. Concurrently, implicit neural representations (INRs) have emerged as fast and lightweight inverse imaging solvers that are amenable to hybrid approaches that combine learned priors with traditional inverse problem formulations. In this paper, we present a diffusive computed tomography (CT) inversion framework for regularizing INRs called Diffusive INR (DINR), designed to enable high-quality reconstruction from sparse-view neutron CT. Pretrained purely on synthetic data, DINR is evaluated on simulated and experimentally obtained observations of concrete microstructures, where traditional reconstruction methods suffer substantial degradation when the number of views is reduced. Our approach delivers superior performance, reduces reconstruction artifacts, and achieves gains in PSNR and SSIM, enabling accurate micro-structural characterization even under extreme data limitations compared to state-of-the-art sparse-view reconstruction techniques.

Hossain, Maliha [ORNL]↗