Search NASA⌕ Search

SEARCH · Search NASA

Results for “sparse data set”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Random Variables with Moment-Matching Staircase Density Functions

This paper proposes a family of random variables for uncertainty modeling. The variables of interest have a bounded support set, and prescribed values for the first four moments. We present the feasibility conditions for the existence of any of such variables, and propose a class of variables that conforms to such constraints. This class is called staircase because the density of its members is a piecewise constant function. Convex optimization is used to calculate their distributions according to several optimality criteria, including maximal entropy and maximal log-likelihood. The flexibility and efficiency of staircases enable modeling phenomena having a possibly skewed and/or multimodal response at a low computational cost. Furthermore, we provide a means to account for the uncertainty in the distribution caused by estimating staircases from data. These ideas are illustrated by generating empirical staircase predictor models. We consider the case in which the predictor matches the sample moments exactly (a setting applicable to large datasets), as well as the case in which the predictor accounts for the sampling error in such moments (a setting applicable to sparse datasets). A predictor model for the dynamics of an aeroelastic airfoil subject to flutter instability is used as an example. The resulting predictor not only describes the system's response accurately, but also enables carrying out a risk analysis for safe flight.

Luis G. Crespo↗
DOE OSTI2021

Assimilation of citizen science data in snowpack modeling using a new snow data set: Community Snow Observations

A physically based snowpack evolution and redistribution model was used to test the effectiveness of assimilating crowd-sourced snow depth measurements collected by citizen scientists. The Community Snow Observations project gathers, stores, and distributes measurements of snow depth recorded by recreational users and snow professionals in high mountain environments. These citizen science measurements are valuable since they come from terrain that is relatively undersampled and can offer in situ snow information in locations where snow information is sparse or nonexistent. The present study investigates (1) the improvements to model performance when citizen science measurements are assimilated, and (2) the number of measurements necessary to obtain those improvements. Model performance is assessed by comparing time series of observed (snow pillow) and modeled snow water equivalent values, by comparing spatially distributed maps of observed (remotely sensed) and modeled snow depth, and by comparing fieldwork results from within the study area. The results demonstrate that few citizen science measurements are needed to obtain improvements in model performance, and these improvements are found in 62 % to 78 % of the ensemble simulations, depending on the model year. Model estimations of total water volume from a subregion of the study area also demonstrate improvements in accuracy after CSO measurements have been assimilated. These results suggest that even modest measurement efforts by citizen scientists have the potential to improve efforts to model snowpack processes in high mountain environments, with implications for water resource management and process-based snow modeling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Parallel beamlet dose calculation via beamlet contexts in a distributed multi‐GPU framework

Purpose Dose calculation is one of the most computationally intensive, yet essential tasks in the treatment planning process. With the recent interest in automatic beam orientation and arc trajectory optimization techniques, there is a great need for more efficient model‐based dose calculation algorithms that can accommodate hundreds to thousands of beam candidates at once. Foundational work has shown the translation of dose calculation algorithms to graphical processing units ( GPU s), lending to remarkable gains in processing efficiency. But these methods provide parallelization of dose for only a single beamlet, serializing the calculation of multiple beamlets and under‐utilizing the potential of modern GPU s. In this paper, the authors propose a framework enabling parallel computation of many beamlet doses using a novel beamlet context transformation and further embed this approach in a scalable network of multi‐ GPU computational nodes. Methods The proposed context‐based transformation separates beamlet‐local density and TERMA into distinct beamlet contexts that independently provide sufficient data for beamlet dose calculation. Beamlet contexts are arranged in a composite context array with dosimetric isolation, and the context array is subjected to a GPU collapsed‐cone convolution superposition procedure, producing the set of beamlet‐specific dose distributions in a single pass. Dose from each context is converted to a sparse representation for efficient storage and retrieval during treatment plan optimization. The context radius is a new parameter permitting flexibility between the speed and fidelity of the dose calculation process. A distributed manager‐worker architecture is constructed around the context‐based GPU dose calculation approach supporting an arbitrary number of worker nodes and resident GPU s. Phantom experiments were executed to verify the accuracy of the context‐based approach compared to Monte Carlo and a reference CPU ‐ CCCS implementation for single beamlets and broad beams composed by addition of beamlets. Dose for representative 4 π beam sets was calculated in lung and prostate cases to compare its efficiency with that of an existing beamlet‐sequential GPU ‐ CCCS implementation. Code profiling was also performed to evaluate the scalability of the framework across many networked GPU s. Results The dosimetric accuracy of the context‐based method displays <1.35% and 2.35% average error from the existing serialized CPU ‐ CCCS algorithm and Monte Carlo simulation for beamlet‐specific PDD s in water and slab phantoms, respectively. The context‐based method demonstrates substantial speedup of up to two orders of magnitude over the beamlet‐sequential GPU ‐ CCCS method in the tested configurations. The context‐based framework demonstrates near linear scaling in the number of distributed compute nodes and GPU s employed, indicating that it is flexible enough to meet the performance requirements of most users by simply increasing the hardware utilization. Conclusions The context‐based approach demonstrates a new expectation of performance for beamlet‐based dose calculation methods. This approach has been successful in accelerating the dose calculation process for very large‐scale treatment planning problems ‐ such as automatic 4 π IMRT beam orientation and VMAT arc trajectory selection, with hundreds of thousands of beamlets ‐ in clinically feasible timeframes. The flexibility of this framework makes it as a strong candidate for use in a variety of other very large‐scale treatment planning tasks and clinical workflows.

Neph, Ryan↗
DOE OSTI2023

A Bayesian model for multivariate discrete data using spatial and expert information with application to inferring building attributes

When modeling sparsely observed multivariate data, strong prior information elicited from experts can be used to bolster predictive accuracy and counteract sampling bias. Similarly, modeling autocorrelation in space can help make use of co-occurrence patterns present in many types of spatial data. To make use of both expert prior information and spatial structure, we propose a novel graphical model for a spatial Bayesian network developed specifically to address challenges in inferring the attributes of buildings from geographically sparse observational data. This model is implemented as the sum of a spatial multivariate Gaussian random field and a tabular conditional probability function in real-valued space prior to projection onto the probability simplex. This modeling form is especially suitable for the usage of prior information in the form of sets of atomic rules obtained from experts. To perform inference with missing data, we implement a Markov chain Monte Carlo scheme composed of alternating steps of Gibbs sampling of missing entries and Hamiltonian Monte Carlo for model parameters. A case study in building attribution is presented to highlight the advantages and limitations of this approach.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2003

Characterization of the Crab Pulsar's Timing Noise

We present a power spectral analysis of the Crab pulsar's timing noise, mainly using radio measurements from Jodrell Bank taken over the period 1982-1989, an interval bounded by sparse data sampling and a large glitch. The power spectral analysis is complicated by nonuniform data sampling and the presence of a steep red power spectrum that can distort power spectra measurement by causing severe power 'leakage'. We develop a simple windowing method for computing red noise power spectra of uniformly sampled data sets and test it on Monte Carlo generated sample realizations of red power-law noise. We generalize time-domain methods of generating power-law red noise with even integer spectral indices to the case of noninteger spectral indices. The Jodrell Bank pulse phase residuals are dense and smooth enough that an interpolation onto a uniform time series is possible. A windowed power spectrum is computed revealing a periodic or nearly periodic component with a period of 568 +/- 10 days and a l/f(exp 3) power-law noise component in pulse phase with a noise strength S(sub infinity)=(1.24 +/- 0.067) x 10(exp 16) cycles(exp 2)/sec(exp 2) over the analysis frequency range f=0.003- 0.1 cycles/day. This result deviates from past analyses which characterized the pulse phase timing residuals as either l/f(sub 4) power-law noise or a quasiperiodic process. The analysis was checked using the Deeter polynomial method of power spectrum estimation that was developed for the case of nonuniform sampling, but has lower spectral resolution. The timing noise is consistent with a torque noise spectrum rising with analysis frequency as f implying blue torque noise, a result not predicted by current models of pulsar timing noise. If the periodic or nearly periodic component is due to a binary companion, we find a mass function f(M) = (6.8 +/- 2.4) x 10(exp -16) solar mass and a companion mass, M(sub c) is greater than or equal to 3.2 solar mass assuming a Crab pulsar mass of 1.4 solar mass.

Scott, D. M.↗
DOE OSTI2022

Poisson hurdle model-based method for clustering microbiome features

Abstract Motivation High-throughput sequencing technologies have greatly facilitated microbiome research and have generated a large volume of microbiome data with the potential to answer key questions regarding microbiome assembly, structure and function. Cluster analysis aims to group features that behave similarly across treatments, and such grouping helps to highlight the functional relationships among features and may provide biological insights into microbiome networks. However, clustering microbiome data are challenging due to the sparsity and high dimensionality. Results We propose a model-based clustering method based on Poisson hurdle models for sparse microbiome count data. We describe an expectation–maximization algorithm and a modified version using simulated annealing to conduct the cluster analysis. Moreover, we provide algorithms for initialization and choosing the number of clusters. Simulation results demonstrate that our proposed methods provide better clustering results than alternative methods under a variety of settings. We also apply the proposed method to a sorghum rhizosphere microbiome dataset that results in interesting biological findings. Availability and implementation R package is freely available for download at https://cran.r-project.org/package=PHclust. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1990

Recognition of simple visual images using a sparse distributed memory: Some implementations and experiments

Previously, a method was described of representing a class of simple visual images so that they could be used with a Sparse Distributed Memory (SDM). Herein, two possible implementations are described of a SDM, for which these images, suitably encoded, will serve both as addresses to the memory and as data to be stored in the memory. A key feature of both implementations is that a pattern that is represented as an unordered set with a variable number of members can be used as an address to the memory. In the 1st model, an image is encoded as a 9072 bit string to be used as a read or write address; the bit string may also be used as data to be stored in the memory. Another representation, in which an image is encoded as a 256 bit string, may be used with either model as data to be stored in the memory, but not as an address. In the 2nd model, an image is not represented as a vector of fixed length to be used as an address. Instead, a rule is given for determining which memory locations are to be activated in response to an encoded image. This activation rule treats the pieces of an image as an unordered set. With this model, the memory can be simulated, based on a method of computing the approximate result of a read operation.

Jaeckel, Louis A.↗
DOE OSTI2021

Using AI to build a hydrobiogeochemical soil model

Soil water content is a function of inputs from precipitation and outputs via evaporation, transpiration, lateral flow, and vertical percolation, and is sensitive to biogeochemical processes. As such, soils serve as an ideal integrator of atmospheric, hydrological, and biogeochemical processes affecting the water cycle. In addition, soil water retention capacity, infiltration rates, and hydraulic conductivity can buffer or exacerbate the effects of extreme precipitation events (e.g., flooding, runoff, subsurface transport, erosion, greenhouse gas emissions) and mitigate the impact of droughts and heat waves on land systems (e.g., fire, crop failure). However, integrating water cycle measurements spanning different land atmosphere compartments across scales is a fundamental barrier for numerical model predictability. A significant challenge is that each domain (soil, hydrology, biology, and atmosphere) typically collects different sets of data at different temporal and spatial frequencies/scales, and even different dimensionalities (2D vs 3D). To implement soil as an integrator of the water cycle in land models, we suggest that novel machine learning (ML) tools can be developed to effectively simulate complex landscapes across various domains and scales, extended to regions with sparse or no data. The ultimate goals are to improve predictive understanding of land-atmosphere interactions and to extend the predictability of current Earth System Models (ESMs) through better integration of hydrological and biogeochemical data. We envision a framework in which: (1) ML-aided data reconstructions enable the merger of data sources into a unified geospatial product; (2) automated detection techniques are used to improve the knowledge of complex soil processes and interactions; and (3) this knowledge is leveraged and incorporated into models through AI-based emulators to distinctly connect the land and atmospheric compartments of the water cycle in models.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2014

Application of Spectral Analysis Techniques in the Intercomparison of Aerosol Data. Part II: Using Maximum Covariance Analysis to Effectively Compare Spatiotemporal Variability of Satellite and AERONET Measured Aerosol Optical Depth

Moderate Resolution Imaging SpectroRadiometer (MODIS) and Multi-angle Imaging Spectroradiomater (MISR) provide regular aerosol observations with global coverage. It is essential to examine the coherency between space- and ground-measured aerosol parameters in representing aerosol spatial and temporal variability, especially in the climate forcing and model validation context. In this paper, we introduce Maximum Covariance Analysis (MCA), also known as Singular Value Decomposition analysis as an effective way to compare correlated aerosol spatial and temporal patterns between satellite measurements and AERONET data. This technique not only successfully extracts the variability of major aerosol regimes but also allows the simultaneous examination of the aerosol variability both spatially and temporally. More importantly, it well accommodates the sparsely distributed AERONET data, for which other spectral decomposition methods, such as Principal Component Analysis, do not yield satisfactory results. The comparison shows overall good agreement between MODIS/MISR and AERONET AOD variability. The correlations between the first three modes of MCA results for both MODIS/AERONET and MISR/ AERONET are above 0.8 for the full data set and above 0.75 for the AOD anomaly data. The correlations between MODIS and MISR modes are also quite high (greater than 0.9). We also examine the extent of spatial agreement between satellite and AERONET AOD data at the selected stations. Some sites with disagreements in the MCA results, such as Kanpur, also have low spatial coherency. This should be associated partly with high AOD spatial variability and partly with uncertainties in satellite retrievals due to the seasonally varying aerosol types and surface properties.

aerosols↗
DOE OSTI2022

Prediction and compression of lattice QCD data using machine learning algorithms on quantum annealer

We present regression and compression algorithms for lattice QCD data utilizing the efficient binary optimization ability of quantum annealers. In the regression algorithm, we encode the correlation between the input and output variables into a sparse coding machine learning algorithm. The trained correlation pattern is used to predict lattice QCD observables of unseen lattice configurations from other observables measured on the lattice. In the compression algorithm, we define a mapping from lattice QCD data of floating-point numbers to the binary coefficients that closely reconstruct the input data from a set of basis vectors. Since the reconstruction is not exact, the mapping defines a lossy compression, but, a reasonably small number of binary coefficients are able to reconstruct the input vector of lattice QCD data with the reconstruction error much smaller than the statistical fluctuation. In both applications, we use D-Wave quantum annealers to solve the NP-hard binary optimization problems of the machine learning algorithms.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Semi-Analytical Hierarchical Bayesian Inference of Nonlinear Model Structure in Stochastic Dynamics: Applied to Compartmental Models of Infectious Diseases

A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

Analysis of airborne imaging spectrometer data for the Ruby Mountains, Montana, by use of absorption-band-depth images

Airborne Imaging Spectrometer-1 (AIS-1) data were obtained for an area of amphibolite grade metamorphic rocks that have moderate rangeland vegetation cover. Although rock exposures are sparse and patchy at this site, soils are visible through the vegetation and typically comprise 20 to 30 percent of the surface area. Channel averaged low band depth images for diagnostic soil rock absorption bands. Sets of three such images were combined to produce color composite band depth images. This relative simple approach did not require extensive calibration efforts and was effective for discerning a number of spectrally distinctive rocks and soils, including soils having high talc concentrations. The results show that the high spectral and spatial resolution of AIS-1 and future sensors hold considerable promise for mapping mineral variations in soil, even in moderately vegetated areas.

Brickey, David W.↗
DOE OSTI2020

Predicting System Response at Unmeasured Locations

Traditional techniques to derive dynamic specification for components have a great deal of uncertainty. One of the major sources of uncertainty is that the number of response measurements in the operational system environment is insufficient to determine the component motion. This inadequacy is due to logistical limitations for data recording in field testing and space limitations for accelerometers, strain gages and associated wiring. Available measurements are often some distance from the component and therefore do not represent component motion. Typical straight-line envelopes of these unrepresentative measurements guarantee an increase in the uncertainty. In this paper multiple methods are attempted to expand a sparse set of field test measurements on a system to responses of interest that cannot be measured in the field due to the limitations. Proof of concept is demonstrated on the Modal Analysis Test Vehicle (MATV). The responses of interest, known as “truth responses”, are measured in a system vibration environment along with an optimized sparse set of 30 field responses. Methods to expand the field responses to the truth responses are demonstrated by comparing the acceleration spectral density of the expanded response to the measured response. Two methods utilize a validated finite element model of the MATV. One is developed from purely experiment based frequency response functions of a laboratory pre-test. Finally, these approaches are designed to drastically reduce the uncertainty of the component in-service motion as a basis for developing specifications that are guaranteed to be conservative with a known (instead of unknown) conservatism.

42 ENGINEERING↗
NASA NTRS1993

The StarView intelligent query mechanism

The StarView interface is being developed to facilitate the retrieval of scientific and engineering data produced by the Hubble Space Telescope. While predefined screens in the interface can be used to specify many common requests, ad hoc requests require a dynamic query formulation capability. Unfortunately, logical level knowledge is too sparse to support this capability. In particular, essential formulation knowledge is lost when the domain of interest is mapped to a set of database relation schemas. Thus, a system known as QUICK has been developed that uses conceptual design knowledge to facilitate query formulation. By heuristically determining strongly associated objects at the conceptual level, QUICK is able to formulate semantically reasonable queries in response to high-level requests that specify only attributes of interest. Moreover, by exploiting constraint knowledge in the conceptual design, QUICK assures that queries are formulated quickly and will execute efficiently.

Semmel, R. D.↗
DOE OSTI2020

A Remote Sensing Technique to Upscale Methane Emission Flux in a Subtropical Peatland

Abstract Quantification of methane (CH 4 ) gas emission from peat is critical to understand CH 4 budget from natural wetlands under a climate warming scenario. Previous studies have focused on prediction and mapping of CH 4 emission flux using process‐based models, while application of statistical‐empirical models for upscaling spatially sparse in situ measurements is scarce. In this study, we developed an empirical remote sensing upscaling approach to estimate CH 4 emission flux in the Everglades using limited in situ point‐based CH 4 emission flux measurements and Landsat data during 2013–2018. We spatially and temporally linked in situ data with Landsat surface reflectance based on temporally composite data sets and developed an object‐based machine learning framework to model and map CH 4 emission flux. An ensemble analysis of two machine learning models, k ‐Nearest Neighbor ( k ‐NN) and Support Vector Machine (SVM), shows that the upscaling approach is promising for predicting CH 4 emission flux with a R 2 of 0.65 and 0.87 based on a fivefold cross‐validation for a dry season and wet season estimation, respectively. We generated emission flux map products that successfully revealed the spatial and temporal heterogeneity of CH 4 emission within the dominant freshwater marsh ecosystem in the Everglades. We conclude that Landsat is promising for upscaling and monitoring CH 4 emission flux and reducing the uncertainty in emission estimates from wetlands.

Zhang, Caiyun↗
DOE OSTI2026

Anomaly Detection for Online Monitoring of Thermocouple Sensors in the Advanced Test Reactor

This study explores data-driven anomaly detection methods to analyze sensor fail- ures in the Advanced Gas Reactor (AGR) nuclear fuel irradiation experiments. Specifically, we examine failures of thermocouples (TCs), which are critical for mon- itoring and controlling in-reactor temperatures during operation. Failures were pri- marily observed during abrupt power transitions and manifested as sensor drop-outs, drifts, or unexplained behavior. We applied three time-series analysis techniques— rolling mean smoothing, matrix profile, and vector auto-regression (VAR)—to de- tect anomalies in TC data prior to failure events. The rolling mean method effec- tively highlighted deviations aligned with reported failures, while the matrix profile provided partial early warning but sometimes flagged normal fluctuations during power-down periods. VAR shows potential in capturing multivariate dependencies but requires further calibration. A rare case of TC drift was also documented, which did not result in failure, underscoring the challenge of building predictive models with sparse positive examples. Our findings demonstrate that traditional statistical tools can aid anomaly detection but have limited predictive power without richer training data. We propose future directions including synthetic data generation, real- time surrogate modeling, and multi-modal feature integration. This work provides a foundation for applying robust anomaly detection frameworks to mission-critical sensor systems in experimental settings.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

AIVT: Inference of turbulent thermal convection from measured 3D velocity data by physics-informed Kolmogorov-Arnold networks

We propose the artificial intelligence velocimetry-thermometry (AIVT) method to reconstruct a continuous and differentiable representation of the temperature and velocity in turbulent convection from measured three-dimensional (3D) velocity data. AIVT is based on physics-informed Kolmogorov-Arnold networks and trained by optimizing a loss function that minimizes residuals of the velocity data, boundary conditions, and governing equations. We apply AIVT to a set of simultaneously measured 3D temperature and velocity data of Rayleigh-Bénard convection, obtained by combining particle image thermometry and Lagrangian particle tracking. This enables us to directly compare machine learning results to true volumetric, simultaneous temperature and velocity measurements. We demonstrate that AIVT can reconstruct and infer continuous, instantaneous velocity and temperature fields and their gradients from sparse experimental data at a high resolution, providing an additional approach for understanding thermal turbulence.

Science & Technology - Other Topics↗
DOE OSTI2019

Software Defined Architectures for Data Analytics

Data analytics applications increasingly are complex workflows composed of phases with very different program behaviors (e.g., graph algorithms and machine learning, algorithms operating on sparse an dense data structures, etc). To reach the levels of efficiency required to process these workflows in real time, upcoming architectures will need to leverage even more workload specialization. If, at one end, we may find even more heterogenous processors composed by a myriad of specialized processing elements, at the other end we may see novel reconfigurable architectures, composed of sets of functional units and memories interconnected with (re)configurable on-chip networks, able to adapt dynamically to adapt the workload characteristics. Field Programmable Gate Arrays are more and more used for accelerating various workloads and, in particular, inferencing in machine learning, providing higher efficiency than other solutions. However, their fine grained nature still leads to issues for the design software and still makes dynamic reconfiguration impractical. Future, more coarse-grained architectures could offer the features to execute diverse workloads at high efficiency while providing better reconfiguration mechanisms for dynamic adaptability. Nevertheless, we argue that the challenges for reconfigurable computing remain in the software. In this position paper, we describe a possible toolchain for reconfigurable architectures targeted at data analytics.

FPGA, Dataflow, Reconfigurable Architectures, Soft↗