Search NASA⌕ Search

SEARCH · Search NASA

Results for “count data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Analysis of overlapping count data

Counts of a specific characteristic were obtained within regions defined on an object that was manufactured in a proprietary setting. The count regions were altered during production and resulted in misaligned or overlapping count data. A closed-formula maximum likelihood estimator (MLE) of the new region means is derived using all of the available count data and an independent Poisson model. The MLE is shown to be preferable to estimators constructed using generalized linear models for the overlapping data setting. This closed-form estimator extends to over-dispersed overlapping count data as the quasi-MLE and also performs well with correlated overlapping count data. Standard errors for the estimator are approximated and are validated with a simulation study. Additionally, the methods are extended to overlapping multinomial data. Illustrative examples of the methods are provided throughout the paper and are reproducible with the supplemental R code. Additionally, proofs of the paper’s results are also included in the supplemental material.

97 MATHEMATICS AND COMPUTING↗

Poisson Log-Normal Process for Count Data Prediction

Modeling count data is important in physics and other scientific disciplines, where measurements often involve discrete, non-negative quantities such as photon or neutrino detection events. Traditional parametric approaches can be trained to generate integer-count predictions but may struggle with capturing complex, non-linear dependencies often observed in the data. Gaussian process (GP) regression provides a robust non-parametric alternative to modeling continuous data; however, it cannot generate integer outputs. We propose the Poisson Log-Normal (PoLoN) process, a framework that employs GP to model Poisson log-rates. As in GP regression, our approach relies on the correlations between data points captured via GP kernel structure rather than explicit functional parameterizations. We demonstrate that the PoLoN predictive distribution is Poisson-LogNormal and provide an algorithm for optimizing kernel hyperparameters. Furthermore, we adapt the PoLoN approach to the problem of detecting weak localized signals superimposed on a smoothly varying background - a task of considerable interest in many areas of science and engineering. Our framework allows us to predict the strength, location and width of the detected signals. We evaluate PoLoN's performance using both synthetic and real-world datasets, including the open dataset from CERN which was used to detect the Higgs boson at the Large Hadron Collider. Our results indicate that the PoLoN process can be used as a non-parametric alternative for analyzing, predicting, and extracting signals from integer-valued data.

Saha, Anushka [Rutgers U., Piscataway]↗

Bayesian Blocks: A New Method to Analyze Photon Counting Data

A Bayesian analysis of photon-counting data leads to a new time-domain algorithm for detecting localized structures (bursts), revealing pulse shapes, and generally characterizing intensity variations. The raw counting data -- time-tag events (TTE), time-to-spill (TTS) data, or binned counts -- is converted to a maximum likelihood segmentation of the observation into time intervals during which the photon arrival rate is perceptibly constant -- i.e. has a fixed intensity without statistically significant variations. The resulting structures, Bayesian Blocks, can be thought of as bins with arbitrary spacing determined by the data. The method itself sets no lower limit to the time scale on which variability can be detected. We have applied the method to RXTE data on Cyg X-1, yielding information on this source's short-time-scale variability.

Scargle, Jeffrey D.↗

Sequential data assimilation for single-molecule FRET photon-counting data

Data assimilation is a statistical method designed to improve the quality of numerical simulations in combination with real observations. Here, we develop a sequential data assimilation method that incorporates one-dimensional time-series data of smFRET (single-molecule Förster resonance energy transfer) photon-counting into conformational ensembles of biomolecules derived from “replicated” molecular dynamics (MD) simulations. A particle filter using a large number of “replicated” MD simulations with a likelihood function for smFRET photon-counting data is employed to screen the conformational ensembles that match the experimental data. We examine the performance of the method using emulated smFRET data and coarse-grained (CG) MD simulations of a dye-labeled polyproline-20. The method estimates the dynamics of the end-to-end distance from smFRET data as well as revealing that of latent conformational variables. The particle filter is also able to correct model parameter dependence in CG MD simulations. We discuss the applicability of the method to real experimental data for conformational dynamics of biomolecules.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Bayesian nonparametric analysis for zero-inflated multivariate count data with application to microbiome study

High-throughput sequencing technology has enabled researchers to profile microbial communities from a variety of environments, but analysis of multivariate taxon count data remains challenging. Here, we develop a Bayesian nonparametric (BNP) regression model with zero inflation to analyse multivariate count data from microbiome studies. A BNP approach flexibly models microbial associations with covariates, such as environmental factors and clinical characteristics. The model produces estimates for probability distributions which relate microbial diversity and differential abundance to covariates, and facilitates community comparisons beyond those provided by simple statistical tests. We compare the model to simpler models and popular alternatives in simulation studies, showing, in addition to these additional community-level insights, it yields superior parameter estimates and model fit in various settings. The model's utility is demonstrated by applying it to a chronic wound microbiome data set and a Human Microbiome Project data set, where it is used to compare microbial communities present in different environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Zero-truncated Poisson regression for sparse multiway count data corrupted by false zeros

Abstract We propose a novel statistical inference methodology for multiway count data that is corrupted by false zeros that are indistinguishable from true zero counts. Our approach consists of zero-truncating the Poisson distribution to neglect all zero values. This simple truncated approach dispenses with the need to distinguish between true and false zero counts and reduces the amount of data to be processed. Inference is accomplished via tensor completion that imposes low-rank tensor structure on the Poisson parameter space. Our main result shows that an $N$-way rank-$R$ parametric tensor $\boldsymbol{\mathscr{M}}\in (0,\infty )^{I\times \cdots \times I}$ generating Poisson observations can be accurately estimated by zero-truncated Poisson regression from approximately $IR^2\log _2^2(I)$ non-zero counts under the nonnegative canonical polyadic decomposition. Our result also quantifies the error made by zero-truncating the Poisson distribution when the parameter is uniformly bounded from below. Therefore, under a low-rank multiparameter model, we propose an implementable approach guaranteed to achieve accurate regression in under-determined scenarios with substantial corruption by false zeros. Several numerical experiments are presented to explore the theoretical results.

97 MATHEMATICS AND COMPUTING↗

Analyzing count data with measurement error

In this article, we analyze observed count data such as the number of defects in a steel product where the observed counts are the true counts measured with errors. We account for the measurement error by using a measurement error model based on a latent lognormal (LLN) distribution. We consider making inference about a single population (e.g., from samples of a production lot) and a regression model (e.g., from runs of a designed experiment), where the measurement system properties are known, that is, the parameters of the LLN distribution are known. Then, we consider simultaneous inference for the single population and regression model as well as the measurement system. We demonstrate the proposed methodology with both simulated and real observed counts.

42 ENGINEERING↗

Performance of a Discrete Wavelet Transform for Compressing Plasma Count Data and its Application to the Fast Plasma Investigation on NASA's Magnetospheric Multiscale Mission

Plasma measurements in space are becoming increasingly faster, higher resolution, and distributed over multiple instruments. As raw data generation rates can exceed available data transfer bandwidth, data compression is becoming a critical design component. Data compression has been a staple of imaging instruments for years, but only recently have plasma measurement designers become interested in high performance data compression. Missions will often use a simple lossless compression technique yielding compression ratios of approximately 2:1, however future missions may require compression ratios upwards of 10:1. This study aims to explore how a Discrete Wavelet Transform combined with a Bit Plane Encoder (DWT/BPE), implemented via a CCSDS standard, can be used effectively to compress count information common to plasma measurements to high compression ratios while maintaining little or no compression error. The compression ASIC used for the Fast Plasma Investigation (FPI) on board the Magnetospheric Multiscale mission (MMS) is used for this study. Plasma count data from multiple sources is examined: resampled data from previous missions, randomly generated data from distribution functions, and simulations of expected regimes. These are run through the compression routines with various parameters to yield the greatest possible compression ratio while maintaining little or no error, the latter indicates that fully lossless compression is obtained. Finally, recommendations are made for future missions as to what can be achieved when compressing plasma count data and how best to do so.

Particles↗

Zero-Truncated Poisson Tensor Decomposition for Sparse Count Data

We propose a novel statistical inference paradigm for zero-inflated multiway count data that dispenses with the need to distinguish between true and false zero counts. Our approach ignores all zero entries and applies zero-truncated Poisson regression on the positive counts. Inference is accomplished via tensor completion that imposes low-rank structure on the Poisson parameter space. Our main result shows that an $\textit{N}$-way rank-R parametric tensor 𝓜 ϵ (0, ∞) $I$Χ∙∙∙Χ$I$ generating Poisson observations can be accurately estimated from approximately $IR^2 \text{log}^2_2(I)$ non-zero counts for a nonnegative canonical polyadic decomposition. Several numerical experiments are presented demonstrating that our zero-truncated paradigm is comparable to the ideal scenario where the locations of false zero counts are known $\textit{a priori}$.

97 MATHEMATICS AND COMPUTING↗

Tensor Decompositions for Count Data that Leverage Stochastic and Deterministic Optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the global maximum likelihood estimator from local minima. Simultaneously, a recent trend in theoretical computer science and numerical linear algebra leverages randomization to solve very large, hard problems. The typical approach is to use randomization for a fast approximation and determinism for refinement to yield effective algorithms with theoretical guarantees. Two popular algorithms for Poisson CPD reflect that emergent dichotomy: CP Alternating Poisson Regression is a deterministic algorithm and Generalized Canonical Polyadic decomposition makes use of stochastic algorithms in several variants. This work extends recent work to develop two new methods that leverage randomized and deterministic algorithms for improved accuracy and performance.

97 MATHEMATICS AND COMPUTING↗

Tensor decompositions for count data that leverage stochastic and deterministic optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the maximum likelihood estimator (MLE) of the Poisson CPD model. Here, this work presents two new algorithms that extend state-of-the-art local methods for Poisson CPD. Hybrid GCP-CPAPR combines Generalized Canonical Decomposition (GCP) with stochastic optimization and CP Alternating Poisson Regression (CPAPR), a deterministic algorithm, to increase the probability of converging to the MLE over either method used alone. Restarted CPAPR with SVDrop uses a heuristic based on the singular values of the CPD model unfoldings to identify convergence toward optimizers that are not the MLE and restarts within the feasible domain of the optimization problem, thus reducing overall computational cost when using a multi-start strategy. We provide empirical evidence that indicates our approaches outperform existing methods with respect to converging to the Poisson CPD MLE.

CPAPR↗

Performance of a Space-Based Wavelet Compressor for Plasma Count Data on the MMS Fast Plasma Investigation

Data compression has been a staple of imaging instruments for years. Recently, plasma measurements have utilized compression with relatively low compression ratios. The Fast Plasma Investigation (FPI) on board the Magnetospheric Multiscale (MMS) mission generates data roughly 100 times faster than previous plasma instruments, requiring a higher compression ratio to fit within the telemetry allocation. This study investigates the performance of a space-based compression standard employing a Discrete Wavelet Transform and a Bit Plane Encoder (DWT/BPE) in compressing FPI plasma count data. Data from the first 6 months of FPI operation are analyzed to explore the error modes evident in the data and how to adapt to them. While approximately half of the Dual Electron Spectrometer (DES) maps had some level of loss, it was found that there is little effect on the plasma moments and that errors present in individual sky maps are typically minor. The majority of Dual Ion Spectrometer burst sky maps compressed in a lossless fashion, with no error introduced during compression. Because of induced compression error, the size limit for DES burst images has been increased for Phase 1B. Additionally, it was found that the floating point compression mode yielded better results when images have significant compression error, leading to floating point mode being used for the fast survey mode of operation for Phase 1B. Despite the suggested tweaks, it was found that wavelet-based compression, and a DWT/BPE algorithm in particular, is highly suitable to data compression for plasma measurement instruments and can be recommended for future missions.

plasma↗

Predicting plug loads with occupant count data through a deep learning approach

Predictive control has gained increasing attention for its ability to reduce energy consumption and improve occupant comfort in buildings. The plug loads prediction is a key component for the predictive building controls, as plug loads is a major source of internal heat gains in buildings. This study proposed a novel method to apply the Long-Short-Term-Memory (LSTM) Network, a special form of Recurrent Neural Network, to predict plug loads. The occupant count and the time have been confirmed to drive the plug load profile and thus selected as the features for the plug load prediction. The LSTM network was trained and tested with ground truth occupant count data collected from a real office building in Berkeley, California. Results from the LSTM network markedly improve the prediction accuracy compared with traditional linear regression methods and the classical Artificial Neural Network. 95% of 1-hour predictions from LSTM network are within ±1 kW of the actual plug loads, given the average plug loads during the office hour is 8.6 kW. The CV(RMSE) of the predicted plug load is 11% for the next hour, and 20% for the next eight hours. Lastly, we compared four prediction approaches with the office building we monitored: LSTM vs. ARIMA, with occupant counts vs. without occupant counts. It was found, the prediction error of the LSTM approach is around 4% less than the ARIMA approach. Using occupant counts as an exogenous input could further reduce the prediction error by 5%-6%. The findings of this paper could shed light on the plug load prediction for building control optimizations such as model-predictive control.

Occupant count↗

The equivalence of three techniques for estimating ground reflectance from LANDSAT digital count data

The equivalence of three separate investigations that related LANDSAT digital count (DC) to ground measured reflectance (R) was demonstrated. One investigator related DC data to the cosZ, where Z is the solar zenith angle, for surfaces of constant R. The second investigator corrected the DC data to the solar zenith angle of 39 degrees before relating to surface R. Both of these investigators used LANDSAT 1 and 2 data from overpass dates 1972 through 1977. A third investigator calculated the relation between DC and R based on atmospheric radiative transfer theory. The equation coefficients obtained from these three investigators for all four LANDSAT MSS bands were shown to be equivalent although differences in ground reflectance measurement procedures have created coefficient variations among the three investigations. These relations should be useful for testing atmospheric radiative transfer theory.

Richardson, A. J.↗

Improved Quantitative Dynamic Range of Time-of-Flight Mass Spectrometry by Simultaneously Waveform-Averaging and Ion-Counting Data Acquisition

Two different types of data acquisition methods, “averaging mode” and “ion-counting mode”, have been used in a time-of-flight (TOF) mass spectrometry. The most common method is an averaging mode that sums waveform signals obtained from each flight cycle. While it is possible to process many ions arriving at the same TOF in one flight cycle, low-abundance ions are difficult to measure because ion signals are overwhelmed by noises from the detection system. An ion-counting mode is suitable for the detection of such low-concentration ions, but counting loss occurs when two or more ions arrive at the detector within the dead time of the acquisition system. In this study, we introduce a technique that combines two methods to measure target ions with a high concentration difference, i.e., averaging mode and ion-counting mode are used simultaneously for high abundant and trace ions, respectively. By processing waveforms concurrently during data acquisition, one can choose to analyze either or both types of data to achieve a highly quantitative mass spectrum over a wide range of sample concentrations. The result of the argon isotope analysis shows that this method provides a more accurate determination of the isotope ratio compared to averaging mode alone at one-twentieth of the analysis time required by ion-counting alone. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Marine Corps Base Camp Lejeune Avian Point Count Data 2011

Critical military training and testing on lands along the nation’s coastal and estuarine shorelines are increasingly placed at risk because of encroachment pressures in surrounding areas, impairments due to other anthropogenic disturbances, and changes in climate and sea level. The U.S. Department of Defense (DoD) intends to enhance and sustain its training and testing assets and also optimize its stewardship of natural resources through the development and application of an ecosystem-based management approach on DoD installations. To accomplish this goal, particularly for installations in estuarine/coastal environments, the Strategic Environmental Research and Development Program (SERDP) launched the Defense Coastal/Estuarine Research Program (DCERP) as a 10-year effort at Marine Corps Base Camp Lejeune (MCBCL) in North Carolina. The results of the second 5 years of the program (DCERP2) are presented in the DCERP2 Final Report.There were four overarching objectives of DCERP2. The first objective was to understand the effects of climate change impacts, including warming temperatures, variability in the hydrological cycle, storm events, and sea level rise on the coastal ecosystems at MCBCL from observations and measurements made over the 10-year program. The second objective was to understand the carbon cycle of the coastal and terrestrial ecosystems at MCBCL through a highly integrated sampling program. The third objective was to develop models, tools, and indicators to evaluate current and projected future ecosystem state changes and translate scientific findings into actionable information for installation managers. The last objective was to recommend adaptive management strategies to sustain ecosystem natural resources within the context of an active military installation.The census points were sampled in 2009 and again in 2010 and 2011 (2009 and 2010 data are in MARDIS). We used point counts to census birds following a methodology from Ralph et al. (1995) by Kirkpatrick et al. (2006). Surveys were conducted between April 10 and July 10 starting at sunrise and going no later than 10 a.m. (or 4 hours after dawn), on days without precipitation and with wind speeds less than 12 km/h (Ralph et al., 1995; Rosenstock et al., 2002). We recorded temperature (°C), wind speed (km/h), cloud cover, and background noise level. After initially arriving at a point, we waited for a 1-minute silent period, and then recorded any birds seen or heard for an 8-minute period (Farnsworth et 14-8 al., 2002; Rosenstock et al., 2002). We estimated the distance to each bird, using a Rangefinder whenever necessary (Buckland, 2001; Rosenstock et al., 2002).

54 ENVIRONMENTAL SCIENCES↗