Search NASA⌕ Search

SEARCH · Search NASA

Results for “data skew”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1983

Redundant asynchronous microprocessor system for fault tolerant flight control and navigation

Unlike their synchronized counterparts, redundant channels in an asynchronous flight system can, under no-fault conditions, exhibit cross-channel data disparities. Sources of these errors are examined in terms of the general, individual functions of the flight control and navigation application in the asynchronous digital environment. The effects of asynchronism on trajectory programmers, dynamic control algorithms and data reconstruction processes are examined in terms of data skews, data latencies, and clock rate uncertainties. An example is presented in which time corrections are applied to reduce the data disparities. Practical limitations of the approach of the example are discussed.

Dunn, W. R.↗
NASA NTRS1979

Program documentation for the data noising program

Problems in the Space Shuttle aerodynamic data that are expected to occur in the actual Orbital Flight Test (OFT) flight data are introduced. The user selects from 1 to 100 channels to be operated on by this program. For each selected channel number, the user may introduce up to three problems in the channel data: time skew, data noise in the frequency domain, and data-quantization. The output data is the input data plus the data for each selected channel number with NOISE selected by the program user.

Nguyen, M. T.↗
DOE OSTI2025

Toward equitable environmental exposure modeling through convergence of data, open, and citizen sciences: an example of air pollution exposure modeling amidst increasing wildfire smoke

Exposure modeling is critical in environmental epidemiology and human health but may face challenges (e.g., skewed data, unequal error, context-insensitive validation, and computational demands). Modeling decisions reflect the intended use of the models and the values that modelers prioritize. We aimed to provide a conceptual framework and machine learning (ML) modeling protocols that address these issues. With 500m-gridded hourly PM 2.5 and O 3 levels in Illinois before, during, and after the 2023 Canadian wildfire season as a motivating example, we conducted modeling experiments to evaluate modeling methods, guided by three domains we propose based on theories of science: 1) Data Diversity, leveraging open and citizen science data to enhance inclusivity, parsimony, and representativeness; 2) Equitable Accuracy, ensuring fairly distributed uncertainties across subpopulations; and 3) Sustainable Modeling, balancing accuracy with reducing computational demands to promote accessibility for under-resourced researchers. Here, we found that ML with publicly available data can achieve high accuracy. Depending on methods, performance may vary substantially, even with identical input data. Large but skewed data may reduce performance. Misuse of cross-validation protocols can underestimate prediction error; although we observed R 2 s of ∼98 %, the modeled estimates varied significantly, indicating the need for careful model validation. By using new modeling protocols including representativeness-considered training and validation data and a new loss function, we achieved high agreement between estimates and ground-based measurements (e.g., R 2 = ∼90 % for PM 2.5 ; ∼80 % for O 3 ), equally distributed errors across sociodemographic strata and urban–rural divides, and reduction in computation time—from several weeks or months to a few days.

Exposure assessment↗
DOE OSTI2019

Distributions of the Mean for Lognormally Distributed Data - 19427

In environmental sciences, a distribution of the mean is often needed for right-skewed data. For example, risk assessments are often based on distributions of the mean to obtain upper confidence limits (UCLs) for analyte concentrations, and, in probabilistic performance assessment (PPA) models, parameters are given distributions rather than deterministic values, and they are usually given distributions of their mean rather than the population distributions to account for the large spatial and temporal domains to which they are applied. How this distribution of the mean is developed depends on the characteristics of the parameter's full distribution and how the parameter relates to its use. In this presentation, I will discuss the implications of a right skew in distribution development. Parameters may vary over orders of magnitude and/or may be right-skewed. The material property of saturated hydraulic conductivity, Ks, is a prime example of such a parameter. Right-skewed parameters are often assumed to follow lognormal distributions. If X follows a lognormal distribution, then log(X) follows a normal distribution. For some right-skewed parameters, practitioners go back and forth between the two forms of the data depending on their needs. For example, it may be preferred to use the untransformed version of X in a model or equation, but it is easier to visualize the parameter values on a logarithmic scale. However, distribution development for the average value of the parameter will differ depending on whether X or log(X) is modeled. This difference is due to Jensen's Inequality, a theorem which states that the expected value of a concave function of a variable is less than or equal to the same function of the expected value of the variable; a logarithm is a concave function. In other words, the mean of log(X) is always less than or equal to the log of the mean of X and therefore the mean estimated from a distribution based on log(X) is less than or equal to the mean estimated from a distribution based on X. If a parameter is naturally considered on a log scale by subject matter experts, then distributions developed for the mean of log(X) may be more consistent with the use of the parameters in the literature than distributions developed for X. The statistical decisions of distribution development will be discussed within the context of right-skewed, lognormal data. These issues include method-of-moments versus maximum likelihood estimation of the distribution parameters and the use of bootstrapping to aid in the visualization of a distribution of the mean. The details surrounding Jensen's Inequality will be explained and visualized within the context of distribution development of a right-skewed parameter such as saturated hydraulic conductivity. (authors)

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

DeStager: feature guided in-situ data management in distributed deep memory hierarchies

In-situ analytics have been increasingly adopted by leadership scientific applications to gain fast insights into massive output data of simulations. With the current practice, systems buffer the output data in DRAM for analytics processing, constraining it to DRAM capacity un-used by the simulation. The rapid growth of data size requires alternative approaches to accommodating data-rich analytics, such as using solid-state disks to increase effective memory capacity. For this purpose, this paper explores software solutions for exploring the deep memory hierarchies expected on future high-end machines. Leveraging the fact that many analytics are sensitive to data features (regions-of-interest) hidden in the data being processed, the approach incorporates the knowledge of the data features into in-situ data management. It uses adaptive index creation/refinement to reduce the overhead of index management. In addition, it uses data features to predict data skew and improve load balance through controlling data distribution and placement on distributed staging servers. The experimental results show that such feature-guided optimizations achieve substantial improvements over state-of-the-art approaches for managing output data in-situ.

Computer Science↗
DOE OSTI2026

Resource-Adaptive Federated Text Generation with Differential Privacy

In cross-silo federated learning (FL), sensitive text datasets remain confined to local organizations due to privacy regulations, making repeated training for each downstream task both communication-intensive and privacy-demanding. A promising alternative is to generate differentially private (DP) synthetic datasets that approximate the global distribution and can be reused across tasks. However, pretrained large language models (LLMs) often fail under domain shift, and federated finetuning is hindered by computational heterogeneity: only resource-rich clients can update the model, while weaker clients are excluded, amplifying data skew and the adverse effects of DP noise. We propose a flexible participation framework that adapts to client capacities. Strong clients perform DP federated finetuning, while weak clients contribute through a lightweight DP voting mechanism that refines synthetic text. To ensure the synthetic data mirrors the global dataset, we apply control codes (e.g., labels, topics, metadata) that represent each client’s data proportions and constrain voting to semantically coherent subsets. This two-phase approach requires only a single round of communication for weak clients and integrates contributions from all participants. Experiments show that our framework improves distribution alignment and downstream robustness under DP and heterogeneity.

Wang, Jiayi [ORNL]↗
DOE OSTI2024

On the measurement of shape: With applications to lunar regolith

With the renewed commitment from NASA and other commercial entities for a presence on the Moon, the importance of understanding the characteristics of lunar regolith and how to utilize it have become the target of increasing scrutiny. Much of what is known about lunar regolith was collected during and immediately after the Apollo program, however, analytical techniques and instrumentation have advanced in leaps and bounds in the subsequent decades. Specifically, dynamic image analysis systems have advanced to the point that millions of particles can have morphological characteristics automatically determined in relatively short time frames. Particle morphological data was collected on several lunar samples and a pair of widely used regolith simulants to ascertain the accuracy of these simulants and to explore statistical analysis methods of these large datasets. It is found that these morphology datasets can vary widely depending on the particle size of the particles, and simple averaging of the data skews the results heavily towards the numerically abundant size ranges, the fines. Different reporting methods are suggested to ameliorate these problems. When applied to the lunar regolith, the particles are noted to be less morphologically complex than initially suspected. Compared to the lunar material, the simulants are found to contain some more morphological variability and angular grains. Such difference is likely due to the wildly different comminution processes that these different powder systems are subjected to.

2D shape↗
NASA NTRS1991

The dissipation-range spectrum and the velocity-derivative skewness in turbulent flows

A dynamical equation for the energy dissipation rate is used together with a model spectrum to predict the velocity derivative skewness at a high Reynolds number. The objective is to determine the best choice of the exponent, m, so that the resulting model spectrum can be used to estimate statistical properties of the fine scales of turbulence. Skewness data suggest that m = 2 is the best model for laboratory-scale flows and m = 1 is a poor model for a broad range of Reynolds numbers.

Smith, L. M.↗
NASA NTRS2012

Development of Thermal Infrared Sensor to Supplement Operational Land Imager

The thermal infrared sensor (TIRS) is a quantum well infrared photodetector (QWIP)-based instrument intended to supplement the Operational Land Imager (OLI) for the Landsat Data Continuity Mission (LDCM). The TIRS instrument is a far-infrared imager operating in the pushbroom mode with two IR channels: 10.8 and 12 m. The focal plane will contain three 640 ~512 QWIP arrays mounted onto a silicon substrate. The readout integrated circuit (ROIC) addresses each pixel on the QWIP arrays and reads out the pixel value (signal). The ROIC is controlled by the focal plane electronics (FPE) by means of clock signals and bias voltage value. The means of how the FPE is designed to control and interact with the TIRS focal plane assembly (FPA) is the basis for this work. The technology developed under the FPE is for the TIRS focal plane assembly (FPA). The FPE must interact with the FPA to command and control the FPA, extract analog signals from the FPA, and then convert the analog signals to digital format and send them via a serial link (USB) to a computer. The FPE accomplishes the described functions by converting electrical power from generic power supplies to the required bias power that is needed by the FPA. The FPE also generates digital clocking signals and shifts the typical transistor-to-transistor logic (TTL) to }5 V required by the FPA. The FPE also uses an application- specific integrated circuit (ASIC) named System Image, Digitizing, Enhancing, Controlling, And Retrieving (SIDECAR) from Teledyne Corp. to generate the clocking patterns commanded by the user. The uniqueness of the FPE for TIRS lies in that the TIRS FPA has three QWIP detector arrays, and all three detector arrays must be in synchronization while in operation. This is to avoid data skewing while observing Earth flying in space. The observing scenario may be customized by uploading new control software to the SIDECAR.

Shu, Peter↗
NASA NTRS2023

What’s That Supposed to Mean? Capturing Micro-Behaviors in Teams

Future long-duration space exploration (LDSE) crews will require extensive coordination, cooperation, and team functioning as they face a myriad of challenges rooted in both taskwork and teamwork (Bell et al., 2015; Landon et al., 2018). While exposed to extreme conditions, crew members must navigate living and working together in prolonged confinement. Moreover, astronaut teams are becoming increasingly diverse, introducing significant variability in team composition. This increasing diversity, alongside traditional constraints of LDSE, introduces additional challenges into effective team functioning. To date, most methods for capturing team functioning rely on self-report measures. Such measures are prone to several limitations, including but not limited to social desirability bias, halo effect, and leniency effects (Trull & Ebner-Priemer, 2013), which skew data and limit nuanced understandings of phenomena at play. Self-report measures broadly capture team functioning, lending the nature of such methods to identifying underlying “macro”-behaviors (i.e., behaviors that are long-standing and last over time). However, team functioning is far more complex than a series of macro-behaviors, rendering reliance on self-report data deficient for accurate measurement. Recent research demonstrates the potential of alternative methods for capturing team functioning, such as speech and physiological data (Chaffin et al., 2017; Murray & Oertel, 2018). Consequently, these methods are more suitable for capturing micro-behaviors: brief, often unconscious expressions that affect the extent to which an individual feels included by others around them (Paletz et al., 2013). Micro-behaviors can be further classified into microaggressions (i.e., subtle, negative exchanges; Keller & Galgay, 2010) or micro-affirmations (i.e., subtle, positive exchanges; Kyte et al. 2020), both of which influence team functioning. Due to the subtle nature of micro-behaviors, contextual factors have a significant impact when determining if it is aggressive or affirmative. Additionally, several iterations of microbehaviors can have lingering effects on team interactions. For example, the use of “mm-hmm” by a crew member can function as both a micro-affirmation and micro-aggression. Specifically, it can be indication of active listening (i.e., micro-affirmation) or as an expression of annoyance (i.e., aggression) depending on the context in which it occurs. Auditory features (e.g., tone, frequency) can help delineate between the two forms; however, the contextual factors (e.g., previous interactions between team members, crew demographics) add a layer of complexity that render auditory features alone as insufficient to capture micro-behaviors. Consequently, this paper seeks to provide a novel approach in which multi-modal data (i.e., auditory features and contextual features) are used in a random-forest model to better identify distinguishing characteristics between micro-affirmations and micro-aggressions. In turn, detected micro-behaviors are used to predict team performance, thereby demonstrating the value of capturing micro-behaviors as supplemental data to macro-behaviors.

Sydney R. Begerowski↗
NASA NTRS2023

What’s That Supposed to Mean? Capturing Micro-Behaviors in Teams

Future long-duration space exploration (LDSE) crews will require extensive coordination, cooperation, and team functioning as they face a myriad of challenges rooted in both taskwork and teamwork (Bell et al., 2015; Landon et al., 2018). While exposed to extreme conditions, crew members must navigate living and working together in prolonged confinement. Moreover, astronaut teams are becoming increasingly diverse, introducing significant variability in team composition. This increasing diversity, alongside traditional constraints of LDSE, introduces additional challenges into effective team functioning. To date, most methods for capturing team functioning rely on self-report measures. Such measures are prone to several limitations, including but not limited to social desirability bias, halo effect, and leniency effects (Trull & Ebner-Priemer, 2013), which skew data and limit nuanced understandings of phenomena at play. Self-report measures broadly capture team functioning, lending the nature of such methods to identifying underlying “macro”-behaviors (i.e., behaviors that are long-standing and last over time). However, team functioning is far more complex than a series of macro-behaviors, rendering reliance on self-report data deficient for accurate measurement. Recent research demonstrates the potential of alternative methods for capturing team functioning, such as speech and physiological data (Chaffin et al., 2017; Murray & Oertel, 2018). Consequently, these methods are more suitable for capturing micro-behaviors: brief, often unconscious expressions that affect the extent to which an individual feels included by others around them (Paletz et al., 2013). Micro-behaviors can be further classified into micro-aggressions (i.e., subtle, negative exchanges; Keller & Galgay, 2010) or micro-affirmations (i.e., subtle, positive exchanges; Kyte et al. 2020), both of which influence team functioning. Due to the subtle nature of micro-behaviors, contextual factors have a significant impact when determining if it is aggressive or affirmative. Additionally, several iterations of micro-behaviors can have lingering effects on team interactions. For example, the use of “mm-hmm” by a crew member can function as both a micro-affirmation and micro-aggression. Specifically, it can be indication of active listening (i.e., micro-affirmation) or as an expression of annoyance (i.e., aggression) depending on the context in which it occurs. Auditory features (e.g., tone, frequency) can help delineate between the two forms; however, the contextual factors (e.g., previous interactions between team members, crew demographics) add a layer of complexity that render auditory features alone as insufficient to capture micro-behaviors. Consequently, this paper seeks to provide a novel approach in which multi-modal data (i.e., auditory features and contextual features) are used in a random-forest model to better identify distinguishing characteristics between micro-affirmations and micro-aggressions. In turn, detected micro-behaviors are used to predict team performance, thereby demonstrating the value of capturing micro-behaviors as supplemental data to macro-behaviors.

Sydney Begerowski↗
NASA NTRSDate not supplied

Towards Surrogate Modeling of Subgrid Turbulent Transport for 3D Radiative Hydrodynamic Simulations of the Quiet Sun

In this work, we investigate the use of deep learn-ing techniques as surrogate models, to enhance the estimationof effects of subgrid turbulent transport for 3D radiatuve hy-drodynamic simulations of the quiet Sun. We develop two dis-tinct 3D Convolutional Neural Networks (3DCNNs) to capturespatio-temporal dependencies in 3D velocity fields, leveragingdifferent activation functions and architectural designs. Thesemodels integrate both averaged velocity vector components andscalar features such as plasma density to enhance predictionaccuracy. Additionally, a Multilayer Perceptron (MLP) modelis employed to approximate complex nonlinear relationships,offering a comparison in performance between convolutionaland fully connected architectures. Logarithmic transformationis applied to the targets to handle heavily skewed data, im-proving model performance. All models are compared againsta physics-based Gradient Model. Results show that the 3DCNNmodels excel at approximating Reynolds stress tensors, makingthem a candidate for assisting in producing reduced resolutionsimulations, and thereby reducing computational overheadwhile maintaining higher accuracy than the baseline. Thesefindings demonstrate the potential of deep learning, particu-larly CNNs, to advance scalable and accurate simulations ofsolar dynamics, offering a promising alternative to traditionalturbulence models.

Heliophysics↗
DOE OSTI2018

Effective Quantification of Wind-Speed Variability at Different Temporal Scales

RCoV, a statistically robust and resistant metric, effectively characterizes and correlates the spreads of the distributions of wind resources and wind-energy productions. Using RCoV, a wind farm with high wind-speed fluctuations possesses high variations in wind-energy generation and vice versa. Because the long-term correlations between the wind-speed and energy-production interannual variabilities are weak and decrease with the length of data, we do not recommend calculating variabilities with yearly mean data. Skewness and kurtosis drastically change with averaging time frames. Nonzero skewness and kurtosis illustrate that the Gaussian assumption is principally inadequate in most of the United States for all averaging time frames of wind speeds.

17 WIND ENERGY↗
NASA NTRS2016

Dynamic Modeling from Flight Data with Unknown Time Skews

A method for estimating dynamic model parameters from flight data with unknown time skews is described and demonstrated. The method combines data reconstruction, nonlinear optimization, and equation-error parameter estimation in the frequency domain to accurately estimate both dynamic model parameters and the relative time skews in the data. Data from a nonlinear F-16 aircraft simulation with realistic noise, instrumentation errors, and arbitrary time skews were used to demonstrate the approach. The approach was further evaluated using flight data from a subscale jet transport aircraft, where the measured data were known to have relative time skews. Comparison of modeling results obtained from time-skewed and time-synchronized data showed that the method accurately estimates both dynamic model parameters and relative time skew parameters from flight data with unknown time skews.

Morelli, Eugene A.↗
DOE OSTI2024

Analysis of BOSS galaxy data with weighted skew-spectra

We present the first application of the weighted skew-spectra to analyze non-Gaussian information in galaxy survey data. Using the tree-level galaxy skew-spectra together with the one-loop power spectrum multipoles, we analyze the Sloan Digital Sky Survey (SDSS)-III Baryon Oscillation Spectroscopic Survey (BOSS) galaxy clustering data, and target our search towards the equilateral bispectrum shape of primordial non-Gaussianity. We use the Effective Field Theory model for the galaxy power spectrum and bispectrum, and account for systematic effects, such as the survey geometry. From our likelihood analysis, we find f NL equil = -34 +296 -334 at 68% CL, consistent with previous works, while systematic errors from our treatment of the survey geometry lead to an unreliable estimation of f NL ortho . As a result, we further constrain the bias and counterterm parameters, while keeping the cosmology fixed to Planck 2018 values. As a check, we also validate our analysis pipeline using theNseries simulation suite.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Clustering High-dimensional Toxicogenomics Data with Rare Signals

Toxicogenomics studies the gene and protein activities to drug treatments or toxic exposures. As the drugs and genes are numerous, toxicogenomics data are naturally high dimensional, with dimension sizes up to millions. In addition, the distribution of toxicogenomics data is oftentimes skewed, and they contain rare but important signals representing a cell or organism’s response to toxicity. The combination of high dimension and extremely skewed distribution of toxicogenomics data makes clustering analysis extremely challenging.We present our study of clustering toxicogenomics data using classical approaches such as principal component analysis as well as deep learning approaches such as auto-encoders. Our experiments show that these approaches fail to preserve rare signals and produce high-quality clusters. We then explore augmenting matrix factorization with deep learning techniques such as attention mechanism to produce latent representations for clustering. Our technique is able to better preserve rare signals after dimensionality reduction than prior approaches. Furthermore, we combine our augmented matrix factorization with a mechanism similar to autoencoder to balance separable clusters and low regeneration errors. Our experiments demonstrate better clustering with our proposed approach.

Cong, Guojing↗
NASA NTRS1972

Design study for multi-channel tape recorder system, volume 2

Skew test data are presented on a tape recorder transport with a double capstan drive for a 100 KHz tone recorded on five tracks simultaneously. Phase detectors were used to measure the skew when the center channel was the 100 KHz reference.

Source record↗