Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data reduction methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

The Capillary Flow Experiments Aboard the International Space Station: Increments 9-15

This report provides a summary of the experimental, analytical, and numerical results of the Capillary Flow Experiment (CFE) performed aboard the International Space Station (ISS). The experiments were conducted in space beginning with Increment 9 through Increment 16, beginning August 2004 and ending December 2007. Both primary and extra science experiments were conducted during 19 operations performed by 7 astronauts including: M. Fincke, W. McArthur, J. Williams, S. Williams, M. Lopez-Alegria, C. Anderson, and P. Whitson. CFE consists of 6 approximately 1 to 2 kg handheld experiment units designed to investigate a selection of capillary phenomena of fundamental and applied importance, such as large length scale contact line dynamics (CFE-Contact Line), critical wetting in discontinuous structures (CFE-Vane Gap), and capillary flows and passive phase separations in complex containers (CFE-Interior Corner Flow). Highly quantitative video from the simply performed flight experiments provide data helpful in benchmarking numerical methods, confirming theoretical models, and guiding new model development. In an extensive executive summary, a brief history of the experiment is reviewed before introducing the science investigated. A selection of experimental results and comparisons with both analytic and numerical predictions is given. The subsequent chapters provide additional details of the experimental and analytical methods developed and employed. These include current presentations of the state of the data reduction which we anticipate will continue throughout the year and culminate in several more publications. An extensive appendix is used to provide support material such as an experiment history, dissemination items to date (CFE publication, etc.), detailed design drawings, and crew procedures. Despite the simple nature of the experiments and procedures, many of the experimental results may be practically employed to enhance the design of spacecraft engineering systems involving capillary interface dynamics.

Jenson, Ryan M.↗
DOE OSTI2025

Data-driven reduced-order models for port-Hamiltonian systems with operator inference

Hamiltonian operator inference has been developed in Sharma et al. (2022) to learn structure-preserving reduced-order models (ROMs) for Hamiltonian systems. The method constructs a low-dimensional model using only data and knowledge of the functional form of the Hamiltonian. The resulting ROMs preserve the intrinsic structure of the system, ensuring that the mechanical and physical properties of the system are maintained. In this work, we extend this approach to port-Hamiltonian systems, which generalize Hamiltonian systems by including energy dissipation, external input, and output. Based on snapshots of the system’s state and output, together with the information about the functional form of the Hamiltonian, reduced operators are inferred through optimization and are then used to construct data-driven ROMs. To further alleviate the complexity of evaluating nonlinear terms in the ROMs, a hyper-reduction method via discrete empirical interpolation is applied. Accordingly, we derive error estimates for the ROM approximations of the state and output. Lastly, we demonstrate the structure preservation, as well as the accuracy of the proposed port-Hamiltonian operator inference framework, through numerical experiments on a linear mass–spring-damper problem and a nonlinear Toda lattice problem.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1975

Image processing and data reduction of Apollo low light level photographs

The removal of the lens induced vignetting from a selected sample of the Apollo low light level photographs is discussed. The methods used were developed earlier. A study of the effect of noise on vignetting removal and the comparability of the Apollo 35mm Nikon lens vignetting was also undertaken. The vignetting removal was successful to about 10% photometry, and noise has a severe effect on the useful photometric output data. Separate vignetting functions must be used for different flights since the vignetting function varies from camera to camera in size and shape.

Alvord, G. C.↗
NASA NTRS2012

A Peculiar Class of Debris Disks from Herschel/DUNES: A Steep Fall Off in the Far Infrared

Context. The existence of debris disks around old main sequence stars is usually explained by continuous replenishment of small dust grains through collisions from a reservoir of larger objects. Aims. We present photometric data of debris disks around HIP 103389 (HD199260), HIP 100350 (HN Peg, HD206860), and HIP 114948 (HD 219482), obtained in the context of our Herschel Open TIme Key Program DUNES (DUst around NEarby Stars). Methods. We used Herschel/PACS to detect the thermal emission of the three debris disks with a 30 sigma sensitivity of a few mJy at l00 micron and 160 micron. In addition, we obtained Herschel/PACS photometric data at 70 micron for HIP 103389. These observations are complemented by a large variety of optical to far-infrared photometric data. Two different approaches are applied to reduce the Herschel data to investigate the impact of data reduction on the photometry. We fit analytical models to the available spectral energy distribution (SED) data using the fitting method of simulated therma1 annealing as well as a classical grid search method. Results. The SEDs of the three disks potentially exhibit an unusually steep decrease at wavelengths >= 70 micron. We investigate the significance of the peculiar shape of these SEDs and the impact on models of the disks provided it is real. Using grain compositions that have been applied successfully for modeling of many other debris disks, our modeling reveals that such a steep decrease of the SEDs in the long wavelength regime is inconsistent with a power-law exponent of the grain size distribution -3.5 expected from a standard equilibrium collisional cascade. In contrast, a steep grain size distribution or, alternatively an upper grain size in the range of few tens of micrometers are implied. This suggests that a very distinct range of grain sizes would dominate the thermal. emission of such disks. However, we demonstrate that the understanding of the data of faint sources obtained with Herschel is still incomplete and that the significance of our results depends on the version of the data reduction pipeline used. Conclusions. A new mechanism to produce the dust in the presented debris disks, deviations from the conditions required for a standard equilibrium collisional cascade (grain size exponent of -3.5), and/or significantly different dust properties would be necessary to explain the potentially steep SED shape of the three debris disks presented.

Ertel, S.↗
NASA NTRS1949

Heat transfer from high-temperature surfaces to fluids II : correlation of heat-transfer and friction data for air flowing in inconel tube with rounded entrance

A heat transfer investigation, which was an extension of a previously reported NACA investigation, was conducted with air flowing through an electrically heated inconel tube with a rounded entrance,an inside diameter of 0.402 inch, and a length of 24 inches over a range of conditions, which included Reynolds numbers up to 500,000, average surface temperatures up to 2050 degrees R, and heat-flux densities up to 150,000 Btu per hour per square foot. Conventional methods of correlating heat-transfer data wherein properties of the air were evaluated at the average bulk, film, and surface temperatures resulted in reductions of Nusselt number of about 38, 46, and 53 percent, respectively, for an increase in surface temperature from 605 degrees to 2050 degrees R at constant Reynolds number. A modified correlation method in which the properties of air were based on the surface temperature and the Reynolds number was modified by substituting the product of the density at the inside tube wall and the bulk velocity for the conventional mass flow per unit cross-sectional area, resulted in a satisfactory correlation of the data for the extended ranges of conditions investigated.

Lowdermilk, Warren H↗
NASA NTRS1989

Monte Carlo simulation of an expanding gas

By application of simple computer graphics techniques, the statistical performance of two Monte Carlo methods used in the simulation of rarefied gas flows are assessed. Specifically, two direct simulation Monte Carlo (DSMC) methods developed by Bird and Nanbu are considered. The graphics techniques are found to be of great benefit in the reduction and interpretation of the large volume of data generated, thus enabling important conclusions to be drawn about the simulation results. Hence, it is discovered that the method of Nanbu suffers from increased statistical fluctuations, thereby prohibiting its use in the solution of practical problems.

Boyd, Iain D.↗
NASA NTRS1973

Study to demonstrate the feasibility of and determine the optimum method of remote haze monitoring by satellite

The author has identified the following significant results. Ground truth data has been obtained in order to determine the amount of haze present at the time of ERTS-1 passes over the Los Angeles area. October 21, 1972 was the haziest day, that was essentially cloud free, to date. At the time of the next pass, November 8, 1972 this area was quite clear. Changes in terrain features are minimized because these were consecutive passes. Digital tapes of the ERTS-1 data from these two days are being analyzed to determine the effects of haze on the data, in order to determine the optimum method for monitoring haze from a satellite. Preliminary results indicate that a slight reduction in contrast and some distortion of spectral signatures are caused by heavy haze. This effect is thought to be sufficient to determine the presence of moderate haze without the benefit of ground truth data. It is significant to note that there is very little degradation of ERTS-1 data caused be a rather heavy haze. For many purposes the data obtained on a hazy day are just as useful as that obtained on a clear day.

Rogers, E. H.↗
NASA NTRS2004

A Wall Correction Program Based on Classical Methods for the National Transonic Facility (Solid Wall or Slotted Wall) and the 14x22-Ft Subsonic Tunnel at NASA LaRC

A Fortran subroutine CMWALL is described, which is an implementation of the collective information from classical methods-based wall corrections. These methods use established closed-form expressions which were developed based on simple linear potential-based methods. This is a simple and rapid tool to calculate corrections due to wall interference in the National Transonic Facility (Solid Wall or Slotted Wall) or the 14x22-Ft Subsonic Tunnel at NASA LaRC. It is designed to be easily implemented in the existing tunnel data reduction programs, either as real-time or post-point. It is however important to realize that the method is based on the simplifying assumptions of linearity, small model and attached flow. The computed results are thus to be viewed as first-cut estimates, to be refined further using more complex methods based on measured wall pressures (known as wall signature methods).

Hemsch, Michael J.↗
NASA NTRS1926

The reduction of airplane flight test data to standard atmosphere conditions

This report was prepared for the National Advisory Committee for Aeronautics in order to supply the need of practical methods of reducing observed performance to standard conditions with a minimum of labor. The first part gives a very simple approximate method of reducing performance in climb, and is particularly adapted to work not requiring extreme accuracy. The second part gives a somewhat more elaborate and more accurate method which is well suited to general flight test reduction. The third part gives the conventional method of calibrating air-speed indicators and reducing the indicated speeds to true air speeds. An appendix gives working tables and charts for the standard atmosphere. (author)

Diehl, Walter S↗
NASA NTRS1985

Constitutive modeling for isotropic materials

The first year of progress on a NASA-Lewis contract with the General Electric Co is documented. The purpose of this contract (NAS3-23927) is to develop and evaluate unified constitutive equations for applications to hot-path components of aircraft gas turbine engines such as high pressure turbine blades and vanes. To accomplish this goal, uniaxial, notched, and multiaxial specimens made of conventionally cast Rene 80 are being tested under conditions that simulate engine operating conditions. To reduce the raw data, automated data reduction techniques are being developed that produce computer files containing the information needed to analyze proposed constitutive theories. Described are the analytical methods being developed to determine the parameters for these nonlinear unified theories by using the reduced data files. In another activity, a dedicated finite-element computer code is being developed to use unified theories in the structural analysis of hot-section components. This code was extensively verified for one such theory by successfully predicting the strain histories measured experimentally at the notch root of complex specimens taken from complex laboratory specimens.

Ramaswamy, V. G.↗
DOE OSTI2025

Denoising Autoencoder for Reconstructing Sensor Observation Data and Predicting Evapotranspiration: Noisy and Missing Values Repair and Uncertainty Quantification

Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE

denoising autoencoder↗
NASA NTRS2021

Trend Detection of Atmospheric Time Series: Incorporating Appropriate Uncertainty Estimates and Handling Extreme Events

This paper is aimed at atmospheric scientists without formal training in statistical theory. Its goal is to, 1) provide a critical review of the rationale for trend analysis of the time series typically encountered in the field of atmospheric chemistry; 2) describe a range of trend-detection methods; and 3) demonstrate effective means of conveying the results to a general audience. Trend detections in atmospheric chemical composition data are often challenged by a variety of sources of uncertainty, which often behave differently to other environmental phenomena such as temperature, precipitation rate, or stream flow, and may require specific methods depending on the science questions to be addressed. Some sources of uncertainty can be explicitly included in the model specification, such as autocorrelation and seasonality, but some inherent uncertainties are difficult to quantify, such as data heterogeneity and measurement uncertainty due to the combined effect of short- and long-term natural variability, instrumental stability, and aggregation of data from sparse sampling frequency. Failure to account for these uncertainties might result in an inappropriate inference of the trends and their estimation errors. On the other hand, the variation in extreme events might be interesting for different scientific questions, for example, the frequency of extremely high surface ozone events and their relevance to human health. In this study we aim to, 1) review trend detection methods for addressing different levels of data complexity in different chemical species; 2) demonstrate that the incorporation of scientifically interpretable covariates can outperform pure numerical curve fitting techniques in terms of uncertainty reduction and improved predictability; 3) illustrate the study of trends based on extreme quantiles that can provide insight beyond standard mean or median based trend estimates; and 4) present an advanced method of quantifying regional trends based on the inter-site correlations of multi-site data. All demonstrations are based on time series of observed trace gases relevant to atmospheric chemistry, but the methods can be applied to other environmental data sets.

Trace gas↗
DOE OSTI2026

From microbial diversity to functional potential using dimensionality reduction

The high dimensionality of microbial diversity data from ‘omics observations can be reduced using Machine Learning, with many recent studies showcasing ML utility for exploratory ecological feature finding and process prediction. Here, we compare the Self Organizing Map (SOM) dimensionality reduction method to the well-documented sample-based Principal Coordinate Analysis (PCoA) and taxa-based Weighted Gene Correlation Network Analysis (WGCNA) using near daily 16S rRNA gene amplicon sequencing data from the 2019 to 2020 MOSAiC International Arctic Drift Expedition. We then map k-means clustering outputs from each method to available metagenomes, extracting functionally distinct seasonal microbial ecotypes in the surface Arctic Ocean. Our results indicate the SOM method better represented expected seasonal transitions and identified a greater number of metabolically distinct functional groups than the more traditional PCoA ordination. Ultimately, we identified four community ecotypes with distinct taxonomic and functional cut-offs driven by seasonality, water mass, and substrate turnover, highlighting the importance of succession in functional diversity for the central Arctic Ocean. These results reinforce ML dimensionality reduction as a meaningful translator in the mining of historical amplicon datasets to address modern mechanistic questions and potentially provide ’omics informed ecotype diversity to leverage in mechanistic biogeochemical models.

Arctic Ocean↗
DOE OSTI2025

CV4Quantum

CV4Quantum is a statistical technique for reducing the sampling overhead in probabilistic error cancellation, which is an error mitigation technique used in quantum computing. CV4Quantum is based on the control variates method, which is a Monte Carlo variance reduction technique. This repository contains the code and data associated with a demonstration of CV4Quantum using simulation experiments.

Shyamsundar, Prasanth [Fermi National Accelerator ↗
NASA NTRS1980

Visual enhancement of images of natural resources: Applications in geology

The principal components technique for use in multispectral scanner LANDSAT data processing results in optimum dimensionality reduction. A powerful tool for MSS IMAGE enhancement, the method provides a maximum impression of terrain ruggedness; this fact makes the technique well suited for geological analysis.

Dejesusparada, N.↗
NASA NTRS1998

Assessment of Uncertainty in the Determination of Activation Energy for Polymeric Materials

An assessment of the experimental uncertainty in obtaining the kinetic activation energy from thermogravimetric analysis (TGA) data is presented. A neat phenolic resin, Borden SC1O08, was heated at three heating rates to obtain weight loss vs temperature data. Activation energy was calculated by two methods: the traditional Flynn and Wall method based on the slope of log(q) versus 1/T, and a modification of this method where the ordinate and abscissa are reversed in the linear regression. The modified method produced a more accurate curve fit of the data, was more sensitive to data nonlinearity, and gave a value of activation energy 75 percent greater than the original method. An uncertainty analysis using the modified method yielded a 60 percent uncertainty in the average activation energy. Based on this result, the activation energy for a carbon-phenolic material was doubled and used to calculate the ablation rate In a typical solid rocket environment. Doubling the activation energy increased surface recession by 3 percent. Current TGA data reduction techniques that use the traditional Flynn and Wall approach to calculate activation energy should be changed to the modified method.

Darby, Stephania P.↗
DOE OSTI2026

QProR: An Efficient Framework for Quantity-of-Interest Based Progressive Retrieval with Guaranteed Error Control

Scientific applications generate an unprecedented volume of data, overwhelming the network and file systems’ bandwidth and posing challenges for efficient and scalable data retrieval and analysis. Progressive data compression offers a promising solution by enabling on-demand retrieval at reduced size. However, existing progressive methods either fail to bound the errors in essential quantities of interest (QoIs) derived from raw data or suffer from suboptimal retrieval efficiency. In this work, we propose QProR, an efficient QoI-based progressive framework that optimizes progressive retrieval for target QoIs. Our key contributions include: (1) a systematic framework that integrates error-controlled lossy compressors with bitplane encoding while decoupling the two processes for high flexibility and adaptability; (2) a novel weighted bitplane encoding method which incorperates QoI knowledge into data refactoring to enhance retrieval efficiency; (3) an optimized retrieval strategy that accounts for the varying impacts of different variables on multivariate QoIs; (4) comprehensive evaluations using six real-world datasets from multiple scientific applications and thorough comparisons against state of the arts. Experimental results demonstrate that QProR achieves up to 80.38% reduction in the retrieval size under the same requested QoI error tolerance, when compared with the best-performing existing methods. When transferring 384 GB of scientific data to remote sites, QProR delivers up to 1.68 × speedup in the end-to-end data transfer performance.

Li, Wenbo [University of Kentucky]↗
NASA NTRSDate not supplied

A Dynamic PCA and Machine Learning Tool for Automated Identification of Solar Wind Disturbances Impacting Earth’s Magnetosphere

Earth’s magnetosphere is continuously impacted by solar wind and interplanetary magnetic field (IMF) disturbances, such as shocks, discontinuities, magnetic clouds and more. Understanding how such disturbances propagate from the Sun and what is their impact on the different magnetospheric domains is key to understanding and forecasting energy transfer from the solar wind to Earth. The large number of overlapping solar wind and magnetospheric missions carrying magnetometers and the recent advances in communications and data storage technologies have enabled an unprecedented quantity of high-fidelity magnetic field data captured by in-situ spacecraft to be available at the click of a button. However, this massive quantity of available data can prove unwieldy for researchers, limiting the identification of interesting phenomena and disturbances to a relatively small percentage of the total dataset. Several techniques have been previously developed for automated identification of specific types of magnetic anomalies, but these methods are typically mission-specific and can be difficult to generalize. We present initial results for a generic method of automated anomaly detection in magnetic field measurements based on dimensionality reduction and unsupervised clustering via machine learning. The benefit of our technique is its high degree of generalizability and flexibility which make it a most useful data survey tool for a wide range of magnetic field datasets. This method can also be applied simultaneously to other observed time-series properties like plasma density, pressure, and velocity for more accurate event identification. Additionally, the application of this method to data captured by multiple spacecraft enables the simultaneous identification of disturbances and the determination of their propagation characteristics. Initial evaluation of this technique has been performed using data from Magnetospheric MultiScale (MMS) and THEMIS-ARTEMIS missions, providing a testbed scenario for the future Heliophysics Environmental and Radiation Measurement Experiment Suite (HERMES) platform instruments that will measure solar wind and IMF properties from lunar orbit onboard the Gateway station.

Miguel Martinez-Ledesma↗