Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

An adaptive data-smoothing routine

An adaptive noise reduction algorithm that can be implemented on a microcomputer is developed. Smoothing polynomials are used where the polynomial coefficients are chosen such that the mean-square-error between the noisy and smoothed data is minimized. This approach is equivalent to the implementation of a low-pass finite impulse response filter. The noise reduction depends on the order of the smoothing polynomial. A whiteness test on the error sequence is incorporated to search for the optimal smoothing. Expansion coefficients may be computed via the fast Fourier transform, and the resulting smoothing process is the equivalent of the implementation of an adaptive ideal low-pass filter. Results are obtained for an analytical signal with added white Gaussian noise. The routine may be applied to any smooth signal with additive random noise.

Taylor, Clayborne D.↗
DOE OSTI2025

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Lifting MGARD: Construction of (pre)wavelets on the interval using polynomial predictors of arbitrary order

MGARD (MultiGrid Adaptive Reduction of Data) is an algorithm for compressing and refactoring scientific data, based on the theory of multigrid methods. The core algorithm is built around stable multilevel decompositions of conforming piecewise linear $C^0$ finite element spaces, enabling accurate error control in various norms and derived quantities of interest. In this work, we extend this construction to arbitrary order Lagrange finite elements $\mathbb{Q}_p$, $p \geq 0$, and propose a reformulation of the algorithm as a lifting scheme with polynomial predictors of arbitrary order. Additionally, a new formulation using a compactly supported wavelet basis is discussed, and an explicit construction of the proposed wavelet transform for uniform dyadic grids is described.

Reshniak, Viktor [Oak Ridge National Laboratory (O↗
NASA NTRS1987

An overview of the mathematical and statistical analysis component of RICIS

Mathematical and statistical analysis components of RICIS (Research Institute for Computing and Information Systems) can be used in the following problem areas: (1) quantification and measurement of software reliability; (2) assessment of changes in software reliability over time (reliability growth); (3) analysis of software-failure data; and (4) decision logic for whether to continue or stop testing software. Other areas of interest to NASA/JSC where mathematical and statistical analysis can be successfully employed include: math modeling of physical systems, simulation, statistical data reduction, evaluation methods, optimization, algorithm development, and mathematical methods in signal processing.

Hallum, Cecil R.↗
NASA NTRS2011

Description and Status of the DC Lightning Mapping Array

The DC Lightning Mapping Array (DC LMA) centered on the Washington, DC metro region has been in operation since 2006. During that time the DC LMA has provided real time data to regional National Weather Service (NSF) Sterling, VA forecast office for operations support and the NOAA Meteorological Development Laboratory (MDL) for new product development and assessment. Data from this network (as well as other from other LMA systems) are now being used to create proxy Geostationary Lightning Mapper (GLM) data sets for GOES-R risk reduction and algorithm development activities. In addition, since spring 2009 data are provided to the Storm Prediction Center in support of Hazardous Weather Testbed and GOES-R Proving Ground activities during the Spring Program. Description, status and plans will be discussed.

Blakeslee, Richard↗
NASA NTRS2011

Description and Status of the North Alabama Lightning Mapping Array

The North Alabama Lightning Mapping Array (LMA) is a network LMA detectors that detects and maps lightning using VHF radiation (TV Channel 5) in a region centered about Huntsville, Alabama that includes North Alabama, Central Tennessee and parts of Georgia and Mississippi. The North Alabama LMA has been in operation since late 2001, and has been providing real time data to regional National Weather Service (NSF) Weather Forecast Offices (WFOs) since mid 2003 through the NASA Short-term Prediction Research and Transition (SPoRT) center. Data from this network (as well as other from other LMA systems) are now being used to create proxy Geostationary Lightning Mapper (GLM) data sets for GOES-R risk reduction and algorithm development activities. In addition, since spring 2009 data are provided to the Storm Prediction Center in support of Hazardous Weather Testbed and GOES-R Proving Ground activities during the Spring Program. Description, status and plans will be discussed.

Blakeslee, Richard J.↗
DOE OSTI2025

Exploring Continuous Seismic Data at an Industry Facility Using Unsupervised Machine Learning

Seismic data recorded at industrial sites contain valuable information on anthropogenic activities. With advances in machine learning and computing power, new opportunities have emerged to explore the seismic wavefield in these complex environments. We applied two unsupervised machine learning algorithms to analyze continuous seismic data collected from an industrial facility in Texas, United States. The Uniform Manifold Approximation and Projection for Dimension Reduction algorithm was used to reduce the dimensionality of the data and generate 2D embeddings. Then, the Hierarchical Density-Based Spatial Clustering of Applications with Noise method was employed to automatically group these embeddings into distinct signal clusters. Our analysis of over 1400 hr (around 59 days) of continuous seismic data revealed five and seven signal clusters at two separate stations. At both stations, we identified clusters associated with background noise and vehicle traffic, with the latter’s temporal patterns aligning closely with the facility’s work schedule. Furthermore, the algorithms detected signal clusters from unknown sources and underline the ability of unsupervised machine learning for uncovering previously unrecognized patterns. Our analysis demonstrates the effectiveness of unsupervised approaches in examining continuous seismic data without requiring prior knowledge or pre-existing labels.

58 GEOSCIENCES↗
NASA NTRS1983

Carbon monoxide mixing ratio inference from gas filter radiometer data

A new algorithm has been developed which permits, for the first time, real time data reduction of nadir measurements taken with a gas filter correlation radiometer to determine tropospheric carbon monoxide concentrations. The algorithm significantly reduces the complexity of the equations to be solved while providing accuracy comparable to line-by-line calculations. The method is based on a regression analysis technique using a truncated power series representation of the primary instrument output signals to infer directly a weighted average of trace gas concentration. The results produced by a microcomputer-based implementation of this technique are compared with those produced by the more rigorous line-by-line methods. This algorithm has been used in the reduction of Measurement of Air Pollution from Satellites, Shuttle, and aircraft data.

Wallio, H. A.↗
NASA NTRS2013

Analysis of Multivariate Experimental Data Using A Simplified Regression Model Search Algorithm

A new regression model search algorithm was developed that may be applied to both general multivariate experimental data sets and wind tunnel strain-gage balance calibration data. The algorithm is a simplified version of a more complex algorithm that was originally developed for the NASA Ames Balance Calibration Laboratory. The new algorithm performs regression model term reduction to prevent overfitting of data. It has the advantage that it needs only about one tenth of the original algorithm's CPU time for the completion of a regression model search. In addition, extensive testing showed that the prediction accuracy of math models obtained from the simplified algorithm is similar to the prediction accuracy of math models obtained from the original algorithm. The simplified algorithm, however, cannot guarantee that search constraints related to a set of statistical quality requirements are always satisfied in the optimized regression model. Therefore, the simplified algorithm is not intended to replace the original algorithm. Instead, it may be used to generate an alternate optimized regression model of experimental data whenever the application of the original search algorithm fails or requires too much CPU time. Data from a machine calibration of NASA's MK40 force balance is used to illustrate the application of the new search algorithm.

Ulbrich, Norbert M.↗
DOE OSTI2025

TensorID v1.0

This Python software package includes new and efficient algorithms for satellite and core interpolative decomposition of tensor data. In general, these algorithms target high-dimensional data reduction and compression. The software is purely numerical and can be applied by others to many important sources of tensor data generated by computation or experiment.

Zhang, Yifan [Lawrence Berkeley National Laborator↗
NASA NTRS1984

LANDSAT instruments characterization

Work performed for the LANDSAT instrument characterization task in the areas of absolute radiometry, coherent noise analysis, and between-date smoothing is reported. Absolute radiometric calibration for LANDSAT-5 TM under ambient conditions was performed. The TM Radiometric Algorithms and Performance Program (TRAPP) was modified to create optional midscan data files and to match the TM Image Processing System (TIPS) algorithm for pulse determination. Several data reduction programs were developed, including a linear regression and its plotted result. A fast Fourier transformation study was conducted on the resequenced TM data. Subscenes of homogeneous water within scenes over Pensacola, Florida were used for testing the FFT on the resequenced data. Finally, a gain and pulse height stability study of LANDSAT 5 TM spectral bands was performed.

Lee, Y.↗
NASA NTRS1994

Acquisition and processing of data for isotope-ratio-monitoring mass spectrometry

Methods are described for continuous monitoring of signals required for precise analyses of 13C, 18O, and 15N in gas streams containing varying quantities of CO2 and N2. The quantitative resolution (i.e. maximum performance in the absence of random errors) of these methods is adequate for determination of isotope ratios with an uncertainty of one part in 10(5); the precision actually obtained is often better than one part in 10(4). This report describes data-processing operations including definition of beginning and ending points of chromatographic peaks and quantitation of background levels, allowance for effects of chromatographic separation of isotopically substituted species, integration of signals related to specific masses, correction for effects of mass discrimination, recognition of drifts in mass spectrometer performance, and calculation of isotopic delta values. Characteristics of a system allowing off-line revision of parameters used in data reduction are described and an algorithm for identification of background levels in complex chromatograms is outlined. Effects of imperfect chromatographic resolution are demonstrated and discussed and an approach to deconvolution of signals from coeluting substances described.

NASA Discipline Exobiology↗
DOE OSTI2025

Genetic algorithm optimization of nuclear criticality experiment for reduction of intermediate-energy 239 Pu nuclear data uncertainties

Nuclear criticality experiments are conducted to investigate specific nuclear data important for safe handling and storage of fissile materials, reactor design and operation, and the validation of radiation transport codes. Incorrect or uncertain nuclear data can prohibitively impact operational safety limits, reactor licensing, and predictive simulation capability; therefore, integral measurements from criticality experiments are necessary and should be performed frequently. To maximize the impact of the integral measurements, it is important to consider experiment geometry, material selection, and component dimensions. When taking these considerations into account, the experiment design process becomes iterative and very time intensive. This work utilizes a genetic algorithm to efficiently explore potential nuclear criticality experiment designs for the Laboratory Directed Research & Development project PARADIGM (PARallel Approach of Differential and InteGral Measurements) at Los Alamos National Laboratory. In this paper, the building blocks of the genetic algorithm are discussed in detail, the genetic algorithm methodology is verified, and the genetic algorithm is used to produce three candidate experiment models for the final PARADIGM design. The three candidate models produced by the genetic algorithm consist of copper-reflected assemblies containing 14 repeating units of alumina, graphite, boron, and plutonium plates. Furthermore, in addition to the optimization results, final design considerations are also discussed for designs with a height and/or weight very close to or slightly above assembly machine operational limits.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1979

Algorithm and program for information processing with the filin apparatus

The reduction of spectral radiation data from space sources is described. The algorithm and program for identifying segments of information obtained from the Film telescope-spectrometer on the Salyut-4 are presented. The information segments represent suspected X-ray sources. The proposed algorithm is an algorithm of the lowest level. Following evaluation, information free of uninformative segments is subject to further processing with algorithms of a higher level. The language used is FORTRAN 4.

Gurin, L. S.↗
NASA NTRS1986

Plate motions and deformations from geologic and geodetic data

Research effort on behalf of the Crustal Dynamics Project focused on the development of methodologies suitable for the analysis of space-geodetic data sets for the estimation of crustal motions, in conjunction with results derived from land-based geodetic data, neo-tectonic studies, and other geophysical data. These methodologies were used to provide estimates of both global plate motions and intraplate deformation in the western U.S. Results from the satellite ranging experiment for the rate of change of the baseline length between San Diego and Quincy, California indicated that relative motion between the North American and Pacific plates over the course of the observing period during 1972 to 1982 were consistent with estimates calculated from geologic data averaged over the past few million years. This result, when combined with other kinematic constraints on western U.S. deformation derived from land-based geodesy, neo-tectonic studies, and other geophysical data, places limits on the possible extension of the Basin and Range province, and implies significant deformation is occurring west of the San Andreas fault. A new methodology was developed to analyze vector-position space-geodetic data to provide estimates of relative vector motions of the observing sites. The algorithm is suitable for the reduction of large, inhomogeneous data sets, and takes into account the full position covariances, errors due to poorly resolved Earth orientation parameters and vertical positions, and reduces baises due to inhomogeneous sampling of the data. This methodology was applied to the problem of estimating the rate-scaling parameter of a global plate tectonic model using satellite laser ranging observations over a five-year interval. The results indicate that the mean rate of global plate motions for that interval are consistent with those averaged over several million years, and are not consistent with quiescent or greatly accelerated plate motions. This methodology was also used to provide constraints on deformation in the western U.S. using very long baseline interferometry observations over a two-year period.

Jordan, T. H.↗
NASA NTRS1975

Satellite on-board processing for earth resources data

Results of a survey of earth resources user applications and their data requirements, earth resources multispectral scanner sensor technology, and preprocessing algorithms for correcting the sensor outputs and for data bulk reduction are presented along with a candidate data format. Computational requirements required to implement the data analysis algorithms are included along with a review of computer architectures and organizations. Computer architectures capable of handling the algorithm computational requirements are suggested and the environmental effects of an on-board processor discussed. By relating performance parameters to the system requirements of each of the user requirements the feasibility of on-board processing is determined for each user. A tradeoff analysis is performed to determine the sensitivity of results to each of the system parameters. Significant results and conclusions are discussed, and recommendations are presented.

Bodenheimer, R. E.↗
NASA NTRS1994

Uncertainty of Five-Hole Probe Measurements

A new algorithm for five-hole probe calibration and data reduction using a non-nulling technique was developed, verified, and reported earlier (Wendt and Reichert, 1993). The new algorithm's simplicity permits an analytical treatment of the propagation of uncertainty in five-hole probe measurement. The objectives of the uncertainty analysis are to quantify the uncertainty of five-hole probe results (e.g., total pressure, static pressure, and flow direction) and to determine the dependence of the result uncertainty on the uncertainty of all underlying experimental and calibration measurands. This study outlines a general procedure that other researchers may use to determine five-hole probe result uncertainty and provides guidance for improving the measurement technique.

Reichert, Bruce A.↗