Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2002

Floating Potential Probe Langmuir Probe Data Reduction Results

During its first five months of operations, the Langmuir Probe on the Floating Potential Probe (FPP) obtained data on ionospheric electron densities and temperatures in the ISS orbit. In this paper, the algorithms for data reduction are presented, and comparisons are made of FPP data with ground-based ionosonde and Incoherent Scattering Radar (ISR) results. Implications for ISS operations are detailed, and the need for a permanent FPP on ISS is examined.

Morton, Thomas L.↗
NASA NTRS1970

Study of sequential decoding

Decoding algorithms for data reduction and transmission through noisy space channels using sequential and hybrid computers

Jelinek, F.↗
NASA NTRS2002

Participation in the Infrared Space Observatory (ISO) Mission

All the Infrared Space Observatory (ISO) data have been transmitted from the ISO Data Centre, reduced, and calibrated. This has been rather labor-intensive as new calibrations for both the ISOPHOT and ISOCAM data have been released and the algorithms for data reduction have improved. We actually discovered errors in the calibration in earlier versions of the software. However the data reduction improvements have now converged and we have a self-consistent, well-calibrated database. It has also been a major effort to obtain the ground-based JHK imaging, 450 micrometer and 850 micrometer imaging and the 1-2.5 micrometer near-infrared spectroscopy for most of the sample galaxies.

Joseph, Robert D.↗
NASA NTRS2012

On Advanced Estimation Techniques for Exoplanet Detection and Characterization using Ground-based Coronagraphs

The direct imaging of planets around nearby stars is exceedingly difficult. Only about 14 exoplanets have been imaged to date that have masses less than 13 times that of Jupiter. The next generation of planet-finding coronagraphs, including VLT-SPHERE, the Gemini Planet Imager, Palomar P1640, and Subaru HiCIAO have predicted contrast performance of roughly a thousand times less than would be needed to detect Earth-like planets. In this paper we review the state of the art in exoplanet imaging, most notably the method of Locally Optimized Combination of Images (LOCI), and we investigate the potential of improving the detectability of faint exoplanets through the use of advanced statistical methods based on the concepts of the ideal observer and the Hotelling observer. We propose a formal comparison of techniques using a blind data challenge with an evaluation of performance using the Receiver Operating Characteristic (ROC) and Localization ROC (LROC) curves. We place particular emphasis on the understanding and modeling of realistic sources of measurement noise in ground-based AO-corrected coronagraphs. The work reported in this paper is the result of interactions between the co-authors during a week-long workshop on exoplanet imaging that was held in Squaw Valley, California, in March of 2012

coronagraphs↗
NASA NTRSDate not supplied

Comparison of two Calibrations of NASA's MC60 Five-component Semi-span Balance

Two separate manual calibrations of NASA’s MC60 five–component semi–span balance were compared. The calibrations were performed in 1999 and 2019using CALSPAN Force Measurement System’s Large Load Rig. The Iterative Method was chosen as the load prediction algorithm and data reduction matrices were generated from both calibration data sets. Then, the applied calibration load schedules, the standard deviations of the calibration load residuals, the prime sensitivities of the five gages, and the maximum outputs at capacity of the five gages were compared. In addition, the calibration data of 1999 was used as check load data for the data reduction matrix that was obtained from the 2019 data. Overall, the agreement between the data analysis results is very good considering the facts that two different load schedules were used for the calibrations, different people performed the calibrations, different instrumentation was used to measure the loads and the outputs, the Large Load Rig was moved to a different site in between the calibrations, and no tare load corrections could be applied to the 1999 data set. Nevertheless, the calibration data of 2019 is clearly the better data set. This observation is no surprise considering that CALSPAN Force Measurement Systems made significant investments in calibration hardware and process improvements during the last 20 years.

wind tunnel balance↗
NASA NTRS2022

Novel Crash Sled with a Translating Support Mass

Background: A novel crash sled has been developed with a translating support, incorporating transducers that allow multiple methods of measuring energy absorption to fully characterize the dynamic crush response of composite components. Objective: The main goal of the current investigation was to demonstrate functionality, repeatability, and accuracy of crush testing using a crash sled with a translating support mass. Methods: A semi-automated algorithm for data reduction was developed based on impact mechanics principles. A preliminary set of tests was initially conducted using aluminum honeycomb specimens with a specified stable crushing force to quantify the accuracy and repeatability of the crush data. Following the success of these tests, triaxially-braided fiber-reinforced polymer (FRP) specimens were evaluated. Results: Crush tests with the aluminum honeycomb specimens showed excellent outcomes for all three specimens. These data provided close agreement with cumulative energy absorption between individual instruments and stable crushing forces at expected values. For the FRP specimens, specific energy absorption (SEA) and force-displacement curves were successfully measured; however, data from the translating support mass accelerometer was excluded from the dataset due to clipping. The SEA of the corrugated specimens was greater than the SEA for the C-channel specimens at both test speeds. Conclusions: The crash sled functionality was verified, the specimen geometry was found to contribute more to SEA than the impact speed in the speed range tested, and the support mass accelerometer will be upgraded to prevent clipping in future tests.

Dynamic Crush↗
NASA NTRS2012

New Techniques for High-Contrast Imaging with ADI: The ACORNS-ADI SEEDS Data Reduction Pipeline

We describe Algorithms for Calibration, Optimized Registration, and Nulling the Star in Angular Differential Imaging (ACORNS-ADI), a new, parallelized software package to reduce high-contrast imaging data, and its application to data from the Strategic Exploration of Exoplanets and Disks (SEEDS) survey. We implement seyeral new algorithms, includbg a method to centroid saturated images, a trimmed mean for combining an image sequence that reduces noise by up to approx 20%, and a robust and computationally fast method to compute the sensitivitv of a high-contrast obsen-ation everywhere on the field-of-view without introducing artificial sources. We also include a description of image processing steps to remove electronic artifacts specific to Hawaii2-RG detectors like the one used for SEEDS, and a detailed analysis of the Locally Optimized Combination of Images (LOCI) algorithm commonly used to reduce high-contrast imaging data. ACORNS-ADI is efficient and open-source, and includes several optional features which may improve performance on data from other instruments. ACORNS-ADI is freely available for download at www.github.com/t-brandt/acorns_-adi under a BSD license

Brandt, Timothy D.↗
NASA NTRS1992

A parameter estimation algorithm for spatial sine testing - Theory and evaluation

This paper presents the theory and an evaluation of a spatial sine testing parameter estimation algorithm that uses directly the measured forced mode of vibration and the measured force vector. The parameter estimation algorithm uses an ARMA model and a recursive QR algorithm is applied for data reduction. In this first evaluation, the algorithm has been applied to a frequency response matrix (which is a particular set of forced mode of vibration) using a sliding frequency window. The objective of the sliding frequency window is to execute the analysis simultaneously with the data acquisition. Since the pole values and the modal density are obtained from this analysis during the acquisition, the analysis information can be used to help determine the forcing vectors during the experimental data acquisition.

Rost, R. W.↗
DOE OSTI2024

Adaptive Methods for Radial Basis Functions

Radial basis functions (RBFs) are a powerful tool for constructing high-order accurate reduced representations of scattered data in arbitrary dimension and on manifolds. We present a method of constructing data approximations in which we utilize a functional tail to capture a global background profile and a RBF neural network (NN) to capture the smaller-scale features. In the RBF NN the RBF centers, matrix shape parameters were selected adaptively for each RBF. We also utilized a geodesic notion of distance on the manifold on which the data lies, e.g., the spherical geodesic for data on the sphere. Although each of these ideas have been been investigated separately in previous works, their combination into a single algorithm is novel. We defined a machine learning problem in which these properties are learned to minimize the data reduction error. We demonstrate the algorithm for applications of scattered data reduction in the plane and on the sphere.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2000

KDD Services at the Goddard Earth Sciences Distributed Active Archive Center

NASA's Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) processes, stores and distributes earth science data from a variety of remote sensing satellites. End users of the data range from instrument scientists to global change and climate researchers to federal agencies and foreign governments. Many of these users apply data mining techniques to large volumes of data (up to 1 TB) received from the GES DAAC. However, rapid advances in processing power are enabling increases in data processing that are outpacing tape drive performance and network capacity. As a result, the proportion of data that can be distributed to users continues to decrease. As mitigation, we are migrating more data mining and mining preparation activities into the data center in order to reduce the data volume that needs to be distributed and to offer the users a more useful and manageable product. This migration of activities faces a number of technical and human-factor challenges. As data reduction and mining algorithms are normally quite specific to the user's research needs, the user's algorithm must be integrated virtually unchanged into the archive environment. Also, the archive itself is busy with everyday data archive and distribution activities and cannot be dedicated to, or even impacted by, the mining activities. Therefore, we schedule KDD 'campaigns' (similar to reprocessing campaigns), during which we schedule a wholesale retrieval of specific data products, offering users the opportunity to extract information from the data being retrieved during the campaign.

Lynnes, Christopher↗
DOE OSTI2025

Developing ML/AI Methods for High-Throughput Characterization of Multiple-Sensor Streams of Tokamak Dynamics for High-Speed Control (Final Report)

This project evaluated and developed new mathematical and algorithmic techniques capable of handling (in real-time) the growing amounts of data generated by modern fusion research. While existing numerical linear algebra (NLA) methods provide the backbone to classical data analysis and algorithms, these methods fundamentally do not port to distributed architectures nor do they allow low-latency data reduction for control. Motivated by the needs for modern fusion reactors, this project explored and implemented new numerical methods to characterize plasma dynamics, respond in real-time to discharge evolution, and to process massive-scale data accurately and rapidly more fully. This project links expertise in multiple-sensor diagnostics of tokamak plasma dynamics from Columbia University’s Plasma Physics Laboratory with expertise in massive-scale data reduction and extreme data control algorithms at Columbia University’s Data Science Institute. This interdisciplinary project (i) applied machine learning methods, (ii) implemented a properly-trained neural-network for very fast processing of high-speed plasma videography, and (ii) developed the applied mathematical methods, based on randomized-NLA (rNLA) routines, for data analysis, reduction, and real-time control. The Columbia University High Beta Tokamak-Extended Pulse (HBT-EP) facility provided data to test new algorithms and partnership with Columbia University's Data Sciences Institute evaluated the broader use of new algorithms for many challenging control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

The ECP ALPINE project: In situ and post hoc visualization infrastructure and analysis capabilities for exascale

A significant challenge on an exascale computer is the speed at which we compute results exceeds by many orders of magnitude the speed at which we save these results. Therefore the Exascale Computing Project (ECP) ALPINE project focuses on providing exascale-ready visualization solutions including in situ processing. In situ visualization and analysis runs as the simulation is run, on simulations results are they are generated avoiding the need to save entire simulations to storage for later analysis. The ALPINE project made post hoc visualization tools, ParaView and VisIt, exascale ready and developed in situ algorithms and infrastructures. The suite of ALPINE algorithms developed under ECP includes novel approaches to enable automated data analysis and visualization to focus on the most important aspects of the simulation. Many of the algorithms also provide data reduction benefits to meet the I/O challenges at exascale. ALPINE developed a new lightweight in situ infrastructure, Ascent.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1979

Data reduction and performance determination

Definitions, algorithms, and procedures for the reduction of monopropellant thruster measurements to performance parameters are provided. A brief discussion of acquisition and recording systems is also included. Emphasis is placed upon monopropellant hydrazine engines, and some parameters relate specifically to the catalytic decomposition of hydrazine (e.g., percent ammonia dissociation). Performance of other types of monopropellant thrusters may, however, be determined by using procedures similar to those discussed. Two appendixes are included: (1) theoretical performance of monopropellant hydrazine; and (2) calculation of rotational performance.

Source record↗
NASA NTRS2012

NASA Tech Briefs, June 2012

Topics covered include: iGlobe Interactive Visualization and Analysis of Spatial Data; Broad-Bandwidth FPGA-Based Digital Polyphase Spectrometer; Small Aircraft Data Distribution System; Earth Science Datacasting v2.0; Algorithm for Compressing Time-Series Data; Onboard Science and Applications Algorithm for Hyperspectral Data Reduction; Sampling Technique for Robust Odorant Detection Based on MIT RealNose Data; Security Data Warehouse Application; Integrated Laser Characterization, Data Acquisition, and Command and Control Test System; Radiation-Hard SpaceWire/Gigabit Ethernet-Compatible Transponder; Hardware Implementation of Lossless Adaptive Compression of Data From a Hyperspectral Imager; High-Voltage, Low-Power BNC Feedthrough Terminator; SpaceCube Mini; Dichroic Filter for Separating W-Band and Ka-Band; Active Mirror Predictive and Requirement Verification Software (AMP-ReVS); Navigation/Prop Software Suite; Personal Computer Transport Analysis Program; Pressure Ratio to Thermal Environments; Probabilistic Fatigue Damage Program (FATIG); ASCENT Program; JPL Genesis and Rapid Intensification Processes (GRIP) Portal; Data::Downloader; Fault Tolerance Middleware for a Multi-Core System; DspaceOgreTerrain 3D Terrain Visualization Tool; Trick Simulation Environment 07; Geometric Reasoning for Automated Planning; Water Detection Based on Color Variation; Single-Layer, All-Metal Patch Antenna Element with Wide Bandwidth; Scanning Laser Infrared Molecular Spectrometer (SLIMS); Next-Generation Microshutter Arrays for Large-Format Imaging and Spectroscopy; Detection of Carbon Monoxide Using Polymer-Composite Films with a Porphyrin-Functionalized Polypyrrole; Enhanced-Adhesion Multiwalled Carbon Nanotubes on Titanium Substrates for Stray Light Control; Three-Dimensional Porous Particles Composed of Curved, Two-Dimensional, Nano-Sized Layers for Li-Ion Batteries 23 Ultra-Lightweight; and Ultra-Lightweight Nanocomposite Foams and Sandwich Structures for Space Structure Applications.

Source record↗
NASA NTRSDate not supplied

Application of Machine Learning Techniques in Calibration and Data Reduction of Multi-Hole Probes

This work presents procedures for implementing machine learning methods into existing algorithms for multi-hole probe calibration and data reduction. It demonstrates that using artificial neural networks (ANNs) can decrease the amount of calibration data needed to achieve a specific calibration uncertainty by over 50%, while also significantly reducing data reduction times. Instead of surface fitting methods, ANNs are employed. Initially, directional calibration coefficients related to flow angles are computed based on pressure measurements, and then these flow angles serve as input parameters for subsequent ANNs to iteratively define Mach number, static pressure, and total pressure. In an alternative approach, new calibration coefficients directly relate pressure measurements from the five-hole probe to the quantities of interest, thereby eliminating the need for iterative algorithms used in conventional surface fitting methods. This method offers several advantages: an average increase of less than 1%in calibration uncertainty for flow angles and a significant reduction in data reduction times to a few seconds on average. Additionally, the methodology is confirmed to avoid both over- and under-fitting.

Machine Learning↗
NASA NTRSDate not supplied

Application of Machine Learning Techniques in Calibration and Data Reduction of Multi-Hole Probes

This work presents procedures to implement machine learning methods in the existing algorithms for multi-hole probe calibrations and data reduction. It is shown here, that utilizing artificial neural networks (ANNs) can reduce the amount of calibration data that needs to be acquired in order to obtain a specific calibration uncertainty, by more than 50% while simultaneously reducing data reduction times significantly. ANNs were used instead of the surface fitting methods, where first, the directional calibration coefficients related to the flow angles are calculated based on the pressure measurements, and then the flow angles are used as a set of the input parameters for the following ANNs to define Mach number and static and total pressure iteratively. In a second approach, novel calibration coefficients were used to directly relate the pressure measurements from five-hole probe to the quantities of interest thus, eliminating the need for iterative algorithms used in the conventional surface fitting methods. The advantageous features of this method are an average increase of less than 1% in the calibration uncertainty for flow angles and significant reduction of the data reduction times (few seconds). In addition, we confirmed the methodology to avoid over-fitting and under-fitting.

Machine Learning↗
DOE OSTI2025

Hybrid learning techniques for scientific data reduction with performance guarantees

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗