Search NASA⌕ Search

SEARCH · Search NASA

Results for “process data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
NASA NTRS2006

EOS MLS Science Data Processing System: A Description of Architecture and Capabilities

This paper describes the architecture and capabilities of the Science Data Processing System (SDPS) for the EOS MLS. The SDPS consists of two major components--the Science Computing Facility and the Science Investigator-led Processing System. The Science Computing Facility provides the facilities for the EOS MLS Science Team to perform the functions of scientific algorithm development, processing software development, quality control of data products, and scientific analyses. The Science Investigator-led Processing System processes and reprocesses the science data for the entire mission and delivers the data products to the Science Computing Facility and to the Goddard Space Flight Center Earth Science Distributed Active Archive Center, which archives and distributes the standard science products.

Microwave Limb Sounder (MLS)↗
DOE OSTI2020

Planck 2018 Results - II. Low Frequency Instrument data processing

We present a final description of the data-processing pipeline for the Planck Low Frequency Instrument (LFI), implemented for the 2018 data release. Several improvements have been made with respect to the previous release, especially in the calibration process and in the correction of instrumental features such as the effects of nonlinearity in the response of the analogue-to-digital converters. We provide a brief pedagogical introduction to the complete pipeline, as well as a detailed description of the important changes implemented. Self-consistency of the pipeline is demonstrated using dedicated simulations and null tests. We present the final version of the LFI full sky maps at 30, 44, and 70 GHz, both in temperature and polarization, together with a refined estimate of the solar dipole and a final assessment of the main LFI instrumental parameters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

PvaPy streaming framework for real-time data processing

User facility upgrades, new measurement techniques, advances in data analysis algorithms as well as advances in detector capabilities result in an increasing amount of data collected at X-ray beamlines. Some of these data must be analyzed and reconstructed on demand to help execute experiments dynamically and modify them in real time. In turn, this requires a computing framework for real-time processing capable of moving data quickly from the detector to local or remote computing resources, processing data, and returning results to users. In this paper, we discuss the streaming framework built on top of PvaPy, a Python API for the EPICS pvAccess protocol. We describe the framework architecture and capabilities, and discuss scientific use cases and applications that benefit from streaming workflows implemented on top of this framework. We also illustrate the framework's performance in terms of achievable data-processing rates for various detector image sizes.

EPICS pvAccess↗
DOE OSTI2023

Demonstrating Cross-Facility Data Processing at Scale with Laue Microdiffraction

In February and April 2023 live, at-scale data processing demonstrations were conducted between the Advanced Photon Source (APS), a synchrotron light source, and the Argonne Leadership Computing Facility (ALCF). These tests were run as part of a novel beamline technique: coded aperture laue micro-diffraction. This technique requires a significant amount of compute to decode appeture patterns embedded in the detector stream. An autonomous system was able to send data to ALCF during an experiment, utilize 50 nodes of the Polaris supercomputer to process 6-12 hour scans, and return the data back to the APS within 12-15 minutes behind the detector. With scan points arriving every 72 seconds, the system kept up with the beamline, potentially enabling in-experiment analysis. The data processing system utilizes Globus infrastructure and an on-demand queue to dynamically acquire nodes on Polaris. The underlying reconstruction algorithms were parallelized via MPI and accelerated with custom CUDA kernels.

Prince, Michael↗
NASA NTRS1984

A simulation of remote sensor systems and data processing algorithms for spectral feature classification

A computational model of the deterministic and stochastic processes involved in multispectral remote sensing was designed to evaluate the performance of sensor systems and data processing algorithms for spectral feature classification. Accuracy in distinguishing between categories of surfaces or between specific types is developed as a means to compare sensor systems and data processing algorithms. The model allows studies to be made of the effects of variability of the atmosphere and of surface reflectance, as well as the effects of channel selection and sensor noise. Examples of these effects are shown.

Arduini, R. F.↗
NASA NTRSDate not supplied

Open Science in Action: The Role of SWxSOC in Expedited Data Release and Cloud-based Data Processing for Heliophysics Missions

NASA's Space Weather Science Operations Center (SWxSOC) is an effort to develop a multi-mission Science Operations Center for the community and specifically Space Weather missions that require data products to be released consistently and quickly. The SWxSOC is committed to Open Science in its approach to software development and data product releases. We are currently supporting the Heliophysics Environmental and Radiation Measurement Experiment Suite (HERMES) that will fly on the Lunar Gateway and the PADRE Small Sat mission. SWxSOC has established an open-source, reusable solution for transitioning data management from on-premises to the cloud, processing data files, and setting up a cloud-based analysis environment to monitor instrument anomalies. Leveraging Amazon Web Services (AWS) and making open-source tools available on GitHub, HERMES exemplifies NASA's commitment to open-source standardization, showcasing the real-world effectiveness of cloud technology in data processing, analysis, and observability. This enhances current operations and ensures faster, standardized deployments for future missions.

hermes↗
NASA NTRS1978

A study of 'onboard' data processing for typical astrophysics payloads

The data, signal flow, and ground processing requirements for a typical set of X-ray and gamma-ray experiments were examined. A set of algorithms for ground processing were derived and these were taken as requirements imposed on 'onboard' spacecraft computers. Five types of CPU microprocessors were evaluated and compared with one another to determine the degree to which the data processing could be performed onboard the spacecraft. These included the NASA Standard Spacecraft Computer (NSSC-1), the Z-80, a Z-80 augmented with an auxiliary fast multiply chip, and an advanced augmented Z-80 with virtually unlimited memory. These CPU were investigated when used in a centralized mode, an experiment dedicated mode, and a hybrid of the first two modes. It was found that the use of onboard processing can significantly reduce downlink data rates for typical payloads, and may lead to greater efficiency of operation. Quantitative data for gamma-ray and X-ray payloads will be presented, and the results of the analysis discussed.

Stout, C. M.↗
NASA NTRS1995

Programmable Iterative Optical Image And Data Processing

Proposed method of iterative optical image and data processing overcomes limitations imposed by loss of optical power after repeated passes through many optical elements - especially, beam splitters. Involves selective, timed combination of optical wavefront phase conjugation and amplification to regenerate images in real time to compensate for losses in optical iteration loops; timing such that amplification turned on to regenerate desired image, then turned off so as not to regenerate other, undesired images or spurious light propagating through loops from unwanted reflections.

Jackson, Deborah J.↗
NASA NTRS1982

Step-oriented pipeline data processing system

Architecture for step-oriented pipeline data processing is disclosed utilizing a plurality of cascaded modules, each module including a programmable general purpose processor and a read/write random access memory. The memory of each module, shared with the next module in cascade serves as an output memory for the processor and the input memory for the next processor. An additional memory is provided to serve as the input memory of the first module, and each module is provided with a memory, which may be a read-out memory, to store a program for the processor. Each module is further provided with a logic network for resolving a potential memory sharing conflict by awarding priority to the processor of the module.

Castleman, Kenneth R.↗
NASA NTRS1995

Millimeter-wave Imaging Radiometer (MIR) data processing and development of water vapor retrieval algorithms

This document describes the progress of the task of the Millimeter-wave Imaging Radiometer (MIR) data processing and the development of water vapor retrieval algorithms, for the second six-month performing period. Aircraft MIR data from two 1995 field experiments were collected and processed with a revised data processing software. Two revised versions of water vapor retrieval algorithm were developed, one for the execution of retrieval on a supercomputer platform, and one for using pressure as the vertical coordinate. Two implementations of incorporating products from other sensors into the water vapor retrieval system, one from the Special Sensor Microwave Imager (SSM/I), the other from the High-resolution Interferometer Sounder (HIS). Water vapor retrievals were performed for both airborne MIR data and spaceborne SSM/T-2 data, during field experiments of TOGA/COARE, CAMEX-1, and CAMEX-2. The climatology of water vapor during TOGA/COARE was examined by SSM/T-2 soundings and conventional rawinsonde.

Chang, L. Aron↗