Search NASA⌕ Search

SEARCH · Search NASA

Results for “data processing and archiving”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

NASA's Earth Observing Data and Information System - Near-Term Challenges

NASA's Earth Observing System Data and Information System (EOSDIS) has been a central component of the NASA Earth observation program since the 1990's. EOSDIS manages data covering a wide range of Earth science disciplines including cryosphere, land cover change, polar processes, field campaigns, ocean surface, digital elevation, atmosphere dynamics and composition, and inter-disciplinary research, and many others. One of the key components of EOSDIS is a set of twelve discipline-based Distributed Active Archive Centers (DAACs) distributed across the United States. Managed by NASA's Earth Science Data and Information System (ESDIS) Project at Goddard Space Flight Center, these DAACs serve over 3 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as the Science Investigator-led Processing systems (SIPS), common metadata and metrics management systems, specialized network systems, standards management, and centralized support for use of commercial cloud capabilities. Given the long-term requirements, and the rapid pace of information technology and changing expectations of the user community, EOSDIS has evolved continually over the past three decades. However, many challenges remain. Challenges addressed in this paper include: growing volume and variety, achieving consistency across a diverse set of data producers, managing information about a large number of datasets, migration to a cloud computing environment, optimizing data discovery and access, incorporating user feedback from a diverse community, keeping metadata updated as data collections grow and age, and ensuring that all the content needed for understanding datasets by future users is identified and preserved.

Remote Sensing↗
NASA NTRS2012

Quality Control Algorithms for the Kennedy Space Center 50-Megahertz Doppler Radar Wind Profiler Winds Database

This paper presents the process used by the Marshall Space Flight Center Natural Environments Branch (EV44) to quality control (QC) data from the Kennedy Space Center's 50-MHz Doppler Radar Wind Profiler for use in vehicle wind loads and steering commands. The database has been built to mitigate limitations of using the currently archived databases from weather balloons. The DRWP database contains wind measurements from approximately 2.7-18.6 km altitude at roughly five minute intervals for the August 1997 to December 2009 period of record, and the extensive QC process was designed to remove spurious data from various forms of atmospheric and non-atmospheric artifacts. The QC process is largely based on DRWP literature, but two new algorithms have been developed to remove data contaminated by convection and excessive first guess propagations from the Median Filter First Guess Algorithm. In addition to describing the automated and manual QC process in detail, this paper describes the extent of the data retained. Roughly 58% of all possible wind observations exist in the database, with approximately 100 times as many complete profile sets existing relative to the EV44 balloon databases. This increased sample of near-continuous wind profile measurements may help increase launch availability by reducing the uncertainty of wind changes during launch countdown

Barbre, Robert E., Jr.↗
DOE OSTI2025

User Guide: A Curated Dataset of Regional Meteor Events with Simultaneous Optical and Infrasound Observations

This user guide supports a curated dataset of 71 meteor events recorded between 2006 and 2011 in Southwestern Ontario, Canada. Each event was simultaneously observed by ground-based optical cameras and an infrasound array, providing a rare opportunity to examine meteor trajectories and acoustic signals from the same atmospheric entry events. The dataset includes raw and processed optical data, meteor trajectories, photometric light curves, infrasound waveforms, and atmospheric specifications relevant for acoustic modeling. The archive is structured to support reproducible research in meteor physics, atmospheric acoustics, and shock wave analysis. It is organized following transparent file naming conventions and structured folders to facilitate scientific reuse, comparison, and integration across research domains. The dataset is freely available on Zenodo, doi: 10.5281/zenodo.15868512.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Observations of surface energy fluxes and meteorology in the seasonally snow-covered high-elevation East River watershed during SPLASH, 2021–2023

Abstract. From autumn 2021 through summer 2023, scientists from the National Oceanic and Atmospheric Administration (NOAA) and partners conducted the Study of Precipitation, the Lower Atmosphere, and Surface for Hydrometeorology (SPLASH) campaign in the East River watershed of Colorado. One objective of SPLASH was to observe the transfer of energy between the atmosphere and the surface, which was done at several locations. Two remote sites were chosen that did not have access to power utilities. These were along the valley floor near the East River in the vicinity of the unincorporated town of Gothic, Colorado. Energy balance measurements were made at these locations using autonomous, single-level flux towers referred to as atmospheric surface flux stations (ASFSs). The ASFSs were deployed on 28 September 2021 at the Kettle Ponds Annex site and on 12 October 2021 at the Avery Picnic site and operated until 19 July and 21 June 2023, respectively. Measurements included basic meteorology; upward and downward longwave and shortwave radiative fluxes and subsurface conductive flux, each at 1 min resolution; 3-D winds from a sonic anemometer and H2O/CO2 from an open-path gas analyzer, both at 20 Hz from which sensible, latent heat, and CO2 fluxes were derived; and profiles of soil properties in the upper 0.5 m (both sites) and temperature profiles through the snow (at Avery Picnic), each reported between 10 min and 6 h. The system uptime was 97 % (Kettle Ponds) and 90 % (Avery Picnic), and collectively 1184 d of data was obtained between the stations. The purpose of this article is to document the ASFS deployment at SPLASH, to document the data acquisition and post-processing of measurements, and to serve as a guide for interested users of the data sets, which are archived at Zenodo (https://doi.org/10.5281/zenodo.10313363, Cox et al., 2023b; https://doi.org/10.5281/zenodo.10327409, Cox et al., 2023c; https://doi.org/10.5281/zenodo.10313894, Cox et al., 2023d; https://doi.org/10.5281/zenodo.10307825, Cox et al., 2023e; https://doi.org/10.5281/zenodo.10310520, Cox et al., 2023f) with the Creative Commons Attribution 4.0 International license.

Cox, Christopher J. (ORCID:0000000322037173)↗
NASA NTRS1994

MODIS. Volume 1: MODIS level 1A software baseline requirements

This document describes the level 1A software requirements for the moderate resolution imaging spectroradiometer (MODIS) instrument. This includes internal and external requirements. Internal requirements include functional, operational, and data processing as well as performance, quality, safety, and security engineering requirements. External requirements include those imposed by data archive and distribution systems (DADS); scheduling, control, monitoring, and accounting (SCMA); product management (PM) system; MODIS log; and product generation system (PGS). Implementation constraints and requirements for adapting the software to the physical environment are also included.

Masuoka, Edward↗
NASA NTRS2019

Sherlock Data Warehouse

This slide deck provides an overview of the data and resources available in the Sherlock Data Warehouse. Sherlock was developed and is currently maintained by the Aviation Systems Division at NASA Ames Research Center. Sherlock contains a valuable collection of flight, air traffic management, and weather data. But Sherlock is not just a data archive. Sherlock also includes tools and resources to access, download, and visualize data, as well as resources to process the data. This overview summarizes Sherlock data sources, demonstrates data analytics and visualization with MicroStrategy, illustrates disparate data integration using the ATM Knowledge graph, and presents a machine learning use case using the Big Data system.

data warehouse↗
NASA NTRS2022

CLASSY. II. A Technical Overview of the COS Legacy Archive Spectroscopic Survey

The COS Legacy Archive Spectroscopic SurveY (CLASSY) is designed to provide the community with a spectral atlas of 45 nearby star-forming galaxies that were chosen to cover similar properties to those seen at high z (z > 6). The prime high-level science product of CLASSY is accurately coadded UV spectra, ranging from ∼1000 to 2000 Å, derived from a combination of archival and new data obtained with HST's Cosmic Origins Spectrograph (COS). This paper details the multistage technical processes of creating this prime data product and the methodologies involved in extracting, reducing, aligning, and coadding far-ultraviolet and near-ultraviolet (NUV) spectra. We provide guidelines on how to successfully utilize COS observations of extended sources, despite COS being optimized for point sources, and best-practice recommendations for the coaddition of UV spectra in general. Moreover, we discuss the effects of our reduction and coaddition techniques in the scientific application of the CLASSY data. In particular, we find that accurately accounting for flux calibration offsets can affect the derived properties of the stellar populations, while customized extractions of NUV spectra for extended sources are essential for correctly diagnosing the metallicity of galaxies via C iii] nebular emission. Despite changes in spectral resolution of up to ∼25% between individual data sets (due to changes in the COS line-spread function), no adverse affects were observed on the difference in velocity width and outflow velocities of isolated absorption lines when measured in the final combined data products, owing in part to our signal-to-noise regime of S/N < 20.

Bethan L. James↗
DOE OSTI2025

Electrical Resistivity Tomography data from 2016 to 2018 at the Lower Montane site in the East River Watershed, Colorado

This dataset contains time-lapse Electrical Resistivity Tomography (ERT) data along a transect located on the northeast-facing hillslope at the lower montane site (Pumphouse site) in the upper East River Watershed. The monitoring dataset covers the period from November 2, 2016, to August 6, 2018. In addition, the archive also contains a baseline dataset from October 9, 2016. The ERT transect consisted of 128 electrodes with an electrode spacing of 1.25 m. The acquisition system was located in the middle of the transect, about 50 m on one side, and included an MPT (Multi-Phase Technologies) ERT system, a mini computer, and batteries with solar panels. Acquisition occurred daily under normal circumstances. The first 16 electrodes (from the upper end of the transect) could not be used after the cable was damaged during the 2017–2018 winter. Also, due to multiple failures in the power system, the temporal resolution of the data is much lower in 2018 compared to 2016 and 2017. The data have been processed and used in Dafflon et al., 2023, and the baseline dataset was used in Falco et al., 2019 (see reference list). This archive contains the measurements (ER.zip containing csv files) for each of the 326 acquisition times and a filtered version where only electrodes 17 to 128 are included (ERT_sm.zip containing csv files). The archive also contains the baseline dataset and two acquisitions with full reciprocals (ERT_RB.zip containing csv files), as well as all the raw MPT files (ERT_raw_MTP.zip). The geometry (electrode position and elevation) is provided in Universal Transverse Mercator (UTM) 13N Geoid2012AB in the file named ERT_Location.csv. The archive contains 1 *.csv data files, four *.zip files, and three metadata *.csv files. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Updates in Developing a Prototype Science Pipeline and Full-Volume, Global Hyperspectral Synthetic Data Sets for NASA’s Earth System Observatory’s Upcoming Surface, Biology and Geology Mission

The Surface Biology and Geology (SBG) mission recently passed mission confirmation review and has entered phase A – design and development. SBG will acquire high resolution solar-reflected spectroscopy and thermal infrared observations at a data rate of ~2.5 TB/day and generate products at ~40 TB/day. Given that the per-day volume is greater than NASA’s total extant airborne hyperspectral data collection, collecting, processing, disseminating, and exploiting the SBG data present new challenges. To meet these challenges, we have developed a prototype science pipeline and a full-volume global hyperspectral synthetic data set to help prepare for SBG’s flight (see poster GC42D-0730). Our science pipeline is based on the science processing technology developed for NASA’s Kepler and TESS planet-hunting missions. The pipeline infrastructure, Ziggy, provides a scalable architecture for robust, repeatable, and replicable science and application products that can be run on a range of systems from a laptop to the cloud or a supercomputer. Ziggy is compliant with NASA Procedural Requirement (NPR) 7150.2C, is at a technical readiness level (TRL) of 7 and has been released to github.com/nasa/ziggy. We integrated Ziggy with EO-1/Hyperion workflows to build a prototype pipeline and ingested the 17-year mission archive that provides globally sampled visible through shortwave infrared spectra that are representative of SBG data types and volumes. We fully implemented the first stage and processed the entire 55 TB Hyperion data set from the raw data (Level 0) to top-of-the-atmosphere radiance (Level 1R). We are currently evaluating the ISOFIT atmospheric correction module to convert the L1R data to surface reflectance (Level 2) before reprocessing the full data set to L2. Crosschecks are being performed with RadCalNet as well as with coincident observations by AVIRIS. We are also investigating modern methods for georectifying the Hyperion scenes. Finally, we describe an analysis of the cost to conduct forward processing and reprocessing campaigns for SBG on HECC with dedicated compute and storage resources using the resurrected Hyperion pipeline as a proxy for full-volume SBG data. The analysis demonstrates that SBG L0 data can be processed to L2 on HECC with full reprocessing campaigns every two years for ~$2.6M over a 7-year lifespan. Moreover, 69% of the system capacity would be available for other activities, possibly enabling future open-source science activities, including algorithm development, L3+ processing, .etc.

ESD↗
DOE OSTI2026

Model Data Archive Associated with Manuscript "Fire-altered Carbon Pools Create Disturbance Memory in Stream Dissolved Organic Carbon"

This data package supports the publication “Fire-altered Carbon Pools Create Disturbance Memory in Stream Dissolved Organic Carbon” by Li et al. (2026). The package contains processed model inputs, configuration files, restart files, simulation outputs, scripts, and visualization products used to evaluate post-fire dissolved organic carbon (DOC) dynamics in the Naches River Watershed, Washington, USA, following the 2021 Schneider Springs Fire. The modeling workflow couples ELM-BGC, the biogeochemistry-enabled Energy Exascale Earth System Model Land Model; ATS, the Advanced Terrestrial Simulator for integrated surface-subsurface hydrology; and PFLOTRAN, a reactive transport model for multicomponent aqueous geochemistry. Together, these models simulate how wildfire-induced changes in vegetation, litter, coarse woody debris, and soil organic matter influence DOC production, transport, and reaction from burned hillslopes to stream networks. The archive includes preprocessed meteorological, geospatial, hydrologic, and biogeochemical forcing data; ELM-BGC-derived DOC source terms; ATS mesh files; PFLOTRAN reactive-transport inputs; model configuration files; spin-up and transient restart files; watershed-scale diagnostic outputs; stream concentration time series; and figures or visualization files used to inspect and reproduce key results. File types include Hierarchical Data Format 5 (HDF5) files for gridded forcing and model-coupling data, model input and configuration files for ELM-BGC, ATS, and PFLOTRAN, restart and simulation-output files generated by the modeling workflow, tabular or time-series diagnostic outputs, scripts for post-processing and figure generation, and image or visualization products associated with the manuscript. Use of the package depends on the intended task. Re-running the simulations requires the relevant modeling software, including ELM-BGC, ATS, and PFLOTRAN as ATS's geochemical engine. Inspecting outputs and reproducing figures requires Python with scientific plotting libraries such as Matplotlib, and three-dimensional model outputs may be viewed with ParaView. Geographic information system files or maps may be inspected with ArcGIS Pro or comparable GIS software. The data package is intended to enable traceability, reuse, and partial reproduction of the coupled land-to-watershed hydro-biogeochemical modeling workflow used to test how wildfire disturbance affects terrestrial carbon pools and downstream DOC dynamics.

ATS↗
NASA NTRS2000

Data Information for Global Change Studies: NASA's Distributed Active Archive Centers and Cooperating Data Centers

The Earth Observing System (EOS) is an integral part of the National Aeronautics and Space Administration's (NASA's) Earth Science Enterprise (ESE). ESE is a long-term global change research program designed to improve our understanding of the Earth's interrelated processes involving the atmosphere, oceans, land surfaces, and polar regions. Data from EOS instruments and other Earth science measurement systems are useful in understanding the causes and processes of global climate change and the consequences of human activities. The EOS Data and Information System (EOSDIS) provides a structure for data management and user services for products derived from EOS satellite instruments and other NASA Earth science data. Within the EOSDIS framework, the Distributed Active Archive Centers (DAACs) have been established to provide expertise in one or more Earth science disciplines. The DAACs and cooperating data centers provide data and information services to support the global change research community. Much of the development of the DAACs has been in anticipation of the enormous amount of data expected from EOS instruments to be launched within the next two decades. Terra, the EOS flagship launched in December 1999, is the first of a series of EOS satellites to carry several instruments with multispectral capabilities. Some data products from these instruments are now available from several of the DAACs. These and other data products can be ordered through the EOS Data Gateway (EDG) and DAAC-specific online ordering systems.

Source record↗
NASA NTRS2014

Multilateral Biomedical Data Sharing in the One-year Joint US-Russian Mission on the International Space Station

The One Year Mission (1YM) by two astronauts on the International Space Station (ISS), starting in March 2015, offers a unique opportunity to expand multilateral collaboration by sharing data and resources among the partner agencies in preparation for planned space exploration missions beyond low Earth orbit. Agreements and protocols will be established for the collection, distribution, analysis and reporting of both research and clinical data. Data will be shared between the agencies sponsoring the investigators, and between the research and clinical medicine communities where common interests are identified. The assignment of only two astronauts, one Russian and the other American, to the 1YM necessitated creativity in bilateral efforts to maximize the biomedical return from the opportunity. Addition of Canadian, European and Japanese investigations make the effort even more integrative. There will be three types of investigations: joint, cross‐participation and data‐exchange. The joint investigations have US and Russian coprincipal investigators, and the data acquired will be their common responsibility. The other two types must develop data sharing agreements and processes specific to their needs. A multilateral panel of ISS partner space agencies will develop policies for international exchange of scientific information to meet their science objectives and priorities. They will promote archiving of space flight data and will inform each other and the scientific community at large about the results obtained from space life sciences studies. Integration tasks for the 1YM are based on current experience from the ISS and previous efforts on the Russian space station Mir. Closer coordination between international partners requires more common approaches to remove barriers to multilateral resource utilization on the ISS. Greater integration in implementation should increase utilization efficiency to benefit all participants in spaceflight human research. This presentation will describe the overarching principles for multilateral data collection, analysis and sharing and for data security for medical and research data shared between ISS partners prior to release in public forums.

Charles, John B.↗
NASA NTRS1988

Requirements for ongoing development of the Pilot Land Data System (PLDS)

The Pilot Land Data System being developed to address the information processing needs of the NASA land sciences research community is presented. The objective of the pilot program is to establish a limited-scale, distributed information system for the archival, location, transfer, integration, and manipulation of data across multiple sites connected by a high-speed communications network. Functional capabilities required for users to create, access, and maintain local and distributed data bases containing various types of data in support of land sciences research are summarized.

Wharton, S. W.↗
NASA NTRS2006

EOS MLS Science Data Processing System: A Description of Architecture and Capabilities

This paper describes the architecture and capabilities of the Science Data Processing System (SDPS) for the EOS MLS. The SDPS consists of two major components--the Science Computing Facility and the Science Investigator-led Processing System. The Science Computing Facility provides the facilities for the EOS MLS Science Team to perform the functions of scientific algorithm development, processing software development, quality control of data products, and scientific analyses. The Science Investigator-led Processing System processes and reprocesses the science data for the entire mission and delivers the data products to the Science Computing Facility and to the Goddard Space Flight Center Earth Science Distributed Active Archive Center, which archives and distributes the standard science products.

Microwave Limb Sounder (MLS)↗
NASA NTRS1976

NASA Global Atmospheric Sampling Program (GASP) data report for tape VL0003

Atmospheric ozone, and related flight and meteorological data for May 1975, obtained during 49 flights of a Pan American World Airways 747, are available as GASP tape VL0003 from the National Climatic Center, Asheville, North Carolina. In addition to the GASP data, tropopause pressure fields obtained from NMC archives for the dates of the GASP flights are included on the data tape. Flight routes and dates, instrumentation, data processing procedures, and data tape specifications are described.

Holdeman, J. D.↗
NASA NTRS1977

NASA Global Atmospheric Sampling Program (GASP) data report for tape VL0005

Atmospheric ozone, water vapor, and related flight and meteorological data were obtained during 214 flights of a United Airlines B-747 and two Pan American World Airways B-747's from March through June 1976. In addition, trichlorofluoromethane data obtained from laboratory analysis of two whole air samples collected in flight are reported. These data are available on GASP tape VL0005 from the National Climatic Center, Asheville, North Carolina. In addition to the GASP data, tropopause pressure fields obtained from NMC archives for the dates of the GASP flights are included on the data tape. Flight routes and dates, instrumentation, data processing procedures, and data tape specifications are described in this report. Selected analyses including ozone and sample bottle data are also presented.

Holdeman, J. D.↗
NASA NTRS1979

NASA Global Atmospheric Sampling Program (GASP) data report for tapes VL0010 and VL0012

The GASP atmospheric trace constituent data currently available are considered. Included on tapes are in-situ measurements of atmospheric ozone, carbon monoxide, water vapor, and clouds, data from laboratory analysis of filters exposed in flight, and related flight and meteorological data. Measurements of ozone levels within the first class cabin of these aircraft are also reported. In addition to the GASP data, tropopause pressures obtained from time and space interpolation of NMC archived data for the dates of the flights are included. Reported herein are the flight routes and dates, instrumentation, data processing procedures, data tape specifications, and analyses of the cabin ozone measurements.

Holdeman, J. D.↗
NASA NTRS1977

NASA Global Atmospheric Sampling Program (GASP) data report for tapes VL0007 and VL0008

The Global Atmospheric Sampling Program (GASP) is obtaining measurements of atmospheric trace constituents in the upper troposphere and lower stratosphere using fully automated air sampling systems on board the NASA CV-990 research aircraft and four commerical B-747 aircraft in routine airline service. In-situ measurements of atmospheric ozone and water vapor, data from laboratory analysis of filters exposed in flight, and related flight and meteorological data obtained from September 1976 through January 1977 are reported. These data are now available on GASP tapes VL0007 & VL0008 from the National Climatic Center, Asheville, North Carolina. In addition to the GASP data, tropopause pressure fields obtained from NMC archives for the dates of the GASP flights are included on the data tape. Flight routes and dates, instrumentation, data processing procedures, and data tape specifications are described.

Holdeman, J. D.↗