SEARCH · Search NASA

Results for “data processing and archiving”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

All sources NASA NTRS DOE OSTI

NASA NTRS2019

Providing Data Access and Analysis Capabilities to SERVIR’s Data-Sparse Regions

In developing regions of the world, the communications infrastructure pose enormous challenges for using Earth observation data. Limited internet bandwidth along with the high costs make it almost impossible to process and extract zonal statistics over large periods of time for even small geographic areas. In such cases, downloading daily rainfall data or dekadal series of NDVI data would take days and consume all the bandwidth allocated to an organization (for reference, internet connections in Niger would cost thousands of dollars per month at a maximum - and unreliable - bandwidth of just 10 Mbps). Running crop models or hydrological models typically require several years of historic data over the area of interest (AOI). In some cases, these AOIs are relatively small compared to the footprint of individual earth observation granules. Hence, systems that let the stakeholders subset the data to download to a user specified area, or even submit processing requests that let them download small result files for the AOI become critical. The SERVIR program has developed a tool to provide this type of access to help decision makers in developing regions use long time series of adjusted rainfall data (CHIRPS), NDVI values, seasonal weather forecasts, evaporative stress indices and others in a very efficient manner. This system, named ClimateSERV (https://climateserv.servirglobal.net) ingests the datasets in an automated fashion and allows interactive access (through a web application), or automated access through a simple API that developers can quickly incorporate in independent applications. This way, the extraction of daily averages of rainfall over a 50 square Km area through 30 years of archived data takes only a few seconds to process, and the results can be presented on an online chart or downloaded in a comma separated file that's only a few Kb.

Ashmall, William↗

NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly.

Lynnes, Christopher↗

NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

Lynnes, Christopher↗

NASA NTRS2006

Archiving Mars Mission Data Sets with the Planetary Data System

This viewgraph presentation reviews the use of the Planetary Data System (PDS) to archive the datasets that are received from the Mars Missions. It reviews the lessons learned in the actual archiving process, and presents an overview of the actual archiving process. It also reviews the lessons learned from the perspectives of the projects, the data producers and the data users.

Planetary Data System (PDS)↗

NASA NTRS1993

EOS ground data systems: A description and interface overview

The Earth Observing System (EOS) is planned as a space-based measurement system, earth-science research program, and data and information system (EOSDIS). It will consist of several high data rate spacecraft with multiple earth sensing instruments which provide investigators with a thorough, longterm view of the earth's environment. Up to seven spacecraft may be supported at once, either in operational, checkout, or testing phases; and the average data rate from the EOS satellites in orbit at any one time is expected to be from 18 to 60 Mbps. Providing the data processing and flight operations support for EOS will be the EOSDIS Core System (ECS). The ECS will command and control the spacecraft; process and store the EOS data; provide access to the data for years; and support researchers. The data processing aspects of the ECS consist of a collection of Distributed Active Archive Centers (DAAC's) which perform the product generation, data archive and distribution, and information management services. Flight operations aspects will be provided by the EOS Operations Center, by instrument control centers, and by widely distributed instrument support terminals. The communications and system management aspects will be provided by the EOSDIS Science Network and the System Management Center. In addition to the EOS satellite data, other data sets from earlier earth science missions are also to be added to designated DAAC's. Other ground data systems which will provide support to EOS for acquiring, transporting, processing, and distributing the transformed spacecraft data are currently being defined or are being upgraded for the EOS era. These systems include the Space Network consisting of the Tracking and Data Relay Satellite System (TDRSS), the TDRSS Ground Terminals, and the Network Control Center as well as the night Dynamics Facility, the EOS Data and Operations System, and EOS Communications. This paper briefly describes data handling by the ECS, the support data systems, their interfaces, and their roles.

Smith, Gene↗

NASA NTRS1995

Summary of OARE flight calibration measurements

To date, the Orbital Acceleration Research Experiment (OARE) has flown on the shuttle orbiter for five missions; namely, STS-40, STS-50, STS-58, STS-62, and STS-65. The OARE instrument system contains a three-axis accelerometer which can resolve accelerations to the nano-g (10(exp -9) g) level and a full calibration station to permit in situ bias and scale factor calibration measurements. This calibration capability eliminates the large uncertainty encountered with accelerometers flown in the past on the orbiter which use ground-based calibrations to provide absolute acceleration measurements. A detailed flight data report presentation is given for the OARE calibration measurements from all missions, along with an estimate of the calibration errors. The main aim is to collect, process, and present the calibration data in one archival report. These calibration data are the necessary key ingredient to produce the absolute acceleration levels from the OARE acceleration flight data.

Blanchard, Robert C.↗

NASA NTRS2001

SIMBIOS Project Data Processing and Analysis Results

The Sensor Intercomparison and Merger for Biological and Interdisciplinary Oceanic Studies (SIMBIOS) Project is concerned with ocean color satellite sensor data intercomparison and merger for biological and interdisciplinary studies of the global oceans. Imagery from different ocean color sensors can now be processed by a single software package using the same algorithms, adjusted by different sensor spectral characteristics, and the same ancillary meteorological and environmental data. This enables cross-comparison and validation of the data derived from satellite sensors and, consequently, creates continuity in ocean color information on both the temporal and spatial scale. The next step in this process is the integration of in situ ocean and atmospheric parameters to enable cross-validation and further refinement of the ocean color methodology. The SIMBIOS Project Office accomplishments during 2000 year are summarized under satellite data processing, data product validation, SeaWiFS Bio-Optical Archive and Storage System (SeaBASS) database, supporting services, sun photometers and calibration activities, and calibration round robins. These accomplishments are described.

Ainsworth, Ewa↗

NASA NTRS2004

The X-Ray Variable Sky

A novel approach to search for rapid time variability of every source, whether persistent or transient, within the entire ROSAT detector field has been developed. Semi-automated software statistically tests the photon arrival time event list for source variability and reports significant (greater than 5 sigma) variable enhancement in the photon count rate on time scales ranging from 5-500 seconds. The computational algorithms have been thoroughly tested, and are able to discriminate between false detections induced by instrumental effects, and real X-ray variable sources (XRVs); when carefully scrutinized by the PI. The analysis has proven to be successful, with both periodic and transient XRVs having been found. Some of these are entirely new discoveries, while others have been detected in previous X-ray missions, yet their nature remained elusive since source identification and analysis had yet to be performed. On average, one new (unidentified) XRV is found per ROSAT PSPC field analyzed. In some cases the XRVs show only a transient signature (a burst type event) with no persistent X-ray or other known (optical) counterpart. The ambitious goal of this project was to process as much of the ROSAT PSPC archive data as feasibly possible to discover XRVs and analyze this data.

Stringfellow, Guy↗

NASA NTRS2009

Development of a Compact, Pulsed, 2-Micron, Coherent-Detection, Doppler Wind Lidar Transceiver; and Plans for Flights on NASA's DC-8 and WB-57 Aircraft

We present results of a recently completed effort to design, fabricate, and demonstrate a compact lidar transceiver for coherent-detection lidar profiling of winds. The novel high-energy, 2-micron, Ho:Tm:LuLiF laser technology developed at NASA Langley was employed to permit study of the laser technology currently envisioned by NASA for global coherent Doppler lidar measurement of winds in the future. The 250 mJ, 10 Hz compact transceiver was also designed for future aircraft flight. Ground-based wind profiles made with this transceiver will be presented. NASA Langley is currently funded to build complete Doppler lidar systems using this transceiver for the DC-8 and WB-57 aircraft. The WB-57 flights will present a more severe environment and will require autonomous operation of the lidar system. The DC-8 lidar system is a likely component of future NASA hurricane research. It will include real-time data processing and display, as well as full data archiving. We will attempt to co-fly on both aircraft with a direct-detection Doppler wind lidar system being prepared by NASA Goddard Space Flight Center.

Kavaya, Michael J.↗

NASA NTRS2015

Rapid, High-Resolution Detection of Environmental Change over Continental Scales from Satellite Data - the Earth Observation Data Cube

The effort and cost required to convert satellite Earth Observation (EO) data into meaningful geophysical variables has prevented the systematic analysis of all available observations. To overcome these problems, we utilise an integrated High Performance Computing and Data environment to rapidly process, restructure and analyse the Australian Landsat data archive. In this approach, the EO data are assigned to a common grid framework that spans the full geospatial and temporal extent of the observations - the EO Data Cube. This approach is pixel-based and incorporates geometric and spectral calibration and quality assurance of each Earth surface reflectance measurement. We demonstrate the utility of the approach with rapid time-series mapping of surface water across the entire Australian continent using 27 years of continuous, 25 m resolution observations. Our preliminary analysis of the Landsat archive shows how the EO Data Cube can effectively liberate high-resolution EO data from their complex sensor-specific data structures and revolutionise our ability to measure environmental change.

Lewis, Adam↗

NASA NTRS2000

BOREAS Level 3-b AVHRR-LAC Imagery: Scaled At-sensor Radiance in LGSOWG Format

The BOREAS Staff Science Satellite Data Acquisition Program focused on providing the research teams with the remotely sensed satellite data products they needed to compare and spatially extend point results. Data acquired from the AVHRR instrument on the NOAA-9, -11, -12, and -14 satellites were processed and archived for the BOREAS region by the MRSC and BORIS. The data were acquired by CCRS and were provided for use by BOREAS researchers. A few winter acquisitions are available, but the archive contains primarily growing season imagery. These gridded, at-sensor radiance image data cover the period of 30-Jan-1994 to 18-Sep-1996. Geographically, the data cover the entire 1,000-km x 1,000-km BOREAS region. The data are stored in binary image format files.

Hall, Forrest G.↗

NASA NTRS2007

A Comprehensive Plan for the Long-Term Calibration and Validation of Oceanic Biogeochemical Satellite Data

The primary objective of this planning document is to establish a long-term capability and validating oceanic biogeochemical satellite data. It is a pragmatic solution to a practical problem based primarily o the lessons learned from prior satellite missions. All of the plan's elements are seen to be interdependent, so a horizontal organizational scheme is anticipated wherein the overall leadership comes from the NASA Ocean Biology and Biogeochemistry (OBB) Program Manager and the entire enterprise is split into two components of equal sature: calibration and validation plus satellite data processing. The detailed elements of the activity are based on the basic tasks of the two main components plus the current objectives of the Carbon Cycle and Ecosystems Roadmap. The former is distinguished by an internal core set of responsibilities and the latter is facilitated through an external connecting-core ring of competed or contracted activities. The core elements for the calibration and validation component include a) publish protocols and performance metrics; b) verify uncertainty budgets; c) manage the development and evaluation of instrumentation; and d) coordinate international partnerships. The core elements for the satellite data processing component are e) process and reprocess multisensor data; f) acquire, distribute, and archive data products; and g) implement new data products. Both components have shared responsibilities for initializing and temporally monitoring satellite calibration. Connecting-core elements include (but are not restricted to) atmospheric correction and characterization, standards and traceability, instrument and analysis round robins, field campaigns and vicarious calibration sites, in situ database, bio-optical algorithm (and product) validation, satellite characterization and vicarious calibration, and image processing software. The plan also includes an accountability process, creating a Calibration and Validation Team (to help manage the activity), and a discussion of issues associated with the plan's scientific focus.

Hooker, Stanford B.↗

NASA NTRS2010

Kepler Data Release 3 Notes

This describes the collection of data and the processing done on it so when researchers around the world get the Kepler data sets (which are a set of pixels from the telescope of a particular target (star, galaxy or whatever) over a 3 month period) they can adjust their algorithms fro things that were done (like subtracting all of one particular wavelength for example). This is used to calibrate their own algorithms so that they know what it is they are starting with. It is posted so that whoever is accessing the publicly available data (not all of it is made public) can understand it .. (most of the Kepler data is under restriction for 1 - 4 years and is not available, but the handbook is for everyone (public and restricted) The Data Analysis Working Group have released long and short cadence materials, including FFls and Dropped Targets for the Public. The Kepler Science Office considers Data Release 3 to provide "browse quality" data. These notes have been prepared to give Kepler users of the Multimission Archive at STScl (MAST) a summary of how the data were collected and prepared, and how well the data processing pipeline is functioning on flight data. They will be updated for each release of data to the public archive and placed on MAST along with other Kepler documentation, at http:// archive.stsci.edu/kepler/documents.html .Data release 3 is meant to give users the opportunity to examine the data for possibly interesting science and to involve the users in improving the pipeline for future data releases. To perform the latter service, users are encouraged to notice and document artifacts, either in the raw or processed data, and report them to the Science Office.

Cleve, Jeffrey E.↗

NASA NTRS2011

Development of the SOFIA Image Processing Tool

The Stratospheric Observatory for Infrared Astronomy (SOFIA) is a Boeing 747SP carrying a 2.5 meter infrared telescope capable of operating between at altitudes of between twelve and fourteen kilometers, which is above more than 99 percent of the water vapor in the atmosphere. The ability to make observations above most water vapor coupled with the ability to make observations from anywhere, anytime, make SOFIA one of the world s premiere infrared observatories. SOFIA uses three visible light CCD imagers to assist in pointing the telescope. The data from these imagers is stored in archive files as is housekeeping data, which contains information such as boresight and area of interest locations. A tool that could both extract and process data from the archive files was developed.

Adams, Alexander N.↗

NASA NTRS1987

SAR image data compression for an on-line archive system

This paper summarizes the investigation of SAR image data compression for an on-line archive data distribution system. This system is planned for the ground processing system of Alaska SAR Facility (ASF) and Shuttle Imaging Radar (SIR-C). The objective of the SAR image data compression is to enable the data archive system to provide the remote users a large data base with good image quality, short response time, low transfer cost, and minimal decoding complexity. The requirements and limitations of the on-line archive data distribution system are presented. The effects of SAR image data characteristics on data compression are addressed. The users' survey results suggest that compression ratios between 10:1 and 20:1 appear suitable. Based on the algorithm evaluation results, the two-level tree-searched vector quantization technique has been recommended as the SAR image data compression algorithm for the on-line archive data distribution system.

Chang, C. Y.↗

NASA NTRS1998

EDCATS: An Evaluation Tool

The purpose of this research is to explore the development of Marshall Space Flight Center Unique Programs. These academic tools provide the Education Program Office with important information from the Education Computer Aided Tracking System (EDCATS). This system is equipped to provide on-line data entry, evaluation, analysis, and report generation, with full archiving for all phases of the evaluation process. Another purpose is to develop reports and data that is tailored to Marshall Space Flight Center Unique Programs. It also attempts to acquire knowledge on how, why, and where information is derived. As a result, a user will be better prepared to decide which available tool is the most feasible for their reports.

Heard, Pamala D.↗

NASA NTRS2017

A Relevancy Algorithm for Curating Earth Science Data Around Phenomenon

Earth science data are being collected for various science needs and applications, processed using different algorithms at multiple resolutions and coverages, and then archived at different archiving centers for distribution and stewardship causing difficulty in data discovery. Curation, which typically occurs in museums, art galleries, and libraries, is traditionally defined as the process of collecting and organizing information around a common subject matter or a topic of interest. Curating data sets around topics or areas of interest addresses some of the data discovery needs in the field of Earth science, especially for unanticipated users of data. This paper describes a methodology to automate search and selection of data around specific phenomena. Different components of the methodology including the assumptions, the process, and the relevancy ranking algorithm are described. The paper makes two unique contributions to improving data search and discovery capabilities. First, the paper describes a novel methodology developed for automatically curating data around a topic using Earthscience metadata records. Second, the methodology has been implemented as a standalone web service that is utilized to augment search and usability of data in a variety of tools.

earth science phenomena↗

NASA NTRS2016

Effects of Polar Stratospheric Clouds in the Nimbus 7 LIMS Version 6 Data Set

The historic Limb Infrared Monitor of the Stratosphere (LIMS) measurements of 1978-1979 from the Nimbus 7 satellite were re-processed with Version 6 (V6) algorithms and archived in 2002. The V6 data set employs updated radiance registration methods, improved spectroscopic line parameters, and a common vertical resolution for all retrieved parameters. Retrieved profiles are spaced about every 1.6 of latitude along orbits and include the additional parameter of geopotential height. Profiles of O3 are sensitive to perturbations from emissions of polar stratospheric clouds (PSCs). This work presents results of implementing a first-order screening for effects of PSCs using simple algorithms based on vertical gradients of the O3 mixing ratio. Their occurrences are compared with the co-located, retrieved temperatures and related to the temperature thresholds needed for saturation of H2O and/or HNO3 vapor onto PSC particles. Observed daily locations where the major PSC screening criteria are satisfied are validated against PSCs observed with the Stratospheric Aerosol Monitor (SAM) II experiment also on Nimbus 7. Remnants of emissions from PSCs are characterized for O3 and HNO3 following the screening. PSCs may also impart a warm bias in the co-located LIMS temperatures, but by no more than 1-2K at the altitudes of where effects of PSCs are a maximum in the ozone; thus, no PSC screening was applied to the V6 temperatures. Minimum temperatures vary between 187 and 194K and often occur 1 to 2 km above where PSC effects are first identified in the ozone (most often between about 21 and 28 hPa). Those temperature-pressure values are consistent with conditions for the existence of nitric acid trihydrate (NAT) mixtures and to a lesser extent of super-cooled ternary solution (STS) droplets. A local, temporary uptake of HNO3 vapor of order 1-3 ppbv is indicated during mid-January for the 550K surface. Seven-month time series of the distributions of LIMS O3 and HNO3 are shown based on their gridded Level 3 data following the PSC screening. Zonal coefficients of both species are essentially free of effects from PSCs on the 550K surface, based on their average values along PV contours and in terms of equivalent latitude. Remnants of PSCs are still present in O3 on the 450K surface during mid- January. It is judged that the LIMS Level 3 data are of good quality for analyzing the larger-scale, stratospheric chemistry and transport processes during the Arctic winter of 1978- 1979.

Remsberg, Ellis↗