SEARCH · Search NASA

Results for “data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

NASA NTRS2003

Arithmetic Data Cube as a Data Intensive Benchmark

Data movement across computational grids and across memory hierarchy of individual grid machines is known to be a limiting factor for application involving large data sets. In this paper we introduce the Data Cube Operator on an Arithmetic Data Set which we call Arithmetic Data Cube (ADC). We propose to use the ADC to benchmark grid capabilities to handle large distributed data sets. The ADC stresses all levels of grid memory by producing 2d views of an Arithmetic Data Set of d-tuples described by a small number of parameters. We control data intensity of the ADC by controlling the sizes of the views through choice of the tuple parameters.

Frumkin, Michael A.↗

NASA NTRS2003

Validation of Envisat Ozone Data Products using Satellite and Ground Based Data

The Envisat chemistry instruments provide an opportunity to continue the ozone data sets produced by TOMS, SBUV, HALOE, and SAGE. These data sets have already been extensively validated and are assured to be of high accuracy. Continuity and consistency of data sets among the various satellite instruments as well as ground networks is essential for detecting an ozone recovery and climate change. Our objective, as part of the Envisat CaWal program, is to validate SCIAMACHY, MIPAS, and GOMOS ozone products using these heritage instruments. In addition we plan to validate SCIAMACHY Level 1 radiance using heritage satellite data and ground observations using a radiative transfer model. Ground based data included US ozonesonde profiles and a double monochromator Brewer instrument located at Goddard Space Flight Center. To date we have tested our intercomparisons algorithms using available Envisat data compared with SBW/2 and the ground based observations. We analyzed the comparisons with respect to ozone amounts and latitude and found the differences that were expected based on the initial release of Envisat data, which had known deficiencies. We will present our latest results based on the most current release of Envisat ozone data products.

Hilsenrath, E.↗

NASA NTRS2004

Inner Radiation Belt Representation of the Energetic Electron Environment: Model and Data Synthesis Using the Salammbo Radiation Belt Transport Code and Los Alamos Geosynchronous and GPS Energetic Particle Data

The highly energetic electron environment in the inner magnetosphere (GEO inward) has received a lot of research attention in resent years, as the dynamics of relativistic electron acceleration and transport are not yet fully understood. These electrons can cause deep dielectric charging in any space hardware in the MEO to GEO region. We use a new and novel approach to obtain a global representation of the inner magnetospheric energetic electron environment, which can reproduce the absolute environment (flux) for any spacecraft orbit in that region to within a factor of 2 for the energy range of 100 KeV to 5 MeV electrons, for any levels of magnetospheric activity. We combine the extensive set of inner magnetospheric energetic electron observations available at Los Alamos with the physics based Salammbo transport code, using the data assimilation technique of "nudging". This in effect input in-situ data into the code and allows the diffusion mechanisms in the code to interpolate the data into regions and times of no data availability. We present here details of the methods used, both in the data assimilation process and in the necessary inter-calibration of the input data used. We will present sample runs of the model/data code and compare the results to test spacecraft data not used in the data assimilation process.

Friedel, R. H. W.↗

NASA NTRS2005

Large Scale Data Mining to Improve Usability of Data: An Intelligent Archive Testbed

Research in certain scientific disciplines - including Earth science, particle physics, and astrophysics - continually faces the challenge that the volume of data needed to perform valid scientific research can at times overwhelm even a sizable research community. The desire to improve utilization of this data gave rise to the Intelligent Archives project, which seeks to make data archives active participants in a knowledge building system capable of discovering events or patterns that represent new information or knowledge. Data mining can automatically discover patterns and events, but it is generally viewed as unsuited for large-scale use in disciplines like Earth science that routinely involve very high data volumes. Dozens of research projects have shown promising uses of data mining in Earth science, but all of these are based on experiments with data subsets of a few gigabytes or less, rather than the terabytes or petabytes typically encountered in operational systems. To bridge this gap, the Intelligent Archives project is establishing a testbed with the goal of demonstrating the use of data mining techniques in an operationally-relevant environment. This paper discusses the goals of the testbed and the design choices surrounding critical issues that arose during testbed implementation.

Ramapriyan, Hampapuram↗

NASA NTRS2006

Effects of data selection on the assimilation of AIRS data

The Atmospheric InfraRed Sounder (AIRS), flying aboard NASA's Earth Observing System (EOS) Aqua satellite with the Advanced Microwave Sounding Unit-A (AMSU-A), has been providing data for use in numerical weather prediction (NWP) and data assimilation systems (DAS) for over three years. The full AIRS data set is currently not transmitted in near-real-time (NRT) to the NWP centers. Instead, data sets with reduced spatial and spectral information are produced and made available in NRT. In this paper, we evaluate the use of different channel selections and error specifications. We achieved significant positive impact from the Aqua AIRS/AMSU-A combination in both hemispheres during our experimental time period of January 2003. The best results were obtained using a set of 156 channels that did not include any in the 6.7micron water vapor band. The latter have a large influence on both temperature and humidity analyses. If observation and background errors are not properly specified, the partitioning of temperature and humidity information from these channels will not be correct, and this can lead to a degradation in forecast skill. We found that changing the specified channel errors had a significant effect on the amount of data that entered into the analysis as a result of quality control thresholds that are related to the errors. However, changing the channel errors within a relatively small window did not significantly impact forecast skill with the 155 channel set. We also examined the effects of different types of spatial data reduction on assimilated data sets and NWP forecast skill. Whether we picked the center or the warmest AIRS pixel in a 3x3 array affected the amount of data ingested by the analysis but had a negligible impact on the forecast skill.

Joiner, Joanna↗

NASA NTRS2007

System and method for transferring data on a data link

A system and method are provided for transferring a packet across a data link. The packet may include a stream of data symbols which is delimited by one or more framing symbols. Corruptions of the framing symbol which result in valid data symbols may be mapped to invalid symbols. If it is desired to transfer one of the valid data symbols that has been mapped to an invalid symbol, the data symbol may be replaced with an unused symbol. At the receiving end, these unused symbols are replaced with the corresponding valid data symbols. The data stream of the packet may be encoded with forward error correction information to detect and correct errors in the data stream.

Cole, Robert M.↗

NASA NTRS2008

The NPOESS Preparatory Project Science Data Segment (SDS) Data Depository and Distribution Element (SD3E) System Architecture

The National Polar-orbiting Operational Environmental Satellite System (NPOESS) Preparatory Project (NPP) Science Data Segment (SDS) will make daily data requests for approximately six terabytes of NPP science products for each of its six environmental assessment elements from the operational data providers. As a result, issues associated with duplicate data requests, data transfers of large volumes of diverse products, and data transfer failures raised concerns with respect to the network traffic and bandwidth consumption. The NPP SDS Data Depository and Distribution Element (SD3E) was developed to provide a mechanism for efficient data exchange, alleviate duplicate network traffic, and reduce operational costs.

Ho, Evelyn L.↗

NASA NTRS2009

Global Land Data Assimilation System (GLDAS) Products, Services and Application from NASA Hydrology Data and Information Services Center (HDISC)

The Global Land Data Assimilation System (GLDAS) is generating a series of land surface state (e.g., soil moisture and surface temperature) and flux (e.g., evaporation and sensible heat flux) products simulated by four land surface models (CLM, Mosaic, Noah and VIC). These products are now accessible at the Hydrology Data and Information Services Center (HDISC), a component of the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). Current data holdings include a set of 1.0 degree resolution data products from the four models, covering 1979 to the present; and a 0.25 degree data product from the Noah model, covering 2000 to the present. The products are in Gridded Binary (GRIB) format and can be accessed through a number of interfaces. Users can search the products through keywords and perform on-the-fly spatial and parameter subsetting and format conversion of selected data. More advanced visualization, access and analysis capabilities will be available in the future. The long term GLDAS data are used to develop climatology of water cycle components and to explore the teleconnections of droughts and pluvial.

Fang, Hongliang↗

NASA NTRS2010

Terrestrial Hydrological Data from NASA's Hydrology Data and Information Services Center (HDISC): Products, Services, and Applications

Terrestrial hydrological variables are important in global hydrology, climate, and carbon cycle studies. The North American and Global Land Data Assimilation Systems (NLDAS and GLDAS, respectively) have been generating a series of land surface states (soil moisture, snow, and temperature) and fluxes (evapotranspiration, radiation, and heat flux) variables. These data, hosted at and available from NASA s Hydrology Data and Information Services Center (HDISC), include the NLDAS hourly 1/8 degree products and the GLDAS 3-hourly 0.25 and 1.0 degree products. HDISC provides easy access and visualization and analysis capabilities for these products, thus reducing the time and resources spent by scientists on data management and facilitating hydrological research. Users can perform spatial and parameter subsetting, data format transformation, and data analysis operations without needing to first download the data. HDISC is continually being developed as a data and services portal that supports weather and climate forecasts, and water and energy cycle research.

Fang, Hongliang↗

NASA NTRS2013

The GLAS Standard Data Products Specification-Data Dictionary, Version 1.0

The Geoscience Laser Altimeter System (GLAS) is the primary instrument for the ICESat (Ice, Cloud and Land Elevation Satellite) laser altimetry mission. ICESat was the benchmark Earth Observing System (EOS) mission for measuring ice sheet mass balance, cloud and aerosol heights, as well as land topography and vegetation characteristics. From 2003 to 2009, the ICESat mission provided multi-year elevation data needed to determine ice sheet mass balance as well as cloud property information, especially for stratospheric clouds common over polar areas. It also provided topography and vegetation data around the globe, in addition to the polar-specific coverage over the Greenland and Antarctic ice sheets.This document contains the data dictionary for the GLAS standard data products. It details the parameters present on GLAS standard data products. Each parameter is defined with a short name, a long name, units on product, type of variable, a long description and products that contain it. The term standard data products refers to those EOS instrument data that are routinely generated for public distribution. These products are distributed by the National Snow and Ice Data Center (NSDIC).

Lee, Jeffrey E.↗

NASA NTRS2015

Rapid, High-Resolution Detection of Environmental Change over Continental Scales from Satellite Data - the Earth Observation Data Cube

The effort and cost required to convert satellite Earth Observation (EO) data into meaningful geophysical variables has prevented the systematic analysis of all available observations. To overcome these problems, we utilise an integrated High Performance Computing and Data environment to rapidly process, restructure and analyse the Australian Landsat data archive. In this approach, the EO data are assigned to a common grid framework that spans the full geospatial and temporal extent of the observations - the EO Data Cube. This approach is pixel-based and incorporates geometric and spectral calibration and quality assurance of each Earth surface reflectance measurement. We demonstrate the utility of the approach with rapid time-series mapping of surface water across the entire Australian continent using 27 years of continuous, 25 m resolution observations. Our preliminary analysis of the Landsat archive shows how the EO Data Cube can effectively liberate high-resolution EO data from their complex sensor-specific data structures and revolutionise our ability to measure environmental change.

Lewis, Adam↗

NASA NTRS2019

Improving Satellite Global Chlorophyll a [alpha] Data Products Through Algorithm Refinement and Data Recovery

A recently developed algorithm to estimate surface ocean chlorophyll a concentrations (Chl in milligrams per cubic meter), namely, the ocean color index (OCI) algorithm, has been adopted by the U.S. National Aeronautics and Space Administration to apply to all satellite ocean color sensors to produce global Chl maps. The algorithm is a hybrid between a band‐difference color index algorithm for low‐Chl waters and the traditional band‐ratio algorithms (OCx) for higher‐Chl waters. In this study, the OCI algorithm is revisited for its algorithm coefficients and for its algorithm transition between color index and OCx using a merged data set of high‐performance liquid chromatography and fluorometric Chl. Results suggest that the new OCI algorithm (OCI2) leads to lower Chl estimates than the original OCI (OCI1) for Chl less than 0.05 milligrams per cubic meter, but smoother algorithm transition for Chl between 0.25 and 0.40 milligrams per cubic meter. Evaluation using in situ data suggests that similar to OCI1, OCI2 has significantly improved image quality and cross‐sensor consistency between SeaWiFS (Sea-viewing Wide Field-of-view Sensor), MODISA (Moderate Resolution Imaging Spectroradiometer on Aqua), and VIIRS (Visible Infrared Imaging Radiometer Suite) over the OCx algorithms for oligotrophic oceans. Mean cross‐sensor difference in monthly Chl data products over global oligotrophic oceans reduced from approximately 10 percent for OCx to 1-2 percent for OCI2. More importantly, data statistics suggest that the current straylight masking scheme used to generate global Chl maps can be relaxed from 7 by 5 to 3 by 3 pixels without losing data quality in either Chl or spectral remote sensing reflectance (R (sub rs) by lambda (sensor wavelength), per steradian (sr (sup −1)) for not just oligotrophic oceans but also more productive waters. Such a relaxed masking scheme yields an average relative increase of 39 percent in data quantity for global oceans, thus making it possible to reduce data product uncertainties and fill data gaps.

Hu, Chuanmin↗

NASA NTRS2019

Building a Data Ecosystem: A New Data Stewardship Paradigm for the Multi-Mission Algorithm and Analysis Platform (MAAP)

New adaptive approaches to Earth observation data stewardship need to be adopted in order to allow for higher data volumes, heterogeneous data and constantly evolving technologies. The data ecosystem approach to stewardship offers a viable solution to this need by placing an emphasis on the relationships between data, technologies and people. In this paper, we present the Joint ESA-NASA Multi-Mission Algorithm and Analysis Platform’s (MAAP) creation of a data ecosystem to support global aboveground terrestrial carbon dynamics research. We present the components needed to support the MAAP data ecosystem along with two data stewardship workflows used in the MAAP and the development of extended metadata for MAAP.

Bugbee, Kaylin↗

NASA NTRS2023

Cloud - Aerosol LIDAR Infrared Pathfinder Satellite Observations (CALIPSO) - Data Management System: Data Products Catalog V4.95

The CALIPSO V4.51 Lidar Level 1 and Level 2 data product is an updated version of an already order-able dataset. The changes were signed off by the CALIPSO Configuration Control Board, versioned, and the code uploaded to a code repository. There is no ITAR/SBU data or code associated with this product. Data will be publicly order-able at the NASA LaRC Atmospheric Sciences Data Center (ASDC). All documentation and web sites will be made public once the data product is released. The data is in HDF4 format and will be generated for majority of the mission (June 2006 - August 2023). The attached Data Products Catalog (v4.95) describes the content of these new data products.

Mark Vaughan↗

DOE OSTI2025

Data about data – when, why and how metadata can support the digital plant

A structured approach for recording data quality and contextual information about how and why a signal exists – i.e. metadata – is central to interpret and use sensor data correctly. This is becoming increasingly important with the global trend with data-driven applications such as digital twins and AI-models. But a structured metadata collection and organization of sensor data is not routine in most plants, which can result in lost information and missed opportunities to make use of the investments made in the data collection. Therefore, the IWA task group on Metadata Collection and Organization in wastewater resource recovery systems (MetaCO) was initiated in 2020 and recently delivered the IWA scientific and technical report number 31. The report gives and in-depth description about metadata in water resources recovery facilities (WRRFs) and is available as open access at IWA publishing. The report is the outcome of the collaboration between more than 80 water professionals with the intention to serve WRRF data users with a guide on how to structure and make use of metadata throughout the data pipeline in order to maximize the value of sensor data.

Alferes, Janelcy [VITO, Belgium]↗

DOE OSTI2026

TSDC: Transportation Secure Data Center: Real-World Data for Planning, Modeling, and Analysis

The Transportation Secure Data Center is a centralized repository for high-resolution transportation data from hundreds of travel and transit surveys and studies. It makes vital transportation data broadly available to users while preserving the privacy of survey participants. It houses surveys and studies conducted by state departments of transportation, metropolitan planning organizations, transit agencies, cities, and other public agencies. Meanwhile, the Livewire Data Platform empowers research, industry, and academic partners to easily and securely preserve, maintain, share, discover, and gain access to transportation and mobility data. Livewire accommodates a range of datasets, including behavioral, experimental, model, analytical, and raw data at the vehicle, traveler, and system levels. Datasets support mobility research and planning spanning urban science, connected and automated vehicles, fueling and charging infrastructure, mobility decision science, multimodal transportation, vehicle efficiency, and more.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2024

Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data‐Driven Parameterizations: Lessons From the Emulation of Gravity Wave Momentum Transport in WACCM

Abstract Neural networks (NNs) are increasingly used for data‐driven subgrid‐scale parameterizations in weather and climate models. While NNs are powerful tools for learning complex non‐linear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are (a) data imbalance related to learning rare, often large‐amplitude, samples; (b) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and (c) generalization to other climates, for example, those with different radiative forcings. Here, we examine the performance of methods for addressing these challenges using NN‐based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics‐based gravity wave (GW) parameterizations as a test case. WACCM has complex, state‐of‐the‐art parameterizations for orography‐, convection‐, and front‐driven GWs. Convection‐ and orography‐driven GWs have significant data imbalance due to the absence of convection or orography in most grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto‐encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria for identifying when an NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4 × CO 2 ). However, their performance is significantly improved by applying transfer learning, for example, re‐training only one layer using ∼1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data‐driven parameterizations for various processes, including (but not limited to) GWs.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2015

Taming Big Data Variety in the Earth Observing System Data and Information System

Although the volume of the remote sensing data managed by the Earth Observing System Data and Information System is formidable, an oft-overlooked challenge is the variety of data. The diversity in satellite instruments, science disciplines and user communities drives cost as much or more as the data volume. Several strategies are used to tame this variety: data allocation to distinct centers of expertise; a common metadata repository for discovery, data format standards and conventions; and services that further abstract the variations in data.

Information Systems↗