Search NASA⌕ Search

SEARCH · Search NASA

Results for “HDF-EOS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

35 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Data are from Mars, Tools are from Venus

Although during the data production phase, the data producers will usually ensure the products to be easily used by the specific power users the products serve. However, most data products are also posted for general public to use. It is not straightforward for data producers to anticipate what tools that these general end-data users are likely to use. In this talk, we will try to help fill in the gap by going over various tools related to Earth Science and how they work with the existing NASA HDF (Hierarchical Data Format) data products and the reasons why some products cannot be visualized or analyzed by existing tools. One goal is for to give insights for data producers on how to make their data product more interoperable. On the other hand, we also provide some hints for end users on how to make tools work with existing HDF data products. (tool category list: check the comments) HDF-EOS tools: HDFView HDF-EOS Plugin, HEG, h4tonccf, hdf-eos2 dumper, NCL, MATLAB, IDL, etc.net; CDF-Java tools: Panoply, IDV, toosUI, NcML, etc.net; CDF-C tools: ArcGIS Desktop, GrADS, NCL, NCO, etc.; GDAL tools: ArcGIS Desktop, QGIS, Google Earth, etc.; CSV tools: ArcGIS Online, MS Excel, Tableau, etc.

hdf↗
NASA NTRS2000

Subsetting and Formatting Landsat-7 LOR ETM+ and Data Products

The Landsat-7 Processing System (LPS) processes Landsat-7 Enhanced Thematic Mapper (ETM+) instrument data into large, contiguous segments called "subintervals" and stores them in Level OR (LOR) data files. The LPS processed subinterval products must be subsetted and reformatted before the Level I processing systems can ingest them. The initial full subintervals produced by the LPS are stored mainly in HDF Earth Observing System (HDF-EOS) format which is an extension to the Hierarchical Data Format (HDF). The final LOR products are stored in native HDF format. Primarily the EOS Core System (ECS) and alternately the DAAC Emergency System (DES) subset the subinterval data for the operational Landsat-7 data processing systems. The HDF and HDF-EOS application programming interfaces (APIs) can be used for extensive data subsetting and data reorganization. A stand-alone subsetter tool has been developed which is based on some of the DES code. This tool makes use of the HDF and HDFEOS APIs to perform Landsat-7 LOR product subsetting and demonstrates how HDF and HDFEOS can be used for creating various configurations of full LOR products. How these APIs can be used to efficiently subset, format, and organize Landsat-7 LOR data as demonstrated by the subsetter tool and the DES is discussed.

Reid, Michael R.↗
NASA NTRS2000

Ocean Data from MODIS at the NASA Goddard DAAC

Terra satellite carrying the Moderate Resolution Imaging Spectroradiometer (MODIS) was successfully launched on December 18, 1999. Some of the 36 different wavelengths that MODIS samples have never before been measured from space. New ocean data products, which have not been derived on a global scale before, are made available for research to the scientific community. For example, MODIS uses a new split window in the four-micron region for the better measurement of Sea Surface Temperature (SST), and provides the unprecedented ability (683 nm band) to measure chlorophyll fluorescence. At full ocean production, more than a thousand different ocean products in three major categories (ocean color, sea surface temperature, and ocean primary production) are archived at the NASA Goddard Earth Sciences (GES) Distributed Active Archive Center (DAAC) at the rate of approx. 230GB/day. The challenge is to distribute such large volumes of data to the ocean community. It is achieved through a combination of public and restricted EOS Data Gateways, the GES DAAC Search and Order WWW interface, and an FTP site that contains samples of MODIS data. A new Search and Order WWW interface at http://acdisx.gsfc.nasa.gov/data/ developed at the GES DAAC is based on a hierarchical organization of data, will always return non-zero results. It has a very convenient geographical representation of five-minute data granule coverage for each day MODIS Data Support Team (MDST) continues the tradition of quality support at the GES DAAC for the ocean color data from the Coastal Zone Color Scanner (CZCS) and the Sea Viewing Wide Field-of-View Sensor (SeaWiFS) by providing expert assistance to users in accessing data products, information on visualization tools, documentation for data products and formats (Hierarchical Data Format-Earth Observing System (HDF-EOS)), information on the scientific content of products and metadata. Visit the MDST website at http://daac.gsfc.nasa.gov/CAMPAIGN DOCS/MODIS/index.html

Leptoukh, Gregory G.↗
NASA NTRS2001

Evaluation of Incremental Releases of ECS User Interfaces and the Development of HDF/HDF-EOS Tutorials

During the reporting period, the PI has continued to serve on numerous review panels, task forces, and committees with the goal of providing input and guidance for the Earth Observing System Data and Information System (EOSDIS) program at NASA Headquarters and NASA Goddard Space Flight Center (GSFC). In addition, the PI has worked together with personnel at Simpson Weather Associates (SWA) to help create an on-line HDF/HDF-EOS tutorial for beginning and non-expert users of both the Hierarchical Data Format (HDF) and HDF-EOS data format and software libraries. Finally, the PI has worked together with personnel at SWA and the Information Technology and Systems Center (ITSC) at the University of Alabama in Huntsville (UAH) on a feasibility study regarding the use of data mining software to ascertain features from the gridded output from numerical meteorological forecast models. A summary of these activities is provided.

Emmitt, G. D.↗
NASA NTRS2000

EOSDIS Terra Data Sampler #1: Western US Wildfires 2000

This CD-ROM contains sample data in HDF-EOS format from the instruments on board the Earth Observing System (EOS) Terra satellite: (1) Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER); (2) Clouds and the Earth's Radiant Energy System (CERES); (3) Multi-angle Imaging Spectroradiometer (MISR); and (4) Moderate Resolution Imaging Spectroradiometer (MODIS). Data from the Measurements of Pollution in the Troposphere (MOPITT) instrument were not available for distribution (as of October 17, 2000). The remotely sensed, coincident data for the Western US wildfires were acquired August 30, 2000. This CD-ROM provides information about the Terra mission, instruments, data, and viewing tools. It also provides the Collage tool for viewing data, and links to Web sites containing other digital data processing software. Full granules of the data on this CD-ROM and other EOS Data and Information System (EOSDIS) data products are available from the NASA Distributed Active Archive Centers (DAACs).

Perkins, Dorothy C.↗
NASA NTRS2002

MODIS Data from the GES DISC DAAC: Moderate-Resolution Imaging Spectroradiometer (MODIS)

The Goddard Earth Sciences (GES) Distributed Active Archive Center (DAAC) is responsible for the distribution of the Level 1 data, and the higher levels of all Ocean and Atmosphere products (Land products are distributed through the Land Processes (LP) DAAC DAAC, and the Snow and Ice products are distributed though the National Snow and Ice Data Center (NSIDC) DAAC). Ocean products include sea surface temperature (SST), concentrations of chlorophyll, pigment and coccolithophores, fluorescence, absorptions, and primary productivity. Atmosphere products include aerosols, atmospheric water vapor, clouds and cloud masks, and atmospheric profiles from 20 layers. While most MODIS data products are archived in the Hierarchical Data Format-Earth Observing System (HDF-EOS 2.7) format, the ocean binned products and primary productivity products (Level 4) are in the native HDF4 format. MODIS Level 1 and 2 data are of the Swath type and are packaged in files representing five minutes of Files for Level 3 and 4 are global products at daily, weekly, monthly or yearly resolutions. Apart from the ocean binned and Level 4 products, these are in Grid type, and the maps are in the Cylindrical Equidistant projection with rectangular grid. Terra viewing (scenes of approximately 2000 by 2330 km). MODIS data have several levels of maturity. Most products are released with a provisional level of maturity and only announced as validated after rigorous testing by the MODIS Science Teams. MODIS/Terra Level 1, and all MODIS/Terra 11 micron SST products are announced as validated. At the time of this publication, the MODIS Data Support Team (MDST) is working with the Ocean Science Team toward announcing the validated status of the remainder of MODIS/Terra Ocean products. MODIS/Aqua Level 1 and cloud mask products are released with provisional maturity.

Source record↗
NASA NTRS2003

Data Access Tools And Services At The Goddard Distributed Active Archive Center (GDAAC)

As one of the largest providers of Earth Science data from the Earth Observing System, GDAAC provides the latest data from the Moderate Resolution Imaging Spectroradiometer (MODIS), Atmospheric Infrared Sounder (AIRS), Solar Radiation and Climate Experiment (SORCE) data products via GDAAC's data pool (50TB of disk cache). In order to make this huge volume of data more accessible to the public and science communities, the GDAAC offers multiple data access tools and services: Open Source Project for Network Data Access Protocol (OPeNDAP), Grid Analysis and Display System (GrADS/DODS) (GDS), Live Access Server (LAS), OpenGlS Web Map Server (WMS) and Near Archive Data Mining (NADM). The objective is to assist users in retrieving electronically a smaller, usable portion of data for further analysis. The OPeNDAP server, formerly known as the Distributed Oceanographic Data System (DODS), allows the user to retrieve data without worrying about the data format. OPeNDAP is capable of server-side subsetting of HDF, HDF-EOS, netCDF, JGOFS, ASCII, DSP, FITS and binary data formats. The GrADS/DODS server is capable of serving the same data formats as OPeNDAP. GDS has an additional feature of server-side analysis. Users can analyze the data on the server there by decreasing the computational load on their client's system. The LAS is a flexible server that allows user to graphically visualize data on the fly, to request different file formats and to compare variables from distributed locations. Users of LAS have options to use other available graphics viewers such as IDL, Matlab or GrADS. WMS is based on the OPeNDAP for serving geospatial information. WMS supports OpenGlS protocol to provide data in GIs-friendly formats for analysis and visualization. NADM is another access to the GDAAC's data pool. NADM gives users the capability to use a browser to upload their C, FORTRAN or IDL algorithms, test the algorithms, and mine data in the data pool. With NADM, the GDAAC provides an environment physically close to the data source. NADM will benefit users with mining or offer data reduction algorithms by reducing large volumes of data before transmission over the network to the user.

Pham, Long↗
NASA NTRS2004

New Earth Science Data and Access Methods

NASA's Earth Science Enterprise, working with its domestic and international partners, provides scientific data and analysis to improve life here on Earth. NASA provides science data products that cover a wide range of physical, geophysical, biochemical and other parameters, as well as services for interdisciplinary Earth science studies. Management and distribution of these products is administered through the Earth Observing System Data and Information System (EOSDIS) Distributed Active Archive Centers (DAACs), which all hold data within a different Earth science discipline. This paper will highlight selected EOS datasets and will focus on how these observations contribute to the improvement of essential services such as weather forecasting, climate prediction, air quality, and agricultural efficiency. Emphasis will be placed on new data products derived from instruments on board Terra, Aqua and ICESat as well as new regional data products and field campaigns. A variety of data tools and services are available to the user community. This paper will introduce primary and specialized DAAC-specific methods for finding, ordering and using these data products. Special sections will focus on orienting users unfamiliar with DAAC resources, HDF-EOS formatted data and the use of desktop research and application tools.

Moses, John F.↗
NASA NTRS2004

Distributive On-line Processing, Visualization and Analysis System for Gridded Remote Sensing Data

The ability to use data stored in the current Earth Observing System (EOS) archives for studying regional or global phenomena is highly dependent on having a detailed understanding of the data's internal structure and physical implementation. Gaining this understanding and applying it to data reduction is a time- consuming task that must be undertaken before the core investigation can begin. This is an especially difficult challenge when science objectives require users to deal with large multi-sensor data sets that are usually of different formats, structures, and resolutions, for example, when preparing data for input into modeling systems. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has taken a major step towards meeting this challenge by developing an infrastructure with a Web interface that allows users to perform interactive analysis online without downloading any data, the GES-DISC Interactive Online Visualization and Analysis Infrastructure or "Giovanni." Giovanni provides interactive, online, analysis tools for data users to facilitate their research. There have been several instances of this interface created to serve TRMM users, Aerosol scientists, Ocean Color and Agriculture applications users. The first generation of these tools support gridded data only. The user selects geophysical parameters, area of interest, time period; and the system generates an output on screen in a matter of seconds. The currently available output options are: Area plot averaged or accumulated over any available data period for any rectangular area; Time plot time series averaged over any rectangular area; Time plots image view of any longitude-time and latitude-time cross sections; ASCII output for all plot types; Image animation for area plot. In the future, we will add correlation plots, GIS-compatible outputs, etc. This allow user to focus on data content (i.e. science parameters) and eliminate the need for expensive learning, development and processing tasks that are redundantly incurred by an archive's user community. The current implementation utilizes the GrADS-DODS Server (GDS), a stable, secure data server that provides subsetting and analysis services across the Internet for any GrADS-readable dataset. The subsetting capability allows users to retrieve a specified temporal and/or spatial subdomain from a large dataset, eliminating the need to download everything simply to access a small relevant portion of a dataset. The analysis capability allows users to retrieve the results of an operation applied to one or more datasets on the server. In our case, we use this approach to read pre-processed binary files and/or to read and extract the needed parts from HDF or HDF-EOS files. These subsets then serve as inputs into GrADS processing and analysis scripts. It can be used in a wide variety of Earth science applications: climate and weather events study and monitoring; modeling. It can be easily configured for new applications.

Leptoukh, G.↗
NASA NTRS2005

NASA GES DISC On-line Visualization and Analysis System for Gridded Remote Sensing Data

The ability to use data stored in the current NASA Earth Observing System (EOS) archives for studying regional or global phenomena is highly dependent on having a detailed understanding of the data's internal structure and physical implementation. Gaining this understanding and applying it to data reduction is a time-consuming task that must be undertaken before the core investigation can begin. This is an especially difficult challenge when science objectives require users to deal with large multi-sensor data sets that are usually of different formats, structures, and resolutions. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has taken a major step towards meeting this challenge by developing an infrastructure with a Web interface that allows users to perform interactive analysis online without downloading any data, the GES-DISC Interactive Online Visualization and Analysis Infrastructure or "Giovanni." Giovanni provides interactive, online, analysis tools for data users to facilitate their research. There have been several instances of this interface created to serve TRMM users, Aerosol scientists, Ocean Color and Agriculture applications users. The first generation of these tools support gridded data only. The user selects geophysical parameters, area of interest, time period; and the system generates an output on screen in a matter of seconds. The currently available output options are: Area plot averaged or accumulated over any available data period for any rectangular area; Time plot time series averaged over any rectangular area; Hovmoller plots image view of any longitude-time and latitude-time cross sections; ASCII output for all plot types; Image animation for area plot. Another analysis suite deals with parameter intercomparison: scatter plots, temporal correlation maps, GIs-compatible outputs, etc. This allow user to focus on data content (i.e. science parameters) and eliminate the need for expensive learning, development and processing tasks that are redundantly incurred by an archive's user community. The current implementation utilizes the GrADS-DODS Server (GDS), and provides subsetting and analysis services across the Internet for any GrADS-readable dataset. The subsetting capability allows users to retrieve a specified temporal and/or spatial subdomain from a large dataset, eliminating the need to download everything simply to access a small relevant portion of a dataset. The analysis capability allows users to retrieve the results of an operation applied to one or more datasets on the server. We use this approach to read pre-processed binary files and/or to read and extract the needed parts directly from HDF or HDF-EOS files. These subsets then serve as inputs into GrADS analysis scripts. It can be used in a wide variety of Earth science applications: climate and weather events study and monitoring; modeling. It can be easily configured for new applications.

Leptoukh, Gregory G.↗
NASA NTRS2006

Processing TES Level-2 Data

TES Level 2 Subsystem is a set of computer programs that performs functions complementary to those of the program summarized in the immediately preceding article. TES Level-2 data pertain to retrieved species (or temperature) profiles, and errors thereof. Geolocation, quality, and other data (e.g., surface characteristics for nadir observations) are also included. The subsystem processes gridded meteorological information and extracts parameters that can be interpolated to the appropriate latitude, longitude, and pressure level based on the date and time. Radiances are simulated using the aforementioned meteorological information for initial guesses, and spectroscopic-parameter tables are generated. At each step of the retrieval, a nonlinear-least-squares- solving routine is run over multiple iterations, retrieving a subset of atmospheric constituents, and error analysis is performed. Scientific TES Level-2 data products are written in a format known as Hierarchical Data Format Earth Observing System 5 (HDF-EOS 5) for public distribution.

Poosti, Sassaneh↗
NASA NTRS2017

Climate Data Guide - Modern Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2)

The Modern-Era Retrospective analysis for Research and Applications, version 2 (MERRA-2) is a global atmospheric reanalysis produced by the NASA Global Modeling and Assimilation Office (GMAO). It spans the satellite observing era from 1980 to the present. The goals of MERRA-2 are to provide a regularly-gridded, homogeneous record of the global atmosphere, and to incorporate additional aspects of the climate system including trace gas constituents (stratospheric ozone), and improved land surface representation, and cryospheric processes. MERRA-2 is also the first satellite-era global reanalysis to assimilate space-based observations of aerosols and represent their interactions with other physical processes in the climate system. The inclusion of these additional components are consistent with the overall objectives of an Integrated Earth System Analysis (IESA). MERRA-2 is intended to replace the original MERRA product, and reflects recent advances in atmospheric modeling and data assimilation. Modern hyperspectral radiance and microwave observations, along with GPS-Radio Occultation and NASA ozone datasets are now assimilated in MERRA-2. Much of the structure of the data files remains the same in MERRA-2. While the original MERRA data format was HDF-EOS, the MERRA-2 supplied binary data format is now NetCDF4 (with lossy compression to save space).

MERRA-2↗
NASA NTRS2017

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗
NASA NTRS2018

Serving NASA GES DISC Multi-Spatiotemporal Earth Science Data to the GIS community

NASA Earth Science (ES) data is essential to a wide range of GIS research and applications. However, for many GIS users, searching, accessing, using and analyzing NASA ES data can be of a great challenge- ranging from the sheer data volumes, types of science parameters, and to the complexity of data encoding formats. As one of the twelve NASA Science Mission Directorate (SMD) Data Centers, Goddard Earth Sciences (GES) Data and Information Services Center (DISC) archives and distributes petabytes of ES parameters covering atmosphere, land, and ocean fields. Most data are multidimensional and multi-spatiotemporal in nature and are encoded in different science data formats (e.g, HDF, HDF-EOS, netCDF, GRIB, binary), which usually contain multiple variables and different metadata information. By far, GES DISC has been developing a number of services and online tools to help GIS users to easily explore our data products. In this presentation, we will describe our ArcGIS-based data accessing and visualization services and portals, which allow users directly exploring the multi-spatiotemporal ES data in ArcGIS clients without having to pre-download/import the data. The ArcGIS services are also compliant with the Open Geospatial Consortium (OGC) Web Coverage Service (WCS) and Web Map Service (WMS) protocols and can be accessed by any other WCS/WMS clients to get customized GES DISC EO data on-the-fly from such services.

Wei, Jennifer↗
NASA NTRS2017

Guided Tour of Pythonian Museum

At http:hdfeos.orgzoo, we have a large collection of Python examples of dealing with NASA HDF (Hierarchical Data Format) products. During this hands-on Python tutorial session, we'll present a few common hacks to access and visualize local NASA HDF data. We'll also cover how to access remote data served by OPeNDAP (Open-source Project for a Network Data Access Protocol). As a glue language, we will demonstrate how you can use Python for your data workflow - from searching data to analyzing data with machine learning.

hdf↗
NASA NTRS2018

Hierarchical Data Format for Earth Observing System Data Product Developer's Guide

The "Hierarchical Data Format for Earth Observing System" talk will address the best practices for creating ESDIS data products. The work presented is done in support of Data Product Developers Guide Working Group with mission "to help data product developers make data usable for end users". During the presentation, we will use some examples of NASA data products and show how to modify them to make data more usable.

Data usability↗