Search NASA⌕ Search

SEARCH · Search NASA

Results for “netCDF”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

NASA Briefing for Unidata

The NASA representative to the Unidata Strategic Committee presented a semiannual update on NASAs work with and use of Unidata technologies. The talk covered the program of cloud computing prototypes being undertaken for the Earth Observing System Data and Information System (EOSDIS). Also discussed were dataset interoperability recommendations ratified via the EOSDIS Standards Office and the HDF Product Designer tool with respect to its possible applicability to data in network Common Data Form (NetCDF) version 4.

data integration↗
NASA NTRS2016

National Climate Assessment - Land Data Assimilation System (NCA-LDAS) Data and Services at NASA GES DISC

The National Climate Assessment-Land Data Assimilation System (NCA-LDAS) is an Integrated Terrestrial Water Analysis, and is one of NASAs contributions to the NCA of the United States. The NCA-LDAS has undergone extensive development, including multi-variate assimilation of remotely-sensed water states and anomalies as well as evaluation and verification studies, led by the Goddard Space Flight Centers Hydrological Sciences Laboratory (HSL). The resulting NCA-LDAS data have recently been released to the general public and include those from the Noah land-surface model (LSM) version 3.3 (Noah-3.3) and the Catchment LSM version Fortuna-2.5 (CLSM-F2.5). Standard LSM output variables including soil moistures temperatures, surface fluxes, snow cover depth, groundwater, and runoff are provided, as well as streamflow using a river routing system. The NCA-LDAS data are archived at and distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). The data can be accessed via HTTP, OPeNDAP, Mirador search and download, and NASA Earth data Search. To further facilitate access and use, the NCA-LDAS data are integrated into the NASA Giovanni, for quick visualization and analysis, and into the Data Rods system, for retrieval of time series of long time periods. The temporal and spatial resolutions of the NCA-LDAS data are, respectively, daily-averages and 0.125x0.125 degree, covering North America (25N 53N; 125W 67W) and the period January 1979 to December 2015. The data files are in self-describing, machine-independent, CF-compliant netCDF-4 format.

data assimiliation↗
NASA NTRS2017

Development of a Multilayer MODIS IST-Albedo Product of Greenland

A new multilayer IST-albedo Moderate Resolution Imaging Spectroradiometer (MODIS) product of Greenland was developed to meet the needs of the ice sheet modeling community. The multiple layers of the product enable the relationship between IST and albedo to be evaluated easily. Surface temperature is a fundamental input for dynamical ice sheet models because it is a component of the ice sheet radiation budget and mass balance. Albedo influences absorption of incoming solar radiation. The daily product will combine the existing standard MODIS Collection-6 ice-surface temperature, derived melt maps, snow albedo and water vapor products. The new product is available in a polar stereographic projection in NetCDF format. The product will ultimately extend from March 2000 through the end of 2017.

MODIS↗
NASA NTRS2017

Development of Gridded Innovations and Observations Supplement to MERRA-2

Atmospheric reanalysis have become an important source of data for weather and climate research, owing to the continuity of the data, but especially because of the multitude of observational data included (radiosondes, commercial aircraft, retrieved data products and radiances). However, the presence of assimilated observations can vary based on numerous factors, and so it is difficult or impossible for a researcher to say with any degree of certainty how many and what type of observations contributed to the reanalysis data they are using at any give point in time or space. For example, quality control, transmission interruptions, and station outages can occasionally affect data availability. While orbital paths can be known, drift in certain instruments and the large number of available instruments makes it challenging to know which satellite is observing any region at any point in the diurnal cycle. Furthermore, there is information from the statistics generated by the data assimilation that can help understand the model and the quality of the reanalysis. Typically, the assimilated observations and their innovations are in observation-space data formats and have not been made easily available to reanalysis users.A test data set has been developed to make the MERRA-2 assimilated observations available for rapid and general use, by simplifying the data format. The observations are binned to a grid similar as MERRA-2 and saved as netCDF. This data collection includes the mean and number of observations in the bin as well as its variance. The data will also include the innovations from the data assimilation, the forecast departure and the analysis increment, as well as bias correction (for satellite radiances). We refer to this proof-of-concept data as the MERRA-2 Gridded Innovations and Observations (GIO). In this paper, we present the data format and its strengths and limitations with some initial testing and validation of the methodology.

SSMI↗
NASA NTRS2017

The Value of Data and Metadata Standardization for Interoperability in Giovanni Or: Why Your Product's Metadata Causes Us Headaches!

Giovanni is a data exploration and visualization tool at the NASA Goddard Earth Sciences Data Information Services Center (GES DISC). It has been around in one form or another for more than 15 years. Giovanni calculates simple statistics and produces 22 different visualizations for more than 1600 geophysical parameters from more than 90 satellite and model products. Giovanni relies on external data format standards to ensure interoperability, including the NetCDF CF Metadata Conventions. Unfortunately, these standards were insufficient to make Giovanni's internal data representation truly simple to use. Finding and working with dimensions can be convoluted with the CF Conventions. Furthermore, the CF Conventions are silent on machine-friendly descriptive metadata such as the parameter's source product and product version. In order to simplify analyzing disparate earth science data parameters in a unified way, we developed Giovanni's internal standard. First, the format standardizes parameter dimensions and variables so they can be easily found. Second, the format adds all the machine-friendly metadata Giovanni needs to present our parameters to users in a consistent and clear manner. At a glance, users can grasp all the pertinent information about parameters both during parameter selection and after visualization.

interoperability↗
NASA NTRS2017

Satellite Level 3 & 4 Data Subsetting at NASA GES DISC

Earth Science data are available in many file formats (NetCDF, HDF, GRB, etc.) and in a wide range of sizes, from kilobytes to gigabytes. These properties have become a challenge to users if they are not familiar with these formats or only want a small region of interest (ROI) from a specific dataset. At NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we have developed and implemented a multipurpose subset service to ease user access to Earth Science data. Our Level 3 & 4 Regridder is capable of subsetting across multiple parameters (spatially, temporally, by level, and by variable) as well as having additional beneficial features (temporal means, regridding to target grids, and file conversion to other data formats). In this presentation, we will demonstrate how users can use this service to better access only the data they need in the form they require.

subsetting↗
NASA NTRS2018

Challenges in Development of Online Visualization and Analysis Tools for Satellite Data

Over the years, various online visualization and analysis tools have been developed to facilitate satellite data access and help scientific users around the world to conduct research and develop applications (e.g., data product evaluation, what-if questions, etc.). For those who are new to satellite data products, using them can be a daunting task due to many obstacles in data processing such as data formats, complex data structures, special software packages, unfamiliar terminology, etc., especially when one is not sure whether a dataset is suitable for his/er research project. Even for experienced users, developing software for data processing and analysis can be a costly and time-consuming task. Online visualization tools can overcome many of these difficulties and allow users to focus on scientific questions. For example, Giovanni (the Geospatial Interactive Online Visualization and Analysis Infrastructure, https://giovanni.gsfc.nasa.gov), developed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), allows access over 1900 satellite and model variables in 82 measurement groups of 8 disciplines without downloading data and software. Main features include basic functions for data analysis and visualization, data provenance, output data in different formats (ASCII, NetCDF, GeoTIFF), and more. Over the years, ~1700 peer-reviewed publications in different disciplines have been benefited from Giovanni in research activities (e.g. initial investigation, what-if questions, product evaluation). Despite the success of online visualization and analysis tools, challenges and new opportunities still exist and more can be done with new requirements and technology. Examples are: a) how to increase the efficiency of dataset search by enhancing intuitive aspects; b) how to facilitate interdisciplinary research; c) how to provide data quality information; d) how to engage users to participate in data quality assessment; and more. NASA Earth Observing System Data and Information System (EOSDIS) satellite-based data products are processed at various levels ranging from Level 0 to Level 4. While most users use data products at higher levels (Level-3 and 4), products at lower levels are still important for case studies, algorithm development, ground validation, etc. In this presentation, we will use Giovanni as an example to present and discuss challenges and near-future opportunities for satellite data online visualization and analysis tools.

Liu, Zhong↗
NASA NTRS2018

The Advantages of Synergy-Quantitative Earth Science Data Visualization and Analysis with Giovanni, Panoply, and Excel

The NASA Giovanni data analysis system provides a multitude of basic analysis capabilities for numerous Earth science data products which are available in the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) archive, as well as for additional selected data products provided by other NASA Distributed Active Archive Center (DAAC) archives. In Giovanni, users can easily generate time-averaged data maps, area-averaged time-series, Latitude-Time and Longitude-Time Hovmoeller diagrams, correlation maps, accumulation maps, and map animations (22 analysis options are available in total). While ASCII text output is available for time-series plots, it is not included as an option for data maps. In order to provide a quantitative, easy-to-use numerical output in ASCII text form, the NetCDF file output from a Giovanni visualization is downloaded and then opened with the free NASA visualization software package Panoply. Panoply provides the capability of translating the Giovanni file into comma-separated-variable (CSV) output. Panoply also provides additional visualization options, including the facile calculation of difference maps and quasi-anomaly maps using Giovanni output files. The CSV files from Panoply can then be imported into an Excel spreadsheet, where an Excel macro converts the CSV files. The output consists of latitude-longitude-data value triads in text form for maps, and either longitude-time-data value or latitude-time-data value triads in text form for Hovmoeller diagrams. This presentation will explicate the basic procedure for the conversion, and then provide several examples where the procedure is applied to Giovanni output from different analysis options.

Earth science↗
NASA NTRS2019

OPeNDAP and HDF5 in the Cloud: Techniques and Best Practices for Serving HDF5 Data

In our talk we will discuss considerations and best practices for organizing data and metadata in HDF5 files when serving ESDIS products in S3 with OPeNDAP. The first part of the talk will be devoted to the best practices for organizing data and user-defined metadata including CF conventions in HDF5. We will also address interoperability with netCDF-4 and its role in accessing data in HDF5. In the second part of the talk we will go over OPeNDAP considerations for S3 access to HDF5 data.

HDF5↗
NASA NTRS2016

NASA WorldWind: Open Source Visualization Technology for Earth Observation

NASA WorldWind: Open Source Visualization Technology for Earth Observation WorldWind, open source virtual globe technology for Java, iOS, Android and Web, is provided by NASA and is architected as API-centric modular componentry. This enable it to be continually optimized and feature-enriched in ways that allow applications based on this SDK (Software Development Kit) to benefit Earth Observation, especially Open Science, with minimal or no adjustment for the decade ahead. The next-generation National Airspace System (NAS) aviation management system for the U.S. Federal Aviation Administration, FAA, uses WorldWind, as do applications currently being developed by the European Space Agency, along with several other US and European government agencies and industry partners. This presentation will demonstrate several NASA open source use cases for WorldWind technology that include advances being made to optimize access to NetCDF and HDF data via WebWorldWind.NASA WorldWind: Multidimensional Geospatial Web Platform The ability to see spatial data in its native context is essential for that data to be appreciated whether by the scientific community, policy and decision-makers or the general public. Recently, the accessibility of spatial data has dramatically improved. Without the need to install an application, spatial data can now be experienced via any web browser, mobile devices included. For developers, by simply updating the app on your server, the latest version of your application is now immediately available to your entire usercommunity. Unlike other virtual globes such as Google Earth, NASA World Wind offers something very special, full control to customize the interface with any features or functionalities you might need. You decide how the data is accessed and experienced. This allows you to provide maximum value of the information to your user community. The web version of NASA WorldWind (WebWorldWind) has made it possible for a whole new suite of applications for managing and sharing spatial data. Apps built with this web version are ideal for immediate social media type activity and also facilitate delivery of sophisticated data exchange scenarios such as weather and climate research, disaster response, personal navigation, and industrial-strength tracking for transportation, supply chain, aviation and satellites. WebWorldWind is an application component, not an app in itself. It is written in JavaScript and provides the real world geographic context for spatial data and information visualization, using a rich set of shapes and graphic primitives. WebWorldWind also provides platform independence, while accommodating any number of data types. Web WorldWind runs on any platform via a browser, i.e., Internet Explorer, Firefox, Chrome and Safari. Features include, 3D virtual globe, 2D map with multiple projection choices (Mercator, Polar, UPS, Equirectangular), imagery and elevation import, extensible, data retrieval (via REST, WMS, WCS, WFS, Bing, User-Defined), decluttering, measurement, accurate line-ofsight, subsurface visualization, and more.

Open Source Mapping↗
NASA NTRS2018

Serving NASA GES DISC Multi-Spatiotemporal Earth Science Data to the GIS community

NASA Earth Science (ES) data is essential to a wide range of GIS research and applications. However, for many GIS users, searching, accessing, using and analyzing NASA ES data can be of a great challenge- ranging from the sheer data volumes, types of science parameters, and to the complexity of data encoding formats. As one of the twelve NASA Science Mission Directorate (SMD) Data Centers, Goddard Earth Sciences (GES) Data and Information Services Center (DISC) archives and distributes petabytes of ES parameters covering atmosphere, land, and ocean fields. Most data are multidimensional and multi-spatiotemporal in nature and are encoded in different science data formats (e.g, HDF, HDF-EOS, netCDF, GRIB, binary), which usually contain multiple variables and different metadata information. By far, GES DISC has been developing a number of services and online tools to help GIS users to easily explore our data products. In this presentation, we will describe our ArcGIS-based data accessing and visualization services and portals, which allow users directly exploring the multi-spatiotemporal ES data in ArcGIS clients without having to pre-download/import the data. The ArcGIS services are also compliant with the Open Geospatial Consortium (OGC) Web Coverage Service (WCS) and Web Map Service (WMS) protocols and can be accessed by any other WCS/WMS clients to get customized GES DISC EO data on-the-fly from such services.

Wei, Jennifer↗
NASA NTRS2018

Development of Gridded Innovations and Observations Supplement to MERRA-2

Atmospheric reanalysis have become an important source of data for weather and climate research, owing to the continuity of the data, but especially because of the multitude of observational data included (radiosondes, commercial aircraft, retrieved data products and radiances). However, the presence of assimilated observations can vary based on numerous factors, and so it is difficult or impossible for a researcher to say with any degree of certainty how many and what type of observations contributed to the reanalysis data they are using at any give point in time or space, or their contribution to the eventual analyzed fields. For example, quality control, transmission interruptions, and station outages can occasionally affect data availability. While orbital paths are known, drift in certain instruments, cloud clearing, and the large number of available instruments makes it challenging to know which satellite is observing any region at any point in the diurnal cycle. Furthermore, there is information from the statistics generated by the data assimilation that can help understand the model and the quality of the reanalysis. Typically, the assimilated observations and their innovations are saved in observation-space data formats and are not easily made available to reanalysis users.Here, we present an early version of a data set has been developed to make the MERRA-2 assimilated observations available for rapid and general use, by simplifying the data format. The observations are binned to a grid similar as MERRA-2 and saved as netCDF. This data collection includes the mean and number of observations in the bin as well as its variance. The data will also include the innovations from the data assimilation, the forecast departure and the analysis increment, as well as bias correction (for satellite radiances). In this paper, we present the data format (called MERRA-2 Gridded Innovations and Observations or GIO) and its strengths and limitations with some initial testing and validation of the methodology.

Bosilovich, Michael G.↗
NASA NTRS2019

Microphone Phased Array NetCDF/HDF5 Archival Files: Application Program Interface Reference

An application program interface (API) has been developed for the creation and access of structured data files generated by microphone phased arrays utilized in aeroacoustics research. Two structured binary file formats are supported, namely NetCDF (Network Common Data Form) and HDF5 (Hierarchical Data Format) files. The API consists of a library of routines callable from C, Fortran or Matlab, with native versions of the API provided for each language. The libraries are divided into categories for file handling, file definition and initialization, data writing, data recovery, and error handling. The API is intended to provide a mechanism for generating self-describing binary files for long-term archiving of raw and processed data generated by phased array systems.

Humphreys, William M., Jr.↗
NASA NTRSDate not supplied

Cloud Optimized Data Formats

Cloud computing offers the promise of being able to analyze Big Data earth Observations at scale, by allowing scientists to deploy many nodes at once to analyze the data. However, in order to take full advantage of cloud scalability, it is often necessary to reorganize and reformat the data to enable fine-grained, parallel access to the data in Web Object Storage. NASA recently conducted a study of several formats that are optimized for analysis in the cloud: Parquet, zarr, HDF (Hierarchical Data Format) in the Cloud, and Cloud-Optimized GeoTIFF (Tagged Image File Format). They were compared against non-cloud-optimized formats, netCDF (network Common Data Form) and GeoTIFF, with criteria based both on stewardship and analysis performance.

Christopher Lynnes↗
NASA NTRSDate not supplied

Providing Long-term Solar and Meteorological Resource Information from NASA Research through the GIS-Enabled POWER Web Services Portal

Solar and other renewable energy systems are optimized using geophysical parameters describing information about the solar resource and meteorological environments and how those environments may be changing in time. The Prediction of Worldwide Renewable Energy Resource (POWER) team at the NASA LaRC Research Center deployed its first version of the revitalized POWER geophysical parameter website that employs Esri Geographic Information System (GIS) tools. This web application provides access to both time series and climatological data sets spanning from a few days behind real-time back to the early 1980’s with a spatial resolution of 0.5 x 0.5 degree, thus extending over 30 years. The GIS tools enable, generate and store climatological averages using spatial queries and calculations in a spatial database resulting in greater accessibility to government agencies, industry and individuals. There is an API that provides data through a URL coding and can also invoked inside user software packages. Output data formats now include ASCII, CSV, geoTiff, JSON and .netCDF. Since May 2018, over 10.4 TB of data have been delivered to fulfill over 110 million data requests from 240,000 unique user IPs (https://power.larc.nasa.gov). This presentation provides an overview of this project and the current version of the new POWER web capabilities through to the end usage. Surface solar parameters are adapted from both the NASA/GEWEX (Global and Energy Water Cycle Exchange) SRB (Surface Radiation Budget) project and the CERES (Clouds and Earth’s Radiant Energy System) data products. The meteorological parameters are adapted from NASA’s MERRA-2 (Modern Era Retrospective-analysis for Research and Applications). We review uncertainty of various basic parameters using surface measurements. After the introduction, we specifically discuss various clusters of parameters for equator pointing tilted surfaces that provide for the estimation of optimal tilt angle by month and year. The discussion will emphasize a few examples of how solar and building engineers are using the data products. We will then preview the capabilities of Version 2 featuring examples of hourly solar and meteorological data products, customized user reports and expanded web mapping services from CERES data products. The web services provide a unique, expandable resource for renewable energy systems engineers to design, evaluate and optimize to environments worldwide.

Solar Resource↗
NASA NTRSDate not supplied

SAGE III/ISS Rapid Data Analysis Through Dashboarding with Jupyter Notebooks

Spaceborne remote sensing observations of Earth’s atmosphere produce significant quantities of data over the life of each mission. In the case of the Stratospheric Aerosol and Gas Experiment III on the International Space Station (SAGE III/ISS) nearly four years of vertical profiles of atmospheric ozone, water vapor, and nitrogen dioxide concentrations as well as aerosol extinction coefficients have been released. The dichotomy of the desire for both long-term trends in the atmospheric state alongside the assessment of short-term impacts of major disruptive events such as volcanic eruptions and pyrocumulus injections requires agile tools to handle these cases in near real-time as new data are produced. The analysis landscape is further complicated by the desire to compare results between the numerous contemporary observations available for a given dataset. The SAGE III/ISS team has developed a suite of tools leveraging modern web-based frameworks allowing members to interact with a dashboard-style interface to load the data record, assess new profiles as they are generated and in ensemble, compare between species, and additionally add in measurements observed by other platforms as necessary. Leveraging a commonly packaged data format of NetCDF alongside the Python Jupyter Notebook framework, the data can be served to interested parties from an analysis server while still runnable on personal systems if required. This presentation illustrates the ecosystem developed by the SAGE III/ISS team, the applicability to measurements made by any limb-observing platform, and the benefit to transforming routine analyses into readily accessible dynamic plots. Frameworks currently exist at larger scales with projects such as GIOVANNI, and this illustration seeks to show that similar frameworks are accessible and possible within the local research environment while simultaneously unloading human processing cycles for more specialized analysis tasks.

Dashboarding↗
NASA NTRS2023

Updated Observations of Clouds By MODIS for Global Model Assessment

This paper describes a new global dataset of cloud properties observed by MODIS relying on the current (collection 6.1) processing of MODIS data and produced to facilitate comparison with results from the MODIS observational proxy used in climate models. The dataset merges observations from the two MODIS instruments into a single netCDF file. Statistics (mean, standard deviation, and number of observations) are accumulated over daily and monthly timescales on an equal-angle grid for viewing and illumination geometry, cloud detection, cloud-top pressure, and cloud properties (optical thickness, effective particle size, and water path) partitioned by thermodynamic phase and an assessment as to whether the underlying observations come from fully or partly cloudy pixels. Similarly partitioned joint histograms are available for (1) optical thickness and cloud-top pressure, (2) optical thickness and particle size, and (3) cloud water path and particle size. Differences with standard data products, caveats for data use, and guidelines for comparison to the MODIS simulator are described.

Robert Pincus↗
NASA NTRS2025

PACE Water Resources: Demonstrating the Use of NASA's PACE Hyperspectral Ocean Color Instrument Data for Enhanced Coastal Management

This project developed tools to support the future use of Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) hyperspectral imagery in water resource monitoring and research by NASA DEVELOP teams and members of the PACE applications community. We sought to address a need for support in processing and visualizing hyperspectral PACE Ocean Color Instrument (OCI) data among researchers and decision-makers working in coastal water quality management and harmful algal bloom (HAB) monitoring. To supplement the day of simulated PACE imagery available, we used Aqua MODIS earth observations with Level 3 processing from March 2022 to build a Python graphical user interface (GUI) for visualizing ocean biogeochemical parameters relevant to the early detection and monitoring of HABs. We used simulated PACE OCI Level 2 data derived from the Python Top of Atmosphere Simulation Tool (PyTOAST) to build Jupyter Notebooks for band subset and selection. The Level 3 PACE Viewer components support users with quick visualizations as well as the creation of geoTIFFs and time-series. The Level 2 Jupyter Notebooks address users’ concerns over the volume and complexity of hyperspectral imagery. The PACE Viewer is useful for visual inspection and netCDF data processing but should not be used for geospatial analysis. Once PACE launches, this tool will alleviate the technical burdens of working with hyperspectral data and support the early detection and monitoring of HABs using PACE satellite imagery.

Python Top of Atmosphere Simulation Tool↗