Using Cloud-Based Storage Technologies for Earth Science Data

Michaelis, Andrew; Readey, John; Votava, Petr

Home/Archive/NASA NTRS

NASA NTRS · 20180008594

Using Cloud-Based Storage Technologies for Earth Science Data

Michaelis, Andrew·Readey, John·Votava, Petr

View original source ↗Export metadata ↓

Abstract

Cloud based infrastructure may offer several key benefits of scalability, built in redundancy and reduced total cost of ownership as compared with a traditional data center approach. However, most of the tools and software systems developed for NASA data repositories were not developed with a cloud based infrastructure in mind and do not fully take advantage of commonly available cloud-based technologies. Object storage services are provided through all the leading public (Amazon Web Service, Microsoft Azure, Google Cloud, etc.) and private (Open Stack) clouds, and may provide a more cost-effective means of storing large data collections online. We describe a system that utilizes object storage rather than traditional file system based storage to vend earth science data. The system described is not only cost effective, but shows superior performance for running many different analytics tasks in the cloud. To enable compatibility with existing tools and applications, we outline client libraries that are API compatible with existing libraries for HDF5 and NetCDF4. Performance of the system is demonstrated using clouds services running on Amazon Web Services.

Explore related subjects

Data Technologies Earth Science

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Michaelis, Andrew, Readey, John, Votava, Petr. 2016-12-12. Using Cloud-Based Storage Technologies for Earth Science Data. https://ntrs.nasa.gov/citations/20180008594

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

NASA NTRS2025

Why We Do What We Do: Data Reuse, Open Access, and Privacy in Data Management at the Life Sciences Data Archive

As custodian of the unique and irreplaceable collections of human subject research data generated by the Human Research Program and its predecessors throughout the agency’s history, the Life Sciences Data Archive (LSDA) is charged with protecting participants’ privacy and implementing their consent decisions as it provides retrospective data for use in new studies. This active, stewardship-focused approach to data management and preservation shapes the products that LSDA provides to researchers and the responsibilities of researchers in using the data and publishing their results. This presentation reviews how federal and agency mandates shape LSDA’s data management procedures and expectations for researchers. Topics covered will include LSDA’s movement towards implementation of the FAIR (Findable, Accessible, Interoperable, Reusable) principles and how the archive’s evolving data management practices support FAIR-ness; collaboration between LSDA and the Lifetime Surveillance of Astronaut Health (LSAH) project (the repository of astronaut medical data); LSDA’s response to the challenges of performing its stewardship role and maintaining trust given the public profiles of the subjects whose data it preserves; and the ever-increasing challenges to expectations of subject privacy stemming from the growing power and ubiquity of of data analysis and aggregation tools.

Data↗

NASA NTRS2021

Microgravity Science Database Development

Throughout NASA’s history, the agency has developed a plethora of complex systems, such as the International Space Station and the space shuttle, and performed research in several fields spanning the gamut from psychology to welding and materials research. Throughout these studies, an extensive amount of data has been generated and unfortunately at times regenerated. As Barend Mons states “Huge sums of taxpayer funds go to waste because such data cannot be reused.”[2] While his comments were directed at the state of data management in the European Union, it is no less valid for data management practices in the United States. The issues surrounding data management, including storage, retrieval, and analysis, will continue to be of utmost importance as the agency aims to responsibly utilize funds and gather the maximum benefit from flight and ground experiments.

Data↗

NASA NTRS2017

Evaluation of Sentinel-1A Data For Above Ground Biomass Estimation in Different Forests in India

Use of remote sensing data for mapping and monitoring of forest biomass across large spatial scales can aid in addressing uncertainties in carbon cycle. Earlier, several researchers reported on the use of Synthetic Aperture Radar (SAR) data for characterizing forest structural parameters and the above ground biomass estimation. However, these studies cannot be generalized and the algorithms cannot be applied to all types of forests without additional information on the forest physiognomy, stand structure and biomass characteristics. The radar backscatter signal also saturates as forest parameters such as biomass and the tree height increase. It is also not clear how different polarizations (VV versus VH) impact the backscatter retrievals in different forested regions. Thus, it is important to evaluate the potential of SAR data in different landscapes for characterizing forest structural parameters. In this study, the SAR data from Sentinel-1A has been used to characterize forest structural parameters including the above ground biomass from tropical forests of India. Ground based data on tree density, basal area and above ground biomass data from thirty-eight different forested sites has been collected to relate to SAR data. After the pre-processing of Sentinel 1-A data for radiometric calibration, geo-correction, terrain correction and speckle filtering, the variability in the backscatter signal in relation tree density, basal area and above biomass density has been investigated. Results from the curve fitting approach suggested exponential model between the Sentinel-1A backscatter versus tree density and above ground biomass whereas the relationship was almost linear with the basal area in the VV polarization mode. Of the different parameters, tree density could explain most of the variations in backscatter. Both VV and VH backscatter signals could explain only thirty and thirty three percent of variation in above biomass in different forest sites of India. Results also suggested saturation of the Sentinel-1A backscatter signal around hundred tonnes per hectare for VV polarization and one hundred and forty five tonnes per hectare for VH polarization. The presentation will highlight the above results in addition to potentials and limitations of Sentinel-1A data for retrieving forest structural parameters. Also, background information on different forest types of India, biomass variations and forest type mapping efforts in the region will be presented.

Data↗