Search NASA⌕ Search

SEARCH · Search NASA

Results for “Earth Science Data Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

The Importance of User Feedback in Sustaining Trusted Repositories

The NASA Earth Observation System Data and Information System (EOSDIS) has been operating since 1994 and is serving a global user community with well-managed Earth science data in a variety of scientific disciplines. EOSDIS processes, archives and distributes data and information products resulting from spaceborne and airborne instruments as well as in situ measurements from field campaigns. The Earth Science Data and Information System (ESDIS) Project at the NASA Goddard Space Flight Center manages EOSDIS with its 12 Distributed Active Archive Centers (DAACs) located across the United States. During the entire life of EOSDIS, the ESDIS Project and the DAACs have deployed many different mechanisms for user feedback, which have proven extremely valuable to their evolution and performance in their service to user communities. Some of the inputs from our user groups have resulted in fundamental changes in the architecture, design and operations of EOSDIS, while others have provided novel ideas for incremental changes. The EOSDIS DAACs have User Working Groups (UWGs) that represent broad user communities in the Earth science disciplines served by the DAACs. The UWGs meet periodically to assess and provide feedback on dataset and service priorities. As regular users of the data and services of the DAAC and experts in the scientific disciplines, the UWG members provide valuable inputs for planning and prioritizing the services, as well as addition of new datasets, for the benefit of the community. The EOSDIS is evaluated annually through an independently administered survey of its users resulting in the American Customer Satisfaction Index (ACSI). The survey provides an ACSI score as well as free-text suggestions from users, which are also helpful in making specific system improvements. In addition, each of the DAACs has a user services group that address on-going requests for help and other comments from users. The ESDIS Project has established a mechanism through the "earthdata" website (http://earthdata.nasa.gov) for users to provide feedback of any kind and these questions/comments are routed to the appropriate individuals in the Project or the DAACs. These various forms of receiving user feedback and responding to them continue to be extremely valuable in evolving and sustaining our Earth Science repository.

Behnke, Jeanne↗

MODIS Science Algorithms and Data Systems Lessons Learned

For almost 10 years, standard global products from NASA's Earth Observing System s (EOS) two Moderate Resolution Imaging Spectroradiometer (MODIS) sensors are being used world-wide for earth science research and applications. This paper discusses the lessons learned in developing the science algorithms and the data systems needed to produce these high quality data products for the earth sciences community. Strong science team leadership and communication, an evolvable and scalable data system, and central coordination of QA and validation activities enabled the data system to grow by two orders of magnitude from the initial at-launch system to the current system able to reprocess data from both the Terra and Aqua missions in less than a year. Many of the lessons learned from MODIS are already being applied to follow-on missions.

Wolfe, Robert E.↗

Benchmark Comparison of Cloud Analytics Methods Applied to Earth Observations

Cloud computing has the potential to bring high performance computing capabilities to the average science researcher. However, in order to take full advantage of cloud capabilities, the science data used in the analysis must often be reorganized. This typically involves sharding the data across multiple nodes to enable relatively fine-grained parallelism. This can be either via cloud-based file systems or cloud-enabled databases such as Cassandra, Rasdaman or SciDB. Since storing an extra copy of data leads to increased cost and data management complexity, NASA is interested in determining the benefits and costs of various cloud analytics methods for real Earth Observation cases. Accordingly, NASA's Earth Science Technology Office and Earth Science Data and Information Systems project have teamed with cloud analytics practitioners to run a benchmark comparison on cloud analytics methods using the same input data and analysis algorithms. We have particularly looked at analysis algorithms that work over long time series, because these are particularly intractable for many Earth Observation datasets which typically store data with one or just a few time steps per file. This post will present side-by-side cost and performance results for several common Earth observation analysis operations.

science data management↗

Assessing the Needs of NASA's Near Real-Time Earth Observation Products

"The 2017-2027 Decadal Survey for Earth Science and Applications from Space stated that NASA's Earth Science with planned implementation of applications provides sustained earth observations for societal benefits [1]. The Decadal Survey indicated that data latency is invaluable for time-sensitive applications including disaster risk reduction, wildland fire carbon emissions quantification, real-time measurements of the state of the hydrologic systems and many more. Data latency refers to the time between earth observation and data products available to users. During the past 13 years, NASA's Land, Atmosphere Near Real-Time Capability for Earth Observing Systems (LANCE) continues to provide free access to earth observation products that are made available much quicker than routine processing allows. The latency of most LANCE data products is Near Real-time (NRT) which is defined as less than three hours from satellite observations [2]. LANCE is managed by the Earth Science Data and Information System (ESDIS) Project at NASA Goddard Space Flight Center [3], and a User Working Group (UWG) is responsible for providing guidance to LANCE. LANCE data are used by direct users and brokers who add value to the data [4]. NASA Earth Applied Sciences Program (ASP) is one of the primary users of LANCE, which collaborates with partner organizations and provides support to scientists to solve problems in applications of earth observations. ASP promotes the use of LANCE NRT data products to demonstrate applications in decision making, facilitates end-user feedback to the science team to improve data products, and provides information on future demands for research. LANCE supports applications that need a rapid response including detecting wildland fires and volcanic eruptions, tracking smoke, ash and dust plumes, monitoring air quality and tracking extreme weather events such as hurricanes, landslides, and floods. To gather feedback regarding the availability, accessibility and actionability of NASA's NRT data products for societal benefit, three surveys and a few discussions with experts involved in the topic within ASP were conducted from the perspective of users. Feedback has been collected from users who are interested in using low latency NASA data within application communities of agriculture, disasters, water resources, health and air quality, ecological conservation, wildland fires and capacity building. Analysis-ready NRT data products in a variety of formats have been mentioned many times in the collected feedback, especially for applied users with little to no experience using research-grade earth observation products. Users prefer to have products that can be easily integrated into their existing workflows and take their analysis to the data. HDF5 is a commonly used data format for research, but typically requires some conversion to a more friendly format for applications and regular use in decision-making. Users prefer the GeoTIFF data format that can be directly ingested into a GIS mapping software and platform for data analysis and visualization. For example, LANCE’s fire, flood, SO2 and Black Marble Nighttime Blue/Yellow Composite data products have been integrated into NASA Disasters Mapping Portal, which is an GIS-based open data portal, for users in the disaster management community. There are 291 LANCE NRT layers available through GIBS and Worldview, where users can download a snapshot in GeoTIFF format. Operational users expect data to be processed as close to the user as possible. The collected feedback indicates that LANCE fire products within 3 hours latency would meet the needs of the wildland fire community. The ideal latency for volcanic application is 10-15 minutes. Users in Volcanic Ash Advisory Centers (VAAC) reported that the first forecast volcanic product should be issued within 75 minutes from the volcano eruption [5]. Overall, for disaster applications, data latency within 3 hours is useful while latency greater than 12 hours is not timely enough for operational use. Capacity building and training are critical for users to be able to access, interpret and use data products and tools for their decision making, especially for applied users with limited experience using earth observation products. LANCE data products have been used in a number of capacity building projects domestically and internationally [6]. As LANCE continues to bring new products into the system, users request training to utilize LANCE new and upcoming data products and capabilities in their applications. Due to the limitation of bandwidth and downstream flow paths, users in some developing countries need tools to select and download data for a specific area of interest instead of bulk downloads. The collected feedback also shows the lack of available SAR satellite low latency data products. The advantages of SAR to monitor conditions and changes on the ground through darkness, clouds, volcanic ash, and other atmospheric conditions, are appealing to low latency users. For example, terabytes of low latency but cloudy optical images are not helpful in rapidly identifying the extent of flood or fire impacts. LANCE could be complemented with low latency measurements via the upcoming NASA-ISRO Synthetic Aperture Radar (NISAR) mission [7]. Requests for higher spatial resolution products are expressed. A user from the wildland fire management community reported that products with 30-m spatial resolution could be used to detect small fires. The 30-m Landsat OLI fire data is now part of NASA’s Fire Information for Resource Management System (FIRMS) US/Canada [8]. Within the open and free NASA resources, LANCE disseminates NRT data products in a manner that allows them to be accessible and understandable to both scientific and applied users. In many application areas, latency plays an important or even decisive role where low latency earth observations help people to observe areas of interest, detect and track changes in the environment and make timely decisions. NASA’s Earth Applied Sciences Program promotes the use of LANCE NRT products and builds a bridge between application users and research teams. The collected feedback indicates data latency within 3 hours is useful for most of the applications, and shows the needs of user-friendly, analysis-ready products, and requests training on LANCE’s new and upcoming data products. User feedback has been provided to LANCE UWG for guidance and recommendations, and for translating findings into something actionable.

Tian Yao↗

Integration of multi-discipline data processing for earth observing systems

The first steps taken to ensure the controlled evolution of existing facilities toward greater interoperability and sharing of resources among NASA-supported earth science and applications data systems (ESADS) are described. Recommendations made by the various panels during the 1987 ESADS Workshop are presented. The panels were concerned with directories and catalogs, data archives, data manipulation software, computational facilities, data storage media, database management, and networking. Consideration was also given to the tracking and tuning of overall development and management coordination issues.

Kahn, Ralph↗

GES DISC Data Recipes in Jupyter Notebooks

The Earth Science Data and Information System (ESDIS) Project manages twelve Distributed Active Archive Centers (DAACs) which are geographically dispersed across the United States. The DAACs are responsible for ingesting, processing, archiving, and distributing Earth science data produced from various sources (satellites, aircraft, field measurements, etc.). In response to projections of an exponential increase in data production, there has been a recent effort to prototype various DAAC activities in the cloud computing environment. This, in turn, led to the creation of an initiative, called the Cloud Analysis Toolkit to Enable Earth Science (CATEES), to develop a Python software package in order to transition Earth science data processing to the cloud. This project, in particular, supports CATEES and has two primary goals. One, to transition data recipes created by the Goddard Earth Science Data and Information Service Center (GES DISC) into an interactive and educational environment using JupyterNotebooks. Two, to acclimate Earth scientists to cloud computing. To accomplish these goals, we create JupyterNotebooks to compartmentalize the different steps of data analysis and help users obtain and parse data from the command line. We also develop a Docker container, comprised of Jupyter Notebooks, Python dependencies, and command line tools, and configure it into an easy-to-deploy package. The end result is an end-to-end product that simulates the use case of end users working in the cloud computing environment.

discoverability↗

NASA Dataset Interoperability Recommendations for Earth Science

NASA ESDS (Earth Science Data and Information System) Dataset Interoperability Working Group has been developing recommendations since 2012 aimed at improving interoperability of EOS (Earth Observing System) datasets. The first set of recommendations were published in 2016 as ESDS RFC-028. The latest set of recommendations is currently undergoing review and will be available as ESDS RFC-036 soon.This talk will inform the ESIP (Earth Science Information Partners) community about the recommendations because their application is relevant to other data producers as well.

Jelenak, Aleksandar↗

Using Selection Pressure as an Asset to Develop Reusable, Adaptable Software Systems

The Goddard Earth Sciences Data and Information Services Center (GES DISC) at NASA has over the years developed and honed several reusable architectural components for supporting large-scale data centers with a large customer base. These include a processing system (S4PM) and an archive system (S4PA) based upon a workflow engine called the Simple Scalable Script based Science Processor (S4P) and an online data visualization and analysis system (Giovanni). These subsystems are currently reused internally in a variety of combinations to implement customized data management on behalf of instrument science teams and other science investigators. Some of these subsystems (S4P and S4PM) have also been reused by other data centers for operational science processing. Our experience has been that development and utilization of robust interoperable and reusable software systems can actually flourish in environments defined by heterogeneous commodity hardware systems the emphasis on value-added customer service and the continual goal for achieving higher cost efficiencies. The repeated internal reuse that is fostered by such an environment encourages and even forces changes to the software that make it more reusable and adaptable. Allowing and even encouraging such selective pressures to software development has been a key factor In the success of S4P and S4PM which are now available to the open source community under the NASA Open source Agreement

Berrick, Stephen↗

Flying the Earth Observing Constellations

Prior to the launch of the Earth Observing System (EOS) Terra and Landsat-7 satellites in 1999, the Project Scientists for the two missions and the Earth Science Data and Information System (ESDIS) Project at the Goddard Space Flight Center signed an inter-project agreement document describing their plan to fly in loose formation, approximately 20 minutes within each other. In November 2000, a technology demonstration satellite, Earth Observer-1 (EO-1), was launched into the same orbit as that of Landsat-7 and Terra, with a goal of flying within a minute from Landsat-7. The SAC-C satellite, developed and operated by the government of Argentina, was launched along with EO-1, with a goal of flying near both Terra and Landsat-7. This formation enables the scientists to make use of the scientific synergy among the instruments on the different spacecraft. This group of satellites constitutes the morning constellation, which is led by the Landsat-7, which has a mean local time (MLT) at 10:00 a.m. In May 2002, the EOS Aqua satellite was launched into an orbit with an altitude of 705 km. and a 1:30 p.m. MLT. Two smaller satellites, CALIPSO (a joint U.S./French mission), and CloudSat (a joint NASA/Colorado State University/Air Force mission), plan to fly in tight formation, within 15 seconds of each other. In addition, CALIPSO and CloudSat also plan to be within 30 to 60 seconds of the Aqua satellite. A third satellite, PARASOL, managed by the French Space Agency, CNES, will be placed within a minute of the CALIPSO satellite. In 2004, the Aura satellite will be launched and phased in relation to the Aqua satellite, such that the instruments on Aura will be able to view the same mass of air no later than 8 minutes after the instruments on Aqua have observed it. Representatives from each mission are currently documenting a plan on how they will coordinate on-orbit operations. Why are all these satellites planning to fly as a constellation? The answer is that as a constellation. the scientists will be able to acquire science data not only from their specific instruments on a single satellite, but science data from the other satellites which will have been taken at approximately the same time, thus resulting in coordinated science observation data. This leads to better quality science. This paper describes how the mission design has been driven by the science requirements. The morning and the afternoon constellations present operational challenges, which had not previously been encountered. Operations planning must address not only how the satellites of each constellation operate safely together, but also, how the two constellations fly on the same orbits without interfering with each other as they downlink data to their respective ground stations. This paper describes the operations experience gained from the morning constellation and the planning for the afternoon constellation.

Kelly, Angelita C.↗

Continuity of MODIS and VIIRS Snow Cover Extent Data Products for Development of an Earth Science Data Record

An Earth Observing System global snow cover extent data products record at moderate spatial resolution (375–500 m) began in February 2000 with the Moderate-resolution Imaging Spectroradiometer (MODIS) instrument onboard the Terra satellite. The record continued with the Aqua MODIS in July 2002, the Suomi-National Polar Platform (S-NPP) Visible Infrared Imaging Radiometer Suite (VIIRS) in January 2012 and continues with the Joint Polar Satellite System-1 (JPSS-1) VIIRS, launched in November of 2017. The objective of this work is to develop a snow cover extent Earth Science Data Record (ESDR) using different satellites, sensors and algorithms. There are many issues to understand when data from different algorithms and sensors are used over a decade-scale time period to create a continuous dataset. Issues may also arise with sensor degradation and even differences in sensor band locations. In this paper we describe development of an ESDR derived from existing MODIS and VIIRS data products and demonstrate continuity among the products. The MODIS and VIIRS snow cover detection algorithms produce very similar daily snow cover maps, with 90–97% agreement in snow cover extent (SCE) in different landscapes. Differences in SCE between products ranged from 2–15% and are attributable to convolved factors of viewing geometry, pixel spread across a scan and time of observation. Compared at a common grid size of 1 km, there is a mean of 95% agreement in SCE and a difference range of 1–10% between the MODIS and VIIRS SCE maps. Mapping sensor observations to a coarser resolution grid reduces the effect of the factors convolved in the 500 m tile to tile comparisons. We conclude that the MODIS and VIIRS SCE data products are reliable constituents of a moderate-resolution ESDR.

snow cover extent↗

Improving Data Discovery, Analysis, and Visualizations With Cloud-Based User Services

The Global Hydrometeorology Resource Center (GHRC) Distributed Active Archive Center (DAAC) is one of 12 DAACs managed by the United States National Aeronautics and Space Administration (NASA) Earth Science Data and Information System (ESDIS) project [1]. GHRC and the other DAACs are designed to process, archive, document, and distribute NASA Earth-observing data, ranging from satellite missions to field campaigns [2]. A major goal of the DAACs is to enable science with these data. Science enabling can be difficult as datasets can be very large, use multiple formats, come from numerous platforms, and require three-dimensional visualization. GHRC is using its expertise with cloud-based technologies to develop open source and open science tools to empower users to explore, coincidentally visualize, and analyze multiple datasets. Being open source, the user community can develop visualizations for their own datasets. This presentation will expand on this objective and highlight the capabilities available to the international community now.

GHRC↗

The Echoes of Earth Science

NASA s Earth Observing System Data and Information System (EOSDIS) acquires, archives, and manages data from all of NASA s Earth science satellites, for the benefit of the Space Agency and for the benefit of others, including local governments, first responders, the commercial remote sensing industry, teachers, museums, and the general public. EOSDIS is currently handling an extraordinary amount of NASA scientific data. To give an idea of the volume of information it receives, NASA s Terra Earth-observing satellite, just one of many NASA satellites sending down data, sends it hundreds of gigabytes a day, almost as much data as the Hubble Space Telescope acquires in an entire year, or about equal to the amount of information that could be found in hundreds of pickup trucks filled with books. To make EOSDIS data completely accessible to the Earth science community, NASA teamed up with private industry in 2000 to develop an Earth science "marketplace" registry that lets public users quickly drill down to the exact information they need. It also enables them to publish their research and resources alongside of NASA s research and resources. This registry is known as the Earth Observing System ClearingHOuse, or ECHO. The charter for this project focused on having an infrastructure completely independent from EOSDIS that would allow for more contributors and open up additional data access options. Accordingly, it is only fitting that the term ECHO is more than just an acronym; it represents the functionality of the system in that it can echo out and create interoperability among other systems, all while maturing with time as industry technologies and standards change and improve.

Source record↗

A Concept of Operations for Earth Science Data Archive and Distribution in the Cloud

Science data systems can enable more comprehensive Earth system research by evolving to take advantage of advances in commercial computer technology services. Since their inception twenty five years ago, NASA's Earth Observing System Data and Information System (EOSDIS) Distributed Active Archive Centers (DAACs) have periodically evolved to utilize new technology and expand research using the exponential growth and diversity of Earth observations. Recently, with the advent of a maturing commercial compute services industry and upcoming high data volume missions such as the Surface Water and Ocean Topography (SWOT) mission and the NASA-Indian Space Research Organization Synthetic Aperture Radar (NISAR) mission, options were explored and a decision made to utilize commercial compute and storage services. This paper presents an overview of the concept of operations under development for the DAACs in the Cloud. We highlight the goals and expected advantages of utilizing Cloud services. We outline EOSDIS operations tenets and driving principles. A high-level view of EOSDIS system of systems target architecture serves as context for describing principle interactions. Concepts for key DAAC system and EOSDIS enterprise functions characterize automated end-to-end operations but mark nominal check and recovery points. Concepts are presented for managing Cloud resources, including organizational roles and responsibilities of the NASA project and DAAC personnel. Scenarios we use to further distinguish between what the system will do and what configuration and controls operators will have. Examples include interactions with data providers and data consumers with both in-cloud and on-premise facilities.

Moses, John F.↗

Proceedings of the NSSDC Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications

The proceedings of the National Space Science Data Center Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications held July 23 through 25, 1991 at the NASA/Goddard Space Flight Center are presented. The program includes a keynote address, invited technical papers, and selected technical presentations to provide a broad forum for the discussion of a number of important issues in the field of mass storage systems. Topics include magnetic disk and tape technologies, optical disk and tape, software storage and file management systems, and experiences with the use of a large, distributed storage system. The technical presentations describe integrated mass storage systems that are expected to be available commercially. Also included is a series of presentations from Federal Government organizations and research institutions covering their mass storage requirements for the 1990's.

Blackwell, Kim↗

NASA's Earth Observing Data and Information System - Near-Term Challenges

NASA's Earth Observing System Data and Information System (EOSDIS) has been a central component of the NASA Earth observation program since the 1990's. EOSDIS manages data covering a wide range of Earth science disciplines including cryosphere, land cover change, polar processes, field campaigns, ocean surface, digital elevation, atmosphere dynamics and composition, and inter-disciplinary research, and many others. One of the key components of EOSDIS is a set of twelve discipline-based Distributed Active Archive Centers (DAACs) distributed across the United States. Managed by NASA's Earth Science Data and Information System (ESDIS) Project at Goddard Space Flight Center, these DAACs serve over 3 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as the Science Investigator-led Processing systems (SIPS), common metadata and metrics management systems, specialized network systems, standards management, and centralized support for use of commercial cloud capabilities. Given the long-term requirements, and the rapid pace of information technology and changing expectations of the user community, EOSDIS has evolved continually over the past three decades. However, many challenges remain. Challenges addressed in this paper include: growing volume and variety, achieving consistency across a diverse set of data producers, managing information about a large number of datasets, migration to a cloud computing environment, optimizing data discovery and access, incorporating user feedback from a diverse community, keeping metadata updated as data collections grow and age, and ensuring that all the content needed for understanding datasets by future users is identified and preserved.

Remote Sensing↗