EOSDIS Zarr Store - Fast, Direct Access to NASA EOSDIS Data in the Cloud
Explore the source record for details and available documents.
Patrick Quinn↗
SEARCH · Search NASA
Results for “EOSDIS”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Panel to review EOSDIS plans
Formed in Jan. 1992, the Panel to Review EOSDIS Plans was charged with advising NASA on its plans for developing the Earth Observing System (EOS) Data and Information System (EOSDIS). Specifically, the panel was asked to do the following: assess the validity of the engineering and technical underpinnings of the EOSDIS; assess its potential value to scientific users; suggest how technical risk can be minimized; and assess whether current plans are sufficiently resilient to be adaptable to changing technology and requirements such as budget environments, data volumes, new users, and new databases. The panel completed an interim report (Addendum A) and transmitted it to NASA and other interested parties in the government on 9 Apr. 1992. Because of a delay in NASA's plans to select the contractor for EOSDIS, the panel was not able to complete its review of the program according to the original government request. With the issuance of a letter report (Addendum B) on 28 Sep. 1992, the panel became inactive until such time as NASA could release the details of the contractor's proposed architecture, schedule, and costs for developing EOSDIS. In early 1993, NASA awarded the contract for the EOSDIS Core System (ECS). On 20 Apr. 1993, NASA asked the panel to reconvene to do the following: ( 1) complete its review of NASA's approach to the EOSDIS architecture and implementation; (2) appraise NASA's responses to the panel's previous recommendations; and (3) review the planning for EOSDIS in the context of NASA's role in the Global Change Data and Information System (GCDIS) implementation plan. To respond to the NASA charge, the panel met three times in 1993 including sessions with NASA officials and the EOSDIS contractor. In addition, several of the panel members visited individual Distributed Active Archive Centers (DAAC's) to obtain additional views of EOSDIS. The panel has now obtained substantial information on the EOSDIS budget, contractor work program, and current baseline architecture that was not previously available, due to procurement restrictions. This report presents the panel's findings and recommendations based on this additional information.
Source record↗
EOSDIS: Archive and Distribution Systems in the Year 2000
Earth Science Enterprise (ESE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of ESE. The EOS Data and Information System (EOSDIS) is another component of ESE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at seven Distributed Active Archive Centers (DAACs) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. The first major set of data to be archived in the EOSDIS is from Landsat-7. Another EOS satellite, Terra, was launched on December 18, 1999. With the Terra launch, the EOSDIS will be required to support approximately one terabyte of data into and out of the archives per day. Since EOS is a multi-mission program, including the launch of more satellites and many other missions, the role of the archive systems becomes larger and more critical. In 1995, at the fourth convening of NASA Mass Storage Systems and Technologies Conference, the development plans for the EOSDIS information system and archive were described. Five years later, many changes have occurred in the effort to field an operational system. It is interesting to reflect on some of the changes driving the archive technology and system development for EOSDIS. This paper principally describes the Data Server subsystem including how the other subsystems access the archive, the nature of the data repository, and the mass-storage I/O management. The paper reviews the system architecture (both hardware and software) of the basic components of the archive. It discusses the operations concept, code development, and testing phase of the system. Finally, it describes the future plans for the archive.
Behnke, Jeanne↗
Evolution of the Earth Observing System (EOS) Data and Information System (EOSDIS)
One of the strategic goals of the U.S. National Aeronautics and Space Administration (NASA) is to "Develop a balanced overall program of science, exploration, and aeronautics consistent with the redirection of the human spaceflight program to focus on exploration". An important sub-goal of this goal is to "Study Earth from space to advance scientific understanding and meet societal needs." NASA meets this subgoal in partnership with other U.S. agencies and international organizations through its Earth science program. A major component of NASA s Earth science program is the Earth Observing System (EOS). The EOS program was started in 1990 with the primary purpose of modeling global climate change. This program consists of a set of space-borne instruments, science teams, and a data system. The instruments are designed to obtain highly accurate, frequent and global measurements of geophysical properties of land, oceans and atmosphere. The science teams are responsible for designing the instruments as well as scientific algorithms to derive information from the instrument measurements. The data system, called the EOS Data and Information System (EOSDIS), produces data products using those algorithms as well as archives and distributes such products. The first of the EOS instruments were launched in November 1997 on the Japanese satellite called the Tropical Rainfall Measuring Mission (TRMM) and the last, on the U.S. satellite Aura, were launched in July 2004. The instrument science teams have been active since the inception of the program in 1990 and have participation from Brazil, Canada, France, Japan, Netherlands, United Kingdom and U.S. The development of EOSDIS was initiated in 1990, and this data system has been serving the user community since 1994. The purpose of this chapter is to discuss the history and evolution of EOSDIS since its beginnings to the present and indicate how it continues to evolve into the future. this chapter is organized as follows. Sect. 7.2 provides a discussion of EOSDIS, its elements and their functions. Sect. 7.3 provides details regarding the move towards more distributed systems for supporting both the core and community needs to be served by NASA Earth science data systems. Sect. 7.4 discusses the use of standards and interfaces and their importance in EOSDIS. Sect. 7.5 provides details about the EOSDIS Evolution Study. Sect. 7.6 presents the implementation of the EOSDIS Evolution plan. Sect. 7.7 briefly outlines the progress that the implementation has made towards the 2015 Vision, followed by a summary in Sect. 7.8.
Ramapriyan, Hampapuram K.↗
NASA EOSDIS Data Usage Metrics- Insight and Assessment
NASA's Earth Science Data and Information System (ESDIS) Project collects Earth science data usage metrics on a daily basis through the ESDIS Metrics System (EMS). This includes metrics on distribution of data products, users, data volumes, and number of files, which are key parameters in evaluating system-level performance of any of the Distributed Active Archive Centers (DAACs) encompassed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS). EOSDIS data usage metrics illustrate the benefits of making NASA data openly available to the public and show a rapid growth in data distribution to a worldwide user community. In fact, each year since 2014 the EOSDIS has distributed over one billion data files of products from EOS satellite, airborne, and in situ observations. An assessment of the long-term trends of data usage metrics and user characterization provides insights into data usability.This study will focus on describing the EMS as a metrics collection tool and will provide a comprehensive analysis of EOSDIS data usage metrics over the last 10 years. This study will also characterize the product distribution metrics by various tools and services, such as Giovanni, the Open-source Project for a Network Data Access Protocol (OPeNDAP), and subsets, to address how these tools/services have extended the usage of data in the EOSDIS collection. Data usage patterns based on discipline and study area will further assist in understanding how EOSDIS data user needs have evolved over time. Results from this study will provide useful information for the DAACs that can help them improve the functionality of their tools and services as well as more efficiently allocate the resources necessary for enhanced access and availability of their data products. Knowledge of these metrics may also benefit user discovery of data in the EOSDIS collection, promote research collaboration, and stimulate new ideas from work and research conducted using specific datasets and data collections.
Kafle, Durga N.↗
Reusability in NASA's Earth Observation System Data and Information System (EOSDIS)
NASA's Earth Observation System Data and Information System (EOSDIS) has been operational since 1994. The term FAIR is relatively recent compared to the operational life of EOSDIS. However, given the evolutionary nature of EOSDIS in response to both the state of the art and community expectations, it is useful to assess where EOSDIS stands relative to the FAIR data management. In this presentation we evaluate EOSDIS against the Reusability principle, which calls for a clear and accessible data usage license, rich description of data, detailed provenance, and compliance with domain-relevant community standards.Achieving consistent compliance for datasets is particularly challenging with the diverse community of missions and scientists providing data to EOSDIS. To address this, EOSDIS's constituent Distributed Active Archive Centers work with data producers to utilize the applicable standards and conventions. In addition, a guidebook is under development to provide comprehensive and understandable guidance on how to construct compliant, and thus more reusable, data products.
Hampapuram (Rama) Ramapriyan↗
Achieving cost reductions in EOSDIS operations through technology evolution
The earth observing system (EOS) data information system (EOSDIS) mission includes the cost-effective management and distribution of large amounts of data to the earth science community. The effect of the introduction of new information system technologies on the evolution of EOSDIS is considered. One of the steps taken by NASA to enable the introduction of new information system technologies into the EOSDIS is the funding of technology development through prototyping. Recent and ongoing prototyping efforts and their potential impact on the performance and cost-effectiveness of the EOSDIS are discussed. The technology evolution process as it related to the effective operation of EOSDIS is described, and methods are identified for the support of the transfer of relevant technology to EOSDIS components.
Newsome, Penny↗
NASA Earth Observing System Data and Information System (EOSDIS): A U.S. Network of Data Centers Serving Earth Science Data: A Network Member of ICSU WDS
NASA's Earth Observing System Data and Information System (EOSDIS) has been in operation since August 1994, and serving a diverse user community around the world with Earth science data from satellites, aircraft, field campaigns and research investigations. The ESDIS Project, responsible for EOSDIS is a Network Member of the International Council for Sciences (ICSU) World Data System (WDS). Nine of the 12 Distributed Active Archive Centers (DAACs), which are part of EOSDIS, are Regular Members of the ICSUWDS. This poster presents the EOSDIS mission objectives, key characteristics of the DAACs that make them world class Earth science data centers, successes, challenges and best practices of EOSDIS focusing on the years 2014-2016, and illustrates some highlights of accomplishments of EOSDIS. The highlights include: high customer satisfaction, growing archive and distribution volumes, exponential growth in number of products distributed to users around the world, unified metadata model and common metadata repository, flexibility provided to uses by supporting data transformations to suit their applications, near-real-time capabilities to support various operational and research applications, and full resolution image browse capabilities to help users select data of interest. The poster also illustrates how the ESDIS Project is actively involved in several US and international data system organizations.
Earth Science↗
NASA EOSDIS 20 Years of Data Usage and User Assessment in Support of Open Science Initiative
NASA EOS Data and Information System (EOSDIS) has been distributing data to world-wide users free with open access. Since the launch of NASA’s Terra satellite in 1999, more than 10,000 distinct EOS data products have been archived and distributed by NASA-funded Earth Science data centers encompassed by the EOSDIS. As of September 30, 2022, more than 90 PB of data archived by EOSDIS have been made available to public users and during FY 2023 over 60 PB have been distributed to public users worldwide. Over these twenty and more years, it has shown significant increase in the distribution of various data products. This has been possible due to free and open access of the data thereby a step towards open science initiative. The purposes of this study are 1) to perform a comprehensive investigation of the archive and distribution patterns of EOSDIS data products for last 20 years, 2) to identify and characterize the global user community for those data, 3) analyze the increased demand for data products, 4) evaluate distribution of higher level products because those are the ones most frequently used in the studies of natural disasters by public users (those data requestors not involved directly in the production or validation of the data products.) and contribute globally to the advance scientific understanding of the Earth-Atmosphere Systems. Funded by the Earth Science Data and Information System (ESDIS) Project, the ESDIS Metrics System (EMS) collects archive, distribution, and user information from EOSDIS data centers. The information (comprising all data products including heritage datasets going back to the 1990s) is stored in a relational database from which it can be analyzed in many ways. We present several metrics analyses that include data distribution patterns for all, as well as the most frequently requested data products; and user characterizations by country, domain, and Earth Science discipline (e.g., Land, Ocean, Cryosphere) of the requested products. Due to the enormous quantity of data handled by EOSDIS data centers and requirements of future data systems to archive increasing amounts of Earth Science data from future and current Earth Science missions effectively, the results of this study can provide insight on how the user communities have accessed the data and provide guidance for open science initiative.
Lalit Wanchoo↗
NASA's EOSDIS, Trust and Certification
NASA's Earth Observing System Data and Information System (EOSDIS) has been in operation since August 1994, managing most of NASA's Earth science data from satellites, airborne sensors, filed campaigns and other activities. Having been designated by the Federal Government as a project responsible for production, archiving and distribution of these data through its Distributed Active Archive Centers (DAACs), the Earth Science Data and Information System Project (ESDIS) is responsible for EOSDIS, and is legally bound by the Office of Management and Budgets circular A-130, the Federal Records Act. It must follow the regulations of the National Institute of Standards and Technologies (NIST) and National Archive and Records Administration (NARA). It must also follow the NASA Procedural Requirement 7120.5 (NASA Space Flight Program and Project Management). All these ensure that the data centers managed by ESDIS are trustworthy from the point of view of efficient and effective operations as well as preservation of valuable data from NASA's missions. Additional factors contributing to this trust are an extensive set of internal and external reviews throughout the history of EOSDIS starting in the early 1990s. Many of these reviews have involved external groups of scientific and technological experts. Also, independent annual surveys of user satisfaction that measure and publish the American Customer Satisfaction Index (ACSI), where EOSDIS has scored consistently high marks since 2004, provide an additional measure of trustworthiness. In addition, through an effort initiated in 2012 at the request of NASA HQ, the ESDIS Project and 10 of 12 DAACs have been certified by the International Council for Science (ICSU) World Data System (WDS) and are members of the ICSUWDS. This presentation addresses questions such as pros and cons of the certification process, key outcomes and next steps regarding certification. Recently, the ICSUWDS and Data Seal of Approval (DSA) organizations merged their Core Trustworthy Data Repositories Requirements and require that members be recertified every three years. Given the rigor with which NASA manages the ESDIS Project and the DAACs, the recertification through WDSDSA, while involving some additional work, is a relatively simple process.
Certification↗
EOSDIS - Its role in the EOS program and its importance to the scientific community
In order to manage the global data sets required to understand the earth as a system, the EOS Data and Information System (EOSDIS) will collect and store satellite, aircraft, and in situ measurements and their resultant data products, and will distribute the data conveniently. EOSDIS will also provide product generation and science computing facilities to support the development, processing, and validation of standard EOS science data products. The overall architecture of EOSDIS, and how the Distributed Active Archive Centers fit into that structure, are shown. EOSDIS will enable users to query data bases nationally, make use of keywords and other mnemonic identifiers, and see graphic images of subsets of available data prior to ordering full (or selected pieces of) data sets for use in their 'home' environment.
Price, Robert D.↗
Case Study of an Object-Oriented System: EOSDIS
Mission to Planet Earth (MTPE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of MTPE. The EOS Data and Information System (EOSDIS) is another component of MTPE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at six Distributed Active Archive Centers (DAACS) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. While there are many segments in EOSDIS (e.g., flight operations, network) this case study discusses the development of the science data processing segment (SDPS). We briefly review the architecture of the system, the goals of the SDPS, and the development progress to date. This study highlights key software development challenges, experiences integrating COTS, and the difficulties of managing a complex system development effort.
Behnke, Jeanne↗
EOSDIS Petabyte Archives: Tenth Anniversary
One of the world s largest scientific data systems, NASA'S Earth Observing System Data and Information System (EOSDIS) has stored over three petabytes of earth science data in a geographically distributed mass storage system. Design for this system began in the early 1990s and included a presentation of the design of the mass storage system at this conference in 1995. Many changes have occurred in the ten years since that presentation, much of it performed while the system was operational. In its first operational year (2000), the EOSDIS system had increased NASA's collection of earth science data holdings eight-fold. Today, EOSDIS collects over 7,000 gigabytes of data per week, almost 60 times more than the Hubble Space Telescope. This load represents major challenges for ingest into the mass storage system, as well as for timely and balanced data distribution out of the mass storage system. This paper discusses the evolution of the EOSDIS focusing primarily on the mass storage system component of the archive. We present the lessons that were learned over the years and some directions that we are taking for the future.
Behnke, Jeanne↗
Managing Sustainable Data Infrastructures: The Gestalt of EOSDIS
EOSDIS epitomizes a System of Systems, whose many varied and distributed parts are integrated into a single, highly functional organized science data system. A distributed architecture was adopted to ensure discipline-specific support for the science data, while also leveraging standards and establishing policies and tools to enable interdisciplinary research, and analysis across multiple scientific instruments. The EOSDIS is composed of system elements such as geographically distributed archive centers used to manage the stewardship of data. The infrastructure consists of underlying capabilities connections that enable the primary system elements to function together. For example, one key infrastructure component is the common metadata repository, which enables discovery of all data within the EOSDIS system. EOSDIS employs processes and standards to ensure partners can work together effectively, and provide coherent services to users.
remote sensing↗
NASA EOSDIS: Enabling Science by Improving User Knowledge
Lessons learned and impacts of applying these newer methods are explained and include several examples from our current efforts such as the interactive, on-line webinars focusing on data discovery and access including tool usage, informal and informative data chats with data experts across our EOSDIS community, data user profile interviews with scientists actively using EOSDIS data in their research, and improved conference and meeting interactions via EOSDIS data interactively used during hyper-wall talks and Worldview application. The suite of internet-based, interactive capabilities and technologies has allowed our project to expand our user community by making the data and applications from numerous Earth science missions more engaging, approachable and meaningful.
EOSDIS↗
EOSDIS core system architecture
The architecture of NASA's Earth Observation System Data Information System (EOSDIS) is described. The role of the EOSDIS Core System (ECS) is described, showing how each ECS functional category (flight operations functions, science data processing functions, and communications and system management functions) supports the five EOSDIS goals and objectives. The baseline configuration of ECS is presented and discussed.
Guzek, Joseph A.↗
The EOS Data and Information System (EOSDIS)
The Earth Observing System (EOS) is the centerpiece of NASA's Mission to Planet Earth initiative. It is a pivotal part of the U.S. Global Change Research Program and hence of the international effort to understand global change and the increasing demands of human activity. EOS consists of a space-based observing system, a Data and Information System (EOSDIS) and a scientific research program. The space component consists of two series of polar-orbiting spacecraft, the first scheduled for launch in 1998, that will collect data for 15 years. EOS is completing its conceptual design phase and is preparing to enter the design phase with the selection and construction of the instruments for the first platform. EOSDIS will allow researchers to quickly and easily access data about the Earth system. Development of EOSDIS has already begun; it will support research and analysis with existing data. Geophysical and biological products will be created from the satellite data to be used to a broad range of the scientific community. NASA has also committed to providing smaller missions - called Earth Probes - dedicated to near-term observations of specific Earth processes. The scientific research program was initiated in 1990, with funding for 28 interdisciplinary teams, to begin development of models that will use EOS data and define the data requirements from the instruments, nine facility instruments and their science teams, 23 instrument investigations and definition studies for a synthetic-aperture radar.
Dozier, J.↗
Interdisciplinary Research Scenario Testing of EOSDIS
During the reporting period, the Principle Investigator (PI) has continued to serve on numerous review panels, task forces and committees with the goal of providing input and guidance for the Earth Observing System Data and Information System (EOSDIS) program at NASA Headquarters and NASA GSFC. In addition, the PI has worked together with personnel at the University of Virginia and the subcontractor (Simpson Weather Associates (SWA)) to continue to evaluate the latest releases of various versions of the user interfaces to the EOSDIS. Finally, as part of the subcontract, SWA has created an on-line Hierarchial Data Format (HDF) tutorial for non-HDF experts, particularly those that will be using EOSDIS and future EOS data products. A summary of these three activities is provided. The topics include: 1) Participation on EODIS Panels and Committees; 2) Evaluation and Tire Kicking of EODIS User Interfaces; and 3) An On-line HDF Tutorial. The report also includes attachments A, B, and C. Attachment A: Report From the May 1999 Science Data Panel. The topics include: 1) Summary of Data Panel Meeting; and 2) Panel's Comments/Recommendations. Attachment B: Survey Requesting Integrated Design Systems (IDS) Teams Input on the Descoping and Rescoping of the EODIS; and Attachment C: An HDF Tutorial for Beginners: EODIS Users and Small Data Providers (HTML Version). The topics include: 1) Tutorial Overview; 2) An introduction to HDF; 3) The HDF Library: Software and Hardware; 4) Methods of Working with HDF Files; 5) Scientific Data API; 6) Attributes and Metadata; 7) Writing a SDS to an HDF file; 8) Obtaining Information on Existing HDF Files; 9) Reading a Scientific Data Set from an HDF file: 10) Example Programs; 11) Browsing and Visualizing HDF Data; and 12) Laboratory (Question and Answer).
Emmitt, G. D.↗