SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Harmony allows users to easily access and transform Earth science data. It employs services from providers across the NASA Earth Science Data and Information System (ESDIS) community, supporting jobs from a single granule to millions of granules, at no cost to users.
The ability of large information systems to support the changing data requirements of active science projects is being tested in a NASA collaborative study. This paper briefly profiles both the active science project and the large information system involved in this effort and offers some observations about the effectiveness of the project support. This is followed by lessons that are important for those participating in large information systems that need to support active science projects or that make available the valuable data produced by these projects. We learned in this work that it is difficult for a large information system focused on long term data management to satisfy the requirements of an on-going science project. For example, in order to provide the best service, it is important for all information system staff to keep focused on the needs and constraints of the scientists in the development of appropriate services. If the lessons learned in this and other science support experiences are not applied by those involved with large information systems of the EOS (Earth Observing System) era, then the final data products produced by future science projects may not be robust or of high quality, thereby making the conduct of the project science less efficacious and reducing the value of these unique suites of data for future research.
The Advanced Spaceborne Thermal Emmission and Reflection Radiometer (ASTER) is designed to provide a high-resolution map of the Earth in both visible, near-infrared, and thermal spectral regions of the electromagnetic spectrum. The ASTER Science Team has developed standard data product algorithms to permit the estimation of surface radiances and reflectance values, to calculate surface temperatures both over water and land, to provide a color enhanced product with a high degree of surface discriminability, in addition to other functions. The ASTER Product Generation System (PGS) Team is implementing these requirements within the constraints of the EOSDIS system, using a rapid development methodology that emphasizes open lines of communication in a team approach using concurrent engineering techniques. This paper describes the critical elements in the development and integration of the ASTER PGS system.
NASA's Earth Observing System Data and Information System (EOSDIS) has been a central component of the NASA Earth observation program since the 1990's. EOSDIS manages data covering a wide range of Earth science disciplines including cryosphere, land cover change, polar processes, field campaigns, ocean surface, digital elevation, atmosphere dynamics and composition, and inter-disciplinary research, and many others. One of the key components of EOSDIS is a set of twelve discipline-based Distributed Active Archive Centers (DAACs) distributed across the United States. Managed by NASA's Earth Science Data and Information System (ESDIS) Project at Goddard Space Flight Center, these DAACs serve over 3 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as the Science Investigator-led Processing systems (SIPS), common metadata and metrics management systems, specialized network systems, standards management, and centralized support for use of commercial cloud capabilities. Given the long-term requirements, and the rapid pace of information technology and changing expectations of the user community, EOSDIS has evolved continually over the past three decades. However, many challenges remain. Challenges addressed in this paper include: growing volume and variety, achieving consistency across a diverse set of data producers, managing information about a large number of datasets, migration to a cloud computing environment, optimizing data discovery and access, incorporating user feedback from a diverse community, keeping metadata updated as data collections grow and age, and ensuring that all the content needed for understanding datasets by future users is identified and preserved.
One of the world s largest scientific data systems, NASA'S Earth Observing System Data and Information System (EOSDIS) has stored over three petabytes of earth science data in a geographically distributed mass storage system. Design for this system began in the early 1990s and included a presentation of the design of the mass storage system at this conference in 1995. Many changes have occurred in the ten years since that presentation, much of it performed while the system was operational. In its first operational year (2000), the EOSDIS system had increased NASA's collection of earth science data holdings eight-fold. Today, EOSDIS collects over 7,000 gigabytes of data per week, almost 60 times more than the Hubble Space Telescope. This load represents major challenges for ingest into the mass storage system, as well as for timely and balanced data distribution out of the mass storage system. This paper discusses the evolution of the EOSDIS focusing primarily on the mass storage system component of the archive. We present the lessons that were learned over the years and some directions that we are taking for the future.
The Laser Interferometer Space Antenna (LISA) is a joint NASA/ESA mission to detect and measure gravitational waves with periods from 1 s to 10000 s. The systems engineering challenges of developing a giant interferometer, 5 million kilometers on a side, an: numerous. Some of the key challenges are presented in this paper. The organizational challenges imposed by sharing the engineering function between three centers (ESA ESTEC, NASA GSFC, and JPL) across nine time zones are addressed. The issues and approaches to allocation of the acceleration noise and measurement sensitivity budget terms across a traditionally decomposed system are discussed. Additionally, using LISA to detect gravitational waves for the first time presents significant data analysis challenges, many of which drive the project system design. The approach to understanding the implications of science data analysis on the system is also addressed.
Cloud computing has the potential to bring high performance computing capabilities to the average science researcher. However, in order to take full advantage of cloud capabilities, the science data used in the analysis must often be reorganized. This typically involves sharding the data across multiple nodes to enable relatively fine-grained parallelism. This can be either via cloud-based file systems or cloud-enabled databases such as Cassandra, Rasdaman or SciDB. Since storing an extra copy of data leads to increased cost and data management complexity, NASA is interested in determining the benefits and costs of various cloud analytics methods for real Earth Observation cases. Accordingly, NASA's Earth Science Technology Office and Earth Science Data and Information Systems project have teamed with cloud analytics practitioners to run a benchmark comparison on cloud analytics methods using the same input data and analysis algorithms. We have particularly looked at analysis algorithms that work over long time series, because these are particularly intractable for many Earth Observation datasets which typically store data with one or just a few time steps per file. This post will present side-by-side cost and performance results for several common Earth observation analysis operations.
Explore the source record for details and available documents.
EOSDIS is a data system created by NASA to manage its collection of Earth Science data. This presentation is a brief description of the data system provided to the general user community. The presentation reviews the data types, management and software development techniques in use to organize the system.
The Real Time Mission Monitor (RTMM) is a visualization and information system that fuses multiple Earth science data sources, to enable real time decision-making for airborne and ground validation experiments. Developed at the NASA Marshall Space Flight Center, RTMM is a situational awareness, decision-support system that integrates satellite imagery, radar, surface and airborne instrument data sets, model output parameters, lightning location observations, aircraft navigation data, soundings, and other applicable Earth science data sets. The integration and delivery of this information is made possible using data acquisition systems, network communication links, network server resources, and visualizations through the Google Earth virtual globe application. RTMM is extremely valuable for optimizing individual Earth science airborne field experiments. Flight planners, scientists, and managers appreciate the contributions that RTMM makes to their flight projects. A broad spectrum of interdisciplinary scientists used RTMM during field campaigns including the hurricane-focused 2006 NASA African Monsoon Multidisciplinary Analyses (NAMMA), 2007 NOAA-NASA Aerosonde Hurricane Noel flight, 2007 Tropical Composition, Cloud, and Climate Coupling (TC4), plus a soil moisture (SMAP-VEX) and two arctic research experiments (ARCTAS) in 2008. Improving and evolving RTMM is a continuous process. RTMM recently integrated the Waypoint Planning Tool, a Java-based application that enables aircraft mission scientists to easily develop a pre-mission flight plan through an interactive point-and-click interface. Individual flight legs are automatically calculated "on the fly". The resultant flight plan is then immediately posted to the Google Earth-based RTMM for interested scientists to view the planned flight track and subsequently compare it to the actual real time flight progress. We are planning additional capabilities to RTMM including collaborations with the Jet Propulsion Laboratory in the joint development of a Tropical Cyclone Integrated Data Exchange and Analysis System (TC IDEAS) which will serve as a web portal for access to tropical cyclone data, visualizations and model output.
The NASA Global Imagery Browse Services (GIBS) and Worldview interactive mapping site leverage scientific and community best practices, open source software, and public standards to provide a scalable, compliant, and authoritative source for NASA Earth Observing System (EOS) Earth science data visualizations. GIBS and Worldview allow end users to easily and quickly interact with more than 800 full resolution pre-generated raster- and vector-based visualizations. This interactive discovery approach relies on visual observation and identification of phenomena that are not as simply identified otherwise. This eLightning presentation will exhibit the broad set of capabilities and visualization layers made possible through the GIBS and Worldview open source software. Specific dependencies on, and contributions to, open source software will be highlighted. Additionally, opportunities for future improvements for better interoperability and reuse through open source software will be discussed.
The Planetary Data System (PDS) information model is a mature but complex model that has been used to capture over 30 years of planetary science data for the PDS archive. As the de-facto information model for the planetary science data archive, it is being adopted by the International Planetary Data Alliance (IPDA) as their archive data standard. However, after seventeen years of evolutionary change the model needs refinement. First a formal specification is needed to explicitly capture the model in a commonly accepted data engineering notation. Second, the core and essential elements of the model need to be identified to help simplify the overall archive process. A team of PDS technical staff members have captured the PDS information model in an ontology modeling tool. Using the resulting knowledge-base, work continues to identify the core elements, identify problems and issues, and then test proposed modifications to the model. The final deliverables of this work will include specifications for the next generation PDS information model and the initial set of IPDA archive data standards. Having the information model captured in an ontology modeling tool also makes the model suitable for use by Semantic Web applications.
This paper presents a comprehensive overview of the Coupled Model Intercomparison Project Phase 7 (CMIP7) request for data pertaining to Earth systems science, and provides justification for the resources needed to produce this data. Topics within the CMIP7 Earth System (CMIP7-ES) theme centre around tracking of flows of energy, carbon, water and other fluxes across domains, and constraining feedbacks between these cycles and the climate system. These topics are summarized in this paper as scientific “opportunities” describing specific model intercomparison experiments and use cases for next-generation Earth System Model (ESM) output. These opportunities were submitted by modelling groups and scientific consortia following an extended public consultation process. Contained within each opportunity are requests for groups of Climate & Forecasting (CF) variables, which are bundled into variable groups representing all data required to address the opportunities' needs. Novel opportunities in CMIP7 compared with previous phases will include running `emissions-driven' simulations that integrate carbon emissions and removal scenarios with updated representations of the global carbon cycle, expanded variable groups needed to model marine trophic interactions and biogeochemistry, and data needed to understand the risk of global tipping points, among others. The production of these variables will close key gaps and uncertainties identified during previous rounds of CMIP, and support the 7th Intergovernmental Panel on Climate Change Assessment Report (AR7). We argue that CMIP7-ES data will be broadly used by scientific, policy, governmental, industry, and other communities that rely on climate model projections for research and decision making. As an author group we also reflect on the evolution of the CMIP7-ES data request as a part of a deliberative process in support of the global CMIP program.
The Tropical Rainfall Measuring Mission (TRMM) satellite was successfully launched in November, 1997. The main purpose of TRMM is to sample tropical rainfall using the first active spaceborne precipitation radar. To validate TRMM satellite observations, a comprehensive Ground Validation (GV) Program has been implemented. The primary goal of TRMM GV is to provide basic validation of satellite-derived precipitation measurements over monthly climatologies for the following primary sites: Melbourne, FL; Houston, TX; Darwin, Australia; and Kwajalein Atoll, RMI. As part of the TRMM GV effort, research analysts at NASA Goddard Space Flight Center (GSFC) generate standardized TRMM GV products using quality-controlled ground-based radar data from the four primary GV sites as input. This presentation will provide an overview of the TRMM GV climatological processing system. A description of the data flow between the primary GV sites, NASA GSFC, and the TRMM Science and Data Information System (TSDIS) will be presented. The radar quality control algorithm, which features eight adjustable height and reflectivity parameters, and its effect on monthly rainfall maps will be described. The methodology used to create monthly, gauge-adjusted rainfall products for each primary site will also be summarized. The standardized monthly rainfall products are developed in discrete, modular steps with distinct intermediate products. These developmental steps include: (1) extracting radar data over the locations of rain gauges, (2) merging rain gauge and radar data in time and space with user-defined options, (3) automated quality control of radar and gauge merged data by tracking accumulations from each instrument, and (4) deriving Z-R relationships from the quality-controlled merged data over monthly time scales. A summary of recently reprocessed official GV rainfall products available for TRMM science users will be presented. Updated basic standardized product results and trends involving monthly accumulation, Z-R relationship, and gauge statistics for each primary GV site will be also displayed.
Efforts to identify high payback functions performed by the Office of Space Science Applications (OSSA) data systems package existing operationally proven implementations as system building blocks and foster their use in the OSSA environment. An overview is given of the current state of OSSA data systems and the major challenges facing system developers over the next five years. Current trends in system commonality are discussed, and development is examined in the context of system design. Central system engineering approaches to solving the commonality and interoperability challenges are described. Efforts to package three diverse building blocks and to foster their reuse are documented.
The NASA Real Time Mission Monitor (RTMM) is a visualization and information system that fuses multiple Earth science data sources, to enable real time decisionmaking for airborne and ground validation experiments. Developed at the National Aeronautics and Space Administration (NASA) Marshall Space Flight Center, RTMM is a situational awareness, decision-support system that integrates satellite imagery and orbit data, radar and other surface observations (e.g., lightning location network data), airborne navigation and instrument data sets, model output parameters, and other applicable Earth science data sets. The integration and delivery of this information is made possible using data acquisition systems, network communication links, network server resources, and visualizations through the Google Earth virtual globe application. In order to improve the usefulness and efficiency of the RTMM system, capabilities are being developed to allow the end-user to easily configure RTMM applications based on their mission-specific requirements and objectives. This second generation RTMM is being redesigned to take advantage of the Google plug-in capabilities to run multiple applications in a web browser rather than the original single application Google Earth approach. Currently RTMM employs a limited Service Oriented Architecture approach to enable discovery of mission specific resources. We are expanding the RTMM architecture such that it will more effectively utilize the Open Geospatial Consortium Sensor Web Enablement services and other new technology software tools and components. These modifications and extensions will result in a robust, versatile RTMM system that will greatly increase flexibility of the user to choose which science data sets and support applications to view and/or use. The improvements brought about by RTMM 2nd generation system will provide mission planners and airborne scientists with enhanced decision-making tools and capabilities to more efficiently plan, prepare and execute missions, as well as to playback and review past mission data. To paraphrase the old television commercial RTMM doesn t make the airborne science, it makes the airborne science easier.
The Earth Science Data and Information System (ESDIS) Project manages twelve Distributed Active Archive Centers (DAACs) which are geographically dispersed across the United States. The DAACs are responsible for ingesting, processing, archiving, and distributing Earth science data produced from various sources (satellites, aircraft, field measurements, etc.). In response to projections of an exponential increase in data production, there has been a recent effort to prototype various DAAC activities in the cloud computing environment. This, in turn, led to the creation of an initiative, called the Cloud Analysis Toolkit to Enable Earth Science (CATEES), to develop a Python software package in order to transition Earth science data processing to the cloud. This project, in particular, supports CATEES and has two primary goals. One, to transition data recipes created by the Goddard Earth Science Data and Information Service Center (GES DISC) into an interactive and educational environment using JupyterNotebooks. Two, to acclimate Earth scientists to cloud computing. To accomplish these goals, we create JupyterNotebooks to compartmentalize the different steps of data analysis and help users obtain and parse data from the command line. We also develop a Docker container, comprised of Jupyter Notebooks, Python dependencies, and command line tools, and configure it into an easy-to-deploy package. The end result is an end-to-end product that simulates the use case of end users working in the cloud computing environment.