Search NASASearch

SEARCH · Search NASA

Results for “analysis ready data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Near 40 Years MERRA-2 Data at NASA GES DISC -Opportunity and Challenge to Support Extremes Study

To the end of 2019, 40 years NASA climate reanalysis data sets from the Modern Era Retrospective-analysis for Research and Applications, Version 2 (MERRA-2) will be available at NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). MERRA-2 consists atmosphere, land, and ocean data, which may be used for the studies ranging from the short scale weather events to the large scale decadal vulnerabilities. The hourly products, such as precipitation, soil moisture, temperature, and aerosols etc., have been used widely to study extreme events.In supporting users from broad communities, GES DISC have developed various data access services, including subsetter for downloading only data of interest with preferred format; OPeNDAP - for machine-to-machine data access; and Giovanni- for online visualization and analysis, etc. A big challenge for extreme study is to downloading and processing long-term hourly or daily data. The data downloading performance is not very satisfied by many users with current services and the native archived data structure. Late June 2019, many people in Europe had experienced extreme heat waves. The temperatures in several countries exceeded 40°C (104°F). For example, MERRA-2 shows that the near surface daily maximum temperature of June 28 2019 over Marseille, a city in southern France, reached 41.1 °C (106°F), which is the record breaking temperature in the last 40 years. GES DISC is working together with domain science experts to improve the performance of long time series access, making analysis ready data sets in supporting application researches, such as extreme study. In this presentation, using Europe heat wave as an example, we will show prototype of the in developing service for finding extremes from near 40 years MERRA-2 data at a given location. MERRA-2 data can be accessed from NASA GES DISC(https://disc.gsfc.nasa.gov/ ) by search keyword "MERRA-2".

Shen, Suhung

The NASA Earth Science Applied Sciences Disasters Program: Making EO Data and Expertise Available through the Disasters Mapping Portal to Inform Decision-Makers throughout the Disaster Cycle

The NASA Earth Science Applied Sciences Disasters Program promotes the use of Earth Observations (EO) to inform disaster risk reduction and resilience throughout the disaster cycle, from local to global scales, by harnessing NASA science and technology capabilities, through engagement with end users to demonstrate the value and impact of EO to support decision-making, and by supporting end users in their use of EO in decision-making while developing relationships to grow as a trusted source of relevant science and useful results. Through the NASA Earth Science Applied Sciences Disasters Program Mapping Portal, an Esri-backed hub of geospatially enabled disaster products, event-based and near-real-time products are hosted to provide end users with analysis-ready data and data services, keeping in mind their expressed EO needs. In this presentation, case studies of past events will be highlighted, showcasing how EO data and services were provided by the Program and utilized by end users during the disaster cycle. Additionally, this presentation will highlight some of the Program’s ongoing collaboration activities to assist end users in understanding and preparing for utilization of EO data from the Mapping Portal to inform their decision-making throughout the disaster cycle. Ongoing efforts to advance the science provided by the Program on the Mapping Portal through new developments and capabilities will be highlighted, and common challenges and data needs end users often encounter when using EO data and our Program’s innovative solutions to these challenges will be addressed. Furthermore, this presentation will touch on the challenges faced and the solutions created by the Mapping Portal team in managing and hosting a plethora of EO data for end users across various disciplines.

Ronan Lucey

Data Cube Application Algorithms For The United Nations Sustainable Development Goals (UN-SDGS)

In 2015, all United Nations (UN) Member States adopted the 2030 Agenda for Sustainable Development. The Agenda provides a shared blueprint for peace and prosperity for people and for the planet, considering our current situation and helping to create a plan. The core of this agenda is a set of seventeen Sustainable Development Goals (SDGs), which represent an urgent call for action by all countries -both developed and developing - in a global partnership. The Committee on Earth Observation Satellites (CEOS) Systems Engineering Office (SEO) team has recently developed and released a set of innovative notebooks addressing UN SDGs 6.6.1 (spatial extents of water-related ecosystems), 11.3.1 (ratio of land consumption rate to population growth rate), and 15.3.1 (proportion of land that is degraded over total land area). These notebooks empower users by providing features that will assist with streamlining analysis ready data retrieval, processing, and visualization. The main contributions in this paper are: (1) briefly describing the framework of the UN SDG notebooks, (2) enumerating the notebooks’ salient features, and (3) discussing current limitations and proposing approaches to overcome these limitations.

Open Data Cube

Assessment of the Performance of the Atmospheric Correction Algorithm MAJA for Sentinel-2 Surface Reflectance Estimates

The correction of atmospheric effects on optical remote sensing products is an essential component of Analysis Ready Data (ARD) production lines. The MAJA processor aims at providing accurate time series of surface reflectances over land for satellite missions, such as Sentinel-2, Venμs, and Landsat 8. The Centre d’Études Spatiales de la Biosphère (CESBIO) and the Centre National d’Études Spatiales (CNES) share a common effort to maintain, validate, and improve the MAJA processor, using state-of-the-art ground measurement sites, and participating in processor inter-comparisons, such as the Atmospheric Correction Intercomparison Exercise (ACIX). While contributing to the second ACIX-II Land validation exercise, it was found that the candidate MAJA dataset could not adequately be compared to the main reference dataset. MAJA reflectances were corrected for adjacency and topography effects while the reference dataset was not, excluding MAJA from a part of the performance metrics of the exercise. The first part of the following study aims at providing complementary performance assessment to ACIX-II by reprocessing MAJA surface reflectances without adjacency nor topographic correction, allowing for an un-biased full resolution comparison with the reference Sentinel-2 dataset. The second part of the study consists of validating MAJA against surface reflectance measurements time series of up to five years acquired at three automated stations. Both approaches provide extensive insights on the quality of MAJA Sentinel-2 Level 2 products.

ROSAS

NASA POWER: Providing Analysis-Ready, Cloud-Optimized Data for AI /ML Training and Applications in Earth Science

As global demand for sustainable development grows, the integration of Earth Observation (EO) data into decision making frameworks has become a primary objective for the scientific community. The NASA Prediction of Worldwide Energy Resources (POWER) project serves as a bridge between NASA EO data and the specialized needs of the renewable energy, sustainable infrastructure and agroclimatology communities. In this poster presentation we will present an overview of POWER data products and services along with its use in diverse research to decision-making workflows. By providing over 40 years of high-resolution historical, hourly and daily solar and meteorological data, POWER transforms satellite observations and global model reanalysis into actionable, Analysis-Ready Dataset (ARD). Currently, the project delivers over 250 industry-friendly parameters to the users from different NASA datasets like CERES SYN1Deg, MERRA-2, and IMERG alongside downscaled CMIP6 climate model data, fulfilling over 16 million requests from 50,000 unique users monthly. To ensure data quality and traceability, these parameters are rigorously validated against the ground-based observations from the Baseline Surface Radiation Network (BSRN) and the Global Surface Summary of the Day (GSOD) – these results will be discussed in the presentation. A newly introduced web-based PaRameter Uncertainty ViEwer (PRUVE) tool will be presented that provides an online validation platform to the users that benchmarks satellite-based and assimilation data products against these surface measurements. To reduce technical barriers to data adoption, POWER data is accessible through RESTful APIs, ESRI ArcGIS Image Services, a web-based Data Access Viewer tool, allowing users to visualize, validate and apply the dataset. For efficient data delivery POWER data is cloud-optimized into Zarr datastore accessible through NASA managed Amazon S3 ensures high-performance allowing users to integrate EO directly into operational pipelines. These customized services will be presented. Use cases from application will be presented from the energy sector - such as for design of generation systems, performance monitoring of solar power plants, in infrastructure sector- optimizing building energy efficiency and thermal comfort, in agriculture – such as driving crop simulation and yield forecasting models to enable climate resilient farming. Furthermore, the shift toward machine learning (ML) in EO research that has positioned POWER as a key provider for training datasets which will be discussed. Use-cases will be presented to showcase how NASA data is enabling the development of predictive tools for climate variability and resource management. The poster will present POWER’s future plans including technology development to enhance data traceability and reproducibility and improving I/O performance to support the rapid integration of new EO products, ensuring that POWER remains a robust scalable backend for the evolving landscape of AI-driven Earth Science. Additionally, POWER is developing an AI Agent and an MCP-Server to enable industry AI-Agentic workflows.

Neha Khadka

Assessing the Needs of NASA's Near Real-Time Earth Observation Products

"The 2017-2027 Decadal Survey for Earth Science and Applications from Space stated that NASA's Earth Science with planned implementation of applications provides sustained earth observations for societal benefits [1]. The Decadal Survey indicated that data latency is invaluable for time-sensitive applications including disaster risk reduction, wildland fire carbon emissions quantification, real-time measurements of the state of the hydrologic systems and many more. Data latency refers to the time between earth observation and data products available to users. During the past 13 years, NASA's Land, Atmosphere Near Real-Time Capability for Earth Observing Systems (LANCE) continues to provide free access to earth observation products that are made available much quicker than routine processing allows. The latency of most LANCE data products is Near Real-time (NRT) which is defined as less than three hours from satellite observations [2]. LANCE is managed by the Earth Science Data and Information System (ESDIS) Project at NASA Goddard Space Flight Center [3], and a User Working Group (UWG) is responsible for providing guidance to LANCE. LANCE data are used by direct users and brokers who add value to the data [4]. NASA Earth Applied Sciences Program (ASP) is one of the primary users of LANCE, which collaborates with partner organizations and provides support to scientists to solve problems in applications of earth observations. ASP promotes the use of LANCE NRT data products to demonstrate applications in decision making, facilitates end-user feedback to the science team to improve data products, and provides information on future demands for research. LANCE supports applications that need a rapid response including detecting wildland fires and volcanic eruptions, tracking smoke, ash and dust plumes, monitoring air quality and tracking extreme weather events such as hurricanes, landslides, and floods. To gather feedback regarding the availability, accessibility and actionability of NASA's NRT data products for societal benefit, three surveys and a few discussions with experts involved in the topic within ASP were conducted from the perspective of users. Feedback has been collected from users who are interested in using low latency NASA data within application communities of agriculture, disasters, water resources, health and air quality, ecological conservation, wildland fires and capacity building. Analysis-ready NRT data products in a variety of formats have been mentioned many times in the collected feedback, especially for applied users with little to no experience using research-grade earth observation products. Users prefer to have products that can be easily integrated into their existing workflows and take their analysis to the data. HDF5 is a commonly used data format for research, but typically requires some conversion to a more friendly format for applications and regular use in decision-making. Users prefer the GeoTIFF data format that can be directly ingested into a GIS mapping software and platform for data analysis and visualization. For example, LANCE’s fire, flood, SO2 and Black Marble Nighttime Blue/Yellow Composite data products have been integrated into NASA Disasters Mapping Portal, which is an GIS-based open data portal, for users in the disaster management community. There are 291 LANCE NRT layers available through GIBS and Worldview, where users can download a snapshot in GeoTIFF format. Operational users expect data to be processed as close to the user as possible. The collected feedback indicates that LANCE fire products within 3 hours latency would meet the needs of the wildland fire community. The ideal latency for volcanic application is 10-15 minutes. Users in Volcanic Ash Advisory Centers (VAAC) reported that the first forecast volcanic product should be issued within 75 minutes from the volcano eruption [5]. Overall, for disaster applications, data latency within 3 hours is useful while latency greater than 12 hours is not timely enough for operational use. Capacity building and training are critical for users to be able to access, interpret and use data products and tools for their decision making, especially for applied users with limited experience using earth observation products. LANCE data products have been used in a number of capacity building projects domestically and internationally [6]. As LANCE continues to bring new products into the system, users request training to utilize LANCE new and upcoming data products and capabilities in their applications. Due to the limitation of bandwidth and downstream flow paths, users in some developing countries need tools to select and download data for a specific area of interest instead of bulk downloads. The collected feedback also shows the lack of available SAR satellite low latency data products. The advantages of SAR to monitor conditions and changes on the ground through darkness, clouds, volcanic ash, and other atmospheric conditions, are appealing to low latency users. For example, terabytes of low latency but cloudy optical images are not helpful in rapidly identifying the extent of flood or fire impacts. LANCE could be complemented with low latency measurements via the upcoming NASA-ISRO Synthetic Aperture Radar (NISAR) mission [7]. Requests for higher spatial resolution products are expressed. A user from the wildland fire management community reported that products with 30-m spatial resolution could be used to detect small fires. The 30-m Landsat OLI fire data is now part of NASA’s Fire Information for Resource Management System (FIRMS) US/Canada [8]. Within the open and free NASA resources, LANCE disseminates NRT data products in a manner that allows them to be accessible and understandable to both scientific and applied users. In many application areas, latency plays an important or even decisive role where low latency earth observations help people to observe areas of interest, detect and track changes in the environment and make timely decisions. NASA’s Earth Applied Sciences Program promotes the use of LANCE NRT products and builds a bridge between application users and research teams. The collected feedback indicates data latency within 3 hours is useful for most of the applications, and shows the needs of user-friendly, analysis-ready products, and requests training on LANCE’s new and upcoming data products. User feedback has been provided to LANCE UWG for guidance and recommendations, and for translating findings into something actionable.

Tian Yao

2022 Power Global Community (Gloco) Summit Summary Report

The Prediction Of Worldwide Energy Resources (POWER) Project 25th year of providing global solar and meteorology data products to the world. The goals of this event were to inform users of new, enhanced features, gather feedback on services, identify new requests and requirements, and capture more thorough user stories while identifying new user communities and partnership opportunities. The event was “virtual” and had over 580 registrants. Over the two-day event, the Summit was attended by 162 unique attendees representing 24 countries.

NASA

Treating gridded geospatial data as point data to simplify analytics

Gridded geospatial remote sensing (satellite) data has traditionally been stored in file-based multidimensional arrays to preserve the locality of data. Measurements from locations that are physically next to each other on earth remain next to each other in the arrays. Maintaining this locality is useful when running calculations like reprojection, but unnecessary for many other calculations. This talk will go through a real world example of a tool redesign at the Goddard Earth Sciences Data and Information Services Center (GES DISC), showing the advantages of using the data frame model for calculating summary statistics, where measurement proximity is unimportant.

Analysis-ready data

NASA Power: Global Solar Insolation, Meteorological Parameter Data, and Web Services to Support Sustainable Building Design and Operations

The buildings industry is currently striving to adopt green solutions to make infrastructure more energy-efficient in order to meet the 2050 net-zero climate goals. This planning requires reliable environmental datasets that are crucial in designing, building, and maintaining our world’s-built environment, as well as other energy-related processes and investments. This webinar for the National Institute of Building Sciences provides an overview of NASA’s Prediction Of Worldwide Energy Resources (POWER) Project that informs decision-making and development for sustainable building design and operations by enabling public open discovery, efficient access, and convenient distribution of NASA’s Earth Observations and global atmospheric model datasets. POWER’s datastore is comprised of solar radiation and surface meteorology parameters, spanning nearly 40 years of hourly data, that are easily accessible via several access methods and tools to support three focus areas: 1) renewable energy deployment and management, 2) sustainable infrastructure, and 3) agroclimatology applications. POWER and NASA Earth Science both plan future data parameters, updated tools, and improved observations that could directly support U.S. and international sustainable development goals, climate strategies, and building information modeling. To this end, solar data from several NASA projects and meteorological data from NASA assimilation models have already been reformatted and disseminated to the public via a user-friendly web GIS-enabled based data portal through the POWER Project. POWER data is analysis-ready and accessible through an Application Programming Interface (API), ArcGIS Image Services, and the project’s Data Access Viewer enhanced (DAVe), an interactive online tool. The POWER DAVe also features data consistent with ASHRAE Climate Design Conditions and has developed web image services showing Building Climate Zones and their variability. Through those tools, the data can be downloaded into multiple formats that support the infrastructure community, including CSV and Energy Plus Weather (EPW). POWER’s entire data product catalog is available through Amazon Web Services (AWS) Open Data Registry (ODR) via a free and publicly accessible Simple Storage Service (S3). This webinar provides a full overview of the NASA POWER Project's data and services developed in collaboration with the sustainable infrastructure community. Examples of how the renewable energy and building communities have utilized POWER data products to make decisions and a preview of future data product expansion, including climate projections, and web services will also be provided. Additionally, use case stories from our broad community of users will be presented.

Paul W. Stackhouse

Processing Satellite Images on Tertiary Storage: A Study of the Impact of Tile Size on Performance

Before raw data from a satellite can be used by an Earth scientist, it must first undergo a number of processing steps including basic processing, cleansing, and geo-registration. Processing actually expands the volume of data collected by a factor of 2 or 3 and the original data is never deleted. Thus processing and storage requirements can exceed 2 terrabytes/day. Once processed data is ready for analysis, a series of algorithms (typically developed by the Earth scientists) is applied to a large number of images in a data set. The focus of this paper is how best to handle such images stored on tape using the following assumptions: (1) all images of interest to a scientist are stored on a single tape, (2) images are accessed and processed in the order that they are stored on tape, and (3) the analysis requires access to only a portion of each image and not the entire image.

Yu, JieBing

Chaos: Understanding and Controlling Laser Instability

In order to characterize the behavior of tunable diode lasers (TDL), the first step in the project involved the redesign of the TDL system here at the University of Tennessee Molecular Systems Laboratory (UTMSL). Having made these changes it was next necessary to optimize the new optical system. This involved the fine adjustments to the optical components, particularly in the monochromator, to minimize the aberrations of coma and astigmatism and to assure that the energy from the beam is focused properly on the detector element. The next step involved the taking of preliminary data. We were then ready for the analysis of the preliminary data. This required the development of computer programs that use mathematical techniques to look for signatures of chaos. Commercial programs were also employed. We discovered some indication of high dimensional chaos, but were hampered by the low sample rate of 200 KSPS (kilosamples/sec) and even more by our sample size of 1024 (1K) data points. These limitations were expected and we added a high speed data acquisition board. We incorporated into the system a computer with a 40 MSPS (million samples/sec) data acquisition board. This board can also capture 64K of data points so that were then able to perform the more accurate tests for chaos. The results were dramatic and compelling, we had demonstrated that the lead salt diode laser had a chaotic frequency output. Having identified the chaotic character in our TDL data, we proceeded to stage two as outlined in our original proposal. This required the use of an Occasional Proportional Feedback (OPF) controller to facilitate the control and stabilization of the TDL system output. The controller was designed and fabricated at GSFC and debugged in our laboratories. After some trial and error efforts, we achieved chaos control of the frequency emissions of the laser. The two publications appended to this introduction detail the entire project and its results.

Blass, William E.

An Overview of the Mock LISA Data Challenges

The LISA International Science Team Working Group on Data Analysis (LIST-WG1B) is sponsoring several rounds of mock data challenges, with the purpose of fostering the development of LISA data-analysis capabilities, and of demonstrating technical readiness for the maximum science exploitation of the LISA data. The first round of challenge data sets were released at this Symposium. We describe the objectives, structure, and timeline of this program.

black holes

How Can NASA Science Benefit Solar Energy Development and Assessment?

This is a keynote presentation for a workshop highlighting the use of remote sensing and modeling products derived with NASA missions and Earth system modeling for solar energy system development. We broadly introduce NASA's fleet of satellite producing publicly available data products that would be useful for the information needed solar energy development. We also introduce the work in Earth system science modeling that provides additional global gridded data products that are also relevant. We identify that one issue is obtaining and using this data is the level of expertise needed to find it. NASA has developed a new Earth Action program to address this gap and bring the data closer to the user. Now under the Energy Resources element, the NASA LaRC's POWER (Prediction of Worldwide Renewable Energy) project has developed a web services platform that provides key information needed for solar energy development as analysis-ready. This means that the data for particular locations are immediately useful for help provide the information needed by engineers and architects to plan for solar systems. Thus, the presentation shows why NASA products would be relevant and then shows examples of finding the needed data through the POWER web portal.

remote sensing

SatCORPS Global Cloud Composite (GCC): the Design and Delivery of A High Quality, High Resolution, Global Cloud Product Available in Near-Real Time

The NASA Satellite ClOud and Radiation Property retrieval System (SatCORPS) supports the development of an analysis ready and cloud-optimized data transformation pipeline and geospatial service enablement of a global cloud composite (GCC) product derived from global geostationary satellite imagery. This geospatial service will be available at high temporal and spatial resolution via the SatCORPS web mapping application for visualization and analysis as well as direct ingestion to common geospatial software and custom programming. The resulting global cloud composite products from the processing pipeline can then be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. Near real time global observations are created through the composition of five geostationary satellites that provides modelling and forecasting communities with the capability to provide high quality and timely information to start the projection process. The Global Cloud Composite product combines information from geostationary satellites, GOES-16, GOES-17, Himawari-8, Meteosat-11 and Meteosat-9 to create a single global composite netcdf file and images using the different products within the netcdf file. The SatCORPS team, though our Global Cloud Composite (GCC) product and web-based visualization tools including Geographical Information System (GIS) services provide near real time global cloud product information to both automated processes and traditional web users that is timely and high quality derived from geostationary satellites. The Global Cloud Composite product takes advantage of the scalable processing resources provided by the AWS batch service to provide new composites every thirty minutes. Because information from each of the low earth orbiting satellites is available on schedules tuned to the specific satellite, the processing algorithm temporally composites the final dataset as each satellite’s information becomes available. The SatCORPS team has leveraged our experience using Amazon Web Services (AWS) to build a low latency high availability tool that allows end users both human and automated to acquire high quality and high-resolution Geostationary Earth Orbiting (GEO) information at zero cost to the end user. This presentation will describe how we architected and implemented the service as well as lessons learned based on our experiences both developing and operating the system. The lessons learned include how we integrated multiple services including Amazon Batch, Amazon S3 and Amazon Lambda service to create a low cost but high-performance processing system that is capable of identifying and processing the most appropriate satellite overpass information into global cloud composites. We will also describe our web-based tools including our Geographic Information System that can be used for visualization and analysis. The products from the processing can be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. The SatCORPS Global Composite Cloud product provides sophisticated global composited cloud research products with very low latency that we see that as filling a rapidly growing need in the research and modelling community with no up-front nor ongoing costs associated with downloading or using the information.

AWS AMCE SMCE GCC SATCORPS GLOBAL CLOUD COMPOSITE