Search NASA⌕ Search

SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Open Source Application of Fusing Aerosol Products from GEO and LEO Satellites

Retrieving aerosol optical depths (AODs) from sun-synchronous polar orbiting (aka low earth orbit, LEO) satellites, such as MODISs, and VIIRSs, OMI, TROPOMI, etc, has become well-established as a tool for extracting information on particulate matter (PM) and related processes in the atmosphere. However, with recently launched geostationary satellites (GEO), such as GOES-16/17/18, and Himawari-8/9, and Meteosat Third Generation (MTG) they provide a much higher temporal resolution (order of 10 minutes), typically an image once or more per hour during daylight compared to LEO once per day. By combining these observations, we may be able to characterize the diurnal cycle of global AOD at the local, regional and global scale. While the science community is still exploring the new data from GEO observations, we have been thinking about how to properly combine/merge/fuse those data considering differences in their spatial and temporal resolutions. However, this poses a “Big Data” challenge. The big data challenge is not just about data storage, but also about data discoverability, and accessibility, and even more, about data migration/mirroring in the cloud-computing environment. This paper is merely showing some of the efforts and approaches we have attempted in fusing six satellites’ Level 2 aerosol data (three are from GEO (GOES-16/17 and Himawari-8), and the other three are from LEO (TERRA/MODIS, AQUA/MODIS, SNPP-VIIRS) from Dark Target (DT) aerosol retrieval algorithm. Having the on-demand capability of fusing remote sensing products onto the desired temporal and spatial domain enables researchers and application practitioners to better manipulate and work with satellite and sensor data. It is our hopeWe hope that by making such an open-source package, and the accompanying functionality, the scientific community will be granted easier access to aerosol data processing resources. The MEaSUREs Program (Making Earth System Data Records for Use in Research Environments) expands our understanding of the Earth's current system through atmospheric and surface measurements. In an effort to aid the scientific research component and improve open source methods, this project developed Python code for fusing six satellite Level 2 aerosol data (three are from geostationary satellites (GEO), and the other three are from low earth orbital satellites (LEO)) from Dark Target Aerosol Retrieval Algorithm.

Jennifer Wei↗
NASA NTRS2016

Addressing the Big-Earth-Data Variety Challenge with the Hierarchical Triangular Mesh

We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. We observe that, in the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. In addition, since HTM is also an indexing scheme, when it is used to index all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. Moreover, our updated HTM encoding turns most geospatial set operations into integer interval operations, gaining further performance advantages.

SciDB↗
NASA NTRSDate not supplied

NASA’s Atmospheric Science Data Center’s Approach to a Cloud-Based Model of Ingest, Archival, and Distribution of TEMPO Data: Methods, Challenges, and Best Practices

The National Aeronautics and Space Administration's (NASA) Atmospheric Science Data Center (ASDC) at NASA Langley Research Center in Hampton, VA provides atmospheric science data products and services to the science community, including enhanced search and subsetting capabilities for numerous datasets. The ASDC is the official Distributed Active Archive Center (DAAC) of record for the upcoming Tropospheric Emissions: Monitoring of Pollution (TEMPO) instrument. TEMPO will be situated on a geostationary satellite positioned at a longitude near the center of the conterminous United States and focused on North America, making hourly swaths of its field of regard from east to west. ASDC’s data products are currently hosted locally and services (e.g., spatial and temporal subsetting) are managed on premises. The ASDC is planning to provide TEMPO data and services in the cloud through the Earthdata Search platform. This presentation will discuss the ASDC’s approach to a cloud-based model of ingest, archival, and distribution of TEMPO data. Methods, challenges, best practices, lessons learned, and future plans will be discussed.

Iman Nasif↗
NASA NTRSDate not supplied

Data Integrity Challenges in NASA Giovanni

The Geospatial Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni) is an online tool developed by the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC), one of 12 NASA Science Mission Directorate Data Centers (DAACs) to analyze and visualize NASA remote sensing and model data without downloading data and software. As of this writing, over 2000 Earth satellite and model variables are available in Giovanni, including several well-known NASA satellite missions (e.g., TRMM, GPM) and projects (e.g., MERRA-2, GPCP). There are twenty-two plots provided by Giovanni that can be used to analyze, compare, and explore Earth data across disciplines. Results can be shared with colleagues and downloaded for further analysis. Giovanni has helped publish over 3000 referral papers over the years. As open science policies roll in, data integrity has become a major challenge for Giovanni and other tools. For integrity, both data and workflows must be transparent. FAIR-compliant data, including input, intermediate, and result products, as well as their associated statistics, metadata, and information, are needed. The NASA Data Product Development Guide for Data Producers provides a key resource on how to develop FAIR-compliant data products. Data quality information is also needed from data producers and analysis services like Giovanni. The workflow part is quite challenging and requires workflow management improvements, such as recording workflows and making them available to users. In this presentation, we will discuss the data integrity challenges in Giovanni.

data analysis, visualization↗
NASA NTRSDate not supplied

Data Science Challenges for Urban Air Mobility

Aviation is a combination of aircraft, airspace and airports. The data science life cycle comprises of five steps - capture, maintain, process, analyze and communicate. The presentation introduces the legacy of conventional aviation research in the context of the data science life cycle to motivate the challenges with Urban Air Mobility, a field that is quite nascent. A summary of recent research will be presented to highlight the innovative ways to address the challenges. Examples provided will include the generation of synthetic data, encounter models from simulations, and leveraging novel and diverse data sets from traditional transportation and non-aviation sources, to analyze problems of operation in urban airspace. Finally, opportunities will be identified for further exploration, niche development and filling the gaps in the field of data science for UAM.

Data Science↗
NASA NTRS2020

Data Science Challenges for Urban Air Mobility

Aviation is a combination of aircraft, airspace and airports. The data science life cycle comprises of five steps - capture, maintain, process, analyze and communicate. The presentation introduces the legacy of conventional aviation research in the context of the data science life cycle to motivate the challenges with Urban Air Mobility, a field that is quite nascent. A summary of recent research will be presented to highlight the innovative ways to address the challenges. Examples provided will include the generation of synthetic data, encounter models from simulations, and leveraging novel and diverse data sets from traditional transportation and non-aviation sources, to analyze problems of operation in urban airspace. Finally, opportunities will be identified for further exploration, niche development and filling the gaps in the field of data science for UAM.

Data Science↗
NASA NTRS2007

Architecture and Data Management Challenges in GEOSS and IEOS

The international Group on Earth Observations (GEO) was initiated in 2003 to engage all the nations of the Earth in building a coordinated, comprehensive, and sustained Earth observation capability, known as the Global Earth Observation System (GEOSS). The GEO website describes GEOSS this way: "GEOSS will build on and add value to existing Earth-observation systems by coordinating their efforts, addressing critical gaps, supporting their interoperability, sharing information, reaching a common understanding of user requirements, and improving delivery of information to users." Each member nation has responded to GEO by establishing some sort of coordinating body; within the United States, that is the United States Group on Earth Observations (USGEO). This paper will describe the establishment of GEO and USGEO, will provide an overview of the activities and challenges in the area of architecture and data management, and will highlight some of the major efforts underway within USGEO today.

Fontaine, Kathleen S.↗
NASA NTRSDate not supplied

High Temperature Material Property Data and Challenges to Thermal Process Model Predictions and In-Situ/Ex-Situ Measurements for Metallic Additive Manufacturing

Understanding and predicting performance properties of parts produced by metallic additive manufacturing has improved significantly over the past decade; however, difficult to measure material properties and process outcomes continue to be challenges. The qualification or certification of aerospace parts require extensive measures to quantify variable part properties in order to buy down the risk of component failure. The variability, inherent to the additive manufacturing, process adds unwanted uncertainty in the production of load critical structural components. Process modeling has proven valuable in providing predictions and context for understanding outcomes of the additive manufacturing process; however, these physically informed process models require material properties at temperatures that are difficult to measure and rarely available. Further, calibrating or validating such models is difficult because the process itself is challenging to measure. This talk will explore some of the challenges resulting from difficult to acquire input data by relating thermal process model predictions to in-situ and ex-situ optical microscopy measurements.

Process Model↗
NASA NTRS2019

Data Analysis Challenges for Multi-Messenger Astrophysics

Recent multi-messenger observations of gravitational-wave and high-energy neutrino sources together with electromagnetic signatures have opened new ways of observing the Universe. These promise a future in which physics and astronomy will be advanced by combining observations and data from across the electromagnetic spectrum with gravitational waves and neutrinos. We consider the challenges the field is facing in fully utilizing data for multi-messenger astrophysics. Such data come from heterogeneous detector networks and standards, and their analysis is often time-critical to guide further observations. In this area, science capabilities depend on the interplay among observation, theory and computational/modeling work. Advances in data science and computing present additional opportunities and considerations in analyzing such data. We invited ADASS participants to a Birds of a Feather session to engage in discussion on the challenges and opportunities in data analysis for multimessenger astrophysics.

Peter S Shawhan↗
NASA NTRS2005

Response to MRO's end-to-end data accountability challenges

(MRO) on August 12, 2005. It carries six science instruments and three engineering payloads. Because MRO will produce an unprecedented number of science products, it will transmit a much higher data volume via high data rate than any other deep space mission to date. Keeping track of MRO products as well as relay products would be a daunting, expensive task without a well-planned data-product tracking strategy. To respond to this challenge, the MRO project developed the End-to- End Data Accountability System by utilizing existing information available from both ground and flight elements. Therefore, a capability to perform first-order problem diagnosis is essential in order for MRO to answer the questions, where is my data? and when will my data be available? This paper details the approaches taken, design and implementation of the tools, procedures and teams that track data products from the time they are predicted until they arrive in the hands of the end users.

data delivery status tracking↗
NASA NTRS1988

Lateral stability and control derivatives extracted from space shuttle Challenger flight data

Flight data taken from six flights of the Space Transportation System shuttle Challenger (STS-6, 7, 8, 11, 13 and 17) during atmospheric entry are analyzed to determine the shuttle lateral aerodynamic characteristics. Maximum likelihood estimation is applied to data derived from accelerometer and rate gyro measurements and trajectory, meteorological and control surface data to estimate lateral-directional stability and control derivatives. The vehicle stability and control surface effectiveness are compared across the flights and to preflight predicted values.

Schiess, James R.↗
NASA NTRS2001

Constituent Data Assimilation: Challenges and Limitations

The assimilation of observations of atmospheric constituents naturally divides into two major pieces. The first is the assimilation of trace gases whose variability is related to atmospheric motions. The second is the assimilation of trace gases which are sharply influenced by chemical exchange between different constituents. In order to advance beyond the initial successes of explorative investigation of assimilation techniques, tremendous challenges must be met to improve the geophysical integrity of assimilated data products. A subject of special interest is ozone near the tropopause. At the tropopause the information from both the observations and the model simulation becomes most uncertain. However a number of important geophysical parameters, e.g. stratosphere-troposphere exchange and tropospheric ozone, require the assimilation to have high accuracy at the tropopause. This talk will review the current status of the quality of assimilated data products near the tropopause, what must be done to improve the assimilation near the tropopause, and the intrinsic limitations that will require additional sources of information in order for the field to advance.

Rood, Richard B.↗
NASA NTRS2006

Scaling Equations for Ballistic Modeling of Solid Rocket Motor Case Breach

This paper explores the development of a series of scaling equations that can take a known nominal motor performance and scale it for small and growing case failures. This model was developed for the Malfunction-Turn Study as part of Return to Flight activities for the Space Shuttle program. To verify the model, data from the Challenger accident (STS- 51L) were used. The model is able to predict the motor performance beyond the last recorded Challenger data and show how the failed right hand booster would have performed if the vehicle had remained intact.

McMillin, Joshua E.↗
NASA NTRS2019

A Web of Data Analytics Services

Cloud Computing has become the ubiquitous approach to our Big Data challenge. However, one will quickly discover that moving (a.k.a. forklifting) existing on-premise data analytics solutions to the Cloud doesn’t always translate to costing saving and performance boost. The Cloud’s elasticity, its availability, and its wide selection of computing options and selections of costing models making Cloud an attractive environment to tackle our Big Data challenge. The fact is Cloud, on its own, is not the silver bullet to our daunting challenge need for analyze and derive scientific inferences through vast collections of multi-sensor measurements. We would like to have all scientific data in one easy to access environment, but getting the world of scientific data in one analytic system is immensely difficult to achieve. This paper describes the data analytics web architecture NASA is developing by infusing instances of Integrated Data Analytics systems next to the data. The goal is to minimize unnecessary data movement through collection of data access and analytics webservices for researchers to interact with and analyze measurements without have to download data to their local computer. These services are RESTful and provisioned by the data centers with the help from subject matter and science experts. These services encapsulate the physical computing infrastructure, which could local computing cluster, on-premise or public Cloud environment.

Huang, Thomas↗