Search NASA⌕ Search

SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
NASA NTRS2011

X-Ray Polarimetry with GEMS

The polarization properties of cosmic X-ray sources are still largely unexplored. The Gravity and Extreme Magnetism SMEX (GEMS) will carry out the first sensitive X-ray polarization survey of a wide range of sources including; accreting compact objects (black holes and neutron stars), AGN, supernova remnants, magnetars and rotation-powered pulsars. GEMS employs grazing-incidence foil mirrors and novel time-projection chamber (TPC) polarimeters leveraging the photoelectric effect to achieve high polarization sensitivity in the 2 - 10 keV band. I will provide an update of the project status, illustrate the expected performance with several science examples, and provide a brief overview of the data analysis challenges

Strohmayer, Tod↗
NASA NTRS2011

Data Analysis And Polarization Measurements With GEMS

The Gravity and Extreme Magnetism SMEX (GEMS) mission was selected by NASA for flight in 2014. GEMS will make the first sensitive survey of X-ray polarization across a wide range of source classes including black hole and neutron star binaries, AGN of different types, rotation and accretion-powered pulsars, magnetars, shell supernova remnants and pulsar wind nebulae. GEMS employs grazing-incidence foil mirrors and novel time-projection chamber (TPC) polarimeters leveraging the photoelectric effect. The GEMS detectors image the charge tracks of photoelectrons produced by 2 - 10 keV X-rays. The initial direction of the photoelectron is determined by the linear polarization of the photon. We present an overview of the data analysis challenges and methods for GEMS, including procedures for producing optimally filtered images of the charge tracks and estimating their initial directions. We illustrate our methods using laboratory measurements of polarized and unpolarized X-rays with flight-like detectors as well as from simulated tracks. We also present detailed simulations exploring the statistics of polarization measurements appropriate for GEMS, and make comparisons with previous work.

Stohmayer, Tod↗
NASA NTRS2014

Analysis of Aviation Safety Reporting System Incident Data Associated with the Technical Challenges of the Atmospheric Environment Safety Technology Project

This study analyzed aircraft incidents in the NASA Aviation Safety Reporting System (ASRS) that apply to two of the three technical challenges (TCs) in NASA's Aviation Safety Program's Atmospheric Environment Safety Technology Project. The aircraft incidents are related to airframe icing and atmospheric hazards TCs. The study reviewed incidents that listed their primary problem as weather or environment-nonweather between 1994 and 2011 for aircraft defined by Federal Aviation Regulations (FAR) Parts 121, 135, and 91. The study investigated the phases of flight, a variety of anomalies, flight conditions, and incidents by FAR part, along with other categories. The first part of the analysis focused on airframe-icing-related incidents and found 275 incidents out of 3526 weather-related incidents over the 18-yr period. The second portion of the study focused on atmospheric hazards and found 4647 incidents over the same time period. Atmospheric hazards-related incidents included a range of conditions from clear air turbulence and wake vortex, to controlled flight toward terrain, ground encounters, and incursions.

Withrow, Colleen A.↗
NASA NTRS2016

A Global Capacity Building Vision for Societal Applications of Earth Observing Systems and Data: Key Questions and Recommendations

Capacity building using Earth observing (EO) systems and data (i.e., from orbital and nonorbital platforms) to enable societal applications includes the network of human, nonhuman, technical, nontechnical, hardware, and software dimensions that are necessary to successfully cross the valley [of death; see NRC (2001)] between science and research (port of departure) and societal application (port of arrival). In many parts of the world (especially where ground-based measurements are scarce or insufficient), applications of EO data still struggle for longevity or continuity for a variety of reasons, foremost among them being the lack of resilient capacity. An organization is said to have resilient capacity when it can retain and continue to build capacity in the face of unexpected shocks or stresses. Stresses can include intermittent power and limited Internet bandwidth, constant need for education on ever-increasing complexity of EO systems and data, communication challenges between the ports of departure and arrival (especially across time zones), and financial limitations and instability. Shocks may also include extreme events such as disasters and losing key staff with technical and institutional knowledge.

shocks↗
NASA NTRS2019

Automatic Anomaly Detection with Machine Learning

This presentation will discuss the following topics: Understanding the benefits of automatic anomaly detection Identifying precursors with algorithms. Overcoming the challenges of data issues in automation.

Data Science↗
NASA NTRS2021

Assembly of the IXPE Mirror Modules

Expected to launch in December 2021, the Imaging X-ray Polarimetry Explorer (IXPE) is a NASA Astrophysics Small Explorer Mission with significant contributions from the Italian space agency (ASI). Three identical x-ray telescopes combine to form the IXPE Observatory. Each is comprised of a 4-m-focal length mirror module assembly (MMA, provided by Marshall Space Flight Center) that focuses x-rays onto a polarization-sensitive, imaging detector (contributed by ASI-funded institutions). This paper describes the now-completed assembly process for the 3 flight and one spare mirror modules, and compares as-tested calibrated performance with as-built metrology data. Unexpected challenges and lessons-learned are also discussed.

X-ray optics IXPE polarimetry↗
NASA NTRS2022

Assessment of Retrieved GMI Emissivity Over Land, Snow, and Sea Ice in the GEOS System

Directly assimilating microwave radiances over land, snow and sea ice remains a significant challenge for data assimilation systems. These data assimilation systems are critical to the success of global numerical weather prediction systems including the Global Earth Observing System-Atmospheric Data Assimilation System (GEOS-ADAS). Extending more surface sensitive microwave channels over land, snow and ice could provide a needed source of data for Numerical Weather Prediction particularly in the Planetary Boundary Layer (PBL). Unfortunately, the accuracy of emissivity models currently available within the GEOS-ADAS along with other data assimilation systems are insufficient to simulate and assimilate radiances. Recently, Munchak et al. (2020) published a 5-year climatological database for retrieved microwave emissivity from the GPM Microwave Imager (GMI) aboard the Global Precipitation Mission (GPM). In this work the database is utilized by modifying the GEOS-ADAS to use this emissivity database in place of the default emissivity value available in the Community Radiative Transfer Model (CRTM), which is the fast radiative transfer model used by the GEOS-ADAS. As a first step, the GEOS-ADAS is run in a so-called “stand-alone” mode to simulate radiances from GMI using the default CRTM emissivity, and replacing the default CRTM emissivity models with values from Munchak et al, 2020. The simulated observations using Munchak et al., 2020 agree more closely with observations from GMI. These results are presented along with a discussion of the implication for GMI observations within the GEOS-ADAS.

GPM↗
NASA NTRSDate not supplied

Observation impacts in the lower troposphere and challenges of Planetary Boundary Layer data assimilation

The Goddard Earth Observing System (GEOS) developed by the NASA Global Modeling and Assimilation Office assimilates a wide range of observations to support various NASA Earth Science missions. To set the stage for follow-on Planetary Boundary Layer (PBL) science and prepare for future observing systems of the next decade, we have assessed the effectiveness of the use of existing observing systems in the lower troposphere in GEOS, and analyzed model responses to the incremental analysis update (IAU) forcing. With a better understanding of the GEOS data assimilation algorithms in the PBL, we have developed strategies for improved PBL data assimilation in GEOS. The strategies to enhance data usages in both the data assimilation system and forecast model will be presented, and the utilization of PBL height data from multiple observing systems will be discussed as well.

Yanqiu Zhu↗
DOE OSTI2024

Exploring the Intersection of AI and Visualization in the Nuclear Industry

This presentation explores the impact of AI and visualization in advancing the nuclear industry by improving safety, operational efficiency, and decision-making processes. It highlights key applications such as real-time monitoring, predictive maintenance, and immersive training, while addressing challenges like data quality, regulatory hurdles, and the need for explainable AI. Additionally, the presentation outlines future directions, emphasizing the potential of AI-driven reactor design, advanced simulation tools, and ethical considerations to drive innovation and sustainability in nuclear operations.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2026

Machine Learning on Heterogeneous, Edge, and Quantum Hardware for Particle Physics (ML-HEQUPP)

The next generation of particle physics experiments will face a new era of challenges in data acquisition, due to unprecedented data rates and volumes along with extreme environments and operational constraints. Harnessing this data for scientific discovery demands real-time inference and decision-making, intelligent data reduction, and efficient processing architectures beyond current capabilities. Crucial to the success of this experimental paradigm are several emerging technologies, such as artificial intelligence and machine learning (AI/ML) and silicon microelectronics, and the advent of quantum algorithms and processing. Their intersection includes areas of research such as low-power and low-latency devices for edge computing, heterogeneous accelerator systems, reconfigurable hardware, novel codesign and synthesis strategies, readout for cryogenic or high-radiation environments, and analog computing. This white paper presents a community-driven vision to identify and prioritize research and development opportunities in hardware-based ML systems and corresponding physics applications, contributing towards a successful transition to the new data frontier of fundamental science.

Gonski, Julia [SLAC]↗
DOE OSTI2025

Capturing Historic Reliability Performance Through Graph Databases: A Model Based System Engineering Approach

With the goal of improving the performance and reliability of high dependable technological systems such as nuclear power plants, advanced monitoring and health management systems are employed to inform system engineers on observed degradation processes and anomalous behaviors of assets and components. This information is captured in the form of large amount of data which can be heterogenous in nature (e.g., numeric, textual). Such large data availability poses challenges when system engineers are required to parse and analyze them in order to track historic reliability performance of assets and components. This paper tackles directly this challenge by providing means to organize data in the form of a graph: a knowledge graph. The presented approach distinguish itself from current knowledge graph-based methods by the fact that model-based system engineering (MBSE) models are used to “put data into context”. In particular, MBSE models are used as skeleton of a knowledge graph; numeric and textual data elements, once processed, are associated to MBSE model elements. Thus, a knowledge graph captures both system architecture (though MBSE models) and health/performance data. Such feature opens the door to new data analytics methods designed to identify causal relations between observed phenomena.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A cost and community perspective on the barriers to microbiome data reuse

Microbiome research is becoming a mature field with a wealth of data amassed from diverse ecosystems, yet the ability to fully leverage multi-omics data for reuse remains challenging. To provide a view into researchers’ behavior and attitudes towards data reuse, we surveyed over 700 microbiome researchers to evaluate data sharing and reuse challenges. We found that many researchers are impeded by difficulties with metadata records, challenges with processing and bioinformatics, and problems with data repository submissions. We also explored the cost constraints of data reuse at each step of the data reuse process to better understand “pain points” and to provide a more quantitative perspective from sixteen active researchers. The bioinformatics and data processing step was estimated to be the most time consuming, which aligns with some of the most frequently reported challenges from the community survey. From these two approaches, we present evidence-based recommendations for how to address data sharing and reuse challenges with concrete actions for future work.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Data Science and Computation for Rapid and Dynamic Compression Experiment Workflows at Experimental Facilities, September 8-11, 2020. Workshop Report

The application of high pressure to materials has enabled discoveries in scientific fields such as planetary science, materials science, and materials synthesis. Recent advances in X-ray user light sources and other facilities, co-location and integration of user facilities with high-pressure drivers, availability of high-performance computing (HPC) platforms, and the development of new data science techniques have created opportunities for, and challenges in, advancing data analytics for rapid and dynamic compression experiments. To address these challenges, harness the emerging technology now available, and expedite scientific discovery, Los Alamos National Laboratory (LANL) hosted a virtual workshop entitled “Data Science and Computation for Rapid and Dynamic Compression Workflows at Experimental Facilities” from September 8 to 11, 2020. The workshop included 95 registered scientists and analytics experts from 15 universities, 9 United States (US) national laboratories, 5 US and European X-ray light sources, neutron sources such as the Los Alamos Neutron Science Center (LANSCE), other big science facilities such as the National Ignition Facility (NIF), and an industry representative. The workshop included 31 invited talks and 4 lightning talks by students and postdocs.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Pseudonymization at Scale: OLCF’s Summit Usage Data Case Study

The analysis of vast amounts of data and the processing of complex computational jobs have traditionally relied upon high performance computing (HPC) systems, which offer reliable and efficient management of large-scale computational and data resources. Understanding these analyses’ needs is paramount for designing solutions that can lead to better science, and similarly, understanding the characteristics of the user behavior on those systems is important for improving user experiences on HPC systems. A common approach to gathering data about user behavior is to extract workload characteristics from system log data available only to system administrators. Recently at Oak Ridge Leadership Computing Facility (OLCF), however, we unveiled user behavior about the Summit supercomputer by collecting data from a user’s point of view with ordinary Unix commands.In this paper, we discuss the process, challenges, and lessons learned while preparing this dataset for publication and submission to an open data challenge. The original dataset contains personal identifiable information (PII) about the users of OLCF which needed be masked prior to publication, and we determined that anonymization, which scrubs PII completely, destroyed too much of the structure of the data to be interesting for the data challenge. We instead chose to pseudonymize the dataset, which reduced the linkability of the dataset to the users’ identities. Pseudonymization is significantly more computationally expensive than anonymization, and the size of our dataset, which is approximately 175 million lines of raw text, necessitated the development of a parallelized workflow that could be reused on different HPC machines. We demonstrate the scaling behavior of the workflow on two leadership class HPC systems at OLCF, and we show that we were able to bring the overall makespan time from an impractical 20+ hours on a single node down to around 2 hours. As a result of this work, we release the entire pseudonymized dataset and make the workflows and source code publicly available.

Maheshwari, Ketan↗
NASA NTRS2020

Near 40 Years MERRA-2 Data at NASA GES DISC -Opportunity and Challenge to Support Extremes Study

To the end of 2019, 40 years NASA climate reanalysis data sets from the Modern Era Retrospective-analysis for Research and Applications, Version 2 (MERRA-2) will be available at NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). MERRA-2 consists atmosphere, land, and ocean data, which may be used for the studies ranging from the short scale weather events to the large scale decadal vulnerabilities. The hourly products, such as precipitation, soil moisture, temperature, and aerosols etc., have been used widely to study extreme events.In supporting users from broad communities, GES DISC have developed various data access services, including subsetter for downloading only data of interest with preferred format; OPeNDAP - for machine-to-machine data access; and Giovanni- for online visualization and analysis, etc. A big challenge for extreme study is to downloading and processing long-term hourly or daily data. The data downloading performance is not very satisfied by many users with current services and the native archived data structure. Late June 2019, many people in Europe had experienced extreme heat waves. The temperatures in several countries exceeded 40°C (104°F). For example, MERRA-2 shows that the near surface daily maximum temperature of June 28 2019 over Marseille, a city in southern France, reached 41.1 °C (106°F), which is the record breaking temperature in the last 40 years. GES DISC is working together with domain science experts to improve the performance of long time series access, making analysis ready data sets in supporting application researches, such as extreme study. In this presentation, using Europe heat wave as an example, we will show prototype of the in developing service for finding extremes from near 40 years MERRA-2 data at a given location. MERRA-2 data can be accessed from NASA GES DISC(https://disc.gsfc.nasa.gov/ ) by search keyword "MERRA-2".

Shen, Suhung↗
DOE OSTI2020

A New Capability of E4D For 3D Parallel Joint Inversion of DC Resistivity And Traveltime Data on Unstructured Mesh

A major challenge in interpreting geophysical data is how to derive consistent three-dimensional (3D) earth models of different physical properties from spatially and temporally limited measurements. Joint inversion with cross-gradient constraints is an approach to find such models by imposing structural similarities between different physical parameters. We have developed a parallel distributed-memory joint inversion code for direct-current (DC) resistivity and traveltime data using the cross-gradient constraint on unstructured mesh. The code utilizes existing E4D framework for parallel forward simulation, distributed storage and computation of the Jacobian matrix of forward operator, and parallel execution of matrix-vector multiplication during inversion. Besides, the joint inversion is solved by nonlinear conjugate gradient algorithm parallelized for DC resistivity and traveltime data. The joint inversion capability of E4D was tested using synthetic data from cross-borehole DC resistivity and traveltime data. The results indicate that the shape and size of the anomalies from the joint inversion are more reliable than those from separate inversions.

58 GEOSCIENCES↗
NASA NTRS2019

Past and Future Operations Concepts of NASA's Earth Science Data and Information System

NASA committed to support the collection and distribution of Earth science data to study global change in the 1990's. A series of Earth science remote sensing satellites, the Earth Observing System (EOS), was to be the centerpiece. The concept for the science data system, the EOS Data and Information System (EOSDIS), created new challenges in the data processing of multiple satellite instrument observations for climate research and in the distribution of global-coverage remote sensor products to a large and growing science research community. EOSDIS was conceived to facilitate easy access to EOS science data for a wide heterogeneous national and international community of users. EOSDIS was to provide a spectrum of services designed for research scientists working on NASA focus areas but open to the general public and international science community. EOSDIS would give researchers tools and assistance in searching, selecting and acquiring data, allowing them to focus on Earth science climate research rather than complex product generation. Goals were to promote exchange of data and research results and expedite development of new geophysical algorithms. The system architecture had to accommodate a diversity of data types, data acquisition and product generation operations, data access requirements and different centers of science discipline expertise. Steps were taken early to make EOSDIS flexible by distributing responsibility for basic services. Many of the system operations concept decisions made in the 90s continued to this day. Once implemented, concepts such as the EOSDIS data model played a critical role developing effective data services, now a hallmark of EOSDIS. In other cases, EOSDIS architecture has evolved to enable more efficient operations, taking advantage of new technology and thereby shifting more resources on data services and less on operating and maintaining infrastructure. In looking to the future, EOSDIS may be able to take advantage of commercial compute environments for infrastructure and further enable large scale climate research. In this presentation, we will discuss key EOSDIS operations concepts from the 1990's, how they were implemented and evolved in the architecture, and look at concepts and architectural challenges for EOSDIS operations utilizing commercial cloud services.

Moses, John F.↗