Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

Data-Driven Buy Clean: Decarbonization and Beyond

This report was compiled to provide recommendations on the availability of public background data from the U.S. Federal life cycle assessment (LCA) Data Commons to be conformant with the Association for Life Cycle Assessment (ACLCA) 2022 Product Category Rule (PCR) Open Standard to build technical tools that can assist industry in creating more comparable Type II Environmental Product Declarations (EPDs) for Federal Buy Clean and sustainability initiatives. The Federal LCA Commons is not only a public data source but also a consistently structured, self-referencing mega-repository for data developed by federal agency experts (in agency repositories) and by academia, nonprofit organizations, and industry (via the US Life Cycle Inventory Database). The Federal LCA Commons Technical Working Group is continuously improving the standardization of data documentation, formatting, and nomenclature to ensure lossless data loading and accurate data representation. This report and appendixes include the following: 1) An introduction to data-driven Buy Clean and decarbonization initiatives at the federal level; 2) The current status and associated challenges with LCA data and EPD standards and comparability; 3) Opportunities for the Federal LCA Commons to support conformance with the ACLCA 2022 PCR Open Standard and provide resources to implement the Federal Sustainability Plan, Buy Clean Program, and Inflation Reduction Act (IRA) sustainability goals and objectives. To date, the Federal LCA Commons is the result of coordinated work by National Renewable Energy Laboratory (NREL), the U.S. Department of Agriculture (USDA), the Environmental Protection Agency (EPA), the National Energy Technology Laboratory (NETL), the Argonne National Laboratory (ANL), the U.S. Army Corps of Engineers (USACE), the Federal Highway Administration (FHWA), the U.S. Forest Service (USFS), the Federal Aviation Administration (FAA), the Department of Defense (DoD) and the National Institute of Standards and Technologies (NIST). The Federal LCA Commons will continue to combine databases from the collaborating agencies while remaining a public resource. There are several initiatives among the collaborating agencies to expand the Federal LCA Commons and dedicated federal funding and resources could accelerate and strengthen these initiatives.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Subsetting Satellite Level 2 Data at NASA GES DISC

Working with Level 2 satellite swath data can prove challenging for many users. The variety of metadata, high resolution, and orbit-based nature of such data makes it difficult to acquire and manage. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has developed a general-purpose Level 2 Subsetting service that facilitates access to Level 2 data products. The Level 2 Subsetter enables users to sub-sample swath and gridded data in a variety of ways: by location (in a lat/lon box, within a circle of given radius, or at a specific point), by time (via swath segments or collections of granules), by any nonspatial dimension, or by selecting specific variables. The Level 2 Subsetting service also allows users to specify output data configuration and file format. In this presentation, we will detail and demonstrate the capabilities of this new service.

Huwe, Paul↗

Design and Construction of a NASA Airborne and Field Investigation Inventory

NASA conducts airborne and field investigations that produce a wealth of valuable research data. Unfortunately, this data is often scattered across individual scientist hard drives or NASA Distributed Active Archive Centers and it can be difficult to locate and retrieve. Although satellite data has been successfully consolidated by tools such as EarthData Search, airborne and field investigation data present unique challenges stemming from the variability of temporal, spatial, platform, and instrument metadata. To address these difficulties with data retrieval and metadata variability, the Interagency Implementation and Concepts Team established an Airborne Data Management Group to improve airborne data search, understanding, access, and use. Surveys have been conducted of end users in order to build query lists that will drive the augmentation and standardization of existing metadata. Detailed metadata was then laboriously compiled from present and historic airborne and field investigations to build a database that will enable intelligent data search and retrieval. The inventory structure and function will be described and demonstrated. The purpose of this presentation is to bring awareness to this effort, to highlight and describe the issues and complications in development, and to increase user interest prior to public release in 2020.

Davis, Carson↗

Extreme digitization for ground-based cosmic microwave background experiments

The large size of the time-ordered data of cosmic microwave background experiments presents challenges for mission planning and data analysis. These issues are particularly significant for Antarctica- and space-based experiments, which depend on satellite links to transmit data. We explore the viability of reducing the time-ordered data to a few bit numbers to address these challenges. Unlike lossless compression, few bit digitization introduces additional noise into the data. We present a set of one, two, and three bit digitization schemes and measure the increase in noise in the cosmic microwave background temperature and polarization power spectra. The digitization noise is independent of angular scale and is well described as a constant percentage of the original detector noise. Three bit digitization increases the map noise level by <2 percent, while reducing the data volume by a factor of 10 relative to 32-bit floats. Extreme digitization is a promising strategy for upcoming experiments.

79 ASTRONOMY AND ASTROPHYSICS↗

High performance multivariate visual data exploration for extremely large data

One of the central challenges in modern science is the need to quickly derive knowledge and understanding from large, complex collections of data. We present a new approach that deals with this challenge by combining and extending techniques from high performance visual data analysis and scientific data management. This approach is demonstrated within the context of gaining insight from complex, time-varying datasets produced by a laser wakefield accelerator simulation. Our approach leverages histogram-based parallel coordinates for both visual information display as well as a vehicle for guiding a data mining operation. Data extraction and subsetting are implemented with state-of-the-art index/query technology. This approach, while applied here to accelerator science, is generally applicable to a broad set of science applications, and is implemented in a production-quality visual data analysis infrastructure. We conduct a detailed performance analysis and demonstrate good scalability on a distributed memory Cray XT4 system.

Rübel, O↗

A Spatiotemporal Indexing Approach for Efficient Processing of Big Array-Based Climate Data with MapReduce

Climate observations and model simulations are producing vast amounts of array-based spatiotemporal data. Efficient processing of these data is essential for assessing global challenges such as climate change, natural disasters, and diseases. This is challenging not only because of the large data volume, but also because of the intrinsic high-dimensional nature of geoscience data. To tackle this challenge, we propose a spatiotemporal indexing approach to efficiently manage and process big climate data with MapReduce in a highly scalable environment. Using this approach, big climate data are directly stored in a Hadoop Distributed File System in its original, native file format. A spatiotemporal index is built to bridge the logical array-based data model and the physical data layout, which enables fast data retrieval when performing spatiotemporal queries. Based on the index, a data-partitioning algorithm is applied to enable MapReduce to achieve high data locality, as well as balancing the workload. The proposed indexing approach is evaluated using the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications (MERRA) climate reanalysis dataset. The experimental results show that the index can significantly accelerate querying and processing (10 speedup compared to the baseline test using the same computing cluster), while keeping the index-to-data ratio small (0.0328). The applicability of the indexing approach is demonstrated by a climate anomaly detection deployed on a NASA Hadoop cluster. This approach is also able to support efficient processing of general array-based spatiotemporal data in various geoscience domains without special configuration on a Hadoop cluster.

big data↗

Automated Extraction of Energy Systems Information from Remotely Sensed Data: A Review and Analysis

We report high quality energy systems information is a crucial input to energy systems research, modeling, and decision-making. Unfortunately, actionable information about energy systems is often of limited availability, incomplete, or only accessible for a substantial fee or through a non-disclosure agreement. Recently, remotely sensed data (e.g., satellite imagery, aerial photography) have emerged as a potentially rich source of energy systems information. However, the use of these data is frequently challenged by its sheer volume and complexity, precluding manual analysis. Recent breakthroughs in machine learning have enabled automated and rapid extraction of useful information from remotely sensed data, facilitating large-scale acquisition of critical energy system variables. Here we present a systematic review of the literature on this emerging topic, providing an in-depth survey and review of papers published within the past two decades. We first taxonomize the existing literature into ten major areas, spanning the energy value chain. Within each research area, we distill and critically discuss major features that are relevant to energy researchers, including, for example, key challenges regarding the accessibility and reliability of the methods. We then synthesize our findings to identify limitations and trends in the literature as a whole, and discuss opportunities for innovation. These include the opportunity to extend the methods beyond electricity to broader energy systems and wider geographic areas; and the ability to expand the use of these methods in research and decision making as satellite data become cheaper and easier to access. We also find that there are persistent challenges: limited standardization and rigor of performance assessments; limited sharing of code, which would improve replicability; and a limited consideration of the ethics and privacy of data.

97 MATHEMATICS AND COMPUTING↗

Joint ESA-NASA Multi-Mission Algorithm and Analysis Platform (MAAP)

The scientific community is faced with a need for greatly improved data sharing, analysis, visualization and advanced collaboration based firmly on open science principles. Recent and upcoming launches of new satellite missions with more complex and voluminous data, as well as the ever more urgent need to better understand the global carbon budget and related ecological processes, provided the immediate rational for the ESA-NASA Multi-mission Algorithm and Analysis Platform (MAAP). This highly collaborative joint project of ESA and NASA established a framework between ESA and NASA to share data, science algorithms and compute resources in order to foster and accelerate scientific research conducted by ESA and NASA EO data users. Presented to the public in October 2021, the current version of MAAP provides a common cloud-based platform with computing capabilities co-located with the data, a collaborative coding and analysis environment, and a set of interoperable tools and algorithms developed to support the estimation and visualization of global above-ground biomass. Data from the Global Ecosystem Dynamics Investigation (GEDI) mission on the International Space Station and the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) have been instrumental in the first products of MAAP including the first comprehensive map of Boreal above-ground Biomass and a current Global Biomass Harmonization Activity, but the platform is also being specifically designed to support the forthcoming ESA Biomass mission and incorporate data from the upcoming NASA-ISRO SAR (NISAR) mission. While these missions and the corresponding research which includes airborne, field, and calibration/validation data collection and analyses, provide a wealth of data and information relating to global biomass estimation, they also present data storing, processing and sharing challenges. The NISAR mission alone will produce about 80TB/day. These large data volumes present a challenge that would otherwise place accessibility limits on the scientific community and impact scientific progress. Other challenges being addressed by MAAP include: 1) Enabling researchers to easily discover, process, visualize and analyze large volumes of data from both agencies; 2) Providing a wide variety of data in the same coordinate reference frame to enable comparison, analysis, data evaluation, and data generation; 3) Providing a version-controlled science algorithm development environment that supports tools, co-located data and processing resources; and 4) Addressing intellectual property and sharing challenges related to collaborative algorithm development and sharing of data and algorithms. MAAP products can be explored on the MAAP Dashboard at https://earthdata.nasa.gov/maap-biomass or the joint platform entrance at scimaap.net. MAAP also can be accessed through individual NASA (https://maap-project.org) and ESA (https://esa-maap.org/) landing pages.

cloud computing↗

Exploring the Utility-Privacy Trade-Off: Impacts of Semantic and Visit Types Ambiguities on Human Mobility Simulation

Humans are in perpetual movement, constantly traversing buildings, cities, waters, oceans, and countries. Mobility stands out as a major driving force shaping our modern societies. Capturing and explaining human behavior in a world of eight billion distinct mobility agendas is a complex challenge. With the rise of interconnected devices and platforms, such as smartphones, wearables, and point-of-interest data, largescale behavioral data has become more accessible, enabling rich insights into mobility patterns. However, the widespread availability of such data introduces significant ethical challenges. Detailed mobility data can inadvertently reveal sensitive personal information, including individuals' locations, habits, social interactions, and even political or religious affiliations. Beyond privacy breaches, the ethical implications of uncovering and potentially manipulating underlying behavioral patterns demand attention. Striking a balance between the utility of mobility models and the protection of individual privacy is therefore paramount. This paper explores the utility-privacy trade-offs in human mobility modeling, focusing on the impacts of introducing semantic and visit type ambiguities. By systematically examining how these ambiguities affect the fidelity of simulated trajectories and privacy risks, we provide a framework for evaluating ethical and privacy-conscious modeling practices. Our findings emphasize the need for methods that safeguard privacy without undermining the usefulness of mobility models, contributing to the responsible advancement of mobility science in alignment with ethical standards and societal expectations.

Amichi, Licia [ORNL] (ORCID:0000000177631394)↗

Cyber Security Concerns for Sharing Distributed Antenna Systems

Wireless access to data is a challenge for environments with poor radio frequency coverage such as nuclear power plants. An 802.11 Wi-Fi data network is often not a cost-effective solution not a cost-effective solution due to the requirement for extensive antenna outlays in large nuclear facilities and poor signal penetration due to thick concrete walls. Combining commercial Long-Term Evolution (LTE) with an existing distributed antenna system’s (DAS) resources and infrastructure is an attractive solution due to and lower cost of implementation and likely coverage increases. A 2017 technical report by EPRI has demonstrated that multiple systems such as Land Mobile Radio (LMR) and LTE can be operated using the same DAS and by sharing the leaky coaxial cabling and indoor antenna systems (Reference 1). Since the radio frequency (RF) bands used by the two wireless systems are specified to be non-overlapping, it is believed that the systems are isolated from each other. No extensive analysis has been conducted to determine whether RF signals from one system sharing antennas and coaxial cable can influence another. This paper investigates the assumption of isolation between the two wireless systems and provides recommendations for future needed work to identify specific vulnerabilities and potential mitigations.

97 MATHEMATICS AND COMPUTING↗

DrSPINE - New approach to data reduction and analysis for neutron spin echo experiments from pulsed and reactor sources

Neutron spin echo (NSE) method at a pulsed neutron source presents new challenges to the data reduction and analysis as compared to the instruments installed at reactor sources. The main advantage of the pulsed source NSE is the ability to resolve the neutron wavelength and collect neutrons over a wider bandwidth. This allows us to more precisely determine the symmetry phase and measure the data for several Q-values at the same time. Based on the experience gained at the SNS NSE - the first, and to date the only one, NSE instrument installed at a pulsed spallation source, we propose a novel and unified approach to the NSE data processing.

Monkenbusch, Michael↗

Modeling Atmospheric Science Knowledge from Research Publications

NASA Earth Science Data Centers contain enormous amounts of remote sensing digital data. It is often a significant challenge for users to find data suitable for their research topic in these vast archives. One of the approaches is the usage-driven dataset discovery, where users seek publications on projects similar to their intended study. For this approach to be effective, users need a clear connection between the underlying data in the publications and the study objectives; this is not often apparent to non-expert users. Tools and methodologies that can help facilitate and organize these connections are therefore valuable for creating improved knowledge mappings, which can be further used by search engines to suggest data or publications best tailored to a user’s specific research goal. As an illustration of these challenges, in this work we focus on the atmospheric chemistry processes related to Earth environmental impacts such as ozone depletion, aerosols, smog formation, acid rain, and radiative forcing. We further limit our study to publications that use data from the Microwave Limb Sounder (MLS) instrument flown on the Aura Earth Observing System. To create knowledge representations of science carried out in these publications, we use existing ontologies such as the Global Change Master Directory (GCMD) and Semantic Web for Earth and Environmental Terminology (SWEET). These ontologies together encompass term dictionaries that include measured variables, names of molecules or radicals, mission and instrument names, locations, action words, among many others. Based on these terms acknowledge graph database was populated with the terms retrieved from scientific publications that study atmospheric chemistry. These databases can be used to further enhance the automation of knowledge discovery and facilitate machine learning and artificial intelligence algorithms or applications. These tools and methods can also be extended to apply to content from other related Earth science domains.

Irina Gerasimov↗

VAST 2009 Challenge: An Insider Threat

The 4th year of the VAST Challenge offered participating teams the opportunity to analyze three mini challenges and to combine the data and analyses and enter the Grand challenge. In this paper we provide a description of the data collections used, the overall scenario and the specific questions asked in the mini challenges. We also discuss the infrastructure developed to evaluate accuracy measures in this year’s social network mini challenge. We provide a few examples of visualizations that merited awards. As in previous year, The VAST 2009 Challenge was held in conjunction with the 2009 IEEE VAST Symposium.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Efficient Data-Driven Geologic Feature Characterization from Pre-stack Seismic Measurements using Randomized Machine-Learning Algorithm

Conventional seismic techniques for detecting the subsurface geologic features are challenged by limited data coverage, computational inefficiency and subjective human factors. We developed a novel data-driven geological feature characterization approach based on pre-stack seismic measurements. Our characterization method employs an efficient and accurate machine learning method to extract useful subsurface geologic features automatically. Specifically, we use kernel ridge regression to account for the nonlinear relationship between seismic data and geological features. We moreover employ kernel tricks to avoid the explicit nonlinear mapping and infinite dimension of feature space. Yet, the conventional kernel ridge regression can be computationally prohibitive because of the large volume of seismic measurements. We employ a data reduction technique in combination with the conventional kernel ridge regression method to improve the computational efficiency and reduce memory usage. In particular, we utilize a randomized numerical linear algebra technique, named Nyström method, to effectively reduce the dimensionality of the feature space without compromising the information content required for accurate characterization. We provide thorough computational cost analysis to show the efficiency of our new geological feature characterization methods. We validate the performance of our method in characterizing geologic fault zones because faults play an important role in various subsurface applications. Our numerical examples demonstrate that our new characterization method significantly improves the computational efficiency while maintaining comparable accuracy. Interestingly, we demonstrate that our approach yields a speedup ratio on the order of ~102 to ~103 in a multicore computational environment.

58 GEOSCIENCES↗

Semi-automated Design of Artificial Intelligence Earth Science Models

Prediction and observation of water cycles involve not only patterns isolated in space and time, but rather modeling complex spatio-temporal relationships across multiple sources of data and domains. For instance, Evapotranspiration (ET) and Leaf Area Indexes (LAI) are two critical components in DOE’s Energy Exascale Earth System Model (E3SM). Accurate assessments of ET and LAI are critical for understanding hydrological processes, deforestation, crop yield, and irrigation impacts. However, current ET estimates for global simulations are available at very coarse spatial resolution. They are usually derived from satellite data based on broad plant functional types (PFT), which fail to capture the fine-scale variations due to change in vegetation type across the globe. Within this context and in light of the data-model integration challenges highlighted in the EESSD Strategic Plan, the new era of AI model development for geosciences calls for data-driven methods that provide domain scientists with estimations of parameters such as PFT and LAI in an efficient, interpretable, and easy-to-operate manner.

Ambrozio Dias, Philipe↗

Attitude Control and Orbital Dynamics Challenges of Removing the First 3-Axis Stabilized Tracking and Data Relay Satellite from the Geosynchronous ARC

Launched on April 4, 1983 onboard STS 6 (Space Shuttle Challenger), the First Tracking and Data Relay Satellite (TDRS 1) was retired above the Geosynchronous Orbit (GEO) on June 27, 2010 after having provided real-time communications with a variety of low-orbiting spacecraft over a 26-year period. To meet NASA requirements limiting orbital debris 1, a team of experts was assembled to conduct an End-Of-Mission (EOM) procedure to raise the satellite 350 km above the GEO orbit. Following the orbit raising via conventional station change maneuvers, the team was confronted with having to deplete the remaining propellant and passivate all energy storage or generation sources. To accomplish these tasks within the time window, communications (telemetry and control links), electrical power, propulsion, and thermal constraints, a spacecraft originally designed as a three-axis stabilized satellite was turned into a spinner. This paper (a companion paper to Innovative Approach Enabled the Retirement of TDRS 1, paper # 1699, IEEE 2011 Aerospace Conference, March 5-12, 2011 sup 2) focuses on the challenges of maintaining an acceptable spinning dynamics, while repetitively firing thrusters. Also addressed are the effects of thruster firings on the orbit characteristics and how they were mitigated by a careful scheduling of the fuel depletion operations. Periodic thruster firings for spin rate adjustment, nutation damping, and precession of the momentum vector were also required in order to maintain effective communications with the satellite. All operations were thoroughly rehearsed and supported by simulations thus lending a high level of confidence in meeting the NASA EOM goals.

Benet, Charles A.↗

Unique & challenging aspects of plutonium metal standards exchange program for actinide measurements

The Los Alamos National Laboratory exchange program is the only program of its kind for the distribution of plutonium (Pu) standards materials with a range of impurity contents to multiple laboratories for destructive measurements of elemental concentration. This paper discusses statistical methods used to address challenges in Pu metal exchange data by way of two case studies. Challenges include how to evaluate a data set when a large fraction of the values are minimum detection limits (MDLs), and how to determine potential outliers with limited in-formation on the true spread of the data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Synchrophasor Data Compression Under Disturbance Conditions via Cross-Entropy-Based Singular Value Decomposition

The increasing deployment of phasor measurement units and the advances of their reporting rates are challenging the present data centers in terms of storing and analyzing large-volume data. Under power system disturbance conditions, it is difficult to retain critical information while compressing the synchrophasor data effectively. This article combines the cross entropy and the singular value decomposition, proposing a novel model to compress the synchrophasor data to an extremely small size yet keep superior accuracy. The proposed model is extensively tested and compared with the state-of-the-art algorithms using the simulated and the FNET/GridEye field-collected data. The result indicates that the proposed algorithm has superior performance in compressing the data while retaining critical information under disturbance conditions.

42 ENGINEERING↗