Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Estimating Species Distributions Across Space Through Time and with Features of the Environment

Complete guidance for mastering the tools and techniques of the digital revolution With the digital revolution opening up tremendous opportunities in many fields, there is a growing need for skilled professionals who can develop data-intensive systems and extract information and knowledge from them. This book frames for the first time a new systematic approach for tackling the challenges of data-intensive computing, providing decision makers and technical experts alike with practical tools for dealing with our exploding data collections. Emphasizing data-intensive thinking and interdisciplinary collaboration, The Data Bonanza: Improving Knowledge Discovery in Science, Engineering, and Business examines the essential components of knowledge discovery, surveys many of the current research efforts worldwide, and points to new areas for innovation. Complete with a wealth of examples and DISPEL-based methods demonstrating how to gain more from data in real-world systems, the book: Outlines the concepts and rationale for implementing data-intensive computing in organizations Covers from the ground up problem-solving strategies for data analysis in a data-rich world Introduces techniques for data-intensive engineering using the Data-Intensive Systems Process Engineering Language DISPEL Features in-depth case studies in customer relations, environmental hazards, seismology, and more Showcases successful applications in areas ranging from astronomy and the humanities to transport engineering Includes sample program snippets throughout the text as well as additional materials on a companion website The Data Bonanza is a must-have guide for information strategists, data analysts, and engineers in business, research, and government, and for anyone wishing to be on the cutting edge of data mining, machine learning, databases, distributed systems, or large-scale computing.

Kelling, S.↗

A Visual Analytics Approach for Correlation, Classification, and Regression Analysis

New approaches that combine the strengths of humans and machines are necessary to equip analysts with the proper tools for exploring today s increasing complex, multivariate data sets. In this paper, a visual data mining framework, called the Multidimensional Data eXplorer (MDX), is described that addresses the challenges of today s data by combining automated statistical analytics with a highly interactive parallel coordinates based canvas. In addition to several intuitive interaction capabilities, this framework offers a rich set of graphical statistical indicators, interactive regression analysis, visual correlation mining, automated axis arrangements and filtering, and data classification techniques. This chapter provides a detailed description of the system as well as a discussion of key design aspects and critical feedback from domain experts.

Steed, Chad A↗

E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials

Abstract This work presents Neural Equivariant Interatomic Potentials (NequIP), an E(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations for molecular dynamics simulations. While most contemporary symmetry-aware models use invariant convolutions and only act on scalars, NequIP employs E(3)-equivariant convolutions for interactions of geometric tensors, resulting in a more information-rich and faithful representation of atomic environments. The method achieves state-of-the-art accuracy on a challenging and diverse set of molecules and materials while exhibiting remarkable data efficiency. NequIP outperforms existing models with up to three orders of magnitude fewer training data, challenging the widely held belief that deep neural networks require massive training sets. The high data efficiency of the method allows for the construction of accurate potentials using high-order quantum chemical level of theory as reference and enables high-fidelity molecular dynamics simulations over long time scales.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Integrating Data Centers and Grid Technologies at Scale

This presentation focuses on the challenge of integrating AI-driven data centers with the power grid at scale. It examines the AI data center capacity challenge and the role of new Medium Voltage Direct Current (MVDC) and other grid-enhancing technologies in enabling efficient and reliable power delivery. The session will highlight the National Laboratory of the Rockies' ARIES capabilities and planning tools, along with collaborative examples involving Verrus, Compass, and Schneider through the Agora test bed for grid-friendly data center evaluations, and ON. Energy for UPS evaluation. It will showcase the NLR Stable Grid Platform for studying oscillations caused by large-scale data centers, along with planning tools to assess grid security and reliability. Additionally, the presentation covers reconductoring strategies to increase grid capacity and explores innovative data center architectures, including the Advanced DC Architectures with Power-electronic Transformers (ADAPT) platform, which enables testing of complete DC architectures for data centers.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Characterizing the subglacial environment of lower Thwaites Glacier using radar modeling

Understanding the spatial heterogeneity beneath Thwaites Glacier, West Antarctica, is vital to projecting its impact on future sea levels. Radar-echo sounding (RES) is commonly used to infer subglacial conditions, but these data can be challenging to interpret. We assess basal heterogeneity across Thwaites Glacier by comparing RES returns to a radar backscattering simulator for over 400 km of RES data. The modeled variations in bed returned power exhibited a strong correlation with actual RES data in 40% of our simulated flight segments, which we consider evidence for a relatively homogeneous glacier bed. Other sites (40%) demonstrated improved fit quality when hydrology or substrate transitions were introduced in the bed material model. The remaining simulated segments (20%) were diagnosed as having more complex basal heterogeneity. The spatial distribution of complex heterogeneity appears to coincide with asymmetric patterns in the RES specularity content, which has been interpreted in previous studies as a signature for channelized hydrology. Conversely, the homogeneous substrate locations coincide with areas of fast-moving ice in western Thwaites. Our simulation method can isolate power variations induced by material heterogeneity vs topography, which is an important limitation of existing RES analysis methods.

Antarctic glaciology↗

A review of artificial intelligence applications in manufacturing operations

Abstract Artificial intelligence (AI) and machine learning (ML) can improve manufacturing efficiency, productivity, and sustainability. However, using AI in manufacturing also presents several challenges, including issues with data acquisition and management, human resources, infrastructure, as well as security risks, trust, and implementation challenges. For example, getting the data needed to train AI models can be difficult for rare events or costly for large datasets that need labeling. AI models can also pose security risks when integrated into industrial control systems. In addition, some industry players may be hesitant to use AI due to a lack of trust or understanding of how it works. Despite these challenges, AI has the potential to be extremely helpful in manufacturing, particularly in applications such as predictive maintenance, quality assurance, and process optimization. It is important to consider the specific needs and capabilities of each manufacturing scenario when deciding whether and how to use AI in manufacturing. This review identifies current developments, challenges, and future directions in AI/ML relevant to manufacturing, with the goal of improving understanding of AI/ML technologies available for solving manufacturing problems, providing decision‐support for prioritizing and selecting appropriate AI/ML technologies, and identifying areas where further research can yield transformational returns for the industry. Early experience suggests that AI/ML can have significant cost and efficiency benefits in manufacturing, especially when combined with the ability to capture enormous amounts of data from manufacturing systems.

Plathottam, Siby Jose↗

Stakeholder analysis for designing an urban air quality data governance ecosystem in smart cities

Cities, the world over, are fuelling economic growth. At the same time, rapid urbanization is a root cause of serious environmental damage. Recent WHO global air pollution guidelines highlight air pollution as a critical environmental threat along with climate change. To address these threats, smart cities and clean air programs are on a rise. In smart cities, data and Information and Communication Technologies (ICT) are major drivers of city transformations. The 4th Industrial Revolution (4IR) technologies such as the Internet of Things (IoT), big data, artificial intelligence (AI), and cloud computing have the potential to accelerate these transformations toward urban resilience. However, the success of smart cities and clean air programs depends on cohesive multi-sector stakeholder contributions. This study conducted interdisciplinary participative stakeholder analysis to understand the data, and sectorial challenges, to outline the technological opportunities to facilitate clean air programs in Indian smart cities. The research highlights gaps due to siloed stakeholder operations, lack of data calibration, non-alignment of smart city and air quality management services, non-availability of health exposure data, and difficulty in translating scientific data into implementable actions. Stakeholders expressed potential ‘fit for the purpose’ use of IoT devices, satellites, smartphones, and mobility data augmented by AI methods in bridging these gaps. In conclusion, the analysis points toward a need to develop an easily accessible and ubiquitous urban data governance ecosystem enabling seamless cross-sector data exchanges to build trusting relationships among the stakeholders across the air quality management value chain.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Transfer learning of neural surrogates on multifidelity groundwater simulations

Multifidelity data used in the paper published in Advances in Water Resources 206 (2025) 105140, https://doi.org/10.1016/j.advwatres.2025.105140 The code used to process the data is openly available on GitHub at https://github.com/Model-Reduction-and-UQ-Group/Transfer_Learning_K_reconstruction Computationally inexpensive surrogates of process-based models, such as deep neural networks, enable ensemble-based computations used in risk assessment, data assimilation, etc. However, generation of large datasets required to train a neural network can be as expensive as the ensemble simulations themselves. We ameliorate this challenge by using data from multifidelity (MF) groundwater simulations and transfer learning (TL) to reduce data generation costs while maintaining model accuracy. As a computational example, we train a deep convolutional neural network (CNN) to reconstruct permeability fields from saturation maps derived from a multiphase flow model. Starting with very low- and low-fidelity data generated on increasingly coarse meshes, we pretrain the CNN, followed by output-layer training and fine-tuning using only a limited number of high-fidelity samples. We demonstrate the surrogate’s robustness when interpreting low-quality inputs—such as interpolated maps or data affected by noise—which has strong implications for the applicability in practical hydrogeological scenarios. This multilevel MF-TL strategy achieves a favorable trade-off between computational efficiency and predictive accuracy, significantly outperforming high-fidelity-only approaches under the same computational budget.

Chiofalo, Alessia [University of Bologna] (ORCID:0↗

Distributed Storage Systems for Data Intensive Computing

In this chapter, the authors present an overview of the utility of distributed storage systems in supporting modern applications that are increasingly becoming data intensive. Their coverage of distributed storage systems is based on the requirements imposed by data intensive computing and not a mere summary of storage systems. To this end, they delve into several aspects of supporting data-intensive analysis, such as data staging, offloading, checkpointing, and end-user access to terabytes of data, and illustrate the use of novel techniques and methodologies for realizing distributed storage systems therein. The data deluge from scientific experiments, observations, and simulations is affecting all of the aforementioned day-to-day operations in data-intensive computing. Modern distributed storage systems employ techniques that can help improve application performance, alleviate I/O bandwidth bottleneck, mask failures, and improve data availability. They present key guiding principles involved in the construction of such storage systems, associated tradeoffs, design, and architecture, all with an eye toward addressing challenges of data-intensive scientific applications. They highlight the concepts involved using several case studies of state-of-the-art storage systems that are currently available in the data-intensive computing landscape.

99 GENERAL AND MISCELLANEOUS↗

Challenges and alternatives to empirical orthogonal functions for earth system data

Empirical orthogonal functions (EOFs) applied to gridded Earth system data enables users to diagnose modes of variability with relative ease. Yet, many challenges to interpretation exist such that they must be used with awareness and intention when applied to gridded climate data, especially with large ensembles. Utilizing data from two different Earth system modelling large ensemble frameworks, the Energy Exoscale Earth System Model and the Community Earth System Model, as well as reanalysis data, common EOF pitfalls are summarized and discussed. Challenges include erroneous mode swapping, sign flipping, and the temporal variability of the centers of action. For modes of variability with similar contribution to variance, mode swapping is not uncommon. Sign flipping can occur with almost any mode where the pattern is correct, but the sign is arbitrary. Although the variability of the center of action is not necessarily problematic, it potentially complicates interpretation over multi-century timescales. A wide variety of alternative methods to EOFs exist, but fitness-for-purpose must be evaluated. Additionally, illustrations of alternative methods and examples of proper use are provided. Alternative methods fit into three categories: EOF variants, linear methods, and multilinear methods.

54 ENVIRONMENTAL SCIENCES↗

In situ Synchrotron X‐ray Metrology Boosted by Automated Data Analysis for Real‐time Monitoring of Cathode Calcination

Abstract Synchrotron X‐ray‐based in situ metrology is advantageous for monitoring the synthesis of battery materials, offering high throughput, high spatial and temporal resolution, and chemical sensitivity. However, the rapid generation of massive data poses a challenge to on‐site, on‐the‐fly analysis needed for real‐time process monitoring. Here, a weighted lagged cross‐correlation (WLCC) similarity approach is presented for automated data analysis, which merges with in situ synchrotron X‐ray diffraction metrology to monitor the calcination process of the archetypal nickel‐based cathode, LiNiO 2 . The WLCC approach, incorporating variables that account for peak shifts and width changes associated with structural transformations, enables rapid extraction of phase progression within 10 seconds from tens of diffraction patterns. Details are captured, from initial precursors to intermediates and the final layered LiNiO 2 , providing information for agile on‐site adjustments during experiments and complementing post hoc diffraction analysis by offering insights into early‐stage phase nucleation and growth. Expanding this data‐powered platform paves the way for real time calcination process monitoring and control, which is pivotal to quality control in battery cathode manufacturing.

36 MATERIALS SCIENCE↗

An optimal merging technique for high-resolution precipitation products: OPTIMAL MERGING OF PRECIPITATION METHOD

Precipitation products are currently available from various sources at higher spatial and temporal resolution than any time in the past. Each of the precipitation products has its strengths and weaknesses in availability, accuracy, resolution, retrieval techniques and quality control. By merging the precipitation data obtained from multiple sources, one can improve its information content by minimizing these issues. However, precipitation data merging poses challenges of scale-mismatch, and accurate error and bias assessment. In this paper we present Optimal Merging of Precipitation (OMP), a new method to merge precipitation data from multiple sources that are of different spatial and temporal resolutions and accuracies. This method is a combination of scale conversion and merging weight optimization, involving performance-tracing based on Bayesian statistics and trend-analysis, which yields merging weights for each precipitation data source. The weights are optimized at multiple scales to facilitate multiscale merging and better precipitation downscaling. Precipitation data used in the experiment include products from the 12-km resolution North American Land Data Assimilation (NLDAS) system, the 8-km resolution CMORPH and the 4-km resolution National Stage-IV QPE. The test cases demonstrate that the OMP method is capable of identifying a better data source and allocating a higher priority for them in the merging procedure, dynamically over the region and time period. This method is also effective in filtering out poor quality data introduced into the merging process.

54 ENVIRONMENTAL SCIENCES↗

AUTOMATIC DETECTION AND TRACKING OF CORONAL MASS EJECTIONS. II. MULTISCALE FILTERING OF CORONAGRAPH IMAGES

Studying coronal mass ejections (CMEs) in coronagraph data can be challenging due to their diffuse structure and transient nature, and user-specific biases may be introduced through visual inspection of the images. The large amount of data available from the Solar and Heliospheric Observatory (SOHO), Solar TErrestrial RElations Observatory (STEREO), and future coronagraph missions also makes manual cataloging of CMEs tedious, and so a robust method of detection and analysis is required. This has led to the development of automated CME detection and cataloging packages such as CACTus, SEEDS, and ARTEMIS. Here, we present the development of a new CORIMP (coronal image processing) CME detection and tracking technique that overcomes many of the drawbacks of current catalogs. It works by first employing the dynamic CME separation technique outlined in a companion paper, and then characterizing CME structure via a multiscale edge-detection algorithm. The detections are chained through time to determine the CME kinematics and morphological changes as it propagates across the plane of sky. The effectiveness of the method is demonstrated by its application to a selection of SOHO/LASCO and STEREO/SECCHI images, as well as to synthetic coronagraph images created from a model corona with a variety of CMEs. The algorithms described in this article are being applied to the whole LASCO and SECCHI data sets, and a catalog of results will soon be available to the public.

79 ASTRONOMY AND ASTROPHYSICS↗

Fusion Energy Science Joint Facilities and Theory Research Target 2011: Final Report for the Period October 1, 2010 through September 30, 2011

Briefly stated, the goal of this combined experiment/theory milestone was to improve our knowledge of the physics processes that control the H-mode pedestal by applying models of these mechanisms to experimental data. A concerted and coordinated effort of experimental, theory and modeling activities has examined several physics mechanisms of interest, and has met the goals of the milestone. A few noteworthy results will be noted here. The activity has applied peeling-ballooning theory to all three machines, and increased confidence that the theory predicts the ultimate limits to pedestal height in an ELMing plasma. The activity has expanded the range of conditions for testing of the EPED model, and increased confidence in the ability of the model to compute the pedestal pressure height in medium aspect ratio tokamaks. Kinetic neoclassical models have been used to verify that an important analytic model for the pedestal bootstrap current has good accuracy in many regimes of interest but needs correction at high collisionality. A benchmarking effort between electromagnetic gyrokinetic codes has provided increased understanding of how to use these codes in the pedestal, and has laid the groundwork for using these codes to study linear stability of gyrokinetic modes in and on top of the pedestal. Comparisons of data and models have led to successful simulations of a quasi-coherent mode in C-Mod and to the tentative identification of a high frequency mode in DIII-D. A kinetic neoclassical model qualitatively agrees with some features of pedestal density profiles. A paleoclassical transport model quantitatively predicts some features of the electron temperature profiles in NSTX and DIII-D. The milestone activity also laid the groundwork for future advancements in understanding and predicting the pedestal. All three machines obtained a wealth of pedestal data under a wide variety of operating conditions. Some of these data have been analyzed already, and the analysis and comparison with models will provide important results over the next year or two. Many codes were improved in significant ways in order to model data under the challenging conditions of the pedestal. In addition, new and strengthened collaborations were established between experiment, modeling and theory, and these efforts include providing mechanisms for sharing experimental data. All of these efforts will continue to provide major benefits to future pedestal studies. Much of the experimental and modeling work performed to support this milestone is reported in detail in the research contributions to this report. Here, a summary of results from the various contributions, as they pertain to several proposed pedestal physics processes, is discussed here. Emphasis is placed on mechanisms for which there are results from one or more machines or one or more models. Work from preceding quarterly reports is used for some of this summary. At the end of this summary, a brief summary of implications for ITER and implications for future work will be presented.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Evaluating the Efficacy of Conditional Variational Autoencoders in Generating Synthetic Single Nuclei RNA-Seq Data for Space Biology Research

Astronauts are subject to unique stressors during spaceflight, leading to changes in their cellular function. However, neither astronauts nor model organisms respond the same to spaceflight, and research implicates a contribution of omics components in differential responses. Understanding how gene expression affects astronaut health is critical for the success of long-term space missions, prompting interest in developing personalized predictive models leveraging artificial intelligence (AI) and machine learning (ML) techniques. Developing such models requires extensive data, which is challenging to obtain and share. This study explores the use of conditional variational autoencoders (CVAEs) to synthetically generate single-nuclei RNA-seq (snRNA-seq) data. CVAEs build on standard variational autoencoders (VAEs) by conditioning data generation on covariates like sample identity and mission parameters, enhancing the relevance of generated data for specific contexts. For our work, we built two CVAEs with varying degrees of sparsity to optimize both interpretability and generative power. We train and validate models on existing snRNA-seq data collected from the brain tissue of mice subjected to spaceflight conditions and their ground control counterparts. We evaluate model performance using statistical tests and visualizations to compare synthetic data to real data. We aim to demonstrate that these prototype CVAE architectures could be used in future space biology work and that this is a method worth further exploring.

Sarah Golts↗

An optimal merging technique for high-resolution precipitation products

Precipitation products are currently available from various sources at higher spatial and temporal resolution than any time in the past. Each of the precipitation products has its strengths and weaknesses in availability, accuracy, resolution, retrieval techniques and quality control. By merging the precipitation data obtained from multiple sources, one can improve its information content by minimizing these issues. However, precipitation data merging poses challenges of scale-mismatch, and accurate error and bias assessment. In this paper we present Optimal Merging of Precipitation (OMP), a new method to merge precipitation data from multiple sources that are of different spatial and temporal resolutions and accuracies. This method is a combination of scale conversion and merging weight optimization, involving performance-tracing based on Bayesian statistics and trend-analysis, which yields merging weights for each precipitation data source. The weights are optimized at multiple scales to facilitate multiscale merging and better precipitation downscaling. Precipitation data used in the experiment include products from the 12-km resolution North American Land Data Assimilation (NLDAS) system, the 8-km resolution CMORPH and the 4-km resolution National Stage-IV QPE. The test cases demonstrate that the OMP method is capable of identifying a better data source and allocating a higher priority for them in the merging procedure, dynamically over the region and time period. This method is also effective in filtering out poor quality data introduced into the merging process.

54 ENVIRONMENTAL SCIENCES↗

Cloud Climate Data Records from Geostationary Satellites to Support CERES: Challenges and Recent Progress

Cloud properties are critical for understanding the Earth’s radiation budget and cloud feedbacks. At NASA Langley Research Center, the Satellite ClOud and Radiative Property retrieval System (SatCORPS) provides real-time and historical analyses of clouds derived from Geostationary satellite (GEOsat) data for weather and climate applications. For the Clouds and the Earth’s Radiant Energy System (CERES) program, the global constellation of GEOsats has been analyzed since 2000 to help characterize and account for the diurnal cycle of clouds and their radiative impacts in the CERES climate data record. Obtaining consistent cloud properties over the GEOsat data record at all times of day during the CERES era is a major objective but a significant challenge considering the diversity of imaging capabilities deployed during that time. A brief overview of the current CERES/GEO cloud property data record constructed from three generations of GEOsat imagers for CERES Edition-4 is presented. In this version, different algorithms were applied to different satellites to take advantage of as much spectral information as possible for each satellite. Under this approach, the derived cloud properties were found to become accurate (and more similar to MODIS) for the GEOsat imagers with more spectral information. However, this approach also led to some discontinuities in cloud properties across platforms. Significant differences were also found between daytime and nighttime cloud properties. This paper will highlight the major issues in the Edition-4 record and discuss new strategies and early results from the next version of the SatCORPS GEOsat cloud property time series in development for CERES Edition-5 with the goal of improving day-night and cross-platform consistency during the CERES record.

clouds↗

Challenges in Obtaining and Visualizing Satellite Level 2 Data in GIS

Satellite data products are important for a wide variety of applications that can bring far-reaching benefits to the science community and the broader society. These benefits can best be achieved if the satellite data are well utilized and interpreted. Unfortunately, this is not always the case, despite the abundance and relative maturity of numerous satellite data products provided by NASA and other organizations. One way to help users better understand the satellite data is to provide data along with Images, including accurate pixel coverage area delineation, and science team recommended quality screening for individual geophysical parameters. However, there are challenges of visualizing remote sensed non-gridded products: (1) different geodetics of space-borne instruments (2) data often arranged in a long-track and a cross-track axes (3) spatially and temporally continuous data chunked into granule files: data for a portion (or all) of a satellite orbit (4) no general rule of resampling or interpolations to a grid (5) geophysical retrieval only based on pixel center location without shape information. In this presentation, we will unravel a new Goddard Earth Sciences Data and Information Services Center (GES DISC) Level 2 (L2) visualization on-demand service. The service's front end provides various visualization and data accessing capabilities, such as overlay and swipe of multiply variables and subset and download of data in different formats. The backend of the service consists of Open Geospatial Consortium (OGC) standard-compliant Web Mapping Service (WMS) and Web Coverage Service. The infrastructure allows inclusion of outside data sources served in OGC compliant protocols and allows other interoperable clients, such as ArcGIS clients, to connect to our L2 WCS/WMS.

GI↗