Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

Integrated Multi-Scale Data Analytics and Machine Learning for the Distribution Grid and Building-to-Grid Interface

This white paper introduces the application of advanced data analytics to the modernized grid. In particular, we consider the field of machine learning and where it is both useful, and not useful, for the particular field of the distribution grid and buildings interface. While analytics, in general, is a growing field of interest, and often seen as the golden goose in the burgeoning distribution grid industry, its application is often limited by communications infrastructure, or lack of a focused technical application. Overall, the linkage of analytics to purposeful application in the grid space has been limited. In this paper we consider the field of machine learning as a subset of analytical techniques, and discuss its ability and limitations to enable the future distribution grid and the building-to-grid interface. To that end, we also consider the potential for mixing distributed and centralized analytics and the pros and cons of these approaches. Machine learning is a subfield of computer science that studies and constructs algorithms that can learn from data and make predictions and improve forecasts. Incorporation of machine learning in grid monitoring and analysis tools may have the potential to solve data and operational challenges that result from increasing penetration of distributed and behind-the-meter energy resources. There is an exponentially expanding volume of measured data being generated on the distribution grid, which, with appropriate application of analytics, may be transformed into intelligible, actionable information that can be provided to the right actors – such as grid and building operators, at the appropriate time to enhance grid or building resilience, efficiency, and operations against various metrics or goals – such as total carbon reduction or other economic benefit to customers. While some basic analysis into these data streams can provide a wealth of information, computational and human boundaries on performing the analysis are becoming significant, with more data and multi-objective concerns. Efficient applications of analysis and the machine learning field are being considered in the loop.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Making NASA GES DISC Level 2 Data GIS Analysis Ready

There are many valuable data hosted by NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) for GIS applications in the areas of extreme weather events, climatic anomaly, and public health. However, using NASA Earth Science Data poses some challenges for GIS users. Many of these users are not experts in Earth Observation and have little knowledge about NASA's Earth science data. In the GIS community, GeoTiff is the most widely used raster format, whereas NASA's data is primarily in complex multidimensional netCDF and HDF formats. This complexity makes it difficult for GIS users, especially those who are unfamiliar with these formats. Although GIS software like ArcGIS has made progress in processing multidimensional netCDF data, certain issues still remain, particularly with level 2 data. In this study, we use TROPSpheric Monitoring instrument (TROPOMI) level 2 data as an example to demonstrate how to make such data GIS analysis ready. The process involves: 1. Creating a feature layer from the TROPOMI level 2 data. 2. Converting the feature layer to a gridded raster dataset. 3. Mosaicking gridded raster datasets into a raster dataset covering the entire desired extent. 4. Generating a symbology with a GIBS-specific style that aligns with the visual standards and requirements of GIBS. 5. Publishing image services. By performing these preprocessing and transformation steps, NASA level 2 data can be made compatible and ready for use within GIS software for various spatial analysis and visualization tasks.

Geographic Information System↗

Designing alloys with process-mapping AI pre-trained on empirical knowledge

<span style="font-family: Calibri, sans-serif; font-size: 12pt;">Accelerated materials design should match the recent trends in the product development cycles. Materials data analytics can be used to significantly shorten development time of specialized alloys needed for next generation energy applications. However, it faces a challenge of scarce data available for training ML models. Incorporation of the domain knowledge into deep-learning graph structure via fuzzy pre-training and causal process imitation presents a viable approach to developing accurate data-driven models and reliable alloy design tools, with limited datasets. Artificial Intelligence (AI) was used in this study to incorporate such knowledge in the domain-specific computational tool, pyroMind. The tool provides not only novel design ideas but also their interpretation via physics and engineering concepts.</span>

Romanov, Vyacheslav↗

Effective Tooling for Linked Data Publishing in Scientific Research

Challenges that make it difficult to find, share, and combine published data, such as data heterogeneity and resource discovery, have led to increased adoption of semantic data standards and data publishing technologies. To make data more accessible, interconnected and discoverable, some domains are being encouraged to publish their data as Linked Data. Consequently, this trend greatly increases the amount of data that semantic web tools are required to process, store, and interconnect. In attempting to process and manipulate large data sets, tools–ranging from simple text editors to modern triplestores– eventually breakdown upon reaching undefined thresholds. This paper offers a systematic approach that data publishers can use to categorize suitable tools to meet their data publishing needs. We present a real-world use case, the Resource Discovery for Extreme Scale Collaboration (RDESC), which features a scientific dataset(maximum size of 1.4 billion triples) used to evaluate a toolbox for data publishing in climate research. This paper also introduces a semantic data publishing software suite developed for the RDESC project.

Purohit, Sumit↗

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗

Analytics-Driven Lossless Data Compression for Rapid In-situ Indexing, Storing, and Querying

The analysis of scientific simulations is highly data-intensive and is becoming an increasingly important challenge. Peta-scale data sets require the use of light-weight query-driven analysis methods, as opposed to heavy-weight schemes that optimize for speed at the expense of size. This paper is an attempt in the direction of query processing over losslessly compressed scientific data. We propose a co-designed double-precision compression and indexing methodology for range queries by performing unique-value-based binning on the most significant bytes of double precision data (sign, exponent, and most significant mantissa bits), and inverting the resulting metadata to produce an inverted index over a reduced data representation. Without the inverted index, our method matches or improves compression ratios over both general-purpose and floating-point compression utilities. The inverted index is light-weight, and the overall storage requirement for both reduced column and index is less than 135%, whereas existing DBMS technologies can require 200-400%. As a proof-of-concept, we evaluate univariate range queries that additionally return column values, a critical component of data analytics, against state-of-the-art bitmap indexing technology, showing multi-fold query performance improvements.

Jenkins, John↗

Adaptive wavelet compression of large additive manufacturing experimental and simulation datasets

New manufacturing technologies such as additive manufacturing require research and development to minimize the uncertainties in the produced parts. The research involves experimental measurements and large simulations, which result in huge quantities of data to store and analyze. We address this challenge by alleviating the data storage requirements using lossy data compression. We select wavelet bases as the mathematical tool for compression. Unlike images, additive manufacturing data is often represented on irregular geometries and unstructured meshes. Thus, we use Alpert tree-wavelets as bases for our data compression method. We first analyze different basis functions for the wavelets and find the one that results in maximal compression and miminal error in the reconstructed data. We then devise a new adaptive thresholding method that is data-agnostic and allows a priori estimation of the reconstruction error. Finally, we propose metrics to quantify the global and local errors in the reconstructed data. One of the error metrics addresses the preservation of physical constraints in reconstructed data fields, such as divergence-free stress field in structural simulations. While our compression and decompression method is general, we apply it to both experimental and computational data obtained from measurements and thermal/structural modeling of the sintering of a hollow cylinder from metal powders using a Laser Engineered Net Shape process. The results show that monomials achieve optimal compression performance when used as wavelet bases. The new thresholding method results in compression ratios that are two to seven times larger than the ones obtained with commonly used thresholds. Overall, adaptive Alpert tree-wavelets can achieve compression ratios between one and three orders of magnitude depending on the features in the data that are required to preserve. Furthermore, these results show that Alpert tree-wavelet compression is a viable and promising technique to reduce the size of large data structures found in both experiments and simulations.

42 ENGINEERING↗

Roadmap for transforming heterogeneous catalysis with artificial intelligence

Artificial intelligence (AI) is poised to transform heterogeneous catalysis, opening avenues for catalytic materials discovery. By uncovering intricate patterns in high-dimensional data, AI has been reshaping our pursuit of sustainable catalytic processes across the energy, environmental and chemical sectors. This promise, however, hinges on overcoming fundamental barriers, including limitations in data availability and quality, challenges in the generalizability and interpretability of data-augmented decisions, and the persistent gap between in silico predictions and experiments. Furthermore, we outline a forward-looking roadmap for deeply integrating AI into heterogeneous catalysis with an AI-ready data ecosystem, multimodal foundation models, and ultimately autonomous laboratories to accelerate the development of next-generation catalytic technologies via AI-empowered human–machine collaboration.

Computational methods↗

Effects of PHENYLALANINE AMMONIA LYASE ( PAL ) knockdown on cell wall composition, biomass digestibility, and biotic and abiotic stress responses in Brachypodium

The phenylpropanoid pathway in plants synthesizes a variety of structural and defence compounds, and is an important target in efforts to reduce cell wall lignin for improved biomass conversion to biofuels. Little is known concerning the trade-offs in grasses when perturbing the function of the first gene family in the pathway, PHENYLALANINE AMMONIA LYASE ( PAL ). Therefore, PAL isoforms in the model grass Brachypodium distachyon were targeted, by RNA interference (RNAi), and large reductions (up to 85%) in stem tissue transcript abundance for two of the eight putative BdPAL genes were identified. The cell walls of stems of BdPAL -knockdown plants had reductions of 43% in lignin and 57% in cell wall-bound ferulate, and a nearly 2-fold increase in the amounts of polysaccharide-derived carbohydrates released by thermochemical and hydrolytic enzymic partial digestion. PAL -knockdown plants exhibited delayed development and reduced root growth, along with increased susceptibilities to the fungal pathogens Fusarium culmorum and Magnaporthe oryzae . Surprisingly, these plants generally had wild-type (WT) resistances to caterpillar herbivory, drought, and ultraviolet light. RNA sequencing analyses revealed that the expression of genes associated with stress responses including ethylene biosynthesis and signalling were significantly altered in PAL knocked-down plants under non-challenging conditions. These data reveal that, although an attenuation of the phenylpropanoid pathway increases carbohydrate availability for biofuel, it can adversely affect plant growth and disease resistance to fungal pathogens. Lastly, the data identify notable differences between the stress responses of these monocot pal mutants versus Arabidopsis (a dicot) pal mutants and provide insights into the challenges that may arise when deploying phenylpropanoid pathway-altered bioenergy crops.

60 APPLIED LIFE SCIENCES↗

Effective Materials Property Information Management for the 21st Century

This paper discusses key principles for the development of materials property information management software systems. There are growing needs for automated materials information management in various organizations. In part these are fuelled by the demands for higher efficiency in material testing, product design and engineering analysis. But equally important, organizations are being driven by the needs for consistency, quality and traceability of data, as well as control of access to proprietary or sensitive information. Further, the use of increasingly sophisticated nonlinear, anisotropic and multi-scale engineering analyses requires both processing of large volumes of test data for development of constitutive models and complex materials data input for Computer-Aided Engineering (CAE) software. And finally, the globalization of economy often generates great needs for sharing a single gold source of materials information between members of global engineering teams in extended supply-chains. Fortunately material property management systems have kept pace with the growing user demands and evolved to versatile data management systems that can be customized to specific user needs. The more sophisticated of these provide facilities for: (i) data management functions such as access, version, and quality controls; (ii) a wide range of data import, export and analysis capabilities; (iii) data pedigree traceability mechanisms; (iv) data searching, reporting and viewing tools; and (v) access to the information via a wide range of interfaces. In this paper the important requirements for advanced material data management systems, future challenges and opportunities such as automated error checking, data quality characterization, identification of gaps in datasets, as well as functionalities and business models to fuel database growth and maintenance are discussed.

36 MATERIALS SCIENCE↗

A Visual Analytics Approach for Correlation, Classification, and Regression Analysis

New approaches that combine the strengths of humans and machines are necessary to equip analysts with the proper tools for exploring today's increasing complex, multivariate data sets. In this paper, a novel visual data mining framework, called the Multidimensional Data eXplorer (MDX), is described that addresses the challenges of today's data by combining automated statistical analytics with a highly interactive parallel coordinates based canvas. In addition to several intuitive interaction capabilities, this framework offers a rich set of graphical statistical indicators, interactive regression analysis, visual correlation mining, automated axis arrangements and filtering, and data classification techniques. The current work provides a detailed description of the system as well as a discussion of key design aspects and critical feedback from domain experts.

97 MATHEMATICS AND COMPUTING↗

Analytic continuation of noisy data using Adams Bashforth residual neural network

We propose a data-driven learning framework for the analytic continuation problem in numerical quantum many-body physics. Designing an accurate and efficient framework for the analytic continuation of imaginary time using computational data is a grand challenge that has hindered meaningful links with experimental data. The standard Maximum Entropy (MaxEnt)-based method is limited by the quality of the computational data and the availability of prior information. Also, the MaxEnt is not able to solve the inversion problem under high level of noise in the data. Here we introduce a novel learning model for the analytic continuation problem using a Adams-Bashforth residual neural network (AB-ResNet). Additionally, the advantage of this deep learning network is that it is model independent and, therefore, does not require prior information concerning the quantity of interest given by the spectral function. More importantly, the ResNet-based model achieves higher accuracy than MaxEnt for data with higher level of noise. Finally, numerical examples show that the developed AB-ResNet is able to recover the spectral function with accuracy comparable to MaxEnt where the noise level is relatively small.

97 MATHEMATICS AND COMPUTING↗

Advanced technology for earth observation - Data processing

The Experimental Land Observing System (ELOS) is to demonstrate major advances in space-based land observation. Program success will depend upon successful demonstration of both space-segment performance and ground-system capabilities. Increases in data-rate, volume, and processing requirements present substantial challenges to the ground data processing system. ELOS is based on the utilization of the multispectral linear array (MLA) sensor, a solid-state, electronically scanned instrument. Current MLA instrument designs offer a number of capabilities beyond those provided in the Landsat imaging devices. These capabilities include greater resolution, inherent registration, onboard calibration, off-nadir pointing capability, and fore/aft pointing capability. The proposed MLA flight mission scenario will be one of continued growth and will include flights on the Space Transportation System. Attention is also given to ground-to-ground data transport, data capture, and the use of high speed computers.

Heffner, P.↗

Data management for community research projects: A JGOFS case study

Since the mid 1980s, much of the marine science research effort in the United Kingdom has been focused into large scale collaborative projects involving public sector laboratories and university departments, termed Community Research Projects. Two of these, the Biogeochemical Ocean Flux Study (BOFS) and the North Sea Project incorporated large scale data collection to underpin multidisciplinary modeling efforts. The challenge of providing project data sets to support the science was met by a small team within the British Oceanographic Data Centre (BODC) operating as a topical data center. The role of the data center was to both work up the data from the ship's sensors and to combine these data with sample measurements into online databases. The working up of the data was achieved by a unique symbiosis between data center staff and project scientists. The project management, programming and data processing skills of the data center were combined with the oceanographic experience of the project communities to develop a system which has produced quality controlled, calibrated data sets from 49 research cruises in 3.5 years of operation. The data center resources required to achieve this were modest and far outweighed by the time liberated in the scientific community by the removal of the data processing burden. Two online project databases have been assembled containing a very high proportion of the data collected. As these are under the control of BODC their long term availability as part of the UK national data archive is assured. The success of the topical data center model for UK Community Research Project data management has been founded upon the strong working relationships forged between the data center and project scientists. These can only be established by frequent personal contact and hence the relatively small size of the UK has been a critical factor. However, projects covering a larger, even international scale could be successfully supported by a network of topical data centers managing online databases which are interconnected by object oriented distributed data management systems over wide area networks.

Lowry, Roy K.↗

Effective Materials Property Information Management for the 21st Century

This paper discusses key principles for the development of materials property information management software systems. There are growing needs for automated materials information management in various organizations. In part these are fueled by the demands for higher efficiency in material testing, product design and engineering analysis. But equally important, organizations are being driven by the need for consistency, quality and traceability of data, as well as control of access to sensitive information such as proprietary data. Further, the use of increasingly sophisticated nonlinear, anisotropic and multi-scale engineering analyses requires both processing of large volumes of test data for development of constitutive models and complex materials data input for Computer-Aided Engineering (CAE) software. And finally, the globalization of economy often generates great needs for sharing a single "gold source" of materials information between members of global engineering teams in extended supply chains. Fortunately, material property management systems have kept pace with the growing user demands and evolved to versatile data management systems that can be customized to specific user needs. The more sophisticated of these provide facilities for: (i) data management functions such as access, version, and quality controls; (ii) a wide range of data import, export and analysis capabilities; (iii) data "pedigree" traceability mechanisms; (iv) data searching, reporting and viewing tools; and (v) access to the information via a wide range of interfaces. In this paper the important requirements for advanced material data management systems, future challenges and opportunities such as automated error checking, data quality characterization, identification of gaps in datasets, as well as functionalities and business models to fuel database growth and maintenance are discussed.

Ren, Weiju↗

Assimilation of Dual-Polarimetric Radar Observations with WRF GSI

Dual-polarimetric (dual-pol) radar typically transmits both horizontally and vertically polarized radio wave pulses. From the two different reflected power returns, more accurate estimate of liquid and solid cloud and precipitation can be provided. The upgrade of the traditional NWS WSR-88D radar to include dual-pol capabilities will soon be completed for the entire NEXRAD network. Therefore, the use of dual-pol radar network will have a broad impact in both research and operational communities. The assimilation of dual-pol radar data is especially challenging as few guidelines have been provided by previous research. It is our goal to examine how to best use dual-pol radar data to improve forecast of severe storm and forecast initialization. In recent years, the Development Testbed Center (DTC) has released the community Gridpoint Statistical Interpolation (GSI) DA system for the Weather Research and Forecasting (WRF) model. The community GSI system runs in independently environment, yet works functionally equivalent to operational centers. With collaboration with the NASA Short-term Prediction Research and Transition (SPoRT) Center, this study explores regional assimilation of the dual-pol radar variables from the WSR-88D radars for real case storms. Our presentation will highlight our recent effort on incorporating the horizontal reflectivity (ZH), differential reflectivity (ZDR), specific differential phase (KDP), and radial velocity (VR) data for initializing convective storms, with a significant focus being on an improved representation of hydrometeor fields. In addition, discussion will be provided on the development of enhanced assimilation procedures in the GSI system with respect to dual-pol variables. Beyond the dual-pol variable assimilation procedure developing within a GSI framework, highresolution (≤1 km) WRF model simulations and storm scale data assimilation experiments will be examined, emphasizing both model initialization and short-term forecast of precipitation fields and processes. Further details of the methodology of data assimilation, the impact of different dual-pol variables, the influence on precipitation forecast will be presented at the conference.

Li, Xuanli↗

NASA World Wind: Infrastructure for Spatial Data

The world has great need for analysis of Earth observation data, be it climate change, carbon monitoring, disaster response, national defense or simply local resource management. To best provide for spatial and time-dependent information analysis, the world benefits from an open standards and open source infrastructure for spatial data. In the spirit of NASA's motto "for the benefit of all" NASA invites the world community to collaboratively advance this core technology. The World Wind infrastructure for spatial data both unites and challenges the world for innovative solutions analyzing spatial data while also allowing absolute command and control over any respective information exchange medium.

Hogan, Patrick↗

Assimilation of Dual-Polarimetric Radar Observations with WRF GSI

Dual-polarimetric (dual-pol) radar typically transmits both horizontally and vertically polarized radio wave pulses. From the two different reflected power returns, more accurate estimate of liquid and solid cloud and precipitation can be provided. The upgrade of the traditional NWS WSR-88D radar to include dual-pol capabilities will soon be completed for the entire NEXRAD network. Therefore, the use of dual-pol radar network will have a broad impact in both research and operational communities. The assimilation of dual-pol radar data is especially challenging as few guidelines have been provided by previous research. It is our goal to examine how to best use dual-pol radar data to improve forecast of severe storm and forecast initialization. In recent years, the Development Testbed Center (DTC) has released the community Gridpoint Statistical Interpolation (GSI) DA system for the Weather Research and Forecasting (WRF) model. The community GSI system runs in independently environment, yet works functionally equivalent to operational centers. With collaboration with the NASA Short-term Prediction Research and Transition (SPoRT) Center, this study explores regional assimilation of the dual-pol radar variables from the WSR-88D radars for real case storms. Our presentation will highlight our recent effort on incorporating the horizontal reflectivity (ZH), differential reflectivity (ZDR), specific differential phase (KDP), and radial velocity (VR) data for initializing convective storms, with a significant focus being on an improved representation of hydrometeor fields. In addition, discussion will be provided on the development of enhanced assimilation procedures in the GSI system with respect to dual-pol variables. Beyond the dual-pol variable assimilation procedure developing within a GSI framework, highresolution (≤1 km) WRF model simulations and storm scale data assimilation experiments will be examined, emphasizing both model initialization and short-term forecast of precipitation fields and processes. Further details of the methodology of data assimilation, the impact of different dual-pol variables, the influence on precipitation forecast will be presented at the conference.

Li, Xuanli↗