Search NASA⌕ Search

SEARCH · Search NASA

Results for “data infrastructures”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

GridDS: Data Science Toolkit for Energy Grid Data

According to the U.S. Energy Information Administration (EIA), the demand for energy is expected to increase 50% by the year 20501. While energy standards, such as the Institute of Electrical and Electronics Engineers (IEEE) Standard 1547, (Basso 2015) and monitoring with wide area management systems (WAMS) (Liu 2017, Zhou 2016) have enabled large scale data collection and storage, the application of this data in mitigating costs associated with increased consumer demand is an ongoing focus for energy research. This ubiquitous data collection presents a promising opportunity for machine learning and data science to improve efficiency of distributed energy resources (DERs). The GridDS software toolkit is designed to leverage advanced metering infrastructure (AMI), outage management systems data (OMS), Supervisory control Data Acquisition (SCADA), and geographic information systems (GIS) to forecast future energy demands and detect incipient grid failures. GridDS is a python software library designed to be modular and generalizable to data recorded by DERs. In adapting to disparate datasets recorded by various WAMS, GridDS provides a range of unique functionality not presently implemented in current WAMS which have highly specific software infrastructure by design. GridDS functionality ranges from data specification and preparation, to training and validation for state of the art machine learning, to interactive data visualization. For data intake, GridDS combines: Pandera: a library for creating data specifications. TimeScaleDB: a postgresSQL database infrastructure for efficient storage of timeseries data. Dataset class: A custom dataset class / interface that ensures modularity between a range of synthetic and live recorded datasets. Is

Ladd, Alexander↗
NASA NTRS1991

The Earth Observing System Data and Information System

The system is described in terms of the broad context of scientific cooperation and research which are facilitated by the architecture designed and the nature of the generated data. The Earth Observing System Data and Information System (EOSDIS) is composed of earth-science and applications data systems which archive global-change data and distribute the information to system users. Several archive centers catalogue selected Pathfinder data sets and generate new data products (such as maps of sea ice, cloud studies, etc.) with consistent calibration of all data. The schedule of three versions of EOSDIS is discussed with reference to the system architecture, programmatic plans, and data policy. The critical issue for EOSDIS is international cooperation to sponsor an infrastructure of global change data of this magnitude for public benefit and scientific research.

Maiden, Martha E.↗
NASA NTRS1998

AXAF: Current Developments and Future Plans for the Scientific Data Analysis

AXAF, the Advanced X-ray Astrophysics Facility, is one of NASA's great observatories scheduled for launch in January of 1999 by a Space Shuttle. This facility will be available to scientists in the United States and to the international astronomical community over an anticipated mission lifetime of at least 5 years. The objective of AXAF is to make astrophysical observations in the 0.09 to 10.0 keV energy range with two imaging detectors, and two sets of transmission gratings giving high spatial (0.5 arcsec) and spectral (E/(Delta)E=100-2000) resolution. The AXAF Science Center (ASC) in Cambridge, MA, USA is responsible for obtaining and reviewing observation proposals submitted by users, providing long range science planning and performing all science data calibration, data analysis, science instrument trend analysis, and science data archiving. The ASC will distribute science data products to the users. We will review here the current developments and future plans for the AXAF scientific data analysis system, its infrastructure and new key features, including the global modelling and fitting environment, integrated data analysis GUIs and new analysis tools.

Fruscione, Antonella↗
NASA NTRS2010

Updates of Land Surface and Air Quality Products in NASA MAIRS and NEESPI Data Portals

Following successful support of the Northern Eurasia Earth Sciences Partner Initiative (NEESPI) project with NASA satellite remote sensing data, from Spring 2009 the NASA GES DISC (Goddard Earth Sciences Data and Information Services Center) has been working on collecting more satellite and model data to support the Monsoon Asia Integrated Regional Study (MAIRS) project. The established data management and service infrastructure developed for NEESPI has been used and improved for MAIRS support.Data search, subsetting, and download functions are available through a single system. A customized Giovanni system has been created for MAIRS.The Web-based on line data analysis and visualization system, Giovanni (Goddard Interactive Online Visualization ANd aNalysis Infrastructure) allows scientists to explore, quickly analyze, and download data easily without learning the original data structure and format. Giovanni MAIRS includes satellite observations from multiple sensors and model output from the NASA Global Land Data Assimilation System (GLDAS), and from the NASA atmospheric reanalysis project, MERRA. Currently, we are working on processing and integrating higher resolution land data in to Giovanni, such as vegetation index, land surface temperature, and active fire at 5km or 1km from the standard MODIS products. For data that are not archived at the GESDISC,a product metadata portal is under development to serve as a gateway for providing product level information and data access links, which include both satellite, model products and ground-based measurements information collected from MAIRS scientists.Due to the large overlap of geographic coverage and many similar scientific interests of NEESPI and MAIRS, these data and tools will serve both projects.

Shen, Suhung↗
DOE OSTI2024

The ATLAS Workflow Management System Evolution in the LHC Run3 and towards the High-Luminosity LHC era

The ATLAS experiment has 18+ years of experience using workload management systems to deploy and develop workflows to process and to simulate data on the distributed computing infrastructure. Simulation, processing and analysis of LHC experiment data require the coordinated work of heterogeneous computing resources. In particular, the ATLAS experiment utilizes the resources of 250 computing centers worldwide, the power of supercomputing centres, and national, academic and commercial cloud computing resources. In this contribution, we present new techniques for cost-effectively improving efficiency introduced in workflow management system software. The evolution from a mesh framework to new types of computing facilities such as cloud and HPCs is described, as well as new types of production and analysis workflows.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Advanced Computing, Data Science, and Artificial Intelligence Research Opportunities for Energy-Focused Transportation Science

The Energy Efficient Mobility Systems (EEMS) technology landscape is complex and rapidly evolving, which provides both tremendous opportunities and formidable challenges. Significant alterations to the mobility landscape are underway due to the advent of vehicle and infrastructure connectivity, autonomous driving, and rapid passenger- and freight-vehicle electrification. Advanced computing will play an increasingly important role in enabling the EEMS program to understand and identify the most important levers to improve the energy productivity of future integrated mobility systems. It is also driving new approaches to mobility and the research to unlock an affordable, efficient, safe, and accessible transportation future. Driving much of this change is the collection, analysis, and strategic use of massive amounts of diverse, complex data from infrastructure and vehicles with on-board sensors and data storage and transmission capabilities. Diverse and representative data are key to implementing approaches to maximize mobility energy productivity. While high-fidelity modeling of integrated transportation networks has strengthened our understanding of dynamic movement and behavior patterns, existing tools must be expanded beyond their current focus. This work necessitates data infrastructure investments (e.g., secure-streaming data platforms driven by ubiquitous sensors and video analytics) as well as investments in critical capabilities for large-scale automated analysis and organization using modern machine learning, statistics, and artificial intelligence. Other chief needs include agile, large-scale storage that can be quickly searched and queried for relevant data to support validation and model development, data-sharing agreements, and formatting standards for key data types. The future of public transit must be explored in greater detail, research must inform design, and opportunities must be identified for improving the mobility productivity of public transit in both urban and rural America.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Carbon Storage Technical Viability Approach (CS TVA) Data Availability Results Database

The Carbon Storage Technical Viability Approach (CS TVA) Data Availability Results Database v1.0 is the output of a data availability assessment workflow that was developed by NETL to create a tool for spatial data availability assessment. This results database is the output of the first version of the workflow performed on the data with the CS TVA Database v2.0.

carbon storage↗
NASA NTRS2000

SHADOZ (Southern Hemisphere ADditional OZonesondes): An Ozonesonde Network and Resource for Remote Sensing Research and Education

Balloon-borne ozone instrumentation (ozonesondes), launched at fixed sites, is used to study local patterns in stratospheric and tropospheric ozone and to provide validation for satellite ozone products and model calculations of ozone. A paucity of coordinated ozonesonde data in the southern hemisphere tropics is being remedied in a 3-year project of coordinated ozonesondes launches at 10 sites. The data are available to the scientific community at the SHADOZ website at NASA/Goddard. Stations and their operational characteristics, with examples of ozone observations, are given. One expectation of SHADOZ is that wide dissemination of data and interaction with users and field projects will leverage local funding to maintain infrastructure and operations. SHADOZ data are well-suited for educational projects in which students learn about regional ozone patterns.

Thompson, Anne M.↗
DOE OSTI2021

Example on how to (intelligently) augment the nuclear-data pipeline with machine learning [Slides]

The presentation discusses how machine learning has helped the Los Alamos National Laboratory (LANL) nuclear-data pipeline. It also discusses the strengths of machine learning as it finds trends in large amounts of data where human brains are overwhelmed and that this information may be crucial to improve our nuclear data. It does stress, however, that machine learning is no "silver bullet" and that it is critical to feed it expert knowledge and use physics intuition to interpret the results. The presentation discusses the need to develop infrastructure and tools to provide data in an easily readable and unambiguously interpretable format (e.g., EXFOR format), to develop experimental data and theory to solve physics questions, and that statisticians and nuclear-data experts must be brought together to correctly interpret the results. The presentation concludes by stating that machine learning is a great tool and that LANL needs to use the algorithms along with developing physics data, tools and infrastructure.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Plan for a Data Base Approach to Evaluating LWR Plant Integration with Hydrogen and Industrial Facilities

The purpose of this report is to provide an outline and scope of a database tool that could be used to identify the most beneficial locations to site a hydrogen, chemical facility or other industrial facility that would be powered by nuclear energy. This energy may be in the form of electricity produced by the plant, heat in the form of steam or both. In addition, this siting tool should consider factors such as economics, geography, industries, residences, geological hydrogen storage, and key infrastructure. The infrastructure to be considered could include pipelines or other storage and transportation facilities to carry the created products from the production facility to the end user. Also considered should be infrastructure that could deliver needed chemicals or gasses to allow the production of desired products such as ammonia or biofuels. This data base tool should allow a screening study of all U.S. nuclear plants to identify opportunities and barriers to creating hydrogen hubs around nuclear power plants. For this effort, it has been decided that the Siting Tool for Advanced Nuclear Development (STAND) would be used to conduct this screening study which is a follow-on research project to this present effort. Currently, plans are being developed to modify the STAND tool to provide the capability necessary to perform screening of each US commercial nuclear plant to identify the best locations to site hydrogen or chemical production facilities. Many features of the current STAND tool will be employed to meet this objective, but additional infrastructure data (such as the location of pipelines) may need to be added to allow these locations to be fully vetted. In addition, new analysis capabilities may need to be added so economic considerations can be understood

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS1999

A Science Information Infrastructure for Access to Earth and Space Science Data through the Nation's Science Museums

In this project, we worked with the University of California at Berkeley/Center for Extreme Ultraviolet Astrophysics and five science museums (the National Air and Space Museum, the Science Museum of Virginia, the Lawrence Hall of Science, the Exploratorium., and the New York Hall of Science) to formulate plans for computer-based laboratories located at these museums. These Science Learning Laboratories would be networked and provided with real Earth and space science observations, as well as appropriate lesson plans, that would allow the general public to directly access and manipulate the actual remote sensing data, much as a scientist would.

Murray, S.↗
NASA NTRS2014

Robust Informatics Infrastructure Required For ICME: Combining Virtual and Experimental Data

With the increased emphasis on reducing the cost and time to market of new materials, the need for robust automated materials information management system(s) enabling sophisticated data mining tools is increasing, as evidenced by the emphasis on Integrated Computational Materials Engineering (ICME) and the recent establishment of the Materials Genome Initiative (MGI). This need is also fueled by the demands for higher efficiency in material testing; consistency, quality and traceability of data; product design; engineering analysis; as well as control of access to proprietary or sensitive information. Further, the use of increasingly sophisticated nonlinear, anisotropic and or multi-scale models requires both the processing of large volumes of test data and complex materials data necessary to establish processing-microstructure-property-performance relationships. Fortunately, material information management systems have kept pace with the growing user demands and evolved to enable: (i) the capture of both point wise data and full spectra of raw data curves, (ii) data management functions such as access, version, and quality controls;(iii) a wide range of data import, export and analysis capabilities; (iv) data pedigree traceability mechanisms; (v) data searching, reporting and viewing tools; and (vi) access to the information via a wide range of interfaces. This paper discusses key principles for the development of a robust materials information management system to enable the connections at various length scales to be made between experimental data and corresponding multiscale modeling toolsets to enable ICME. In particular, NASA Glenn's efforts towards establishing such a database for capturing constitutive modeling behavior for both monolithic and composites materials

Mutli-scale models↗
DOE OSTI2025

OLCF’s Advanced Computing Ecosystem (ACE): FY25 Update for Ongoing Efforts

The advent of widespread use of artificial intelligence (AI) and machine learning (ML) models in science, coupled with fast data production rates of scientific instruments strain the traditional batch-oriented high-performance computing (HPC) environment. As scientific exploration continues to require more data and faster processing and analysis, new emerging technologies and capabilities to enable cross-facility and time-sensitive workflows are required for seamless integration of HPC and experimental facilities. The Advanced Computing Ecosystem (ACE) is a strategic initiative within the Oak Ridge Leadership Computing Facility (OLCF) established in 2024 to support the development of cutting-edge technologies to advance computational research and infrastructure at OLCF and across the Department of Energy (DOE). Several DOE initiatives are spearheading the evolution of the scientific landscape by blurring facility boundaries and connecting the user facilities to advance scientific capabilities and ensure energy dominance. The DOE Integrated Research Infrastructure (IRI) program is one example that is laying a foundation to support complex cross-facility workflows. The IRI program aims to integrate diverse computational resources, data infrastructures, and scientific instruments to facilitate collaboration and accelerate scientific discovery. The Interconnected Science Ecosystem (INTERSECT) initiative at Oak Ridge National Laboratory (ORNL) is another example that aims to revolutionize scientific research through AI-driven, interconnected autonomous laboratories and research facilities. Finally, the American Science Cloud (AmSC), recently announced in the “One Big Beautiful Bill”, aims to leverage prior infrastructure efforts of the IRI and automation and AI efforts of INTERSECT (and others) to build a federated, AI-augmented AmSC platform to unify the DOE’s computing, experimental, and data resources to catalyze scientific innovation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Materials data science using CRADLE: A distributed, data-centric approach

Abstract There is a paradigm shift towards data-centric AI, where model efficacy relies on quality, unified data. The common research analytics and data lifecycle environment (CRADLE™) is an infrastructure and framework that supports a data-centric paradigm and materials data science at scale through heterogeneous data management, elastic scaling, and accessible interfaces. We demonstrate CRADLE’s capabilities through five materials science studies: phase identification in X-ray diffraction, defect segmentation in X-ray computed tomography, polymer crystallization analysis in atomic force microscopy, feature extraction from additive manufacturing, and geospatial data fusion. CRADLE catalyzes scalable, reproducible insights to transform how data is captured, stored, and analyzed. Graphical abstract

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Case Study: Field Evaluation of a Low-Cost Circuit-Level Electrical Submetering System

Circuit-level metering technologies provide the ability to monitor individual circuits within an electrical panel in a building, providing detailed power and energy consumption data at a much more granular level than was previously achievable in a cost-effective manner. While the fundamental hardware components of circuit-level—split-core current transformers (CTs) and power monitoring meters—have existed for some time, the new offerings in the market have tightly integrated these components, lower costs, and have streamlined data organization, transport, and access via software solutions accessible through web and application programming interfaces (API). NREL evaluated Meazon’s single-circuit metering system. The evaluation of the equipment under test (EUT) focused on three aspects: (1) accuracy of the data provided by the system, (2) ease of installation of this technology and ease of data integration into existing U.S. General Services Administration (GSA) analytics platforms, and (3) total cost of ownership and cost-effectiveness of the technology. The metering technology was tested in a field deployment where it was installed in low and high-voltage commercial electrical panels in the César E. Chávez Memorial Building in Denver, Colorado. The goals were to assess the metering accuracy, the ease of installation and data retrieval, and the total cost of ownership and cost-effectiveness. This technology proved easy to install by a certified electrician who completed the install of 6 meters in 2 separate panels (120/208 V and 277/480 V) and 2 circuit disconnects (along with associated commissioning) at the César E. Chávez Memorial Building in less than one day. The technology was installed in high- and low-voltage panels and with limited space in the electrical room, demonstrating the applicability of this technology to almost all commercial buildings in the GSA portfolio. Primary considerations for this technology include appropriate sizing of CTs for each circuit, selection of loads/circuits/panels that are of high value for detailed submetering (e.g., tailored for high-load devices), and how GSA would like to integrate these data with its existing energy management infrastructure. Through the field evaluation, we demonstrated data integration from the vendor system into the primary software component on the GSA enterprise-level energy management and information system, GSA Link. Data from the circuit-level metering were integrated into this platform and used to perform FDD. This demonstrated the ability of the EUT to augment existing energy management and information systems in buildings where GSA Link is deployed and provides a pathway for delivering FDD at other buildings throughout the portfolio.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRSDate not supplied

Selecting Approaches for Enabling Enterprise Data Search: NASA’s Science Mission Directorate (SMD) Catalog

NASA’s Science Mission Directorate (SMD) is working to build an open-source science infrastructure to accelerate open, collaborative and interdisciplinary science. One key component in the open-source science infrastructure is the SMD data catalog. In this paper, we present our process for selecting a technical approach to building a NASA SMD enterprise-wide integrated search capability for science users across multiple science disciplines to support discovery and access to complex scientific data.

Kaylin Bugbee↗
NASA NTRSDate not supplied

Open-Source Data Engineering at NASA: CCMC's Approach to Managing Petabyte-Scale Heliophysics Data

The Community Coordinated Modeling Center (CCMC) at NASA Goddard Space Flight Center (GSFC) leads heliophysics research by providing open access to numerous models and their outputs. Our resources are available on-demand and continuously updated with real-time data, covering sun-earth interactions across multiple domains. These domains include coronal, heliosphere, inner and global magnetosphere, ionosphere, thermosphere, and lower atmosphere interactions. Operating in a hybrid environment, CCMC utilizes both self-owned hardware and Amazon Web Services (AWS) cloud infrastructure. Managing petabytes of data across multiple locations necessitates robust data engineering solutions. To address this challenge, CCMC has adopted industry-standard and open-source tools. We use Apache Airflow as our primary data engineering platform, Python for scripting and data processing, and GitLab for version control and CI/CD. Additionally, we employ Kubernetes for containerized services, Grafana and Prometheus for metrics and monitoring, and Terraform and Puppet for reproducible infrastructure as code. This presentation will discuss lessons learned from our data engineering experiences, platforms evaluated but found unsuitable for our scientific data requirements, and specific techniques developed to enhance data transfer speed and reliability. By using these technologies effectively, CCMC continues to advance heliophysics research through efficient data management and open-access modeling.

space weather↗
NASA NTRS2006

SimScape Terrain Modeling Toolkit

This paper describes the SIMSCAPE middleware toolkit that has been developed recently to provide a common infrastructure for importing terrain model data from multiple data sources and making them available to simulation applications. The SIMSCAPE infrastructure simplifies the overall simulation design by eliminating the traditional need for custom terrain model interfaces to terrain data sources and simulation users. SIMSCAPE provides a collection of libraries and tools to use and manage terrain environment models within a wide range of simulation applications.

terrain↗