Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Management Platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Value of Information and Data Management Platform Promoting Modern Data Analysis Utilization. STTR Phase I, Final Technical Report

The immense growth in weather and river sensor data availability along with increases in computational power and algorithmic sophistication have enabled the widespread application of physical, machine learning, and artificial intelligence algorithms to river forecasting. The forecasts produced by widely varying model approaches are inherently limited in different ways, such as by the number and quality of training observations, producing biased predictions that achieve optimal skill only over specific ranges of weather and river conditions. Thus, despite the growing availability and new sources of forecasts, hydropower owners cannot fully leverage the information because they lack an effective mechanism to determine the “best” forecast for given conditions or objectives. Tools are needed to consolidate forecast information, demonstrate skill, and extract optimal forecast guidance from the range of possibilities. Such tools will increase uptake and beneficial use of the growing wealth of forecast information, and lead to improved operating decisions, optimal power generation, and enhanced system value. This research proposes to improve the value of forecasts by developing a broadly accessible verification framework and applying machine learning techniques to identify the ‘best’ forecast from an ensemble of sources given the current weather, river, and operating conditions.

13 HYDRO ENERGY↗

Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data

To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.

data management platform↗

Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data: Preprint

To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.

data management↗

FY 2024 Multidimensional Data Correlation Platform Data Management Infrastructure Progress: Materials Laboratory

This report provides an inventory of the equipment available at the ORNL Manufacturing Demonstration Facility (MDF) for sample preparation and material characterization, including both destructive and non-destructive techniques that generate critical data to support the development of the Multi-Dimensional Data Correlation (MDDC) framework. The success of the MDDC framework depends heavily on the quality and completeness of the data it can access. Therefore, it is essential to establish a comprehensive inventory of the technologies available to the Advanced Materials and Manufacturing Technologies (AMMT) multi-laboratory team. This starts by gathering information about the types of data they produce, the data collection and transfer protocols used, file formats, and data storage requirements for experiments. This information is then carefully evaluated to create the operations and trackables elements of the Damara Tern platform, which is the foundation of the MDDC framework.

36 MATERIALS SCIENCE↗

FY 2025 Multidimensional Data Correlation Platform: Unified Software Architecture for Advanced Materials and Manufacturing Technologies Data Management and Processing

The Advanced Materials and Manufacturing Technologies (AMMT) program continues to advance a data-driven approach to demonstrate the utility of additive manufacturing for fabricating components for nuclear applications. A key scientific goal is to leverage data to better understand manufacturing outcomes and thereby improve the performance, reliability, and lifespan of nuclear components. Ultimately, this effort supports the development of standards for certification and qualification of additively manufactured components, enabling broader industry adoption. In support of this objective, the AMMT program is building and deploying a data management platform to record, index, analyze, and make available the manufacturing data generated across the AMMT program. In FY 2023, the team conceptualized the architecture of the platform and, in FY 2024, deployed the first functional version at the Oak Ridge National Laboratory (ORNL) Manufacturing Demonstration Facility (MDF). In FY 2025, the platform was officially opened to all AMMT members. To enable this expansion, core modifications and enhancements were developed, including improvements to the user interface and workflows for data entry and retrieval. Most notably, robust security and access control mechanisms were implemented to protect data and manage information sharing. This effort featured a logging system, protected views, and controlled access mechanisms. This report documents these enhancements and the transition of the platform into program-wide use.

36 MATERIALS SCIENCE↗

Grid Operator Analytics and Assessment Tools for Inverter- Based Resources Dominated Grid (GOAAT-IBR) Project Update

This presentation provides an update on the OPTIMA GOAAT project, with emphasis on the cloud-native data platform developed in-house to ingest, manage, and operationalize high-resolution power system data. Since our last NASPI presentation, accessible via OSTI ID #2671437, the project team advanced the design and deployment of a scalable architecture capable of handling both synchronized and non-synchronized streams, including PMU, point-on-wave (POW), COMTRADE, and SCADA data. These materials review the project status, recent progress, and key lessons learned. The core of the presentation examines the architecture and engineering of our cloud-native ingestion and data management platform. We then explain how pipelines were designed to collect, normalize, time-align, store, and serve heterogeneous data at scale. We will discuss design choices such as data models, streaming versus batch ingestion, storage tiers, and interoperability with analytics applications. Practical experiences with cloud-native technologies were shared during the event, including benefits, limitations, and integration challenges in a utility environment, along with methods used to improve performance, reduce latency, and optimize resource usage. The presentation also showcases user interface designs and visualization tools that convert raw measurements and analytics results into intuitive, actionable insights for operators and engineers. During the presentation examples were provided demonstrating how visualization, event views, and summarized analytics enhance situational awareness and support operational decision-making. These use cases illustrate how a well-designed data infrastructure can bridge the gap between high-volume measurements and practical grid operations.

Aminifar, Farrokh↗

DeepLynx Ecosystem 2025

Poor data integration and governance continue to plague complex engineering projects, resulting in missed cost, schedule, and performance targets. Departments operate in isolated systems with manual data exchange, creating fragmented information that compounds errors and leads to significant delays and cost overruns. The DeepLynx ecosystem addresses these challenges through an open-source, modular data management platform that transforms fragmented project data into an integrated digital thread. Built on a federated microservice architecture, the ecosystem comprises seven specialized tools centered around DeepLynx Nexus, a unified data catalog with hierarchical organization and graph-based navigation capabilities. The ecosystem includes: DeepLynx Stream for real-time timeseries data ingestion from industrial sources; DeepLynx Ingest for governed data uploads with formal review workflows; DeepLynx Lattice for ontology-based entity and relationship extraction; DeepLynx Run for workflow orchestration and secure AI/ML compute; DeepLynx Visualize for 3D digital twin visualization; and DeepLynx Insight for AI-assisted document analysis with traceable, grounded responses. Deployable in cloud, on-premise, or hybrid environments using containerized Docker applications and Helm charts, the DeepLynx ecosystem provides flexible infrastructure that adapts to organizational requirements. By consolidating project data into a unified data lake with role-based access controls and OAuth2 authentication, DeepLynx enables digital thread and digital twin capabilities that improve decision-making, reduce risk, and support complex engineering workflows throughout the project lifecycle.

42 - ENGINEERING↗

Digital Twin for Hydropower System Object Modeling: Alder Dam (FY2023)

Hydropower is the world's largest source of renewable electricity, and hydropower plants are distributed all over the world. Typical major components of a hydropower plant are the governor, excitation, generator, thrust bearing, hydraulic turbine, transformer, the main lead, metering and control, tailwater depression, and dissolved oxygen. For each component, various measures are taken. The measurements are acquired by various heterogeneous systems, including standalone sensors, programmable logic controllers (PLC), Supervisory control and data acquisition (SCADA), Internet of Things (IoT), and data acquisition and integration platforms such as OSI/PI. The measured data are often archived within the plant by a data management platform, and many institutions have cloud-based archive systems, such as Hydropower Research Institution (HRI), U.S. Army Corps of Engineers (USACE), and Columbia River Data Access in Real Time (DART). Object Modeling is a general framework for designing information systems. It focuses on objects, the actions they perform, and the messages they send to one another to cause those actions to be taken. The major differences among object modeling, network modeling, data modeling, and process modeling are that in the first we focus on the actions in response to information, objects which form the system, the actions they perform, and how they pass information to one another, while in the second we concentrate on where, when and how much information is moved, while in the third we focus on what information is moved and where it is moved, while in the last we focus on how it is moved and when it is moved. Object modeling was developed basically as a method to develop object-oriented systems and to support object-oriented programming. It describes the static structure of the system. The object Modeling Technique is easy to draw and use. That is why we choose object modeling to connect physical hydropower plants to Digital Twin. It recognizes the objects and the relationship between them. It identifies the attributes and functions of each class. Dynamic Modeling: It explains how objects respond to events. Functional Modeling indicates the processes executed in an object and how data changes when it moves to objects. It has been used in many applications like telecommunication, transportation, etc.

13 HYDRO ENERGY↗

An open-source data storage and visualization platform for collaborative qubit control

Developing collaborative research platforms for quantum bit control is crucial for driving innovation in the field, as they enable the exchange of ideas, data, and implementation to achieve more impactful outcomes. Furthermore, considering the high costs associated with quantum experimental setups, collaborative environments are vital for maximizing resource utilization efficiently. However, the lack of dedicated data management platforms presents a significant obstacle to progress, highlighting the necessity for essential assistive tools tailored for this purpose. Current qubit control systems are unable to handle complicated management of extensive calibration data and do not support effectively visualizing intricate quantum experiment outcomes. In this paper, we introduce Qubit Control Storage and Visualization ( QubiCSV ), a platform specifically designed to meet the demands of quantum computing research, focusing on the storage and analysis of calibration and characterization data in qubit control systems. As an open-source tool, QubiCSV facilitates efficient data management of quantum computing, providing data versioning capabilities for data storage and allowing researchers and programmers to interact with qubits in real time. The insightful visualization are developed to interpret complex quantum experiments and optimize qubit performance. QubiCSV not only streamlines the handling of qubit control system data but also improves the user experience with intuitive visualization features, making it a valuable asset for researchers in the quantum computing domain.

97 MATHEMATICS AND COMPUTING↗

CO 2 Storage Site Screening Platform Development and CO 2 Storage Resource Analysis in SECARB Offshore Reservoirs Using SAS Viya

A major goal of the SECARB Offshore Partnership (DE-FE0031557) is to screen deep saline aquifers and hydrocarbon reservoirs in the central Gulf of Mexico for CO 2 sequestration and CO 2 -enhanced oil and gas recovery (EOR/EGR) and estimate the corresponding CO 2 storage resources for select reservoirs. CO 2 storage potential associated with offshore CO 2 -EOR is considerable and likely represents “low hanging fruit” for near-term CO 2 storage given the in-place infrastructure in the region. It is for these reasons that this assessment focuses on oil and gas fields. To this end, three major objectives have been completed and include (1) managing geological data derived from different sources, (2) building a reservoir screening platform for CO 2 storage, and (3) ranking the reservoirs based on the estimated CO 2 storage resources. The SAS ® Viya platform was used for data management and analytics. The Viya platform is a cloud service platform that provides data integration, data management, quick analytics, data visualization, machine learning functions, and application programming interfaces (APIs) for multi-programming languages. Different sources of data containing geologic information, reservoir properties, and EOR/EGR information were collected, cleaned, formatted, and loaded into the SAS ® Viya platform for evaluation. The major geological characteristics of both shelf and deep-water areas of the central Gulf were examined and compared to define the appropriate reservoir screening criteria. Next, a CO 2 storage site screening system was built in the SAS ® Viya platform with the pre-defined criteria. Finally, the CO 2 storage resources of the screened reservoirs were calculated and reported at the BOEM field level to identify fields with the highest estimated CO 2 storage resource. The fields with the largest total estimated CO 2 storage resource are located in the Mississippi Canyon protraction area. Due to proximity to the Mississippi Delta (indicative of less infrastructure) and large estimated CO 2 storage resources, future development activities may wish to focus efforts in the Mississippi Canyon protraction area.

02 PETROLEUM↗

Unified Software Architecture for Advanced Materials and Manufacturing Technologies Data Management and Processing: FY 2023 Multidimensional Data Correlation Platform

This report details the various digital manufacturing activities ongoing at ORNL as part of the Advanced Materials and Manufacturing Technologies (AMMT) program. The AMMT program is exploring a data-driven approach to demonstrate the use of AM for the fabrication of components for nuclear applications, with the goal of providing a greater understanding of manufacturing quality outcomes that would pave the way toward the development of standards for certification and qualification. The objective of this work package is to establish a digital manufacturing discipline common to all participants of the AMMT program to improve the performance, reliability, and lifetime of nuclear components. As part of this effort, we will develop a unified software architecture for AMMT data management and processing, deploy the digital platform across AMMT participants’ facilities, and generate pedigreed datasets in a common format across multiple labs and facilities. To this end, the MDDC work package has focused on three activities during FY23. First, the MDF Digital Tool was overhauled to better serve the needs of the AMMT program. Next, multiple laser powder bed fusion (L-PBF) systems at the MDF were upgraded to a common sensor package for collecting comparable in situ data across machines. Finally, various improvements relevant to the AMMT program were implemented in the ORNL-developed software tool, Peregrine. This report marks the completion of FY23 milestone M3CR-22OR0403051: Report Describing the Architecture of the Digital Platform to Support AMMT Activities.

36 MATERIALS SCIENCE↗

A National Infrastructure for Artificial Intelligence on the Grid (NI4AI) (Final Scientific/Technical Report)

Electric utilities have traditionally taken a very pragmatic yet myopic approach with grid sensors and the resulting collected data. Sensors are purchased and deployed to solve a specific, known problem that has risen to sufficient awareness as to justify the effort of deploying sensors and the needed capital investment. This sensor data flows into proprietary software packages with limited functionality intended only to address the initial problem. This approach aligns with the financial incentives of the utility to deploy capital into fixed hardware assets for which the corporations earn a rate of return. This mentality stands in stark contrast to the big data revolution that started nearly 25 years ago with the rise of Google. In this worldview, data is a fundamental business asset; successful organizations collect, store, explore, merge, and exploit as much data as possible to not only solve problems well understood today but also to tackle new problems that will inevitably rise tomorrow. The ARPA-E Open Innovation 2018 project entitled A National Infrastructure for Artificial Intelligence on the Grid or NI4AI for short was designed to demonstrate this alternative paradigm for using data. To do this, the project was composed of three key thrust areas. The first major component deployed a variety of high-frequency grid sensors and captured terabytes of both wide-scale and localized grid measurements, generating high-value datasets for grid research and algorithm development. The second aspect made available PingThings’ PredictiveGridTM, a horizontally scalable, cloud-based data management and AI platform built for time series data to explore and exploit the collected data. Finally, the project fostered a diverse and open research community composed of experts from numerous fields through focused educational content, code sharing, and data science competitions. Shifting away from “single use” sensors and closed data silos within electric utilities is a major benefit to the public at large. This legacy approach to data is incredibly (1) capital intensive (new sensors must be deployed for each new problem and problems tend to arise continuously) and (2) painfully slow (new problems must be identified first and then new sensors must be deployed to collect data to begin to address the issue). The transition to a carbon neutral grid requires a massive transformation of the existing grid infrastructure and will continue to challenge the legacy grid in unforeseen ways. The only way to make the energy transition cost effective is for utilities to abandon this dated data paradigm and adopt more contemporary approaches. NI4AI has shown that it is technically possible and economically feasible to ingest, explore, and exploit grid data collected from even very high frequency sensing, such as continuous point on wave sensors collecting measurements 10,000 times a second. In fact, the PredictiveGrid platform used is commercially available and deployed at several utilities in the United States. Project accomplishments were numerous and included (1) making available a state of the art time series platform to the community, (2) collecting over 520 streams of time series data from grid sensors totaling over 1 trillion grid measurements, and (3) developing and nurturing a community within the industry focused on the use of data to create value for utilities and, ultimately, end consumers.

97 MATHEMATICS AND COMPUTING↗

National Renewable Energy Laboratory, South Table Mountain Campus, Golden, Colorado

The National Renewable Energy Laboratory (NREL) South Table Mountain campus, located in Golden, Colorado, has multiyear datasets from buildings with approximately 1,100,000 sq ft of floor area. These 16 buildings include a large office, nine laboratories, and other education or public assembly facilities such as an education center, a warehouse, quick service restaurants, two site entrance security buildings, and a parking garage—all constructed and improved from 1982 to 2014. These datasets are also available from the NREL data storage and management platform SkySparks. The datasets contain electricity meter data (poser, voltage, and current) with 1-minute resolution and building automation system (BAS) data with resolutions of 5 or 15 minutes. Some subsystems (e.g., lighting) or equipment electricity data have the same intervals as above. Some mechanical equipment state or condition data, such as flow and temperature, are captured with 5- to 15-minute intervals. The types of measurement points (i.e., electrical meter power and zone temperature) and associated data quality across the campus are typical, which helps reduce the unreliability of data quality.

STM, Systems and equipment operational, Energy use↗

DOE Data Days 2023 Report

The DOE Data Days (D3) workshop brings together data managers, developers, researchers, and program managers across the Department of Energy (DOE) and national laboratories to highlight data management successes, identify potential synergies and common problems, and establish channels for collaboration across the DOE data management community. The fourth D3 workshop was held on October 24th to 26th, 2023 held entirely in-person at Lawrence Livermore National Laboratory (LLNL). The workshop was organized by a multi laboratory committee in an effort to bring data management practitioners at the DOE laboratories together to share their work and results, facilitating knowledge transfers and best practices across project teams. Tools and platforms to support data management and analysis are rapidly evolving and provide enormous opportunities. This report summarizes the important discussions and recommendations from the different working sessions and contains the agenda, submitted abstracts, presentations with links to recorded presentations, breakout session summaries, list of registered attendees, and lessons learned for future organizing committee. The report will be distributed to the DOE, each participating institution’s programmatic stakeholders, and attendees. The dedicated D3 website will host presentations, agenda, and report that is accessible by all labs.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Instantiation of the Damara Tern Platform for Advanced Materials and Manufacturing Technologies (AMMT) Program Collaborative Data Management

This work package focused on deploying an instance of the Damara Tern platform to support AMMT collaborative research activities. The objectives were to provide selected AMMT collaborators with access to a shared environment for capturing operations, trackables, and associated metadata, and to implement data entry functionalities that reflect site-specific procedures. Key activities included creating configurable, schema-driven entry forms and validating the data collection process. The report details the deployment process, the platform infrastructure, and the implemented data entry workflows, providing a reference for end users and establishing a foundation for future production-scale deployments.

36 MATERIALS SCIENCE↗

Advances in building data management for building performance standards using the SEED platform

Reducing energy consumption and greenhouse gas emissions in the built environment is a critical step in achieving emission goals to mitigate climate change impacts. Local, federal, and international jurisdictions are deploying several methods to reduce energy and emissions such as voluntary and mandatory benchmarking and building performance standards, requiring building owners to reach energy and emission targets. Jurisdictions leveraging benchmarking and building performance standards require knowledge of the buildings covered; which is a large task due to staffing constraints, limited information on building characteristics and tax parcel data, and the need for advanced data management techniques to align datasets. This paper describes an open-source platform's recent advances to create consistent taxonomies, identify erroneous data, enable auditability, and track building performance. The paper concludes with two use cases on how the platform has been used by jurisdictions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The IsoGenie database: an interdisciplinary data management solution for ecosystems biology and environmental research

Modern microbial and ecosystem sciences require diverse interdisciplinary teams that are often challenged in “speaking” to one another due to different languages and data product types. Here we introduce the IsoGenie Database, a de novo developed data management and exploration platform, as a solution to this challenge of accurately representing and integrating heterogenous environmental and microbial data across ecosystem scales. The IsoGenieDB is a public and private data infrastructure designed to store and query data generated by the IsoGenie Project, a ~10 year DOE-funded project focused on discovering ecosystem climate feedbacks in a thawing permafrost landscape. The IsoGenieDB provides (i) a platform for IsoGenie Project members to explore the project’s interdisciplinary datasets across scales through the inherent relationships among data entities, (ii) a framework to consolidate and harmonize the datasets needed by the team’s modelers, and (iii) a public venue that leverages the same spatially explicit, disciplinarily integrated data structure to share published datasets. The IsoGenieDB is also being expanded to cover the NASA-funded Archaea to Atmosphere (A2A) project, which scales the findings of IsoGenie to a broader suite of Arctic peatlands, via the umbrella A2A Database (A2A-DB). The IsoGenieDB’s expandability and flexible architecture allow it to serve as an example ecosystems database.

54 ENVIRONMENTAL SCIENCES↗