Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Data Management in the Continuum: Cross-facility Object-based Data Transfers

Scientific workflows are evolving from relying on a monolithic storage subsystem at a single High-Performance Computing (HPC) facility to using geographically distributed file systems, repositories, and cloud storage. As a result, storing, accessing, transferring, and managing scientific data have become highly complex and prone to performance inefficiencies. This paper delves into these challenges by exploring an optimized end-to-end interface designed to seamlessly connect various local and remote storage systems, enabling efficient data movement of objects across HPC–Cloud and HPC–HPC environments. We showcase this capability through an object-focused data management runtime system, discuss the effects of relaxed consistency semantics in distributed object scenarios, and illustrate its application in an earthquake simulation workflow. Besides reducing the amount of data by selectively transferring regions of interest, our facility-local results achieved a speedup of 45 × over an optimized HDF5 usage and 15 × over the HDF5 with caching by using the new interface in PDC-XF.

Bez, Jean Luca↗
DOE OSTI2025

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗
DOE OSTI2025

Electron cloud predictions for the Hadron Storage Ring of the Electron-Ion Collider and planned mitigations

This paper reports on a collection of electron cloud studies to determine the electron cloud threshold for different sections along the beampipe of the Hadron Storage Ring (HSR) for the Electron-Ion Collider (EIC), presents the results of a study of the interaction of the beam with the electron clouds, and discusses the limitation of potential solutions like scrubbing and Landau damping.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

ROOT RNTuple and EOS: The Next Generation of Event Data I/O

For several years, the ROOT team is developing the new RNTuple I/O subsystem in preparation of the next generation of collider experiments. Both HL-LHC and DUNE are expected to start data taking by the end of this decade. They pose unprecedented challenges to event data I/O in terms of data rates, event sizes, and event complexity. At the same time, the I/O landscape is becoming more diverse. HPC cluster file systems and object stores, NVMe disk cache layers in analysis facilities, and S3 storage on cloud resources are mixing with traditional XRootD-managed spinning disk pools.The ROOT team will finalize a first production version of the RNTuple binary format by the end of 2024. After this point, ROOT will provide backward compatibility for RNTuple data. This contribution provides an overview of the RNTuple feature set, the related R&D activities and the long-term vision for RNTuple. We report on performance, interface design, tooling, robustness, integration with experiment frameworks, and validation results, as well as recent R&D on parallel reading and writing and exploitation of modern hardware and storage systems. We will give an outlook on possible future features after a first production release.Collaboratively, the IT and EP departments at CERN have launched a formal project within the Research and Computing sector to evaluate the novel data format for physics analysis data utilized in LHC experiments and other fields. This part of the project focuses on validating the scalability of the EOS storage backend during the transition from the over 25 years old TTree production format to the newly developed RNTuple format, using both replicated and erasure-coded storage profiles.

Blomer, Jakob [CERN]↗
DOE OSTI2024

Nuclear Quadrupole Resonance for Substance Detection

This review paper provides a comprehensive overview of recent advances in nuclear quadrupole resonance (NQR) spectroscopy for substance detection, highlighting its principles, methodologies, and applications. The paper elucidates the fundamental physics underlying NQR spectroscopy, emphasizing the interaction between nuclear quadrupole moments and electric field gradients. It explores the various experimental techniques and instrumentation developments that have enabled the sensitive detection and precise characterization of substances containing quadrupolar nuclei. A significant portion of the survey is dedicated to discussing the diverse applications of NQR spectroscopy, including the detection of explosives, drug pharmaceuticals, and material authentication. Furthermore, the survey examines the challenges and limitations associated with NQR spectroscopy, including issues related to signal-to-noise ratio (SNR), temperature dependency, and substance restrictions. Strategies to overcome these challenges are discussed, offering insights into the future directions of NQR spectroscopy research that includes artificial intelligence (AI), internet of things (IoT) integration, incorporating a cloud database for NQR parameter storage, and multi-modal analysis.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Enhancing Cloud Cybersecurity: Prescriptive Controls for Operational Technology

This whitepaper provides strategic insights and recommendations into security cloud-based solutions for electric utilities, encompassing operational technology (OT), virtual power plants (VPP), distributed energy resources (DERs), applications, networks, and data storage as they transition to and leverage cloud infrastructure through managed service providers (MSPs) and cloud service providers (CSPs). Principles derived from established frameworks serve as a foundation for best practices across cybersecurity projects and remove the constraints of settling on a single framework. For organizations that prefer not to integrate a specific framework altogether, elements of the proposed approach could be adopted or tailored to best fit defined requirements and expected functionalities. The Cirrus assessment, a utility cloud feasibility tool, and the roadmap it provides serve as a precursor to this paper, which seeks to be a valuable resource for defining next steps following cloud technology integration feasibility appraisal. With its comprehensive approach to adoption, the Cirrus framework offers strategic guidance on responsibly preparing for or deploying a utility cloud solution. The previously published whitepaper, “Use Case-Informed Framework for Utility Cloud Migration,” details the guiding strategy, research, and deployment of cloud solutions within electric and interconnected grid systems. Before implementing the controls suggested in this document, it is recommended that stakeholders complete Cirrus's cloud integration assessment and pair the results with their unique cybersecurity controls to form a comprehensive cloud-based utility cybersecurity plan. The Cirrus outcome will consider a series of future architectures for the grid before and after the energy transition and evaluate the arguments for and against cloud applications for each electric and interconnected grid layer. This document is a companion to the original whitepaper, "Use Case-Informed Framework for Utility Cloud Migration" to further identify and recommend security controls based on Cirrus’s cloud integration assessment output. The following whitepaper outlines the cybersecurity controls that secure cloud-service models pertinent to the electric sector using the predefined categories identify, protect, detect, and respond and recover. The objective is to outline prescriptive security controls based on the type of architecture and data stored in the cloud. The focus includes dissecting the shared responsibility model and elucidating what on-premises Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) entail. A pivotal consideration in this context is allocating responsibility for foundational cybersecurity aspects—having used Cirrus for the cloud integration assessment. The ensuing controls detailed herein also represent a checklist of controls necessary for a secure cloud transition, equipping utilities with the knowledge to navigate this digital transformation with confidence and strategic foresight in a safe and responsible manner.

42 ENGINEERING↗
DOE OSTI2024

Hybrid Storage Solution

With the rise of artificial intelligence and machine learning, data sets used to train models have become increasingly large. The availability, accessibility and integrity of large data sets has become important to the research conducted at Los Alamos National Laboratory. Ceph is a storage solution suitable for use with critical data because of its distributed nature and ability to keep multiple copies of a file in different locations. The amount of data means that bandwidth, latency, and cost are important factors and the reason most storage solutions are on-premises. However, there are distinct advantages to hosting services in the cloud, namely scalability and ease-of-use. In this paper, we explore the possibility of provisioning a hybrid Ceph cluster that leverages the benefits of both cloud architectures and on-premise performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

EDX ClaiMM

EDX ClaiMM is a centralized data & analytical platform designed to revolutionize U.S. critical minerals and materials (CMM) activities. By providing a robust digital infrastructure, ClaiMM will accelerate the combination, leveraging, and rapid utilization of vital data, advanced tools, and cutting-edge research advancements in CMM. This adaptive digital research hub connects the CMM community to essential knowledge products and offers access to interoperable datasets, databases, models, software, and tools from the National Energy Technology’s (NETL’s) Energy Data eXchange (EDX) and other authoritative sources, serving both public and private sectors. EDX ClaiMM delivers AI-informed solutions to address fundamental knowledge gaps and fosters the innovation of new techniques for enhanced characterization and recovery of CMMs within the U.S. By leveraging cloud-hosted, scalable digital infrastructure, ClaiMM meets public–private applied energy needs. It equips the CMM community with priority digital resources that harness on-site and cloud compute capabilities, enabling big data storage, advanced processing, analytics, and visualization.

Critical Materials; Critical Minerals; Rare Earth ↗
DOE OSTI2026

Grid Operator Analytics and Assessment Tools for Inverter- Based Resources Dominated Grid (GOAAT-IBR) Project Update

This presentation provides an update on the OPTIMA GOAAT project, with emphasis on the cloud-native data platform developed in-house to ingest, manage, and operationalize high-resolution power system data. Since our last NASPI presentation, accessible via OSTI ID #2671437, the project team advanced the design and deployment of a scalable architecture capable of handling both synchronized and non-synchronized streams, including PMU, point-on-wave (POW), COMTRADE, and SCADA data. These materials review the project status, recent progress, and key lessons learned. The core of the presentation examines the architecture and engineering of our cloud-native ingestion and data management platform. We then explain how pipelines were designed to collect, normalize, time-align, store, and serve heterogeneous data at scale. We will discuss design choices such as data models, streaming versus batch ingestion, storage tiers, and interoperability with analytics applications. Practical experiences with cloud-native technologies were shared during the event, including benefits, limitations, and integration challenges in a utility environment, along with methods used to improve performance, reduce latency, and optimize resource usage. The presentation also showcases user interface designs and visualization tools that convert raw measurements and analytics results into intuitive, actionable insights for operators and engineers. During the presentation examples were provided demonstrating how visualization, event views, and summarized analytics enhance situational awareness and support operational decision-making. These use cases illustrate how a well-designed data infrastructure can bridge the gap between high-volume measurements and practical grid operations.

Aminifar, Farrokh↗
DOE OSTI2024

Lessons Learned for Responsible Use of Cloud in the Cirrus Project, Following the CrowdStrike Outage Event

A disruption in CrowdStrike’s Falcon cybersecurity platform on July 19th, 2024, caused worldwide chaos. This event highlights the imperative need for cloud security measures for networks that are critically reliant on cloud technology. This incident negatively impacted air travel, government networks, and critical infrastructure sectors such as hospitals and financial institutions. While no electric utilities had a physical impact, and few had an IT impact, there were issues created by loss of cloud services, and other interrelated industries. For utilities and energy distribution organizations, understanding and mitigating these risks is essential. The Cirrus tool offers a strategic solution engineered to weave cloud integration seamlessly into the fabric of operational management, thereby enhancing resilience and streamlining efficiency in the face of digital challenges.

25 ENERGY STORAGE↗
DOE OSTI2025

Validation of Photovoltaic Modeling Tool Solargraf Against Measured Data

Solargraf is a cloud-based 3D design tool by Enphase Energy that allows users to design solar and storage systems with a variety of elements. Through a Cooperative Research and Development Agreement (CRADA), Enphase Energy collaborated with the National Renewable Energy Laboratory (NREL) to validate Solargraf's 3D design simulation against measured PV system performance. This study follows the same methodology of similar validation studies completed at NREL. The predicted performance results from simulations in both Solargraf and NREL's System Advisor Model (SAM) tool were compared with measured data to evaluate performance predictions.

14 SOLAR ENERGY↗
DOE OSTI2024

Experimental investigation of Multi-Mode heat transfer to a Free-Falling dilute particle cloud in a heated vertical tube

The development of dilute particle heat exchangers and reactors for advanced energy systems requires an understanding of the multi-mode heat transfer from a heated wall to falling particles. This study presents experimental results of the overall heat transfer coefficient for a free-falling, dilute flow of particles with solid volume fraction from 0.0005 to 0.006 corresponding to feed rates from 3.7 kg s -1 m -2 to 44 kg s -1 m -2 in a vertical, heated tube containing quiescent air at atmospheric pressure. Tube wall temperatures are varied between 300°C to 900°C while keeping the particle inlet temperature constant. The experimental results show that the overall heat transfer coefficient is a strong function of particle feed rate and surface temperature. Good agreement was obtained with prior studies conducted at comparable temperatures but lower particle feed rates (< 4 kg m -2 s -1 ). The established correlations for particle-to-wall radiation and particle-to-gas convection were used to estimate the wall-to-gas convective contribution from the measured overall heat transfer coefficient. The experimental results indicated a 4 to 6 times improvement in the wall convection in the solid-gas mixture compared to that expected from natural convection in a single-phase gas. Furthermore, the data presented here are applicable to characterize heat transfer in dilute particle heat exchangers, furnaces, and solar receivers.

14 SOLAR ENERGY↗
DOE OSTI2025

Reducing Data Center Peak Cooling Demand and Energy Costs with Underground Thermal Energy Storage (UTES)

By recent estimates, data center energy demands are projected to consume between 6.7% and 12% of U.S. annual electricity generation by the year 2028, driven primarily by expanded demands from cloud services, big data analytics, and Artificial Intelligence (AI) (Shehabi et al., 2024). As much as 40% of data center total energy consumption are loads associated with the site infrastructure cooling systems, and these are often highly water consumptive (Aljbour et al., 2024). For energy system planners, this presents significant challenges to meeting and managing the anticipated loads, and especially the peak loads of projected data center deployments. Geothermal technologies offer two unique solutions to these challenges: 1) by serving loads through the deployment of new conventional and/or next-generation geothermal power technologies such as EGS and 2) through an often-overlooked opportunity to reduce data center peak cooling loads. The latter is the focus of this paper which explores Cold Underground Thermal Energy Storage ("Cold UTES") as an emerging industrial-scale geothermal cooling solution. This cooling solution is energy efficient, non-water-consumptive, and utilizes long duration energy storage (LDES) on both diurnal and seasonal time scales. Cold UTES has the potential to also function as a virtual power plant (VPP). The US Department of Energy's Geothermal Technologies Office is supporting R&D to understand the grid and system-wide value, costs, and impacts of deploying this emergent cooling solution at scale.

AI↗
DOE OSTI2025

Workflow for Process Automation of Soil Gas Results from an Automated Soil Gas-Sampling System for Application in Carbon Storage Projects

Conference presentation at Geoconvention, Calgary, Alberta, Canada, May 12–14, 2025. The Energy & Environmental Research Center (EERC) developed an automated workflow for processing soil gas measurements collected from the automated soil gas-sampling systems deployed across the project site. Raw soil gas measurements are collected from each station every 4 hours and automatically uploaded to a cloud database. The workflow begins by writing code to download the data to a workstation automatically, then the data are published to an online dashboard that visualizes the measurements in time-series plots and a process-based decision-making framework. This automated workflow accelerates the time from data acquisition to decision-making. It supports carbon storage project operators by preparing and delivering a live, standardized dataset for quick analysis and source attribution to provide assurance of containment and overall permit compliance.

02 PETROLEUM↗
DOE OSTI2025

Workflow for Process Automation of Soil Gas Results from an Automated Soil Gas-Sampling System for Application in Carbon Storage Projects

Extended abstract for Geoconvention, Calgary, Alberta, Canada, May 12–14, 2025. The Energy & Environmental Research Center (EERC) developed an automated workflow for processing soil gas measurements collected from the automated soil gas-sampling systems deployed across the project site. Raw soil gas measurements are collected from each station every 4 hours and automatically uploaded to a cloud database. The workflow begins by writing code to download the data to a workstation automatically, then the data are published to an online dashboard that visualizes the measurements in time-series plots and a process-based decision-making framework. This automated workflow accelerates the time from data acquisition to decision-making. It supports carbon storage project operators by preparing and delivering a live, standardized dataset for quick analysis and source attribution to provide assurance of containment and overall permit compliance.

02 PETROLEUM↗
DOE OSTI2026

Data Center Market Report

The data center market is poised to explode in the coming decade due to undeniable drivers such as continued adoption of generative AI, increased data storage needs, and enterprise integration of AI in numerous industries [1] [2] [3]. Scalable power and increased computational capacity are at the forefront of considerations for hyperscalers, the major cloud service providers in this space. Lawrence Livermore National Laboratory is uniquely poised to help with informed decision making for data center market leaders during this phase of explosive expansion. National grid modeling expertise and cutting edge innovations in computer cooling systems place LLNL in an enviable position for creating economic impact in the data center industry by leveraging its expertise in these areas which can help the data center market keep up with growing demand.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Geospatial Data Workflow Orchestration and Architecture

In an era characterized by explosive growth in geospatial data, the selection of appropriate technologies for data storage, processing, and orchestration is critical for organizations aiming to maintain competitive advantages. This white paper provides a comprehensive analysis of how Oak Ridge National Laboratory (ORNL) has effectively employed various cloud technologies, including containerized applications, container orchestrators, and workflow orchestrators, to develop robust geospatial data processing solutions. We explore the fundamental concepts behind these technologies and compare multiple deployment models tailored to diverse use cases. Our findings conclude that while Kubernetes has emerged as the preferred platform for truly scalable and fault-tolerant production workflows, the choice of workflow orchestration tool requires careful consideration of team needs, pipeline complexity, and deployment environments. This paper aims to serve as a strategic guide for organizations leveraging geospatial data, articulating the balance between technology choices and practical implementation to enhance workflow efficacy and scalability.

97 MATHEMATICS AND COMPUTING↗