Search NASA⌕ Search

SEARCH · Search NASA

Results for “architecture data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Pipeline for Integrated Projects in Energy Systems (PIPES): A Tool for Integrated System Planning [Slides]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. This presentation introduces PIPES a multi-model tool for integrated system planning; it describes the underlying architecture, deep dives into common user workflows, and outlines the upcoming development roadmap beyond its current alpha state.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

Metal additively manufactured wavy fin cold-plate architecture for improved thermal-hydraulic performance

Rapid growth in artificial intelligence and data center workloads demands high-performance liquid cooling to manage increasing chip power. This study presents two metal-additive-manufactured cold plates with sinusoidal fins, constant-amplitude wavy fins and linearly variable-amplitude wavy fins and compares them against metal-additive-manufactured straight fins using experiments conducted at 1 kW heat dissipation as well as high-fidelity 3D conjugate computational fluid dynamic simulations. The cold plates were printed in AlSi10Mg material and underwent design using a Python-automated workflow prior to manufacture and testing. The experiments show that wavy fins reduce the normalized thermal resistance by 35 to 45 % at water flow rates from 1 to 4 LPM. At a fixed 20 kPa pressure drop, the variable-waviness design lowered peak surface temperature by 9 °C and thermal resistance by 51 %, while edge-channel maldistribution in the constant wavy fin design limited gains. A thermal resistance breakdown revealed that 55–63 % of the total thermal resistance in wavy designs comes from base heat conduction, 27–33 % from fin heat conduction, and 9–13 % from fin heat convection, indicating the need to address conduction bottlenecks. Parametric sweeps identify a 3 mm fin pitch as optimal, and that horizontal inlet/outlet manifolds further reduce pressure drop by 30–60 % and thermal resistance by 9–16 % relative to vertical inlet-outlet manifolds. The results yield comprehensive guidelines for fin geometry, manifold alignment, material selection and additive-manufacturing constraints to realize high-performance liquid-cooled cold plates for power-dense electronics.

3d printing↗
DOE OSTI2025

Evolution of the ATLAS TDAQ online software framework towards Phase-II upgrade: Use of Kubernetes as an orchestrator of the ATLAS Event Filter computing farm

The ATLAS experiment at the LHC at CERN continuously evolves its TDAQ system to meet the challenges of new physics goals and technological advancements. As ATLAS prepares for the Phase-II Run 4 of the LHC, significant enhancements in the TDAQ Controls and Configuration (TDAQ-CC) tools have been designed to ensure efficient data collection, processing, and management. This abstract presents the evolution of ATLAS TDAQ-CC system leading up to Phase-II Run 4. As part of the evolution towards Phase-II, Kubernetes has been chosen to orchestrate the Event Filter (EF) farm. By leveraging Kubernetes, ATLAS can dynamically allocate computing resources, scale processing capacity in response to changing data taking conditions and ensure high availability of data processing services. The integration of the Kubernetes with the TDAQ Run Control framework enables perfect synchronisation between the experiment’s data acquisition components and the computing infrastructure. We will discuss the architectural considerations and implementation challenges involved in Kubernetes integration with the ATLAS TDAQ-CC system. We will highlight the benefits of using Kubernetes as an EF farm orchestrator, including improved resource utilization, enhanced fault tolerance, and simplified deployment and management of data processing workflows. In addition, we will report on the extensive testing of Kubernetes that was conducted using a farm of 2500 servers within the experiment data taking environment, demonstrating its scalability and robustness in handling the demands of the ATLAS TDAQ system for Phase-II. The adoption of Kubernetes represents a significant step forward in the evolution of ATLAS TDAQ-CC system, aligning with industry best practices in container orchestration.

Corso Radu, Alina [Univ. of California, Irvine, CA↗
DOE OSTI2025

APOLLO: a facility-scale differentiable virtual accelerator at Fermilab FAST/IOTA

As the design complexity of modern accelerators grows, there is more interest in using advanced simulations that have fast execution time or yield additional insights like gradients. The FAST/IOTA facility has been working on implementing and experimentally validating an end-to-end digital twin that is both fast and gradient-aware, allowing for rapid prototyping of new software and experiments with minimal beam time costs. Our framework integrates physics and ML codes for linac and ring simulation through a set of generic interfaces between surrogate and physics-based sections. To reproduce device inputs and outputs, system state is exposed as a deterministic event loop in a specialized discrete event simulator architecture. Because Fermilab is undergoing control system transition, several APIs were implemented as final user interfaces - a fully asynchronous EPICS soft IOC, a gRPC-based Data Pool Manager (DPM), and legacy ACNET protocols. We discuss implementation details as well as challenges handling live data assimilation and future plans to extend modelling to main complex proton accelerators like PIPII and Booster.

Kuklev, Nikita [Fermilab]↗
DOE OSTI2024

Temporal sequence transformer to advance long-term streamflow prediction

Accurate streamflow prediction is crucial for understanding climate change impacts on water resources and for effective management of extreme hydrological events. While Long Short-Term Memory (LSTM) networks have been the dominant data-driven approach for streamflow forecasting, recent advancements in transformer architectures for time series tasks have shown promise in outperforming traditional LSTM models. This study introduces a transformer-based model that integrates historical streamflow data with climatic variables to enhance streamflow prediction accuracy. We evaluated our transformer model against a benchmark LSTM across five diverse basins in the United States. Results demonstrate that the transformer architecture consistently outperforms the LSTM model across all evaluation metrics, highlighting its potential as a more effective tool for hydrological forecasting. This research contributes to the ongoing development of advanced AI techniques for improved water resource management and climate change adaptation strategies.

Singh, Ruhaan [Farragut High School]↗
DOE OSTI2024

On a Simplified Approach to Achieve Parallel Performance and Portability Across CPU and GPU Architectures

This paper presents software advances to easily exploit computer architectures consisting of a multi-core CPU and CPU+GPU to accelerate diverse types of high-performance computing (HPC) applications using a single code implementation. The paper describes and demonstrates the performance of the open-source C++ matrix and array (MATAR) library that uniquely offers: (1) a straightforward syntax for programming productivity, (2) usable data structures for data-oriented programming (DOP) for performance, and (3) a simple interface to the open-source C++ Kokkos library for portability and memory management across CPUs and GPUs. The portability across architectures with a single code implementation is achieved by automatically switching between diverse fine-grained parallelism backends (e.g., CUDA, HIP, OpenMP, pthreads, etc.) at compile time. The MATAR library solves many longstanding challenges associated with easily writing software that can run in parallel on any computer architecture. This work benefits projects seeking to write new C++ codes while also addressing the challenges of quickly making existing Fortran codes performant and portable over modern computer architectures with minimal syntactical changes from Fortran to C++. We demonstrate the feasibility of readily writing new C++ codes and modernizing existing codes with MATAR to be performant, parallel, and portable across diverse computer architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

MatRIS: Addressing the Challenges for Portability and Heterogeneity Using Tasking for Matrix Decomposition (Cholesky)

The ubiquitous in-node heterogeneity of HPC and cloud computing platforms makes software portability and performance optimization extremely challenging. Described here, the MatRIS multilevel math library abstraction framework employs tasking to alleviate these difficulties. MatRIS includes the IRIS task-based runtime on the bottom level and exposes different layers of abstraction to render algorithms architecturally agnostic. MatRIS ensures the decomposition and creation of tasks that represent the necessary encapsulation of the optimized kernels from both vendor and open-source math libraries. Once built, MatRIS can select different combinations of accelerators at runtime, making it portable even on diverse heterogeneous architectures. By leveraging the IRIS runtime’s features for managing heterogeneity, MatRIS deploys algorithms that remove the need to specify orchestration and data transfer. This study describes how the serial task abstraction of a tiled Cholesky factorization is made portable and scalable in the case of multi-device and multi-vendor heterogeneity on a node with NVIDIA and AMD GPUs by using MatRIS. First, we demonstrate that Cholesky in MatRIS provides multi-GPU scalability that offers competitive performance versus cuSolverMG. Then, we present the challenges and opportunities for heterogeneous execution.

Monil, M. A. H.↗
DOE OSTI2024

Accelerating science: The usage of commercial clouds in ATLAS Distributed Computing

The ATLAS experiment at CERN is one of the largest scientific machines built to date and will have ever growing computing needs as the Large Hadron Collider collects an increasingly larger volume of data over the next 20 years. ATLAS is conducting R&D projects on Amazon Web Services and Google Cloud as complementary resources for distributed computing, focusing on some of the key features of commercial clouds: lightweight operation, elasticity and availability of multiple chip architectures. The proof of concept phases have concluded with the cloud-native, vendoragnostic integration with the experiment’s data and workload management frameworks. Google Cloud has been used to evaluate elastic batch computing, ramping up ephemeral clusters of up to O(100k) cores to process tasks requiring quick turnaround. Amazon Web Services has been exploited for the successful physics validation of the Athena simulation software on ARM processors. We have also set up an interactive facility for physics analysis allowing endusers to spin up private, on-demand clusters for parallel computing with up to 4 000 cores, or run GPU enabled notebooks and jobs for machine learning applications. The success of the proof of concept phases has led to the extension of the Google Cloud project, where ATLAS will study the total cost of ownership of a production cloud site during 15 months with 10k cores on average, fully integrated with distributed grid computing resources and continue the R&D projects.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Enabling Command-and-Control in Advanced In Situ Workflows

Scientific discovery is progressing towards autonomous science with the combination of scientific instruments, high-performance computing, and artificial intelligence in complex workflows. This evolution introduces new requirements for managing scientific workflows, including feedback loops, near real-time constraints, and the ability to dynamically control workflow execution. In situ workflows that analyze and visualize data as it is generated are well-suited to satisfy stringent time constraints and their iterative nature offers greater opportunities for command-and-control. However, only a few of the many workflow management systems available have been specifically designed to manage in situ workflows and often lack support for automated feedback loops that allow analysis and visualization components to interact with the main scientific data producer. To address this need, we present in this paper how to add command-and-control capabilities to a workflow management system. We identify the functional design requirements of such a command-and-control system, detail its architecture, interface, and core mechanisms, and illustrate how advanced in situ workflows can leverage command-and-control in three use cases: graceful termination with checkpoint, dynamic and adaptive data reduction, and event-triggered analysis.

Mehta, Kshitij [ORNL] (ORCID:0000000297149981)↗
DOE OSTI2024

A Microservices Architecture Toolkit for Interconnected Science Ecosystems

Microservices architecture is a promising approach for developing reusable scientific workflow capabilities for inte- grating diverse resources, such as experimental and observational instruments and advanced computational and data management systems, across many distributed organizations and facilities. In this paper, we describe how the INTERSECT Open Architec- ture leverages federated systems of microservices to construct interconnected science ecosystems, review how the INTERSECT software development kit eases microservice capability develop- ment, and demonstrate the use of such capabilities for deploying an example multi-facility INTERSECT ecosystem.

Brim, Michael↗
DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗
DOE OSTI2024

Machine Learning-Enabled Image Classification for Automated Electron Microscopy

Abstract Traditionally, materials discovery has been driven more by evidence and intuition than by systematic design. However, the advent of “big data” and an exponential increase in computational power have reshaped the landscape. Today, we use simulations, artificial intelligence (AI), and machine learning (ML) to predict materials characteristics, which dramatically accelerates the discovery of novel materials. For instance, combinatorial megalibraries, where millions of distinct nanoparticles are created on a single chip, have spurred the need for automated characterization tools. This paper presents an ML model specifically developed to perform real-time binary classification of grayscale high-angle annular dark-field images of nanoparticles sourced from these megalibraries. Given the high costs associated with downstream processing errors, a primary requirement for our model was to minimize false positives while maintaining efficacy on unseen images. We elaborate on the computational challenges and our solutions, including managing memory constraints, optimizing training time, and utilizing Neural Architecture Search tools. The final model outperformed our expectations, achieving over 95% precision and a weighted F-score of more than 90% on our test data set. This paper discusses the development, challenges, and successful outcomes of this significant advancement in the application of AI and ML to materials discovery.

Materials Science↗
DOE OSTI2024

Enhancing Discoverability and Management of Atmospheric Data at Scale: Solutions from the ARM Data Center

The Atmospheric Radiation Measurement (ARM) is a multi-laboratory and multi-institutional U.S. Department of Energy (DOE) Office of Science National User Facility. The ARM Data Center (ADC), located at Oak Ridge National Laboratory, collects, archives, and shares vast atmospheric data crucial for climate research. The ADC manages over 7 PB of data from 460 instruments worldwide, processing it into more than 11,000 diverse data products using the Network Common Data Form (NetCDF) for machine-independent accessibility. The primary challenge addressed in this paper is the efficient management and distribution of vast and diverse datasets essential for the climate research community, enhancing accessibility through advanced tools like Data Discovery. The ADC has developed advanced infrastructure and software architecture to handle the continuous influx of heterogeneous data to enhance data discoverability, resulting in increased scientific collaboration. In 2023, users from over 34 countries downloaded and utilized ARM data, resulting in 1,455 publications. The ADC’s efforts have significantly improved the discoverability and usability of atmospheric data, fostering extensive scientific research and collaboration. This paper details the solutions implemented by the ADC team for efficient data discovery and distribution, and it demonstrates ARM’s capability of staging processed data for scientific analysis.

Shah, Chirag [ORNL] (ORCID:0000000203145737)↗
DOE OSTI2024

MAPLE v.1.0

SAND2025-00659O MAPLE is a software tool that uses epigenomic data to predict gene expression. MAPLE uses a set of epigenomic modifications to determine the effect on gene expression in a specific subset of species. The algorithm can be trained on additional species and epigenomic modifications, enhancing its predictive capabilities. EAGLE employs a hybrid neural network architecture, featuring a convolutional front-end and a multi-head attention layer, to process pre-processed signal data as input. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Davis IV, Warren↗
DOE OSTI2024

Scalability of Real-time Distribution Models

This work will focus on developing the capabilities and validating the models for a sub transmission network with multiple feeders and microgrids. To achieve this scale of Hardware-in-the-loop (HitL) simulation, it is necessary to federate and collaborate. The work aims to design the large-scale feeder models to allow federation with complementary testbeds in the future. The feeder would be designed to be reconfigurable to put the system into a variety of modes. Aggregators models will be included in each distribution network’s federate to take control actions and interact with the management systems. Lastly, the feeder model will support large scale resilience studies involving complex Distributed Energy Resources (DER) controls, microgrid studies and emulation of complex data flows in future grid architectures.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Demonstrating the data center as a flexible grid asset using a C-HIL setup

Increasing data center demand is outpacing grid infrastructure development. Artificial intelligence workloads and hyperscale cloud growth are creating unprecedented demand for power, while traditional grid expansion faces multiyear development timelines. Verrus is developing an innovative datacenter solution for this challenge, data centers that act as active grid-supportive assets rather than passive loads. Our approach integrates a novel grid-aware power flow management system with battery energy storage systems(BESS) into a microgrid-controlled, medium-voltage power distribution architecture that delivers critical capabilities, such as: * Fast response to grid disturbances such over/ under voltage or over/ under frequency * Demand flexibility that can service requests from the utility within 10 s * Uninterrupted transition to islanded operation during grid outages * Continuous uptime assurance for compute loads while maintaining all customer service level agreements. Through Verrus' strategic partnership with the National Renewable Energy Laboratory (NREL), these capabilities were validated using NREL's Advanced Research on Integrated Energy Systems (ARIES) virtual emulation environment to model a 70-MW grid-interactive data center. This paper outlines the design, methodology, and results of this emulated deployment, demonstrating that data centers can provide both critical load resilience and ancillary grid support without compromising uptime requirements. Specifically, we present a digital real time simulation of a 70 MW data center integrated with a physical microgrid controller, and demonstrate the data center response in the event of a grid voltage and frequency event, utility demand response request and utility outage.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

System-Level Integration of Modular Language Models for Real-Time Risk Assessment in Third-Party Risk Management Systems

Large enterprises typically rely on dedicated teams to govern and implement security measures throughout their supply chains, ensuring compliance with enterprise security procedures. There is a significant reliance on Third-Party Risk Management (TPRM) platforms, which often require complete, highly structured information from potential vendors. The review and compliance assurance processes are time- and labor intensive, often requiring several rounds of review between the supply chain security risk management teams, business users, and potential vendors, leading to delays in the supply chain processing and consumer experience. Significant challenges in the risk management paradigm include handling unstructured data in various formats and providing real-time feedback to users to reduce the required review time. This paper presents a novel solution to these challenges. A modular multi-step system architecture is proposed using advances in language processing, specifically for unstructured responses and provides real-time feedback (i.e., 3 seconds) so that users can improve their responses before the TPSRM team review. This novel system architecture will increase information accuracy and significantly reduce time and labor during the review process.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2026

Integrated System Planning: Emerging Software Requirements in the Power Industry

Power system planning software remains fragmented across organizational boundaries, with specialized tools for capacity expansion, production cost modeling, power flow, and dynamic analysis operating on incompatible data models and assumptions. This article argues that the fragmentation is not merely a technical problem but a predictable consequence of Conway's law: software architectures mirror the departmental structures within which they are developed. Regulatory milestones like Federal Energy Regulatory Commission (FERC) Order 888 formalized these divisions, but the roots trace back to the distinct engineering disciplines-mechanical, chemical, and electrical-that staffed generation and transmission planning departments in vertically integrated utilities. As the industry moves toward integrated system planning (ISP) that coordinates generation, transmission, and distribution investment decisions, the software ecosystem must evolve accordingly. We identify five categories of software requirements to enable this transition: coherent data inputs decoupled from individual applications, unified and extensible data schemas, modular component representations that support multiple abstraction levels, lifecycle management of planning datasets, and well-defined application programming interface (API) contracts that separate data exchange from algorithmic control. We examine how these requirements interact with three common workflow patterns-serial gate clearing, sequential multiapplication, and convergence oriented-and discuss the interface design principles each demands. We then outline a vision for platform-based planning architectures where specialized analytical services compose through standardized interfaces and where artificial intelligence (AI)/machine learning (ML) tools augment decision support within a disciplined software infrastructure. The practices proposed here offer a path from today's siloed tool collections toward collaborative planning ecosystems capable of handling the complexity of modern power system transformation.

24 POWER TRANSMISSION AND DISTRIBUTION↗