SEARCH · Search NASA

Results for “hierarchical storage management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2023

I/O Access Patterns in HPC Applications: A 360-Degree Survey

The high-performance computing I/O stack has been complex due to multiple software layers, the inter-dependencies among these layers, and the different performance tuning options for each layer. In this complex stack, the definition of an “I/O access pattern” has been reappropriated to describe what an application is doing to write or read data from the perspective of different layers of the stack, often comprising a different set of features. It has become common to have to redefine what is meant when discussing a pattern in every new study, as no assumption can be made. This survey aims to propose a baseline taxonomy, harnessing the I/O community’s knowledge over the past 20 years. This definition can serve as a common ground for high-performance computing I/O researchers and developers to apply known I/O tuning strategies and design new strategies for improving I/O performance. We seek to summarize and bring a consensus to the multiple ways to describe a pattern based on common features already used by the community over the years.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Position Papers for the ASCR Workshop on the Management and Storage of Scientific Data

The purpose of this workshop is to identify priority research directions in the area of data management for high-performance and scientific computing above and beyond HPC’s traditional "the parallel file system is the data-management system" model. Supporting the breadth of the DOE mission, including the explosion of AI uses and the growing needs of experimental and observational science, motivates revisiting our assumptions about data management. There are many facets of this topic to explore including: (1) Interfaces for accessing data that resides on traditional persistent storage as well as memory devices; (2) Storage-system architecture design that supports scientific workflows on varied hierarchical storage and networking devices; (3) Devising metadata management infrastructure to support FAIR principles (Findability, Accessibility, Interoperability, and Reusability); (4) Capturing provenance information about scientific data; (5) Utilizing AI to learn I/O patterns of emerging workloads for efficient data management; (6) Providing data management support for AI and complex workflows; and (7) Understanding the overlap between traditional storage systems and I/O (SSIO) efforts and data management. While the program committee has identified these topics as important areas for discussion, we welcome position papers from the community that propose additional topics of interest for discussion at the workshop. The workshop agenda will include breakout sessions for discussing these and selected topic areas to inform priority research directions for data management for high-performance and scientific computing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Report for the ASCR Workshop on the Management and Storage of Scientific Data

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

"PoliMOR: A Policy Engine \"Made-to-Order\" for Automated and Scalable Data Management in Lustre"

Modern supercomputing systems are increasingly reliant on hierarchical, multi-tiered file and storage system architectures due to cost-performance-capacity trade-offs. Within such multi-tiered systems, data management services are required to maintain healthy utilization, performance, and capacity levels. We present PoliMOR, a pragmatic and reliable policy-driven data management framework. PoliMOR is composed of modular, single-purpose agents that gather file system metadata and enforce policies on storage systems. PoliMOR facilitates automated and scalable data management with customizable agents tailored to HPC facility-specific storage systems and policies. Our evaluations demonstrate the scalability and performance of PoliMOR both by its individual agents and as a collective entity. We believe PoliMOR is widely applicable across HPC facilities with large-scale data management challenges and will garner interest from the HPC community, given its flexible and open-source nature.

George, Anjus↗

DOE OSTI2021

EMOS (Energy Management Optimization System) [SWR-21-46]

EMOS software performs a real-time hierarchal optimal control for energy systems like multi-port electric vehicles charging site with distributed energy resources (DERs) and energy storage systems (ESSs). It gathers information in real-time from electric vehicles, power grid, and DERs, solves a multi-objective energy management optimization problem, and output setpoint for chargers, ESS converters, DERs converters, and grid converters. The software incorporates a novel integration of two control tasks: a) An Energy Management Optimization (EMO), which is the brain of EMOS controller that gathers information in real-time from EVs [e.g., battery size, state-of-charge (SOC), desired SOC, and charge acceptance curve], grid (e.g., electricity price, allowed feeder capacity, ramp rate limit, and reactive power), and DERs (e.g., prediction for solar generation for PV systems). It solves a control optimization problem in real-time to find optimal setpoint for ESSs power dispatch, EVs charging rate, and grid inverters. The objectives are to (1) minimize the charging cost considering grid energy, demand charges, and battery energy, (2) minimize charging time to meet fast charging criterion, (3) keep high energy level on ESSs at the end of an operating period, while satisfying constraints related to grid, EVs, and power converters. b) Real-Time Energy Management System (RT-EMS) is a rule-based algorithm that has faster response than EMO. It receives optimal setpoint from EMO and actual measurements from the system and modify the setpoint to compensate for any fast disturbance in the system, until a new optimum solution is received. Fast disturbances may include vehicle connect/disconnect, unpredicted variation in DERs profiles, variation in grid voltage, errors in PV generation prediction, and others. In addition, RT-EMS regulates voltage at point of common coupling (PCC) by managing reactive power of grid converters.

Mohamed, Ahmed↗

DOE OSTI2022

Hierarchical Control of Megawatt-Scale Charging Stations for Electric Trucks with Distributed Energy Resources

Electrifying medium- and heavy-duty trucks is critical to decarbonizing the transportation sector. Energy needs of electric trucks will likely require megawatt-scale charging stations, which could significantly stress the electric distribution grid. Distributed energy resources (DER) can alleviate this stress and reduce charging costs with proper management. To that end, this work develops a hierarchical predictive control algorithm for future multi-port megawatt-scale charging stations that can provide real-time energy management for stations, decide charging rates, dispatch energy storage system (ESS), and provide grid voltage support. We integrate three algorithmic components: (i) an energy management optimization (EMO) that provides supervisory control to DER assets and charging loads at minute scale, (ii) a real-time energy management system (RT-EMS) that heuristically compensates for fast disturbances at sub-second scale, and (iii) a model predictive control (MPC)-based battery management system (BMS) that communicates future charging demands to the EMO, to manage the overall megawatt-scale site. Additionally, validation in a controller hardware-in-the-loop (CHIL) environment shows that the hierarchical controller can reduce the total energy consumption from the grid by approximately 28% compared to an uncontrolled case for the station configuration in this paper, without impacting charging time.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

Solid State Power Substations (SSPS): A Multi-Hierarchical Architecture from Substation to Grid Edge

With the growing deployment of distributed generation, or power electronic interfaced renewable energy and storage technologies, the nature and behavior of the grid is changing. Synchronous machine-driven asset contributions to the generation mix are shrinking, leading to concerns regarding grid stability. Furthermore, the scale of smaller distributed PE resources needed for managing the electrical network could dwarf the existing system leading to more complex optimization problems and communication interconnections. This paper introduces the concept of a hierarchal system of controllers that spans the grid edge or the customer end to distribution scale substations or solid-state power substation (SSPS). This concept focuses on minimizing the number of interfaces and optimization considerations in the grid by clustering resources into nodes and hubs. The work validates the concept in a controller hardware-in-the-loop (cHIL) platform.

Chinthavali, Madhu Sudhan↗

DOE OSTI2021

Closing the Gap between FAIR Data Repositories and Hierarchical Data Formats

Many in the scientific community, particularly in publicly funded research, are pushing to adhere to more accessible data standards to maximize the findability, accessibility, interoperability, and reusability (FAIR) of scientific data, especially with the growing prevalence of machine learning augmented research. Online FAIR data repositories, such as the Open Science Framework (OSF), help facilitate the adoption of these standards by providing frameworks for storage, access, search, APIs, and other features that create organized hubs of scientific data. However, the wider acceptance of such repositories is hindered by the lack of support of hierarchical data formats, such as Technical Data Management Streaming (TDMS) and Hierarchical Data Format 5 (HDF5), that many researchers rely on to organize their datasets. Various tools and strategies should be used to allow hierarchical data formats, FAIR data repositories, and scientific organizations to work more seamlessly together. A pilot project at Los Alamos National Laboratory (LANL) addresses the disconnect between them by integrating the OSF FAIR data repository with hierarchical data renderers, extending support for additional file types in their framework. The multifaceted interactive renderer displays a tree of metadata alongside a table and plot of the data channels in the file. This allows users to quickly and efficiently load large and complex data files directly in the OSF webapp. Users who are browsing files can quickly and intuitively see the files in the way they or their colleagues structured the hierarchical form and immediately grasp their contents. This solution helps bridge the gap between hierarchical data storage techniques and FAIR data repositories, making both of them more viable options for scientific institutions like LANL which have been put off by the lack of integration between them.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Stress- and interface-compatible red phosphorus anode for high-energy and durable sodium-ion batteries

Sodium-ion battery is a promising candidate for energy storage application, but the absence of high-capacity and low-cost anode materials significantly limit their practical specific energy and cost. Red phosphorus (RP) possesses a high theoretical specific capacity but suffers from large volume change, low electronic conductivity and unstable solid-electrolyte interphase (SEI). Herein, a hierarchical micro/nanostructured antimony-doped RP/carbon anode was developed, which demonstrate extraordinary electrochemical performance with high initial Coulombic efficiency of ~90%, high areal (~1.7 mAh cm-2) capacity together with good cycle stability and rate capability. Combined experimental and computational studies consistently revealed that such a unique structural design can dramatically accommodate the mechanical stress and moreover effectively restrain the undesired decomposition of electrolyte solvents regardless of electrolytes formulation, resulting in superior structural integrity and thin/robust SEI formation during cycling. The present finding has offered an alternative strategy for stress management and interface engineering on high-capacity alloying-based anode materials.

Liu, Xiang↗

DOE OSTI2022

LANL Contributions to pyiron: An integrated development environment for computational materials science

To support and accelerate the development of simulation protocols in atomistic modelling, we introduce an integrated development environment (IDE) for computational materials science called pyiron (http://pyiron.org). The pyiron IDE combines a web based source code editor, a job management system for build automation, and a hierarchical data management solution. The core components of the pyiron IDE are pyiron objects based on an abstract class, which links application structures such as atomistic structures, projects, jobs, simulation protocols and computing resources with persistent storage and an interactive user environment. The simulation protocols within the pyiron IDE are constructed using the Python programming language. To highlight key concepts of this tool as well as to demonstrate its ability to simplify the implementation and testing of simulation protocols we discuss two applications. In these examples we show how pyiron supports the whole life cycle of a typical simulation, seamlessly combines ab initio with empirical potential calculations, and how complex feedback loops can be implemented. While originally developed with focus on ab initio thermodynamics simulations, the concepts and implementation of pyiron are general thus allowing to employ it for a wide range of simulation topics.

Janssen, Jan↗

DOE OSTI2023

Beneficial Integration of Energy Storage and Load Management with Photovoltaic (PV)

In recent years, a number of industry activities have aimed at addressing the integration challenges posed by the variability and uncertainty of higher penetration of renewable generation sources, like solar photovoltaic (PV) – one of the key objectives of the Sustainable and Holistic Integration of Energy Storage and Solar PV (SHINES) program launched by the U.S. Department of Energy's Office of Energy Efficiency and Renewable Energy (EERE). This EPRI led Beneficial Integration of Energy Storge, and Load Management with PV project aimed to design, develop, and demonstrate end-to-end distributed energy resource (DER) integration solution to build on these activities. EPRI led project team designed and implemented a local controller that uses model predictive control (MPC) algorithm to optimally manage DERs on site by planning for a receding horizon while executing the control settings for the first step of its plan. The team has also developed a system controller to interface with the local controller to demonstrate the hierarchical control and how it can leverage DER capabilities to address challenges like over voltage and thermal limit violations which typically limits the DER hosting capacity of distribution feeders. Team has demonstrated how the local controller with optimization algorithm can effectively manage controllable loads like HVAC, water heater, and pool pumps to allow for greater integration of PV with relatively smaller energy storage system requirements. Optimal utilization of the load control can also reduce the depth of discharge of batteries to meet grid export/import limit from behind-the-meter (BTM) DERs. Proper utilization of DER capabilities via local control intelligence, like the one developed and demonstrated in this project can help the industry to address integration challenges of higher penetration of solar PV in economically efficient manner. This can help to accelerate deployment of clean renewable energy systems at lower societal cost.

14 SOLAR ENERGY↗

DOE OSTI2023

Hierarchical Control and Stability Analysis for a Nonisolated Grid-Tied DC Energy Router Integrating Energy Storage and Partial Distributed Generation

This article proposes a nonisolated dc converter-based energy router (dc-ER) and its operating strategy. Here, the intent is to integrate energy storage (ES), distributed generation (DG), local load, and dc power grid in an autonomous and more efficient way. The ES/DG power ports are coupled with each other in partial-type connections, which obtains higher voltage supply gain and direct input–output power transmission. High-voltage supply gain allows a wide solar power tracking range and the possibility of optimal battery charging/discharging. Direct input–output power transmission improves the energy conversion efficiency. In this article, the operating modes of dc-ER are first analyzed, followed by the optimal hardware design counting the DG current ripple minimization and the limitation caused by the power flow direction. Second, the mathematical model of dc-ER is deduced. The hierarchical control is then proposed to manage port energy in a flexible manner. The stability is analyzed by using impedance modeling. Finally, experimental and simulation results verify the superiorities of the proposed topology in terms of flexible operating mode transition and high-power conversion efficiency.

25 ENERGY STORAGE↗

DOE OSTI2023

Energy Storage Sizing, Placement and Valuation for Distribution Networks

SAND2023-05682O The Energy Storage Sizing, Placement, and Valuation for Distribution software allows a user to simulate sizing, placement, and operation of one or more storage assets on a distribution grid. Among its features are measuring differences to determine the best combinations, and options for configuring and comparing results. This new simulation capability leverages the Open Distribution System Simulator and the Hierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS). Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Tamrakar, Ujjwol↗

DOE OSTI2021

All-day fresh water harvesting by microstructured hydrogel membranes

Abstract Solar steam water purification and fog collection are two independent processes that could enable abundant fresh water generation. We developed a hydrogel membrane that contains hierarchical three-dimensional microstructures with high surface area that combines both functions and serves as an all-day fresh water harvester. At night, the hydrogel membrane efficiently captures fog droplets and directionally transports them to a storage vessel. During the daytime, it acts as an interfacial solar steam generator and achieves a high evaporation rate of 3.64 kg m −2 h −1 under 1 sun enabled by improved thermal/vapor flow management. With a homemade rooftop water harvesting system, this hydrogel membrane can produce fresh water with a daily yield of ~34 L m −2 in an outdoor test, which demonstrates its potential for global water scarcity relief.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Risk-informed Hierarchical Control of Behind-the-Meter DERs with AMI Data Integration (Final Technical Report)

This project addresses several key barriers to implement the next generation demand response applications and provides a clear understanding of implementing hierarchical and standalone control using AMI data. Through this program, Eaton has developed and tested a meter-as-a-controller prototype with the help of other partners--- National Renewable Energy Laboratory (NREL), Electric Power Research Institute (EPRI), Pecan St Inc. (PSI), and Delaware Electric Cooperative (DEC). The controller can utilize residential controllable loads such as heating, ventilation, and air conditioner (HVAC), electric water heater and distributed energy resources like solar PV and battery energy storage systems for off-setting the demand that is required from the grid, thus providing reliable grid-services for demand reduction or peak shaving. The controller is also capable of coordinating the resources of the premises for better management and energy efficiency while meeting the comfort bound of the premises owner as quality-of-service. The development has been demonstrated in a three virtual-home setup at system performance lab of NREL with real appliances (HVAC, electric water heater, solar PV, and battery). The technology has also been proved through laboratory and field demonstration with successful interconnectivity (e.g., end-to-end communication and data exchange) between the residential appliances and utility through the RF network at Delaware Electric Co-op (DEC) in Delaware.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2026

Scientific Data Compression for Large Scale Computational Fluid Dynamics (CFD) Simulations

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and General Electric (GE) investigated methods for reducing the size of large computational fluid dynamics (CFD) simulation datasets using scientific data compression techniques. The work focused on adapting the MultiGrid Adaptive Reduction of Data (MGARD) compression framework and integrating it with high-performance I/O and visualization tools used in CFD workflows. MGARD uses hierarchical multilevel decomposition to enable error-controlled compression of floating-point scientific data while preserving quantities of interest. During the project, MGARD compression was integrated with the ADIOS I/O framework and visualization tools such as ParaView to enable efficient storage, transfer, and analysis of simulation data. The collaboration also explored approaches for improving compression performance for CFD data defined on unstructured meshes. Results demonstrate that scientific data compression can significantly reduce storage requirements and improve data management for large-scale CFD simulations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Multi-Timescale Optimal Operation Framework for Integrated Economic and Reliability Analysis of Hybrid Power Plants

This paper introduces a hierarchical modeling framework for hybrid power plants (HPP) to facilitate the operation of HPP in power systems similar to conventional generators (Congens) in the integrated multi-timescale optimal operation framework. To consider the uncertainties of HPP renewable power in the day-ahead scheduling, distributionally robust optimization (DRO) is used. To ensure that the state-of-charge (SOC) of energy storage systems in HPPs aligns closely with the planned value for long-term reliability, real-time SOC management is incorporated. In addition, an adjustable real-time control is designed for the robust delivery of HPP real-time services. Case studies performed on a revised IEEE 39-bus system demonstrate the effectiveness of the proposed framework for HPP operation. Simulation results highlight that the proposed framework not only can help operators schedule HPP similar to Congens in varying weather conditions but can also maintain the frequency reliability of the system.

frequency stability↗

DOE OSTI2020

ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

Scientific applications at exascale generate and analyze massive amounts of data. A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems. As such, parallel I/O, the key technology enables moving data between compute nodes and storage, faces monumental challenges from new applications, memory, and storage architectures considered in the designs of exascale systems. As the storage hierarchy is expanding to include node-local persistent memory, burst buffers, etc., as well as disk-based storage, data movement among these layers must be efficient. Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond. In this paper, we describe new capabilities we have developed in Hierarchical Data Format version 5 (HDF5), the most popular parallel I/O library for scientific applications. HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems. The state-of-the-art features we describe include: Virtual Object Layer (VOL), Data Elevator, asynchronous I/O, full-featured single-writer and multiple-reader (Full SWMR), and parallel querying. In this paper, we introduce these features, their implementations, and the performance and feature benefits to applications and other libraries.

97 MATHEMATICS AND COMPUTING↗