Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed dynamic resource management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Electric Access System Enhancement (EASE): Assessment of a Distributed Energy Resource Management System for Enabling Dynamic Hosting Capacity

The Electric Access System Enhancement (EASE) project demonstrated a scalable, interoperable, and cost-effective means of integrating high penetration of distributed energy resources (DERs). The control architecture developed leveraged a Distributed Energy Resource Management System, Distribution System Operator for transacting energy, and 3rd Party DER Aggregator platform. The project identified ways to enhance the customer interconnection process to the grid and improve access to information from DERs and optimize the usage of DERs to provide energy services in a simulated day-ahead shadow market while maintain grid reliability. By integrating these capabilities into a scalable system of systems, the DSO can effectively balance DER generation and customer demand on the distribution network. This capability allows the grid to host more DER than traditionally possible on wires alone. Hosting more DER has the added benefit of supplying increased demand growth, sometimes beyond the capacity limits of the distribution network itself. This is known as a dynamic hosting capacity, which could help utilities manage the forecasted growth in electricity demand as California switch to electric vehicles and appliances. This could help to establish energy storage and PV generation as a “pseudo firm” generation resource mix if managed appropriately and provide sufficient resource adequacy for distribution capacity upgrade deferrals.

14 SOLAR ENERGY↗
DOE OSTI2022

Design and implementation of dynamic I/O control scheme for large scale distributed file systems

In this paper, we have analyzed the input/output (I/O) activities of Cori, which is a high-performance computing system at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory. Our analysis results indicate that most users do not adjust storage configurations but rather use the default settings. In addition, owing to the interference from many applications running simultaneously, the performance varies based on the system status. To configure file systems autonomously in complex environments, we developed DCA-IO, a dynamic distributed file system configuration adjustment algorithm that utilizes the system log information to adjust storage configurations automatically. Our scheme aims to improve the application performance and avoid interference from other applications without user intervention. Moreover, DCA-IO uses the existing system logs and does not require code modifications, an additional library, or user intervention. To demonstrate the effectiveness of DCA-IO, we performed experiments using I/O kernels of real applications in both an isolated small-sized Lustre environment and Cori. Our experimental results shows that our scheme can improve the performance of HPC applications by up to 263% with the default Lustre configuration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM). Final Report

Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM) is a proposed system that will enable data from an infinite number of diverse sources to be organized and accessed from anywhere using any handheld or other computer device. The approach offers a powerful roadmap for the creation and integration of a unified knowledge base of an entire ecosystem, including its many geophysical, geographical, social, political, agricultural, energy, transportation, and cyber aspects. The resulting aggregation of data has the potential to generate an informational universe of unprecedented size that has never before been possible due to the prohibitive costs, managerial complexity, and technical barriers associated with ever-changing exponential-growth data flows. We envision that DREAM will accelerate discovery by enabling climate researchers, among other types of researchers, to manage, analyze, and visualize data from earth-scale measurements and simulations. DREAM’s success will be built on proven components that leverage existing services and resources. A key building block for DREAM will be the ESGF, chaired by Dean N. Williams. Expanding on the existing ESGF, the project will ensure that the access, storage, movement, and analysis of the large quantities of data that are processed and produced by diverse science projects can be dynamically distributed with proper resource management. Much of the Office of Science data is currently generated by multiple stand-alone facilities. DREAM can collect data accumulated from these facilities and incorporate it into a fully integrated network accessible from anywhere in the world. The result is a completely new paradigm shift for data management, analysis, and visualization enabling researchers to: Manage their calculations, data, tools, and research results; Ensure that all data are sharable, reproducible and (re)usable—accompanied by appropriate metadata describing its provenance, syntax, and semantics at creation; Advance application performance by selectively adapting APIs and services in response to scientific requirements and architectural complexities; and Provide scalable interactive resource management—navigate data and metadata at multiple levels, provide architecture-aware data integration, analysis and visualization tools. We will engage closely with DOE, NASA, and NOAA science groups working at the leading edge of computing. These engagements—in domains such as biology, climate, and hydrology—will allow us to advance disciplinary science goals and inform our development of technologies that can accelerate discovery across DOE more broadly. We will advertise and promote our technologies via dedicated workshops, tutorials, and sessions at conferences, stand-alone events with broad inter-disciplinary invitation, and engagements with leadership facilities.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Methods and Experiences for Developing Abstractions for Data-intensive, Scientific Applications

Developing software for scientific applications that require the integration of diverse types of computing, instruments, and data present challenges that are distinct from commercial software. These applications require scale, and the need to integrate various programming and computational models with evolving and heterogeneous infrastructure. Pervasive and effective abstractions for distributed infrastructures are thus critical; however, the process of developing abstractions for scientific applications and infrastructures is not well understood. While theory-based approaches for system development are suited for well-defined, closed environments, they have severe limitations for designing abstractions for scientific systems and applications. The design science research (DSR) method provides the basis for designing practical systems that can handle real-world complexities at all levels. In contrast to theory-centric approaches, DSR emphasizes both practical relevance and knowledge creation by building and rigorously evaluating all artifacts. In this work, we show how DSR provides a well-defined framework for developing abstractions and middleware systems for distributed systems. Specifically, we address the critical problem of distributed resource management on heterogeneous infrastructure over a dynamic range of scales, a challenge that currently limits many scientific applications. We use the pilot-abstraction, a widely used resource management abstraction for high-performance, high throughput, big data, and streaming applications, as a case study for evaluating the DSR activities. For this purpose, we analyze the research process and artifacts produced during the design and evaluation of the pilot-abstraction. We find DSR provides a concise framework for iteratively designing and evaluating systems. Finally, we capture our experiences and formulate different lessons learned.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Scaling Ensembles of Data-Intensive Quantum Chemical Calculations for Millions of Molecules

Deep learning models are efficient computational tools that can accelerate the inverse design of molecules with desired functional properties by generating predictions at a fraction of the time required by traditional quantum chemical approaches. To ensure that a model maintains accuracy and transferability across broad regions of the chemical space explored during the inverse design, it must be trained on massively large volumes of simulation data. This requires running large-scale ensemble quantum chemical calculations on high-performance computing (HPC) systems for data collection. However, the efficient execution of such large ensemble calculations and the management of large volumes of output data require tools that can judiciously utilize computational resources and manage metadata overhead on the file system. Therefore, we present a high-performance, scalable, ensemble management framework for performing data-intensive quantum chemical electronic structure calculations for organic molecules. This framework provides abstractions to plug different ab initio, first principles, and first principles-based semi-empirical methods and executes them efficiently at large scale on HPC systems. It dynamically distributes tasks to resources and uses tiered storage for managing large collections of files. We employed this framework to process over ten million organic molecules and generate open-source datasets that provide UV-vis absorption spectra by running time-dependent density-functional tight-binding calculations. It is the largest database containing molecular optical spectra that were simulated with quantum chemical methods in a consistent manner.

Mehta, Kshitij↗
DOE OSTI2020

How fast can one resize a distributed file system?

Efficient resource utilization becomes a major concern as large-scale distributed computing infrastructures keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and costs. However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited. In this study, we evaluate the viability of malleability as a design principle for a distributed storage system. We specifically model the minimal duration of the commission and decommission operations. To show how our models can be used in practice, we evaluate the performance of these operations in HDFS, a relevant state-of-the-art distributed file system. We show that the existing decommission mechanism of HDFS is good when the network is the bottleneck, but can be accelerated by up to a factor 3 when storage is the limiting factor. We also show that the commission in HDFS can be substantially accelerated. With the highlights provided by our model, we suggest improvements to speed both operations in HDFS. We discuss how the proposed models can be generalized for distributed file systems with different assumptions and what perspectives are open for the design of efficient malleable distributed file systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Power Grid Simulation Testbed for Transactive Energy Management Systems

To effectively engage demand-side and distributed energy resources (DERs) for dynamically maintaining the electric power balance, the challenges of controlling and coordinating building equipment and DERs on a large scale must be overcome. Although several control techniques have been proposed in the literature, a significant obstacle to applying these techniques in practice is having access to an effective testing platform. Performing tests at scale using real equipment is impractical, so simulation offers the only viable route to developmental testing at scales of practical interest. Existing power-grid testbeds are unable to model individual residential end-use devices for developing detailed control formulations for responsive loads and DERs. Furthermore, they cannot simulate the control and communications at subminute timescales. To address these issues, this paper presents a novel power-grid simulation testbed for transactive energy management systems. Detailed models of primary home appliances (e.g., heating and cooling systems, water heaters, photovoltaic panels, energy storage systems) are provided to simulate realistic load behaviors in response to environmental parameters and control commands. The proposed testbed incorporates software as it will be deployed, and enables deployable software to interact with various building equipment models for end-to-end performance evaluation at scale.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Post-Disturbance Dynamic Distribution System Restoration with DGs and Mobile Resources

Distributed generations (DGs) can act as emergency power supplies when distribution systems suffer from outages. However, the generation capabilities of DGs are generally limited by a number of factors including weather conditions, fuel limitations, etc. In this context, mobile resources that are able to reallocate resources to desired locations are regarded as important complements to conventional fixed DGs in assisting distribution system restoration. In this paper, a distribution system restoration model with DGs and mobile resources is proposed. Firstly, the dispatch and allocation of mobile resources are modeled with respect to the characteristics of the traffic network. Then the developed mobile resource models are integrated into the distribution system restoration model to co-optimize the scheduling of DGs and mobile resources. Uncertainty factors are managed by a model predictive control approach so that system operators can dynamically adjust the restoration strategy with the up-to-date information. The effectiveness of the proposed method is validated through an IEEE 13-bus test system.

distributed generations (DGs)↗
DOE OSTI2024

From Resilient and Ready to Used and Useful: Managing Temporal and Locational Uncertainty in Electrification, DER Adoption, and Climate Adaptation

Grid planning decisions involve weighing risks against benefits. The best decisions will facilitate development of electrical infrastructure that minimizes risk and maximizes benefits. With the rapidly evolving energy landscape, today's planner must discern new loads and demand cycles; embrace the operational complexity of climate risk; revise settled standards; and anticipate and manage the system impacts of distributed energy resource (DER) adoption. Only by managing the combined uncertainty of these dynamic processes can a planner hope to make effective decisions. Our analysis focuses on the management of temporal and locational uncertainty, and particularly on the risks presented to customers by the mismanagement of the factors that are the sources of these uncertainties.

climate↗
DOE OSTI2021

Coordinated Inverter Control to Increase Dynamic PV Hosting Capacity: A Real-Time Optimal Power Flow Approach

High penetrations of distributed photovoltaics (PV) could cause adverse grid impacts, such as voltage violations. The recent development in inverter technologies provides the opportunity to develop control systems to realize effective PV governance and thus to improve dynamic PV hosting capacity for distribution grids. In this paper, a novel distributed energy resource management system (DERMS) solution is proposed by adopting the real-time optimal power flow (OPF) approach for coordinated control of the distributed PV inverters. The proposed approach eliminates the dependence on load knowledge via measurement feedback correction, and it can be implemented in real time. One challenge is that the technique is sensitive to the data availability and integrity of voltage measurements. Therefore, a decentralized DERMS approach is developed by leveraging the concepts of state estimation. The framework and the effectiveness of the solution approach are numerically demonstrated on a real distribution feeder in Southern California.

14 SOLAR ENERGY↗
DOE OSTI2021

Voltage Regulation Performance Evaluation of Distributed Energy Resource Management via Advanced Hardware-in-the-Loop Simulation

This paper evaluates the performance of coordinated control across advanced distribution management systems (ADMS), distributed energy resources (DERs), and distributed energy resource management systems (DERMS) using an advanced hardware-in-the-loop (HIL) platform. This platform provides a realistic laboratory testing environment, including accurate dynamic modeling of a real-world distribution system from a utility partner, real controllers (ADMS and DERMS), physical power hardware (DERs), and standard communications protocols. One grid service—voltage regulation—is evaluated to show the performance of the coordinated grid automation system. The testing results demonstrate that the coordinated DERMS and ADMS system can effectively regulate system voltages within target operation limits using DERs. The realistic laboratory HIL testing results give utilities confidence in adopting the grid automation systems to manage DERs to achieve system-level control and operation objectives (e.g., voltage regulation). This helps utilities mitigate potential risks (e.g., instability) prior to field deployment.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Final Report for ARPA-E NODES "Real-Time Optimization and Control of Next-Generation Distribution Infrastructure" Project

The project examined next-generation power distribution systems with high levels of power- electronics-interfaced distributed energy resources (DERs) that include: renewable energy systems (RESs), energy-storage devices, electric vehicles, fuel cells, small-scale diesel generators, and controllable loads. The project developed a comprehensive distribution- network management framework that unifies real-time voltage and frequency control with network-wide energy management under an integrated framework. The aim of the project was to: i) systematically optimize network-wide operation and congruently regulate voltages and frequency in the face of high volatility induced by rapidly varying RES and uncontrolled loads; ii) facilitate high renewable integration beyond 50% on an energy basis; iii) meet power-quality requirements and operational constraints; and, iv) respect response, ramp- time, and hold-time requirements for synthetic frequency and regulating reserves via fast- and longer-term dynamic energy management.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Peak Load Management in Distribution Systems Using Legacy Utility Equipment and Distributed Energy Resources: Preprint

The ability to perform peak load management in distribution systems has several benefits for utilities, including reduced demand charges and improved reliability, efficiency, and utilization of the network infrastructure. This paper demonstrates the coordinated operation of an advanced distribution management system (ADMS) and a distributed energy resource management system (DERMS) to achieve peak load management using a realistic laboratory test bed. A commercial ADMS reduces the peak demand by reducing system voltages using a dynamic voltage regulation (DVR) application. A prototype DERMS—based on real-time optimal power flow—controls distributed battery energy storage systems to further reduce the feeder power. Results from the experiments conducted using a model of a real distribution feeder show that the coordinated operation of the ADMS and DERMS is effective in accomplishing peak load management.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2025

Optimization of distributed compute resources utilization in the CMS Global Pool

The CMS Submission Infrastructure is the primary system for managing computing resources for CMS workflows, including data processing, simulation, and analysis. It integrates geographically distributed resources from Grid, HPC, and cloud providers into federated pools managed by HTCondor and Glidein- WMS, for a total of around 500k CPU cores. This system dynamically manages workloads based on priorities defined by the collaboration. Additionally, CMS scheduling strategies must be flexible to handle multiple concurrent workloads while considering changing processing demands and resource availability from various providers.Efficient utilization of vast amounts of distributed compute resources is a key element for the success of the scientific programs of the LHC experiments. Optimizing the system is essential to maximize resource efficiency and fully utilize the distributed computing power. The CMS Submission Infrastructure team thus systematically investigates sources of inefficiency in workload scheduling to reduce their impact. In addition, a strategy of pilot overloading has been introduced to compensate for other inefficiency sources, thereby optimizing resource utilization and enhancing computational throughput.

Mascheroni, Marco [UC, San Diego (main)]↗
DOE OSTI2026

Multi-Agent Control Planes for Quantum Networks: A Scalable Architecture for Autonomous Quantum Internet Management

Quantum networks are expected to enable distributed quantum computing, secure communication, and global entanglement distribution. However, operating such networks presents significant challenges, including stochastic quantum processes, fragile entanglement resources, dynamic topology, and cross-layer control requirements. Current quantum network control architectures largely rely on centralized or hierarchical controllers inspired by classical software-defined networking (SDN). While effective for small testbeds, these approaches face scalability, latency, and reliability limitations as quantum networks grow. This paper proposes a multi-agent control plane architecture for quantum networks. In this design, intelligent software agents operate at quantum nodes, repeaters, and orchestration layers, collectively managing entanglement generation, routing, purification, and scheduling. The distributed intelligence of the agent system allows the network to adapt dynamically to quantum hardware variability and environmental noise. We argue that multi-agent systems provide significant advantages over centralized control approaches, including scalability, resilience, local autonomy, and real-time adaptation. The paper discusses architectural design principles, agent coordination mechanisms, and research challenges in deploying multi-agent control planes for the emerging quantum Internet.

Alnajjar, Anees [ORNL] (ORCID:0000000237101601)↗
DOE OSTI2020

Exploiting CRIC to streamline the configuration management of GlideinWMS factories for CMS support

GlideinWMS is a workload management and provisioning system that allows sharing computing resources distributed over independent sites. Based on the requests made by GlideinWMS frontends, a dynamically sized pool of resources is created by GlideinWMS pilot factories via pilot job submission to resource sites’ CEs. More than 400 CEs are currently serving more than ten virtual organizations through GlideinWMS, with CMS being the biggest user with 230 CEs. The complex configurations of the parameters defining resource requests, as submitted to those CEs, have been historically managed by manually editing a set of different XML files. New possibilities arise with CMS adopting the CRIC, an information system that collects, aggregates, stores, and exposes, among other things, computing resource data coming from various data providers. The paper will describe the challenges faced when CMS started to use CRIC to automatically generate the GlideinWMS factory configurations. The architecture of the prototype, and the ancillary tools developed to ease this transition, will be discussed. Finally, future plans and milestones will be outlined.

Dost, Jeffrey↗
DOE OSTI2021

Peak Load Management in Distribution Systems Using Legacy Utility Equipment and Distributed Energy Resources

The ability to perform peak load management in distribution systems has several benefits for utilities, including reduced demand charges and improved reliability, efficiency, and utilization of the network infrastructure. This paper demonstrates the coordinated operation of an advanced distribution management system (ADMS) and a distributed energy resource management system (DERMS) to achieve peak load management using a realistic laboratory test bed. A commercial ADMS reduces the peak demand by reducing system voltages using a dynamic voltage regulation (DVR) application. A prototype DERMS-based on real-time optimal power flow-controls distributed battery energy storage systems to further reduce the feeder power. Results from the experiments conducted using a model of a real distribution feeder show that the coordinated operation of the ADMS and DERMS is effective in accomplishing peak load management.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗