Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Modeling Capabilities to Quantify the Benefits of DERMS (Cooperative Research and Development Final Report)

The funds-in under the CRADA will fund a team of National Renewable Energy Laboratory (NREL) researchers to participate in Energy I-Corps (formerly known as Lab-Corps). Energy I-Corps pairs teams of researchers with industry mentors for an intensive two-month training where the researchers define technology value propositions, conduct customer discovery interviews, and develop viable market pathways for their technologies. FedIMPACT, LLC and its affiliate IP Group, Inc., will evaluate the work completed at Energy I-Corps to determine whether it would like to pursue further commercialization and development of related technologies and background intellectual property.

24 POWER TRANSMISSION AND DISTRIBUTION↗

International Cybersecurity: Capabilities and Overview [Slides]

Innovations in clean energy technology are beginning to transform electric grids around the world. It is more important than ever to understand and improve the resiliency and security of the grid against natural and human disruptions as our energy systems become more distributed, intelligent, and interconnected. Through its advanced cybersecurity technical assistance portfolio, experts at the National Renewable Energy Laboratory (NREL) work with international governments to support secure and resilient deployment of renewable energy assets and address grid interconnection challenges. Cybersecurity technical assistance is tailored to the needs of our international partners.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Powered By CADET

The Capacity Expansion Decision Support for Distribution Networks (CADET) is a Python-based library and framework for creating electrical distribution system capacity planning tools for cost-effective, reliable power delivery. It enables the creation of modular, scalable, and extensible distribution capacity planning tools by providing a high-level optimization interface, parameter and options data managers, optimization constraint and objective libraries, generalized nomenclature, a system for tracking and modifying distribution network changes, optimization solution validation, and other capabilities. This webinar will describe 1) the motivation for creating CADET, 2) key designs, and 3) several use cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Modernizing GlideinWMS Factory Monitoring with Prometheus & Grafana

Large-scale scientific experiments like CMS and DUNE rely on the distributed workload management system GlideinWMS to efficiently utilize computing resources across heterogeneous computing environments. GlideinWMS currently records Factory statistics using Round Robin Databases (RRDBs), XML, and JSON files, and these statistics are displayed via custom monitoring Web pages, thereby limiting integration with modern observability platforms. This project investigates the use of Prometheus-based instrumentation to expose Factory metrics using OpenTelemetry principles. Factory statistics related to Glidein submission and job execution are exported as Prometheus metrics through the Prometheus Python Client Library and are served via an HTTP metrics endpoint. The collected metrics are inspected using the Prometheus web-based interface and are visualized through Grafana dashboards within the Landscape monitoring infrastructure at Fermilab. This project significantly streamlines the integration of modern monitoring technologies into GlideinWMS and establishes a framework for extending observability across additional system components.

Appiah, Gideon [Grambling State U.]↗

Systems, methods and computer program products for electric grid control

Systems, methods and computer program products for maintaining and controlling distributed energy resources (DERs), such as wind, solar, water and storage renewable energy resources, within an acceptable error margin are disclosed. The systems and processes control the energy output of these DERs in real time within the error margin.

Johnson, Jay↗

An Energy Service Interface for Distributed Energy Resources

Renewable energy resources, particularly wind and solar photovoltaic, are becoming significant contributors to electric power generation. These re-sources will contribute towards achieving sustainable electric power systems. However, renewable resources will dramatically increase the demand for flexible power system operations. This paper proposes an energy service interface that will allow aggregated distributed energy resources, such as residential loads and inverter-based systems, to participate in NERC-defined smart energy reliability services. Such cyber-physical systems will increase system flexibility by ensuring match between energy supply and energy demand.Aggregation and coordinated dispatch of millions of distributed energy resources will require development of large-scale computing networks. Several smart grid interface-enabling technologies, including IEEE 2030.5, Common Smart Inverter Profile, SunSpec Modbus, and CTA 2045, are discussed. Residential loads are categorized by their static and dynamic energy characteristics to identify services in which they can participate. The business model for the energy services interface as well as probabilistic modeling for resource estimation are highlighted as future considerations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Netload Range Cost Curves for Coordinated Transmission-Distribution Planning Under DER Growth Uncertainty

The increasing penetration of distributed energy resources (DERs) requires better coordination between transmission and distribution (T&D) planning to ensure system security and cost efficiency. However, misaligned planning horizons, computational burdens, and privacy concerns hinder effective coordination, leading to either underutilized resources caused by overinvestments or reliability risks due to underinvestment. To address this challenge, we introduce netload range cost curves (NRCCs), a novel approach for managing long-term DER growth uncertainty through T&D coordination, while preserving existing data-sharing and regulatory structures. NRCCs provide pairs of (i) peak substation netload guarantees and (ii) corresponding distribution upgrade options and costs, enabling their seamless integration into transmission planning workflows. To compute NRCCs efficiently, we develop a transmission-aware distribution network planning (TADNP), which is subsequently integrated to an iterative computation procedure. These NRCCs are then embedded into an NRCC-informed transmission planning model to enable resource-efficient coordination. We illustrate our proposed approach with a case study based on realistic distribution and transmission systems in the San Francisco Bay Area, California. Our results indicate the possibility of dramatic savings in transmission investments by incorporating the proposed NRCC-integrated T&D coordination framework.

Li, Yujia↗

Continuous and Time-Domain Coherent Signal Conversion between Optical and Microwave Frequencies

A quantum network consisting of computational nodes connected by high-fidelity communication channels could expand information-processing capabilities significantly beyond those of classical networks. Superconducting qubits hold promise for scalable and high-fidelity quantum computation at microwave frequencies but must operate in an isolated cryogenic environment, obviating the potential for practical long-range communication. Quantum communication has, however, been demonstrated with optical photons. A fast efficient quantum-coherent interface between superconducting qubits and optical photons would provide a key resource for a large-scale quantum network or distributed quantum computer. Here, we describe the design and experimental operation of a device incorporating a silicon optomechanical nanobeam combined with an aluminum-nitride-based electromechanical transducer. We experimentally demonstrate classical continuous-wave operation of this device at room temperature with external conversion efficiencies of (2.5 +/- 0.4) x 10 -5 (microwave to optical) and (3.8 +/- 0.4) x 10 -5 (optical to microwave), corresponding to internal efficiencies of 2.4% and 3.7%, respectively. Finally, this device also has a larger bandwidth than previous efficient microwave-optical transducers, allowing us to operate in the time domain with 20-ns pulses.

74 ATOMIC AND MOLECULAR PHYSICS↗

Toward a smart data transfer node

Scientific computing systems are becoming significantly more complex, with distributed teams and complex workflows spanning resources from telescopes and light sources to fast networks and Internet of Things sensor systems. In such settings, no single, centralized administrative team and software stack can coordinate and manage all resources used by a single application. Indeed, we have reached a critical limit in manageability using current human-in-the-loop techniques. We therefore argue that resources must begin to respond automatically, adapting and tuning their behavior in response to observed properties of scientific workflows. Over time, machine learning methods can be used to identify effective strategies for autonomic, goal-driven management behaviors that can be applied end-to-end across the scientific computing landscape. Using the data transfer nodes that are widely deployed in modern research networks as an example, we explore the architecture, methods, and algorithms needed for a smart data transfer node to support future scientific computing systems that self-tune and self-manage.

97 MATHEMATICS AND COMPUTING↗

WindWatts Computational Framework and Web UI [SWR-20-100]

This software provides a collection of algorithms, an API, and a functional Web UI for the Distributed Wind’s WindWatts project. DW WindWatts is a DOE WETO-funded project aimed at the development of tools to supporting the distributed wind industry, particularly with respect to siting and resource assessment. The computational framework and the back-end API are powering the easy to use front-end services available at: https://windwatts.nrel.gov. https://github.com/NREL/dw-tap-api; https://github.com/NREL/dw-tap; https://github.com/NREL/windwatts-data For reference, this software was previously known as DW TAP Computational Framework.

Phillips, Caleb↗

Sample Complexity of Power System State Estimation Using Matrix Completion

In this paper, we propose an analytical framework to quantify the amount of data samples needed to obtain accurate state estimation in a power system - a problem known as sample complexity analysis in computer science. Motivated by the increasing adoption of distributed energy resources into the distribution-level grids, it becomes imperative to estimate the state of distribution grids in order to ensure stable operation. Traditional power system state estimation techniques mainly focus on the transmission network which involve solving an overdetermined system and eliminating bad data. However, distribution networks are typically underdetermined due to the large number of connection points and high cost of pervasive installation of measurement devices. In this paper, we consider the recently proposed state-estimation method for underdetermined systems that is based on matrix completion. In particular, a constrained matrix completion algorithm was proposed, wherein the standard matrix completion problem is augmented with additional equality constraints representing the physics (namely power-flow constraints). We analyze the sample complexity of this general method by proving an upper bound on the sample complexity that depends directly on the properties of these constraints that can lower number of needed samples as compared to the unconstrained problem. To demonstrate the improvement that the constraints add to state estimation, we test the method on a 141-bus distribution network case study and compare it to the traditional least squares minimization state estimation method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Sample Complexity of Power System State Estimation using Matrix Completion: Preprint

In this paper, we propose an analytical framework to quantify the amount of data samples needed to obtain accurate state estimation in a power system-a problem known as \emph{sample complexity} analysis in computer science. Motivated by the increasing adoption of distributed energy resources into the distribution-level grids, it becomes imperative to estimate the state of distribution grids in order to ensure stable operation. Traditional power system state estimation techniques mainly focus on the transmission network which involve solving an overdetermined system and eliminating bad data. However, distribution networks are typically \emph{underdetermined} due to the large number of connection points and high cost of pervasive installation of measurement devices. In this paper, we consider the recently proposed state-estimation method for underdetermined systems that is based on \emph{matrix completion}. In particular, a \emph{constrained} matrix completion algorithm was proposed, wherein the standard matrix completion problem is augmented with additional equality constraints representing the physics (namely power-flow constraints). We analyze the sample complexity of this general method by proving an upper bound on the sample complexity that depends directly on the properties of these constraints that can lower number of needed samples as compared to the unconstrained problem. To demonstrate the improvement that the constraints add to state estimation, we test the method on a 141-bus distribution network case study and compare it to the traditional least squares minimization state estimation method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Big Biomedical data as the key resource for discovery science

Modern biomedical data collection is generating exponentially more data in a multitude of formats. This flood of complex data poses significant opportunities to discover and understand the critical interplay among such diverse domains as genomics, proteomics, metabolomics, and phenomics, including imaging, biometrics, and clinical data. The Big Data for Discovery Science Center is taking an “-ome to home” approach to discover linkages between these disparate data sources by mining existing databases of proteomic and genomic data, brain images, and clinical assessments. In support of this work, the authors developed new technological capabilities that make it easy for researchers to manage, aggregate, manipulate, integrate, and model large amounts of distributed data. Guided by biological domain expertise, the Center’s computational resources and software will reveal relationships and patterns, aiding researchers in identifying biomarkers for the most confounding conditions and diseases, such as Parkinson’s and Alzheimer’s.

Alzheimer's disease (ID)↗

Resilient Information Architecture Platform for Smart Grid (RIAPS)

A number of emerging trends will substantially alter the operation and control of the electric grid over the next several decades. These trends include ensuring resiliency under severe weather events, increasing integration of renewable electricity generation, supporting changing electricity demand patterns, and the improving cost effectiveness of distributed energy resources. To address these challenges, the future “Smart Grid” management will need to transition from centralized to coordinated distributed control paradigm. Reliable operation of the Smart Grid depends on distributed intelligence realized through software applications that run on distributed computing devices attached to the power system to collect data and collaboratively manage resources. However, much of the existing software for Smart Grid-enabled devices is either proprietary or developed with custom solutions, which limits interoperability among the heterogeneous devices and hinders the ability to manage system-level reliability, security, and resiliency requirements. Additionally, this approach makes Smart Grid applications hard to maintain, evolve, verify, and replace; resulting in high development and deployment costs. Further development of the Smart Grid requires a reusable software base-layer to move from hard-coded functionality to a plug-and-play architecture capable of managing system-level objectives and constraints in addition to providing consistent common services across heterogeneous devices and applications. Vanderbilt University, in collaboration with North Carolina State University and Washington State University has developed a foundation ‘software platform’ for developing and deploying robust, reliable, effective and secure software applications for the Smart Grid. The Resilient Information Architecture Platform for the Smart Grid (RIAPS) provides core services for building effective and powerful smart grid applications. It offers unique services for real-time data dissemination, fault tolerance, and coordination across apps distributed over the network.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗

Is it Worth Relaxing Fault Tolerance to Speed Up Decommission in Distributed Storage Systems?

Efficient resource utilization is a major concern for large-scale computer platforms. One method used to lower energy consumption and operational cost is to reduce the amount of idle resources. This can be achieved by using malleability, namely, the possibility for resource managers to dynamically increase or decrease the amount of resources of jobs while they are running. Decommissioning (i.e., removing from the cluster) the idle nodes as soon as possible allows the resource manager to quickly reallocate those nodes to other jobs. Challenges appear when such nodes host part of a distributed storage system. Such storage systems may need to transfer large amounts of data before releasing the nodes, in order to ensure data availability and a certain level of fault tolerance. In this paper, we model and evaluate the performance of the decommission operation when relaxing the level of fault tolerance (i.e., the number of replicas) during this operation. Intuitively, this is expected to reduce the amount of data transfers needed before nodes are released, and thus allow nodes to be returned to the resource manager faster. We quantify theoretically how much time and resources are saved by such a fast decommission strategy compared with a standard decommission that does not temporarily reduce the fault-tolerance level. We establish lower bounds for the duration of the different phases of a fast decommission. We use the lower bounds to estimate when fast decommission would be useful to reduce the usage of core-hours and when not. We implement a prototype for fast decommission and experimentally validate the lower bounds on the duration of the operation and confirm in practice our theoretical findings.

Decommission↗

DGaaS: GPU as a Service on Distributed Computing System

In the rapidly evolving landscape of scientific computing, Graphics Processing Units (GPUs) have become indispensable for their unparalleled ability to handle parallel tasks in complex calculations, simulations, and data analysis. Their utility is further magnified in machine learning and AI applications, where they significantly accelerate model training and predictive analytics. Within this context, the Triton Inference Server emerges as a pivotal open-source tool, specializing in AI inferencing and optimizing GPU utilization across various platforms and frameworks. This paper presents an in-depth study on distributed High Throughput Computing (HTC), specifically focusing on the HTCondor framework and its resource provisioning tools, GlideinWMS and HEPCloud. These systems enable large-scale scientific experiments like CMS and DUNE to efficiently access and utilize vast computational resources. The paper explores the core architectural components of GlideinWMS, including jobs, user pools, and worker nodes, and discusses their integration with GPUs and the Triton server. The primary aim of this research is to develop a solution that optimizes GPU utilization by leveraging Glideins and containers. This approach allows computational jobs, particularly those involving AI models, to use GPUs only when essential, thereby facilitating efficient sharing of limited GPU resources. To validate this architecture, the study conducted three key tests involving custom scripts, container-based servers, and Triton server deployments. However, the study faces challenges, notably in locating the Triton server and ensuring secure remote access. To address these issues, future work will focus on developing a proxy mechanism and enhancing security protocols. In conclusion, this study offers a comprehensive roadmap for effective and efficient GPU utilization in distributed High Throughput Computing. It aims to contribute significantly to the scientific community by solving pressing problems and implementing robust solutions in collaboration with the GlideinWMS and HEPCloud teams. The research sets the stage for a more efficient, scalable, and cost-effective paradigm in scientific computing.

97 MATHEMATICS AND COMPUTING↗