Results for “Distributed Software Development” — Page 3

DOE OSTI2019

Spack-Based Packaging and Development for HEP

The art event-based analysis framework is used by multiple Intensity Frontier and other experiments and projects in High Energy Physics (HEP). A system of tools and scripts based around a Fermilab-originated package management system called UNIX Product Support (UPS) has evolved to provide development, packaging, release management, and distribution of the art suite, related experimental software and external dependencies in relocatable binary form. The existing system has limitations in environments such as macOS with System Integrity Protection (SIP) enabled and the various High Performance Computing (HPC) systems that the field of HEP is increasingly looking to leverage. This has led to a search for other ways of providing experiments with a packaging and build system that meets their needs. We describe our efforts to develop a packaging and release management system based on the Spack HPC package management tool to replace the existing UPS-based system. We additionally describe a companion system: SpackDev and cetmodules, allowing simultaneous development of multiple HEP software packages in a consistent environment. A successful implementation of the intended system would have applicability across HEP rather than merely to the users of the art framework.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2020

CyDER: A Cyber Physical Co-simulation Platform for Distributed Energy Resources in Smartgrids

The CyDER project aimed at developing an open-source, modular and scalable co-simulation platform for power grids with large shares of Distributed Energy Resources (DERs). The project partners are the Lawrence Berkeley National Lab (LBNL), Lawrence Livermore National Lab (LLNL), PG&E, SolarCity, and ChargePoint. The prime recipient is LBNL; SolarCity and ChargePoint were partners for the project’s first two years. Increased DER integration introduces a number of challenges in power grid operation including a more dynamic interaction between the transmission grid and distribution grids, and increased modeling complexity. Although specialized software exists to precisely model different components of the power system, it is far from trivial to integrate all various models and perform a holistic simulation. Instead of replicating all models in a common simulation program, a commonly accepted approach to tackle this model diversity is to couple third-party simulators and models through a co-simulation platform that coordinates information exchange among the various components. Following this line of research, this project’s objective was to develop a co-simulation platform based on a widely accepted industrial standard called Functional Mockup Interface (FMI). Within this process, the project developed models compliant with the FMI standard, called Functional Mockup Units (FMUs), and used them to perform various operational and planning power system analyses. Relying and building upon an industrial standard is the main differentiation of this project compared with previous or parallel efforts in the co-simulation area. Particular emphasis was put on delivering software utilities to facilitate setting up and running co-simulations by end-users. Furthermore, a strong aspect of this project is demonstrating that co-simulation techniques can be used to perform Hardware-in-the-Loop (HIL) simulations that couple software components (e.g., simulated models) with hardware components (e.g., real devices such PV systems and batteries). The long-term goal of CyDER project is to help establish FMI as a powerful standard for co-simulation and promote adoption by electric utilities and other interested stakeholders. The main accomplishments of the project include the development of several FMUs including distribution and transmission grid models, PV inverters with Volt/Var/Watt controllers, batteries, and predictive optimal controllers. Additionally, a unique software package was developed, called SimulatorToFMU, which is capable of exporting any Python-driven simulator or Python script as an FMU. This is an important contribution towards establishing FMI as one of the main co-simulation standards, because more and more third-party programs for sub-system modeling and simulation are delivered with Python APIs. The CyDER platform was used to perform PV hosting capacity analyses in real utility feeders with and without smart inverter controls, battery storage, and EV charging. Smart inverter controls include conventional Volt/Var/Watt controls for reactive power support and active power curtailment, but also predictive controls that optimize the charging and discharging profile of the battery connected on the DC side in order to minimize the customer’s economic benefit. Finally, an important result of this project is delivering an experimental setup that consists of residential-scale PV inverters with battery storage, a real-time grid simulator with an ideal voltage source as grid emulator, and micro Phasor Measurement Units (PMUs). All these components and additional software modules are coupled to one another using the FMI standard and can be co-simulated with the CyDER platform.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Grid Architecture Mapping to Understand Transformation (GAMUT): Concept Definition

The electric power system is undergoing significant transformation driven by changes in the generation mix, increasing reliance on information and communication technologies, evolving customer expectations, and a dynamic cyber-physical environment. To address these challenges, substantial investments will be made over the next decade to create a flexible, affordable, secure, reliable, and resilient electric system. In response, the U.S. Department of Energy's Office of Electricity is developing knowledge management tools aimed at systematically documenting technology pilots and demonstration projects. This initiative seeks to improve understanding of operating contexts, planning steps, and integration requirements for innovative grid technologies. The Department of Energy project leverages Grid Architecture principles to collect and organize insights into the interdependencies, requirements, and capabilities of various grid solutions. By employing a systematic approach, the project aims to perform a feasibility study for the development of a software tool that will support stakeholders, including regulators, transmission and distribution system operators, distributed energy resources aggregators, and technology providers. By offering a systematic, software-based framework to document technologies, understand their benefits and impacts, and inform deployment strategies, the proposed Grid Architecture Mapping to Understand Transformation (GAMUT) Tool is intended to reduce costs, enhance decision-making, and facilitate scaling and effective deployment of new technologies. The feasibility study will evaluate the technical and financial viability of the GAMUT Tool, identifying stakeholder needs and assessing risks and mitigation strategies. Ultimately, the GAMUT team aims to develop an initial minimal viable product, followed by staged improvements, to aid in the modernization of the electric grid, contributing to a more sustainable and resilient energy future.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Performance-Aligned LLMs for Generating Fast HPC Code

Optimizing scientific software is a difficult task because codebases are often large and complex, and performance can depend upon several factors including the algorithm, its implementation, and hardware among others. Causes of poor performance can originate from disparate sources and be difficult to diagnose. Recent years have seen a multitude of work that use large language models (LLMs) to assist in software development tasks. However, these tools are trained to model the distribution of code as text, and are not specifically designed to understand performance aspects of code. In this work, we introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. This allows us to build upon the current code modeling capabilities of LLMs and extend them to generate better performing code. Here, we demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks from 0.9 to 1.6 for serial code and 1.9 to 4.5 for OpenMP parallel code.

Computer science↗

DOE OSTI2021

VTK-m User's' Guide (V.1.7)

High-performance computing relies on ever finer threading. Advances in processor technology include ever greater numbers of cores, hyperthreading, accelerators with integrated blocks of cores, and special vectorized instructions, all of which require more software parallelism to achieve peak performance. Traditional visualization solutions cannot support this extreme level of concurrency. Extreme scale systems require a new programming model and a fundamental change in how we design algorithms. To address these issues we created VTK-m: the visualization toolkit for multi-/many-core architectures. VTK-m supports a number of algorithms and the ability to design further algorithms through a top-down design with an emphasis on extreme parallelism. VTK-m also provides support for finding and building links across topologies, making it possible to perform operations that determine manifold surfaces, interpolate generated values, and find adjacencies. Although VTK-m provides a simplified high-level interface for programming, its template-based code removes the overhead of abstraction. VTK-m simplifies the development of parallel scientific visualization algorithms by providing a framework of supporting functionality that allows developers to focus on visualization operations. Consider the listings in Figure 1.1 that compares the size of the implementation for the Marching Cubes algorithm in VTK-m with the equivalent reference implementation in the CUDA software development kit. Because VTK-m internally manages the parallel distribution of work and data, the VTK-m implementation is shorter and easier to maintain. Additionally, VTK-m provides data abstractions not provided by other libraries that make code written in VTK-m more versatile.This book includes contributions from the VTK-m community including the VTK-m development team and the user community.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

The VTK-m Users' Guide (V.1.9)

High-performance computing relies on ever finer threading. Advances in processor technology include ever greater numbers of cores, hyperthreading, accelerators with integrated blocks of cores, and special vectorized instructions, all of which require more software parallelism to achieve peak performance. Traditional visualization solutions cannot support this extreme level of concurrency. Extreme scale systems require a new programming model and a fundamental change in how we design algorithms. To address these issues we created VTK-m: the visualization toolkit for multi-/many-core architectures. VTK-m supports a number of algorithms and the ability to design further algorithms through a top-down design with an emphasis on extreme parallelism. VTK-m also provides support for finding and building links across topologies, making it possible to perform operations that determine manifold surfaces, interpolate generated values, and find adjacencies. Although VTK-m provides a simplified high-level interface for programming, its template-based code removes the overhead of abstraction. VTK-m simplifies the development of parallel scientific visualization algorithms by providing a framework of supporting functionality that allows developers to focus on visualization operations. Consider the listings in Figure 1.1 that compares the size of the implementation for the Marching Cubes algorithm in VTK-m with the equivalent reference implementation in the CUDA software development kit. Because VTK-m internally manages the parallel distribution of work and data, the VTK-m implementation is shorter and easier to maintain. Additionally, VTK-m provides data abstractions not provided by other libraries that make code written in VTK-m more versatile.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

The VTK-m Users' Guide (V.2.0)

High-performance computing relies on ever finer threading. Advances in processor technology include ever greater numbers of cores, hyperthreading, accelerators with integrated blocks of cores, and special vectorized instructions, all of which require more software parallelism to achieve peak performance. Traditional visualization solutions cannot support this extreme level of concurrency. Extreme scale systems require a new programming model and a fundamental change in how we design algorithms. To address these issues we created VTK-m: the visualization toolkit for multi-/many-core architectures. VTK-m supports a number of algorithms and the ability to design further algorithms through a top-down design with an emphasis on extreme parallelism. VTK-m also provides support for finding and building links across topologies, making it possible to perform operations that determine manifold surfaces, interpolate generated values, and find adjacencies. Although VTK-m provides a simplified high-level interface for programming, its template-based code removes the overhead of abstraction. VTK-m simplifies the development of parallel scientific visualization algorithms by providing a framework of supporting functionality that allows developers to focus on visualization operations. Consider the listings in Figure 1.1 that compares the size of the implementation for the Marching Cubes algorithm in VTK-m with the equivalent reference implementation in the CUDA software development kit. Because VTK-m internally manages the parallel distribution of work and data, the VTK-m implementation is shorter and easier to maintain. Additionally, VTK-m provides data abstractions not provided by other libraries that make code written in VTK-m more versatile.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

A Review of Software for Designing and Operating Quantum Networks

Quantum networks development is crucial to realizing a production-grade network that can support distributed sensing, secure communication, and utility-scale quantum computation. However, the transition from laboratory demonstration to deployable networks requires software implementations of architectures and protocols tailored to the unique constraints of quantum systems. This paper reviews the current state of software implementations for quantum networks, organized around a three-plane abstraction of infrastructure, logical, and control/service planes. We cover software for both designing quantum network protocols (e.g., SeQUeNCe, QuISP, and NetSquid) and operating testbeds, with a focus on essential control/service plane functions such as entanglement, topology, and resource management, in a proposed taxonomy. Our review highlights a persistent gap between theoretical architecture and protocol proposals and their realization in simulators or testbeds, particularly in dynamic topology and network management. We conclude by outlining open challenges and proposing a roadmap for developing scalable software architectures to enable hybrid, large-scale quantum networks.

Network Design↗

DOE OSTI2021

Robust Distributed State Estimator for Interconnected Transmission and Distribution Networks (Final Report RPPR-1)

This project’s objective is to develop a combined transmission and distribution state estimator which accounts for very large system size and model complexity (by way of distributing the computations) and large number of solar PV units connected to the distribution system on multiple feeders. The project not only provides a robust formulation and solution to this problem but also tests the solution by implementing it on a well-established large utility system. It introduces several improvements with respect to the state of the art in existing state estimation software: (a) The developed state estimator (SE) allows robust and accurate monitoring of bidirectional flows in distribution systems which result due to the distributed energy sources which are not observable and thus not incorporated in generation dispatch; (b) Large utility systems with tens of thousands of transmission buses and hundreds of thousands of distribution nodes are difficult to model as a single integrated system. This shortcoming is addressed by developing a “scalable distributed computational framework” which allows splitting the ultra large system models into several small subsystems and coordinating their solution by a robust and practical state estimation formulation; (c) Measurement errors irrespective of their locations are detected and removed by the developed state estimator. Historically, transmission and distribution systems were analyzed and operated as two independent systems. Given the non-transposed short feeder sections, unevenly loaded phases, strictly radial configuration and unidirectional power flows in the absence of remote generation, distribution system analysis was customized to account for these characteristics. However, some of these assumptions are no longer valid (non-radial configuration, bidirectional power flows) and thus distribution system analysis should be revisited. Furthermore, in the past, the interaction between the transmission and distribution systems was quite passive, where distribution substations were modeled as lumped loads in the transmission system model. With substantial generation injected by renewable generation located in the distribution systems, such modeling will no longer be accurate. The developed state estimator facilitates proper monitoring of the interactions between the transmission and distribution systems and enables smart dispatch of these units which are made observable by the state estimator.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

A Modern Ethernet Data Acquisition Architecture for Fermilab Beam Instrumentation

The Fermilab Accelerator Division, Instrumentation Department is adopting an open-source framework to replace our embedded VME-based data acquisition systems. Utilizing an iterative methodology, we first moved to embedded Linux, removing the need for VxWorks. Next, we adopted Ethernet on each data acquisition module eliminating the need for the VME backplane in addition to communicating with a rack mount server. Development of DDCP (Distributed Data Communications Protocol), allowed for an abstraction between the firmware and software layers. Each data acquisition module was adapted to read out using 1 GbE and aggregated at a switch which up linked to a 10 GbE network. Current development includes scaling the system to aggregate more modules, to increase bandwidth to support multiple systems and to adopt MicroTCA as a crate technology. The architecture was utilized on various beamlines around the Fermilab complex including PIP2IT, FAST/IOTA and the Muon Delivery Ring. In summary, we were able to develop a data acquisition framework which incrementally replaces VxWorks & VME hardware as well as increases our total bandwidth to 10 Gbit/s using off the shelf Ethernet technology.

43 PARTICLE ACCELERATORS↗

DOE OSTI2023

Qthreads Support for MPICH

SAND2024-00944O Qthread Support for MPICH is software that provides additions needed to enable the use of the Qthreads library. The high-performance message passing interface (MPICH) is an open-source implementation of MPI mainly developed and distributed by Argonne National Laboratory. Qthreads is a lightweight, user-level threading library developed and distributed by Sandia National Laboratories. MPICH currently supports Posix threads, Windows threads, and Argobots. This software enables parallel programs built with the MPICH implementation of MPI to use Qthreads user-level threads rather than Posix system-level threads. The software uses existing infrastructure in MPICH to interface to the Qthreads library. The existing interfaces in MPICH allow for creating, destroying, and managing the execution of multiple threads within a process and this software translates these calls to the equivalent Qthreads library functions. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Ciesko, Jan↗

DOE OSTI2026

ACDC (Automated Campbell Diagram Code) [SWR-26-042]

This application provides a web-based graphical user interface to generating Campbell Diagrams and visualizing mode shapes for OpenFAST turbine models. Determining the aeroelastic stability and dynamic characteristics of wind turbines is a critical step in turbine design and analysis. Historically, extracting natural frequencies and mode shapes from OpenFAST—the industry-standard whole-turbine simulation code—has been a fragmented and tedious process. It required manual model configuration, command-line linearization execution, and complex post-processing via proprietary scripts to handle rotating-frame dynamics. To address these workflow bottlenecks, we present the Automated Campbell Diagram Code (ACDC), an open-source graphical software tool developed by the National Laboratory of the Rockies (NLR) under the DOE-funded Distributed Wind Aeroelastic Modeling (dWAM) project. ACDC streamlines the end-to-end linearization and stability analysis workflow into a single, intuitive cross-platform application. The software guides users through OpenFAST model configuration, definition of operating points, and the automated execution of steady-state trim and linearization simulations. Under the hood, ACDC automates the complex mathematical post-processing steps required for rotating systems, including Multi-Blade Coordinate (MBC) transformations, eigenanalysis, and advanced modal tracking utilizing the Modal Assurance Criterion (MAC) and spectral clustering. Finally, ACDC processes these results to automatically generate Campbell diagrams and features a robust 3D visualization engine to animate full-system mode shapes. By eliminating the reliance on external post-processing environments and manual data manipulation, ACDC significantly accelerates dynamic analysis and lowers the barrier to entry for wind energy researchers and engineers.

Summerville, Brent [National Laboratory of the Roc↗

DOE OSTI2019

New method for analysis of X-ray computed tomography scans of TRISO fuel forms

X-ray computed tomography (XCT) is a powerful characterization tool that has been used for examination of both tristructural-isotropic (TRISO) coated particles and fuel forms. Rapid, nondestructive imaging of complete specimens by XCT provides significant characterization advantages in both speed and comprehensiveness over traditional materialographic methods. This paper discusses a method that has been developed to analyze XCT images of pebble fuel forms containing ~18,000 TRISO particles each. The XCT images are three-dimensional with contrast based on relative x-ray transparency. In the case of TRISO fuel forms, XCT image intensity distinguishes between the high-density kernel, the medium-density SiC, and the low-density matrix and carbon layers. Image analysis software has been developed to process XCT data sets to extract spatial distribution information, including particle nearest-neighbor distances, local packing fraction, and fuel-free zone thickness in TRISO fuel forms. Finally, although this type of high-resolution XCT analysis is likely not feasible as a quality control measurement for commercial TRISO fuel, it is highly useful in generating feedback for the development of fuel form fabrication processes.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2024

Distribution Integration Solution Cost Options (DISCO) [Slides]

DISCO is a Python-based NREL software tool for conducting scalable, repeatable distribution analyses. Although DISCO was originally developed to support photovoltaic impact analyses, it can also be used to understand the impact of other distributed energy resources and load changes on distribution systems. This presentation took place on April 9, 2024, and included NREL power grid researchers Sherin Ann Abraham and Shibani Ghosh.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2021

Hardware-in-the-Loop Evaluation of an Advanced Distributed Energy Resource Management Algorithm

This paper presents the laboratory performance evaluation of voltage regulation under a new distributed energy resource management system (DERMS) algorithm via an advanced hardware-in-tbe-loop (HIL) platform. The HIL platform provides realistic testing in a laboratory environment, including the accurate modeling of a full-scale real-world distribution system from a utility partner, the DERMS software controller, and power hardware photovoltaic (PV) inverters. The new DERMS algorithm is developed based on online multiobjective optimization (OMOO) algorithms that perform fast dispatch of distributed solar PV simulated in a real-time digital simulator and real physical hardware devices. Experimental tests confirm the correct functioning of the HIL platform for evaluating controller algorithms and satisfactory voltage regulation performance of the developed OMOO algorithms.

41 EE - Solar Energy Technologies Office (EE-4S)↗

DOE OSTI2021

Hardware-in-the-Loop Evaluation of an Advanced Distributed Energy Resource Management Algorithm: Preprint

This paper presents the laboratory performance evaluation of voltage regulation under a new distributed energy resource management system (DERMS) algorithm via an advanced hardware-in-tbe-loop (HIL) platform. The HIL platform provides realistic testing in a laboratory environment, including the accurate modeling of a full-scale real-world distribution system from a utility partner, the DERMS software controller, and power hardware photovoltaic (PV) inverters. The new DERMS algorithm is developed based on online multiobjective optimization (OMOO) algorithms that perform fast dispatch of distributed solar PV simulated in a real-time digital simulator and real physical hardware devices. Experimental tests confirm the correct functioning of the HIL platform for evaluating controller algorithms and satisfactory voltage regulation performance of the developed OMOO algorithms.

41 EE - Solar Energy Technologies Office (EE-4S)↗

DOE OSTI2020

An Extensible Software and Communication Platform for Distributed Energy Resource Management

This paper introduces a novel Distributed Extensible Grid Control (DEGC) software and communication platform to facilitate the control of distributed energy resources on electric grids. The DEGC software platform leverages state-of-the-art advances in secure, distributed communication and decentralized authorization and authentication. We discuss how these advances enable the kind of robust and secure communication required for a distributed grid control platform, and show how DEGC applies these technologies to the agile development and deployment of grid software through an extensible and flexible API. Here, we describe how DEGC can implement both Volt-VAR voltage magnitude control and Phasor-Based Control as sample applications and demonstrate the DEGC platform in hardware with the demanding Phasor-Based Control test case, and provide performance metrics.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

Hardware-in-the-Loop Evaluation of an Advanced Distributed Energy Resource Management Algorithm

This paper presents the laboratory performance evaluation of voltage regulation under a new distributed energy resource management system (DERMS) algorithm via an advanced hardware-in-the-loop (HIL) platform. The HIL platform provides realistic testing in a laboratory environment, including the accurate modeling of a full-scale real-world distribution system from a utility partner, the DERMS software controller, and power hardware photovoltaic (PV) inverters. The new DERMS algorithm is developed based on online multi-objective optimization (OMOO) algorithms that perform fast dispatch of distributed solar PV simulated in a real-time digital simulator and real physical hardware devices. Experimental tests confirm the correct functioning of the HIL platform for evaluating controller algorithms and satisfactory voltage regulation performance of the developed OMOO algorithms.

41 EE - Solar Energy Technologies Office (EE-4S)↗