Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

Scaling Up Data-Parallel Analytics Platforms: Linear Algebraic Operation Cases

Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are key to supporting large scale data analysis that require efficient processing over millions of data samples. To this end, we present, ARION, a hardware acceleration based approach for scaling-up individual tasks of Spark, a popular data-parallel analytics platform. We support both linear algebraic operations of between two dense matrices, and between sparse and dense matrices in distributed environments. ARION provides a flexible control of acceleration according to matrix density, along with efficient scheduling based on runtime resource utilization. We demonstrate the benefit of our approach for general matrix multiplication operations over large matrices with up to four billion elements by using Gramian matrix computation that is commonly used in machine learning. Experiments show that our approach achieves more than 2× and 1.5× end-to-end performance speedups for dense and sparse matrices, respectively, and up to 57.04× faster computation compared to MLlib, a state of the art Spark-based implementation. This work is sponsored in part by the NSF under the grants: CNS-1565314, CNS-1405697, and CNS-1615411. The manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Xu, Luna↗

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗

Utilization of Renewable Energy to Meet New National Challenges in Energy and Climate Change

The project aims to design a microgrid system to promote utilization of renewable energy resources such as wind and solar to address the national challenges in energy and climate change. Different optimization techniques and simulation software are used to study the performance of the renewable energy system under study. A series of research works performed under the grant Department of Energy (DOE) is presented. This grant opportunity affords Howard faculty, students, graduates, undergraduates, K-12, postdocs and visiting scholars to benefit state of the art research work. The research work has led to improve or advance understanding of new hardware technologies, software development and engineering optimization methods necessary and sufficient for handling probabilistic models and real-time computation and functions necessary for development of microgrid system. Consistent with State of Project Objective Howard University has partitioned the task into the following integrated activities: 1. Stochastic Model for RER and Load • Development of modeling Renewable Energy Resources (RER) and load which is used to perform distribution power flow study which leads to publication in refereed journals and conferences. The work was also published at the IEEE conference. 2. Stochastic optimization for voltage/Var • The development of voltage VAr optimization based on a review of existing knowledge in optimization led to the use of stochastic program and evolution of programming optimization method for V/VAr optimization. Papers were presented at the North America Power Systems Conference and the IEEE PES general meeting. 3. Modeling RER and Storage • Extending the concept of optimization method an RER with storage, such as the development of microgrid V/VAr and storage is performed. Several papers were published at the North America Power Systems Conference and the IEEE PES general meeting. 4. Power Game • Development of power game experiment using Labvolt to allow for hands on understanding of design and development of microgrid functions is performed. Publication were done by students at the end of their summer program. 5. Designing Microgrid Testbed • Example microgrid test bed is developed. In addition, function of the test bed are developed. The papers were presented at the North America Power Systems Conference and the IEEE general meeting. 6. Outreach Program • From the outreach program, topics from the project have been included in the revision of courses at Howard University, new book called Energy Processing and Smartgrid has being developed. • Hosted masters students from University of Denver to complete their projects with us. • Hosted high school students for early exposure for careers in STEM • Representations made in IEEE conferences to share the lessons learned in the use of micro grid to expose students to STEM education and research.

54 ENVIRONMENTAL SCIENCES↗

Dynamic Phasor Modeling of Three Phase Voltage Source Inverters

With the increase in the development and implementation of distributed energy resources, application of parallel connected inverters is increasing and as a result having accurate modeling and simulation tools that can help in the design, analysis and stability assessment of the grid is of great importance. Development of fundamental methods that can achieve accurate, reliable, and computationally efficient results can be very beneficial. Application of Dynamic phasor (DP) modeling method has been limited to study of limited harmonics and small combination of interconnected converters due to the complexity associated with developing models that describe larger systems. In this paper, application of DP modeling method is expanded to model any number of parallel connected three phase voltage source inverter (VSI) with inclusion of a wider harmonic content including fundamental, subharmonics, inter-harmonics, switching frequency and their sidebands. Results achieved from this modeling method is compared with conventional average model as well as detailed switching model and the effect of inclusion of wider harmonic content on accuracy of DP modeling method is demonstrated.

Xue, Yaosuo↗

Distributed Energy Resource Cybersecurity Standards Development [Final Report]

Currently, the solar industry is operating with little application-specific guidance on how to protect and defend their systems from cyberattacks. This 3-year Department of Energy (DOE) Solar Energy Technologies Office-funded project helped advance the distributed energy resource (DER) cybersecurity state-of-the-art by (a) bolstering industry awareness of cybersecurity concepts, risks, and solutions through a webinar series and (b) developing recommendations for DER cybersecurity standards to improve the security performance of DER products and networks. Drafting DER standards is a lengthy, consensus-based process requiring effective leadership and stakeholder participation. This project was designed to reduce standard and guide writing times by creating well-researched recommendations that could act as a starting place for national and international standards development organizations. Working within the SunSpec/Sandia DER Cybersecurity Workgroup, the team produced guidance for DER cybersecurity certification, communication protocol standards, network architecture s, access control, and patching. The team also led subgroups within the IEEE P 1547.3 Guide for Cybersecurity of Distributed Energy Resources Interconnected with Electric Power Systems committee and pushed a draft to ballot in October 2021.

14 SOLAR ENERGY↗

Hybrid Communication Architectures for Distributed Smart Grid Applications

Wired and wireless communications both play an important role in the blend of communications technologies necessary to enable future smart grid communications. Hybrid networks exploit independent mediums to extend network coverage and improve performance. However, whereas individual technologies have been applied in simulation networks, as far as we know there is only limited attention that has been paid to the development of a suite of hybrid communication simulation models for the communications system design. Hybrid simulation models are needed to capture the mixed communication technologies and IP address mechanisms in one simulation. To close this gap, we have developed a suite of hybrid communication system simulation models to validate the critical system design criteria for a distributed solar Photovoltaic (PV) communications system, including a single trip latency of 300 ms, throughput of 9.6 Kbps, and packet loss rate of 1%. In conclusion, the results show that three low-power wireless personal area network (LoWPAN)-based hybrid architectures can satisfy three performance metrics that are critical for distributed energy resource communications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DER Cybersecurity R&D

The National Renewable Energy Laboratory (NREL) conducted more than 30 assessments for utilities across the United States with a cybersecurity assessment tool based on the U.S. Department of Energy (DOE) Cybersecurity Capability Maturity Model (C2M2) and the National Institute of Standards and Technology (NIST) Cybersecurity Framework (CSF) and focused on business process. With funding from the DOE Office of Renewable Energy and Energy Efficiency Federal Energy Management Program, NREL modified the current cyber governance assessment tool to include an assessment process specifically for distributed energy resources (DERs). The Distributed Energy Resources Cybersecurity Framework (DER-CF) was developed to help federal agencies mitigate gaps in their cybersecurity posture for distributed energy systems.

cybersecurity valuation↗

WPTO Navigator

Protecting hydroelectric plants from incidents that adversely impact their cyber-physical systems presents unique challenges due to the plants’ widely dispersed geographic locations and varied configurations as well as the relative nascent nature of the cyberattacks targeting these facilities. To help hydroelectric plants better respond to and mitigate cybersecurity incidents, this Department of Energy Water Power Technologies Office Navigator aligns the processes within the National Institute of Standards 800-61r2 Computer Security Incident Handling Guide with the emergency actions within the FEMA 64 Emergency Action Plan Framework. This work is to be used at a hydroelectric plant to quickly understand how the plant responds to a cyber incident in relationship to the actions involved in an emergency action plan involving a hydroelectric plant. In addition to this product, there are three other products meant to be distributed to a hydroelectric plant to assist in their cyber incident response and recovery. The first, a report on the processes of building a R&R flip book based on a large set of existing guidance. The second, a handy flip book meant to be distributed to hydroelectric plants to assist them during a cybersecurity incident occurring on a hydroelectic plant. And the third is a comprehensive set of resources to assist an operator in locating appropriate guidance during the recovery process.

13 HYDRO ENERGY↗

Using Swarming Agents for Scalable Security in Large Network Environments

The difficulty of securing computer infrastructures increases as they grow in size and complexity. Network-based security solutions such as IDS and firewalls cannot scale because of exponentially increasing computational costs inherent in detecting the rapidly growing number of threat signatures. Hostbased solutions like virus scanners and IDS suffer similar issues, and these are compounded when enterprises try to monitor these in a centralized manner. Swarm-based autonomous agent systems like digital ants and artificial immune systems can provide a scalable security solution for large network environments. The digital ants approach offers a biologically inspired design where each ant in the virtual colony can detect atoms of evidence that may help identify a possible threat. By assembling the atomic evidences from different ant types the colony may detect the threat. This decentralized approach can require, on average, fewer computational resources than traditional centralized solutions; however there are limits to its scalability. This paper describes how dividing a large infrastructure into smaller managed enclaves allows the digital ant framework to effectively operate in larger environments. Experimental results will show that using smaller enclaves allows for more consistent distribution of agents and results in faster response times.

agents↗

Post-Disaster Microgrid Formation for Enhanced Distribution System Resilience

This paper proposes a deep reinforcement learning (DRL) based approach for post-disaster critical load restoration in active distribution systems to form microgrids through network reconfiguration to minimize critical load curtailments. Distribution networks are represented as graph networks, and optimal network configurations with microgrids are obtained by searching for the optimal spanning forest. The constraints to the research question being explored are the radial topology and power balance. Unlike existing analytical and population-based approaches, which necessitate the repetition of entire analyses and computation for each outage scenario to find the optimal spanning forest, the proposed approach, once properly trained, can quickly determine the optimal, or near-optimal, spanning forest even when outage scenarios change. When multiple lines fail in the system, the proposed approach forms microgrids with distributed energy resources in active distribution systems to reduce critical load curtailment. The proposed DRL-based model learns the action-value function using the REINFORCE algorithm, which is a model-free reinforcement learning technique based on stochastic policy gradients. A case study was conducted on a 33-node distribution test system, demonstrating the effectiveness of the proposed approach for post-disaster critical load restoration.

active distribution systems↗

Emerging Technologies for Privacy Preservation in Energy Systems

This study explores the intersection of digitalization and privacy within the energy sector, focusing on the emerging challenges and opportunities presented by integrating Distributed Energy Resources (DERs) and advanced metering infrastructure. The need for robust digital privacy measures has become crucial as the energy industry evolves towards a more decentralized, digitalized, and decarbonized future. This study delves into four cutting-edge privacy-preserving technologies—Homomorphic Encryption (HE), Secure Multiparty Computation (SMPC), Differential Privacy (DP), and Federated Learning (FL)—each offering unique solutions to safeguard consumer data by increasing digital connectivity and data exchange. Through a detailed examination of these methods, the study explains how each technology operates, its applications within the energy sector, and the specific privacy challenges it addresses. Homomorphic Encryption allows for secure computations on encrypted data, enabling data analysis without compromising privacy. Secure Multiparty Computation enables collaborative data analysis across different entities while protecting the confidentiality of the inputs. Differential Privacy introduces randomness into the assembled data set, preventing the identification of individual records in statistical databases. Lastly, Federated Learning offers a paradigm shift in data analysis, where machine learning models are trained at the edge, minimizing the centralization of sensitive data. The research underscores the significance of implementing these privacy-enhancing technologies to comply with strict data protection regulations, foster consumer trust, and enhance the security of the energy infrastructure. By providing a comprehensive overview of these methodologies and their practical implications for the energy sector, this study aims to contribute to the ongoing discourse on digital privacy, offering insights into how the energy industry can navigate the complexities of data privacy in the digital age.

Cali, Umit↗

Emergency Voltage Regulation in Power Systems via Ripple-Type Control

With increasing penetrations of volatile renewable generation and cyber-physical disruptions, ensuring the safe operation of bulk power systems has become unprecedentedly challenging. Because communication and computational costs restrict centralized system dispatch to being called upon every few minutes, and because purely local schemes are shown to be insufficient, distributed controls have been advocated for handling unanticipated system conditions in real time. The applicability of distributed control schemes, however, is fundamentally limited by their need for widespread communication and model cognizance. In this context, we put forth a hybrid, low-communication, saturation-driven protocol for the coordination of control agents that are distributed over a physical system and are allowed to communicate with peers over a "hotline" communication network. Under this protocol, when agents observe a constraint violation based on local measurements, they respond locally until their control resources saturate, in which case they send a beacon for assistance to peer agents. The scheme ensures that minor violations are efficiently mitigated via fast local controls, whereas severe violations can be handled by collaboration among a relatively small set of agents. We evaluate the performance of this scheme via numerical tests on the IEEE 14-bus test feeder, where agents act upon noisy measurements under diverse scenarios of load variations and severe low-/high-voltage events.

collaboration↗

Evaluating MPI resource usage summary statistics

The Message Passing Interface (MPI) remains the dominant programming model for scientific applications running on today’s high-performance computing (HPC) systems. This dominance stems from MPI’s powerful semantics for inter-process communication that has enabled scientists to write applications for simulating important physical phenomena. MPI does not, however, specify how messages and synchronization should be carried out. Those details are typically dependent on low-level architecture details and the message characteristics of the application. Therefore, analyzing an application’s MPI resource usage is critical to tuning MPI’s performance on a particular platform. The result of this analysis is typically a discussion of the mean message sizes, queue search lengths and message arrival times for a workload or set of workloads. While a discussion of the arithmetic mean in MPI resource usage might be the most intuitive summary statistic, it is not always the most accurate in terms of representing the underlying data. In this paper, we analyze MPI resource usage for a number of key MPI workloads using an existing MPI trace collector and discrete-event simulator. Our analysis demonstrates that the average, while easy and efficient to calculate, is a useful metric for characterizing latency and bandwidth measurements, but may not be a good representation of application message sizes, match list search depths, or MPI inter-operation times. Additionally, we show that the median and mode are superior choices in many cases. We also observe that the arithmetic mean is not the best representation of central tendency for data that are drawn from distributions that are multi-modal or have heavy tails. Furthermore, the results and analysis of our work provide valuable guidance on how we, as a community, should discuss and analyze MPI resource usage data for scientific applications.

97 MATHEMATICS AND COMPUTING↗

HPC Node Performance and Energy Modeling with the Co-Location of Applications

Multicore processors have become an integral part of modern large-scale and high-performance parallel and distributed computing systems. This advance became necessary as the demand for increased system performance has exceeded the limits that single core processors can provide. Unfortunately, applications co-located on multicore processors can suffer from decreased performance and increased dynamic energy use as a result of access to shared resources, such as memory. Consequently, it is increasingly important to characterize the performance of applications that execute on these architectures. This work investigates some of the disadvantages of co-location, and presents a methodology for building models capable of utilizing varying amounts of information about a target application and its co-located applications to make predictions about the target application’s execution time and the system’s energy use under arbitrary co-locations of a wide range of application types. The proposed methodology is validated on three different server class Intel Xeon multicore processors using eleven applications from two scientific benchmark suites. The model’s utility for scheduling is also demonstrated in a simulated large-scale high-performance computing environment through the creation of a co-location aware scheduling heuristic. This heuristic demonstrates that scheduling using information generated with the proposed modeling methodology is capable of making significant improvements over a scheduling heuristic that is naive to co-location interference.

performance modeling, energy modeling, resource ma↗

CDF GlideinWMS usage in grid computing of high energy physics

Many members of large science collaborations already have specialized grids available to advance their research in the need of getting more computing resources for data analysis. This has forced the Collider Detector at Fermilab (CDF) collaboration to move beyond the usage of dedicated resources and start exploiting Grid resources. Nowadays, CDF experiment is increasingly relying on glidein-based computing pools for data reconstruction. Especially, Monte Carlo production and user data analysis, serving over 400 users by central analysis farm middleware (CAF) on the top of Condor batch system and CDF Grid infrastructure. Condor is designed as distributed architecture and its glidein mechanism of pilot jobs is ideal for abstracting the Grid computing by making a virtual private computing pool. We would like to present the first production use of the generic pilot-based Workload Management System (glideinWMS), which is an implementation of the pilot mechanism based on the Condor distributed infrastructure. CDF Grid computing uses glideinWMS for its data reconstruction on the FNAL campus Grid, user analysis and Monte Carlo production across Open Science Grid (OSG). We review this computing model and setup used including CDF specific configuration within the glideinWMS system which provides powerful scalability and makes Grid computing working like in a local batch environment with ability to handle more than 10000 running jobs at a time.

99 GENERAL AND MISCELLANEOUS↗

New Basis Set Exchange: An Open, Up-to-date Resource for the Molecular Sciences Community

In computational quantum chemistry, the basis set is a fundamental input to many types of simulations, and the choice of basis set can have dramatic effects on both the accuracy and performance of the simulation. Due to this fundamental nature, basis sets have been developed starting at the beginning of the computational chemistry field, and extending to the present day, with new basis sets taking into account new theories and methods now accessible due to increases in computing power. Practically, computations are most comparable when they use the same basis set. However, early distribution channels often led to differing incorporation of basis sets indifferent computational chemistry codes, harming reproducibility and introducing errors in calculations that can go unnoticed. Such errors can be due to many causes, including transcription errors, neglecting publication of the full precision of numerical values, updated values not being incorporated in a timely manner, or development of an ‘improved’ version by different developers that retains the same name as the original basis set. Therefore, there was a need to store basis sets in a central location and make them easily and freely accessible by any researcher. The Basis Set Exchange (BSE) is widely considered by the field to be such a resource.

basis set, ab initio, web application, web interfa↗

CG-Kit: Code Generation Toolkit for performant and maintainable variants of source code applied to Flash-X hydrodynamics simulations

CG-Kit is a new Code Generation tool-Kit that we have developed as a part of the solution for portability and maintainability for multiphysics computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. To efficiently use computing resources on a heterogeneous node, an application must have a map of computation to resources and a mechanism to move the data and computation to the resources according to the map. Most existing performance portability solutions are focussed on abstracting the expression of computations so that a unified source code can be specialized to run on different resources. However, such an approach is insufficient for a code like Flash-X, which has a multitude of code components that can be assembled in various permutations and combinations to form different instances of applications. Similar challenges apply to any code that has composability, where a single specified way of apportioning work among devices may not be optimal. Additionally, use cases arise where the optimal control flow of computation may differ for different devices while the underlying numerics remain identical. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed and shared memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. All of these challenges demand that scientific software developers apply existing knowledge about domain applications, algorithms, and computing platforms to determine custom abstractions and granularity for code generation. There is a critical lack of tools to tackle those problems. CG-Kit is designed to fill this gap by providing a user with the ability to express their desired control flow and computation-to-resource map in the form a pseudocode-like recipe. It consists of standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability toolchains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. They are agnostic to the programming language of the source code targeted for code generation. In conclusion, we demonstrate the capabilities of the toolkit with two examples, first, multithreaded variants of the basic AXPY operation, and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes.

Algorithmic portability↗

Advanced Photovoltaic Inverter Control Development and Validation in a Controller-Hardware-in-the-Loop Test Bed

Penetration of solar photovoltaic (PV) installations into the grid has increased in recent years. In the past, photovoltaic installations have not included grid-support functionality. Grid support functions including Volt-VAR (VVAR), Frequency-Watt (FW), Volt-Watt (VW), frequency/voltage-ride through, as well as other inverter functions such as anti-islanding are recommended by standards such as IEEE 1547 to allow PV inverters to contribute to acceptable operation and stability of distribution systems with high penetrations of PV and to ensure predictable response of the PV inverters. First, low-level and high-level grid support inverter control functions were developed and tested in simulation. The developed controller is able to integrate advanced controller functions in a coordinated approach, avoiding any conflict between the different control objectives. The algorithms were then programmed on an off-the-shelf embedded controller that included a dual-core CPU and FPGA. The details of the high-level grid support function and the lower level switching control will be presented. Second, this programmed controller is tested using a controller-hardware-in-the-loop (CHIL) test bed setup using an FPGA-based real-time simulator. The CHIL setup has the capability to run at a 500 nano-second time-step to accommodate the 20 kHz switching frequency of the developed controller. The advanced control function and CHIL test bed details provided here will aide future researchers to design, implement, and test advanced inverter functions for PV inverters.

14 SOLAR ENERGY↗