Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

The Distributed Energy Resource Risk Manager

Organizations need a comprehensive approach to managing security and privacy risks, especially for energy resources that are becoming increasingly distributed. A tool by the National Renewable Energy Laboratory (NREL) makes it possible to manage these risks and maintain the highest standards of cybersecurity. To simplify risk management for facilities and distributed energy resources, NREL has created the Distributed Energy Resource Risk Manager, an automated, user-friendly tool that helps navigate and implement one of the most widely trusted frameworks for information security, the National Institute of Standards and Technology Risk Management Framework.

compliance↗

Welcome to the Special Issue on Grid-Edge Computing With Behind-the-Meter Resources [Guest Editorial]

The integration of distributed energy resources (DERs), such as solar photovoltaic systems, as well as other synergistic assets, including electric vehicles, energy storage, and smart appliances, in electric power systems has been dramatically increasing in the past few years. These assets have the capability to provide much-needed flexibility to electric power systems for improved grid reliability, resilience, and economic efficiency; however, most of these resources are located behind the meter (BTM) on customer premises, and their flexibility is not fully used in current grid operations. Grid-edge computing plays an important role in unlocking the great benefits and potential that BTM resources could provide to electric power systems by enhancing visibility and controllability at the grid edge.

behind-the-meter resources↗

Supporting shared resource usage for a diverse user community: The OSG experience and lessons learned

The Open Science Grid (OSG) supports a diverse community of new and existing users in adopting and making effective use of the Distributed High Throughput Computing (DHTC) model. The LHC user community has deep local support within the experiments. For other smaller communities and individual users the OSG provides consulting and technical services through the User Support area. We describe these sometimes successful and sometimes not so successful experiences and analyze lessons learned that are helping us improve our services. The services offered include forums to enable shared learning and mutual support, tutorials and documentation for new technology, and troubleshooting of problematic or systemic failure modes. For new communities and users, we bootstrap their use of the distributed high throughput computing technologies and resources available on the OSG by following a phased approach. We first adapt the application and run a small production campaign on a subset of friendly sites. Only then do we move the user to run full production campaigns across the many remote sites on the OSG, adding to the community resources up to hundreds of thousands of CPU hours per day. This scaling up generates new challenges - like no determinism in the time to job completion, and diverse errors due to the heterogeneity of the configurations and environments - so some attention is needed to get good results. We cover recent experiences with image simulation for the Large Synoptic Survey Telescope (LSST), small-file large volume data movement for the Dark Energy Survey (DES), civil engineering simulation with the Network for Earthquake Engineering Simulation (NEES), and accelerator modeling with the Electron Ion Collider group at BNL. We will categorize and analyze the use cases and describe how our processes are evolving based on lessons learned.

Garzoglio, Gabriele↗

Orchestrating Distributed Resource Ensembles for Petascale Science

Distributed, data-intensive computational science applications of interest to DOE scientific com- munities move large amounts of data for experiment data management, distributed analysis steps, remote visualization, and accessing scientific instruments. These applications need to orchestrate ensembles of resources from multiple resource pools and interconnect them with high-capacity multi- layered networks across multiple domains. It is highly desirable that mechanisms are designed that provide this type of resource provisioning capability to a broad class of applications. It is also important to have coherent monitoring capabilities for such complex distributed environments. In this project, we addressed these problems by designing an abstract API, enabled by novel semantic resource descriptions, for provisioning complex and heterogeneous resources from multiple providers using their native provisioning mechanisms and control planes: computational, storage, and multi-layered high-speed network domains. We used an extensible resource representation based on semantic web technologies to afford maximum flexibility to applications in specifying their needs. We evaluated the effectiveness of provisioning using representative data-intensive ap- plications. We also developed mechanisms for providing feedback about resource performance to the application, to enable closed-loop feedback control and dynamic adjustments to resource allo- cations (elasticity). This was enabled through development of a novel persistent query framework that consumes disparate sources of monitoring data, including perfSONAR, and provides scalable distribution of asynchronous notifications.

97 MATHEMATICS AND COMPUTING↗

How fast can one resize a distributed file system?

Efficient resource utilization becomes a major concern as large-scale distributed computing infrastructures keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and costs. However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited. In this study, we evaluate the viability of malleability as a design principle for a distributed storage system. We specifically model the minimal duration of the commission and decommission operations. To show how our models can be used in practice, we evaluate the performance of these operations in HDFS, a relevant state-of-the-art distributed file system. We show that the existing decommission mechanism of HDFS is good when the network is the bottleneck, but can be accelerated by up to a factor 3 when storage is the limiting factor. We also show that the commission in HDFS can be substantially accelerated. With the highlights provided by our model, we suggest improvements to speed both operations in HDFS. We discuss how the proposed models can be generalized for distributed file systems with different assumptions and what perspectives are open for the design of efficient malleable distributed file systems.

97 MATHEMATICS AND COMPUTING↗

Object Proxy Patterns for Accelerating Distributed Applications

Workflow and serverless frameworks have empowered new approaches to distributed application design by abstracting compute resources. However, their typically limited or one-size-fits-all support for advanced data flow patterns leaves optimization to the application programmer—optimization that becomes more difficult as data become larger. The transparent object proxy, which provides wide-area references that can resolve to data regardless of location, has been demonstrated as an effective low-level building block in such situations. Here we propose three high-level proxy-based programming patterns—distributed futures, streaming, and ownership—that make the power of the proxy pattern usable for more complex and dynamic distributed program structures. We motivate these patterns via careful review of application requirements and describe implementations of each pattern. As a result, we evaluate our implementations through a suite of benchmarks and by applying them in three meaningful scientific applications, in which we demonstrate substantial improvements in runtime, throughput, and memory usage.

Distributed Computing↗

Cybersecurity Value-at-Risk Framework: Preprint

As more variable renewable energy sources are added to the grid, the critical role of hydropower as a reliable baseline and firming resource is rapidly growing. However, the U.S hydropower fleet is not fully prepared to face modern issues such as cybersecurity threats. Hydropower accounts for 37% of U.S. utility-scale renewable electricity but is challenged by diverse infrastructure and legacy devices that predate modern security practices. While new cybersecurity solutions cannot simply be added to current hydropower generation and operation technologies, custom cybersecurity assessments can reveal system-specific threats and risk probabilities and identify mitigating enhancements.

cybersecurity valuation↗

Sustaining and Extending the Open Science Grid: Science Innovation on a PetaScale Nationwide Facility (DE-FC02-06ER41436) SciDAC-2 Closeout Report

Under this SciDAC-2 grant the project’s goal w a s t o stimulate new discoveries by providing scientists with effective and dependable access to an unprecedented national distributed computational facility: the Open Science Grid (OSG). We proposed to achieve this through the work of the Open Science Grid Consortium: a unique hands-on multi-disciplinary collaboration of scientists, software developers and providers of computing resources. Together the stakeholders in this consortium sustain and use a shared distributed computing environment that transforms simulation and experimental science in the US. The OSG consortium is an open collaboration that actively engages new research communities. We operate an open facility that brings together a broad spectrum of compute, storage, and networking resources and interfaces to other cyberinfrastructures, including the US XSEDE (previously TeraGrid), the European Grids for ESciencE (EGEE), as well as campus and regional grids. We leverage middleware provided by computer science groups, facility IT support organizations, and computing programs of application communities for the benefit of consortium members and the US national CI.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Open Problems in Network-aware Data Management in Exa-scale Computing and Terabit Networking Era

Accessing and managing large amounts of data is a great challenge in collaborative computing environments where resources and users are geographically distributed. Recent advances in network technology led to next-generation high-performance networks, allowing high-bandwidth connectivity. Efficient use of the network infrastructure is necessary in order to address the increasing data and compute requirements of large-scale applications. We discuss several open problems, evaluate emerging trends, and articulate our perspectives in network-aware data management.

97 MATHEMATICS AND COMPUTING↗

Commuting embeddings for parallel strategies in non-local games

Non-local games provide a versatile framework for probing quantum correlations and for benchmarking the power of entanglement. In finite dimensions, the standard method for playing several games in parallel requires a tensor product of the local Hilbert spaces, which scales additively in the number of qubits. In this work, we show that this additive cost can be reduced by exploiting algebraic embeddings. We introduce two forms of compressions. First, when a referee selects one game from a finite collection of games at random, the game quantum strategy can be implemented using a maximally entangled state of dimension equal to the largest individual game, thereby eliminating the need for repeated state preparations. Second, we establish conditions under which several games can be played simultaneously in parallel on fewer qubits than the tensor product baseline. These conditions are expressed in terms of commuting embeddings of the game algebras. Moreover, we provide a constructive framework for building such embeddings. Using tools from Lie theory, we show that aligning the various game algebras into a common Cartan decomposition enables such a qubit reduction. Beyond the theoretical contribution, our framework casts NLGs as algebraic primitives for distributed and resource-constrained quantum computations and suggested NLGs as a comparable device-independent dimension witness.

Commuting embeddings↗

Calibers: A Bandwidth Calendaring Paradigm for Science Workflows

Many scientific workflows require large data transfers between distributed instrument facilities, storage and computing resources. To ensure that these resources are maximally utilized, R&E networks connecting these resources must ensure that an inherently unpredictable network behaves predictably. In practice, this amounts to the per-application over-provisioning of network resources in an attempt to guarantee that adequate throughput is provided to users. This often results in resource under-utilization over time. One promising solution is the use of deadlines and bandwidth calendaring. In this approach, "fair" resource allocation is replaced with deadline-based resource allocation. However, these approaches often suffer from issues in efficiently regulating resource allocation and failure modes. Therefore, our solution, Calibers, approaches bandwidth calendaring and deadline-awareness in a different way. Calibers uses shaping, metering, and pacing at the edge of the network and end-system to provide participating clients the ability to schedule bandwidth reservations without having to worry about network noise from non-participating clients. Calibers can also fail back to the fair resource allocation of underlying transport protocols if necessary. For example, if a non-participating flow somehow enters the core of the network, or a sudden network change causes the available bandwidth to be exceeded, the underlying transport protocol congestion avoidance implementation will be able to handle the congestion as it normally would. Furthermore, Calibers provides a novel simulation method and resource allocation algorithm.

97 MATHEMATICS AND COMPUTING↗

Rucio beyond ATLAS: experiences from Belle II, CMS, DUNE, EISCAT3D, LIGO/VIRGO, SKA, XENON

For many scientific projects, data management is an increasingly complicated challenge. The number of data-intensive instruments generating unprecedented volumes of data is growing and their accompanying workflows are becoming more complex. Their storage and computing resources are heterogeneous and are distributed at numerous geographical locations belonging to different administrative domains and organisations. These locations do not necessarily coincide with the places where data is produced nor where data is stored, analysed by researchers, or archived for safe long-term storage. To fulfil these needs, the data management system Rucio has been developed to allow the high-energy physics experiment ATLAS at LHC to manage its large volumes of data in an efficient and scalable way. But ATLAS is not alone, and several diverse scientific projects have started evaluating, adopting, and adapting the Rucio system for their own needs. As the Rucio community has grown, many improvements have been introduced, customisations have been added, and many bugs have been fixed. Additionally, new dataflows have been investigated and operational experiences have been documented. In this article we collect and compare the common successes, pitfalls, and oddities that arose in the evaluation efforts of multiple diverse experiments, and compare them with the ATLAS experience. This includes the high-energy physics experiments Belle II and CMS, the neutrino experiment DUNE, the scattering radar experiment EISCAT3D, the gravitational wave observatories LIGO and VIRGO, the SKA radio telescope, and the dark matter search experiment XENON.

Lassnig, Mario↗

Quantum computation over the butterfly network

In order to investigate distributed quantum computation under restricted network resources, we introduce a quantum computation task over the butterfly network where both quantum and classical communications are limited. We consider deterministically performing a two-qubit global unitary operation on two unknown inputs given at different nodes, with outputs at two distinct nodes. By using a particular resource setting introduced by M. Hayashi [Phys. Rev. A 76, 040301(R) (2007)], which is capable of performing a swap operation by adding two maximally entangled qubits (ebits) between the two input nodes, we show that unitary operations can be performed without adding any entanglement resource, if and only if the unitary operations are locally unitary equivalent to controlled unitary operations. Our protocol is optimal in the sense that the unitary operations cannot be implemented if we relax the specifications of any of the channels. We also construct protocols for performing controlled traceless unitary operations with a 1-ebit resource and for performing global Clifford operations with a 2-ebit resource.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Time Sequence Machine Learning-Based Data Intrusion Detection for Smart Voltage Source Converter-Enabled Power Grid

Smart inverters of distributed energy resources can enable cloud computing, condition monitoring, result visualization, remote control, and peer-to-peer energy trading in advanced power systems. However, the advent of data injection attacks in the communication architecture can alter measurement characteristics of power grids and have devastating consequences. In this article, we propose a time sequence machine learning-based anomaly detection methodology for detecting cyber intrusion into control signal setpoints and dc voltage signal measurement bias of the voltage source converter (VSC) in wind generators. We first investigated the effects of four types of denial of service, tampering signal, and stealthy-type data intrusion attacks on smart VSCs and overall wind farms. We then proposed a novel time sequence machine learning-based intrusion detection framework that can be implemented to detect different cyberattacks in the VSCs. The performance of the proposed framework has been compared with that of autoencoder and clustering-based intrusion detection framework. The proposed framework was validated by using the IEEE 39 bus power system in the presence of four wind farms in different locations. Using several metrics for intrusion detection performance, we validated the effectiveness of the proposed framework.

42 ENGINEERING↗

Costa - Introduction to 2015 Annual Report

In parallel with Sandia National Laboratories having two major locations (NM and CA), along with a number of smaller facilities across the nation, so too is the distribution of scientific, engineering and computing resources. As a part of Sandia’s Institutional Computing Program, CA site-based Sandia computer scientists and engineers have been providing mission and research staff with local CA resident expertise on computing options while also focusing on two growing high performance computing research problems. The first is how to increase system resilience to failure, as machines grow larger, more complex and heterogeneous. The second is how to ensure that computer hardware and configurations are optimized for specialized data analytical mission needs within the overall Sandia computing environment, including the HPC subenvironment. All of these activities support the larger Sandia effort in accelerating development and integration of high performance computing into national security missions. Sandia continues to both promote national R&D objectives, including the recent Presidential Executive Order establishing the National Strategic Computing Initiative and work to ensure that the full range of computing services and capabilities are available for all mission responsibilities, from national security to energy to homeland defense.

97 MATHEMATICS AND COMPUTING↗

Drugsniffer: An Open Source Workflow for Virtually Screening Billions of Molecules for Binding Affinity to Protein Targets

The SARS-CoV2 pandemic has highlighted the importance of efficient and effective methods for identification of therapeutic drugs, and in particular has laid bare the need for methods that allow exploration of the full diversity of synthesizable small molecules. While classical high-throughput screening methods may consider up to millions of molecules, virtual screening methods hold the promise of enabling appraisal of billions of candidate molecules, thus expanding the search space while concurrently reducing costs and speeding discovery. Here, we describe a new screening pipeline, called drugsniffer, that is capable of rapidly exploring drug candidates from a library of billions of molecules, and is designed to support distributed computation on cluster and cloud resources. As an example of performance, our pipeline required ~40,000 total compute hours to screen for potential drugs targeting three SARS-CoV2 proteins among a library of ~3.7 billion candidate molecules.

59 BASIC BIOLOGICAL SCIENCES↗

Priority-enabled Load Balancing for Dispersed Computing

Opportunistic managed access to local in-network compute resources can improve the performance of distributed applications and reduce the dependence on shared network resources. Instead of backhauling application data to a centralized cloud data center for processing, networked services may be adaptively and continuously dispersed into shared compute resources that are closer to the source of need. While this approach has several benefits, support for mission-aware access to computation is often an afterthought, and is implemented as a brittle extension over traditional load-balancer solutions.In this work, we investigate the design of two priority-aware resource allocation strategies and two load-balancing dispatching strategies as first class citizens in an open-source dispersed computing middleware. We present a control theoretic analysis of these load-balancing primitives to identify weaknesses and strengths in our design, and recommend future directions. In parallel, we prototype two priority-aware allocation algorithms to validate our priority predictions. In initial experiments our prototype shows substantial gains in processing prioritized load. Finally, we make our source-code and experimental configurations open source.

Paulos, Aaron↗

Peer-to-Peer Energy Systems for Connected Communities: A Review of Recent Advances and Emerging Challenges

After a century of relative stability of the electricity industry, extensive deployment of distributed energy resources and recent advances in computation and communication technologies have changed the nature of how we consume, trade, and apply energy. The power system is facing a transition from its traditional hierarchical structure to a more deregulated model by introducing new energy distribution models such as peer-to- peer (P2P) sharing for connected communities. The proven effectiveness of P2P sharing in benefiting both prosumers and the grid has been demonstrated in many studies and pilot projects. However, there is still no extensive implementation of such sharing models in today’s electricity markets. This paper aims to shed some light on this gap through a comprehensive overview of recent advances in the P2P energy system and an insightful discussion of the challenges that need to be addressed in order to establish P2P sharing as a viable energy management option in today’s electricity market. To this end, in this article, we provide some background on different aspects of P2P sharing. Then, we discuss advances in P2P sharing through a systematic domain-based classification. We also review different pilot projects on P2P sharing across the globe. Finally, we identify and discuss a number of challenges that need to be addressed for scaling up P2P sharing in the electricity market followed by concluding remarks at the end of the paper.

Tushar, Wayes↗