Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Using containers to speed up development, to run integration tests and to teach about distributed systems

GlideinWMS is a workload manager provisioning resources for many experiments including CMS and DUNE. The software is distributed both as native packages and specialized production containers. Following an approach used in other communities like web development we built our workspaces, system-like containers to ease development and testing. Developers can change the source tree or check out a different branch and quickly reconfigure the services to see the effect of their changes. In this paper, we’ll talk about what differentiates workspaces from other containers. We’ll describe our base system composed of three containers. A one-node cluster including a compute element and a batch system. A GlideinWMS Factory controlling pilot jobs. And a scheduler and Frontend, to submit jobs and provision resources. Additional containers can be used for optional components. This system can easily run on a laptop and we’ll share our evaluation of different container runtimes, with an eye for ease of use and performance. Finally, we’ll talk about our experience as developers and with students. The GlideinWMS workspaces are easily integrated with IDEs like VS Code, simplifying debugging and allowing development and testing of the system also when offline. They simplified the training and onboarding of new team members and Summer interns. And they were useful in workshops where students could have first-hand experience with the mechanisms and components that, in production, run millions of jobs.

Mambelli, Marco↗
DOE OSTI2025

Using Containers to Speed Up Development, to Run Integration Tests and to Teach About Distributed Systems

GlideinWMS is a workload manager provisioning resources for many experiments, including CMS and DUNE. The software is distributed both as native packages and specialized production containers. Following an approach used in other communities like web development, we built our workspaces, system-like containers to ease development and testing. Developers can change the source tree or check out a different branch and quickly reconfigure the services to see the effect of their changes. In this paper, we will talk about what differentiates workspaces from other containers. We will describe our base system, composed of three containers: a one-node cluster including a compute element and a batch system, a GlideinWMS Factory controlling pilot jobs, and a scheduler and Frontend to submit jobs and provision resources. Additional containers can be used for optional components. This system can easily run on a laptop, and we will share our evaluation of different container runtimes, with an eye for ease of use and performance. Finally, we will talk about our experience as developers and with students. The GlideinWMS workspaces are easily integrated with IDEs like VS Code, simplifying debugging and allowing development and testing of the system even when offline. They simplified the training and onboarding of new team members and summer interns. And they were useful in workshops where students could have first-hand experience with the mechanisms and components that, in production, run millions of jobs.

Mambelli, Marco [Fermilab] (ORCID:0000000294892681↗
NASA NTRS2009

Towards PCC for Concurrent and Distributed Systems (Work in Progress)

We outline some conceptual challenges in extending the PCC paradigm to a concurrent and distributed setting, and sketch a generalized notion of module correctness based on viewing communication contracts as economic games. The model supports compositional reasoning about modular systems and is meant to apply not only to certification of executable code, but also of organizational workflows.

Henriksen, Anders S.↗
DOE OSTI2024

Model-Based Detection of Coordinated Attacks (DCA) in Distribution Systems

The fast-paced growth in digitization of smart grid components enhances system observability and remote-control capabilities through efficient communication. However, enhanced connectivity results in heightened system vulnerability towards cybersecurity risks in the cyber-physical power system. Coordinated cyber-attacks (CCA), when undetected, lead to system-wide impact in terms of large disturbances or widespread outages. Detecting CCA in the cyber layer is critical to thwart cyber-attacks in real-time before the attack impacts the physical system. The challenge of locating CCA stems from the complex grid dynamics, making it difficult to distinguish between normal operational variations and cyber-attack impact. CCA often employs multiple attack vectors targeting geographically distributed components, further complicating CCA identification. Existing research in intrusion detection is primarily focused on the transmission network and limited to detecting individual attacks. In this paper, a novel proactive DCA strategy is proposed for early detection of CCA by establishing correlations among distinct attack events through model-based reinforcement learning that utilizes abductive reasoning to conclude the attacker goal. The solution includes understanding the system model, learning the system dynamics, and correlating individual cyber-attacks to extract the attacker’s objective. The developed learning algorithm identifies the most probable attack path to reach the attacker’s objective by predicting the next attack steps. A DNP3-based cyber-physical co-simulation testbed is developed to test the proposed algorithm using the IEEE 13-node test feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Self-Assembling Microgrids for Resilient Distribution Systems of the Future: Implementation in a Commercial DERMS Platform

Microgrids have long provided resilience to critical facilities such as hospitals and military installations, and they are now increasingly being looked at as a building block for future grids to support the energy resilience needs of the grid of the future. State-of-the-art technologies, such as blackstart algorithms using renewable distributed energy resources (DERs) to effectively and seamlessly form microgrids, have been produced by national labs over the years. Their adoption by the utility industry would be critical to reap the most benefits toward energy and climate resilience, and the pathway is via commercialization of these self-assembling microgrid algorithms by integrating them in a commercial product platform. This project brings a national labs team (LLNL, LANL) together with a vendor (Smarter Grid Solutions) to perform proof-of-concept integration of the algorithms into the vendor’s commercial Distributed Energy Resources Management System (DERMS). The project provides a strong pathway to commercialization of the algorithms thereby promoting adoption of resilient microgrid technology by utilities to offer resilience benefits to all customers and especially to disadvantaged and underserved communities.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1991

Integrating CLIPS applications into heterogeneous distributed systems

SOCIAL is an advanced, object-oriented development tool for integrating intelligent and conventional applications across heterogeneous hardware and software platforms. SOCIAL defines a family of 'wrapper' objects called agents, which incorporate predefined capabilities for distributed communication and control. Developers embed applications within agents and establish interactions between distributed agents via non-intrusive message-based interfaces. This paper describes a predefined SOCIAL agent that is specialized for integrating C Language Integrated Production System (CLIPS)-based applications. The agent's high-level Application Programming Interface supports bidirectional flow of data, knowledge, and commands to other agents, enabling CLIPS applications to initiate interactions autonomously, and respond to requests and results from heterogeneous remote systems. The design and operation of CLIPS agents are illustrated with two distributed applications that integrate CLIPS-based expert systems with other intelligent systems for isolating and mapping problems in the Space Shuttle Launch Processing System at the NASA Kennedy Space Center.

Adler, Richard M.↗
DOE OSTI2024

Data-Informed Synthetic Networks of Water Distribution Systems for Resilience Analysis in Puerto Rico

The increasing potential of infrastructure disruptions calls for high-quality infrastructure models to be used in resilience analysis and decision making. Unfortunately, many utilities and communities do not have access to accurate and detailed models due to a lack of data and resources. Furthermore, security restrictions on sharing infrastructure models present roadblocks to research, analysis, and decision making. Recent advances in the development of synthetic water distribution models provide a potential solution to this problem. There is an opportunity to improve these methods by leveraging incomplete pipe datasets to aid synthetic network generation. To address this gap, we developed a methodology for synthetic network generation that incorporates partial pipe data using a modification of the minimum cost flow algorithm for network generation and pipe sizing. This methodology demonstrates how partial pipe data can be leveraged to improve site-specific synthetic network generation. For the study area of Mayagüez, Puerto Rico, a synthetic model generated using 50% of real pipe data matches the pressure of the validation system with an average error of 23.5 m of head, which improves upon the average error of 31.6 m of head produced by a synthetic model generated using no data of the real pipes. Additionally, synthetic networks are shown to replicate the pressure response under a disruption scenario of the validation network, suggesting potential use in resilience analysis.

resilience analysis↗
NASA NTRS1975

Finite difference identification of noisy distributed systems using scanning measurements

Most of the present-day literature concerned with identification theory and techniques is directed toward lumped parameter systems, and many comprehensive surveys of the field are available. Relatively little has appeared in the literature concerning distributed identification, and even more noticeable is the scarcity of papers dealing with systems described by the one-dimensional wave equation. Perdeauville and Goodson were perhaps the first researchers with a workable but time consuming method for the identification of coefficients of the wave equation. Fairman and Shen, also considering the wave equation, used the technique of finite differencing to approximate spatial derivatives, and Poisson filter chains to approximate temporal derivatives.

Hughes, R. O.↗
DOE OSTI2024

Custom Accessors: Enabling Scalable Data Ingestion, (Re-)Organization, and Analysis on Distributed Systems

The emerging class of high velocity and high volume data analytic workflows comprise interwoven data ingestion, organization, and processing stages, with ingestion and organization steps often contributing comparable or even higher computational costs than actual processing steps. Since complex workflows consist of a variety of phases that view and use data differently, being able to construct efficient, scalable, distributed data structures (arrays, vectors, sets, maps, and multi-maps) is essential and requires custom methods to extend and shrink containers, analyze and position data, and, maintain globallyconsistent meta-data. In this paper, we propose a novel datastructure access paradigm based on the concept of Accessors. At a high level, accessors are customizable callable objects that can modify the behavior of insert, read, update, and delete operations for distributed containers while preserving atomicity guarantees. Accessors provide a very clean and natural way to implement a variety of programming patterns, e.g., conditional insertion/deletion and cascading computations, which would be otherwise hard (or even impossible) to express in parallel and distributed settings without using locks. We demonstrate the practicality and usefulness of our approach with two representative use cases and study the performance of these applications on a distributed High-Performance Computing system. Our analysis highlights that our proposed abstraction allows for an effective overlapping and concurrent execution of different workflow steps (e.g., data ingestion and analysis), which in a conventional analytics pipeline would execute sequentially, contributing cumulatively to the overall latency.

Castellana, Vito G. [BATTELLE (PACIFIC NW LAB)] (O↗
DOE OSTI2024

Technical Impacts of Light-Duty and Heavy-Duty Transportation Electrification on a Coordinated Transmission and Distribution System

In this study, we propose a strategy to model the required spatiotemporal charging demand from light-duty (LD) and medium- and heavy-duty (MHD) electric vehicles (EVs) using actual transportation data by mapping the demand for the required EV charging to a realistic and coordinated distribution and transmission electric grid at the predicted times of the day to study their impact on the power system in a variety of load, weather, and EV penetration scenarios. This work is the first study that includes the actual weather data and transportation data with realistic and coordinated distribution and transmission grid data in a large industry-scale level study. The main goal of this study is to identify possible issues and required upgrades in the electric grid, caused by an increase in EV integration. The transmission case study is a large grid with 6717 buses over a Texas footprint, and the distribution grid is over Houston, a city in Texas, covering over three million customers. The resulting overloads and voltage violations experienced in the system are discussed, and required planning upgrades to avoid these issues are suggested.

AC optimal power flow (AC-OPF)↗
NASA NTRS1986

Power conversion distribution system using a resonant high-frequency AC link

Static power conversion systems based on a resonant high frequency (HF) link offers a significant reduction in the size and weight of the equipment over that achieved with conventional approaches, especially when multiple sources and loads are to be integrated. A faster system response and absence of audible noise are the other principal characteristics of such systems. A conversion configuration based on a HF link which is suitable for applications requiring distributed power is proposed.

Sood, P. K.↗
NASA NTRS1989

Log-Based Recovery in Asynchronous Distributed Systems

A log-based mechanism is described for restoring consistent states to replicated data objects after failures. Preserving a causal form of consistency based on the notion of virtual time is focused upon in this report. Causal consistency has been shown to apply to a variety of applications, including distributed simulation, task decomposition, and mail delivery systems. Several mechanisms have been proposed for implementing causally consistent recovery, most notably those of Strom and Yemini, and Johnson and Zwaenepoel. The mechanism proposed here differs from these in two major respects. First, a roll-forward style of recovery is implemented. A functioning process is never required to roll-back its state in order to achieve consistency with a recovering process. Second, the mechanism does not require any explicit information about the causal dependencies between updates. Instead, all necessary dependency information is inferred from the orders in which updates are logged by the object servers. This basic recovery technique appears to be applicable to forms of consistency other than causal consistency. In particular, it is shown how the recovery technique can be modified to support an atomic form of consistency (grouping consistency). By combining grouping consistency with casual consistency, it may even be possible to implement serializable consistency within this mechanism.

Kane, Kenneth Paul↗
DOE OSTI2024

Distribution System Behind-the-Meter DERs: Estimation, Uncertainty Quantification, and Control

This paper summarizes the three-year technical activities of the IEEE Task Force (TF) on behind-the-meter (BTM) distributed energy resources (DERs): estimation, uncertainty quantification, and control. The potential grid services from BTM DERs are discussed in detail. The paper also reviews the state-of-the-art for BTM DERs visibility, uncertainty quantification, and, optimization and control. Furthermore, different aspects of the market structures associated with BTM DERs are covered, including emerging market and business models. Finally, needs and recommendations are provided for additional areas such as system protection, computing capabilities, algorithm development, market structure design, cyberinfrastructure and security, and hardware and software developments.

behind-the-meter↗
NASA NTRS1987

The implementation and use of Ada on distributed systems with high reliability requirements

Performance analysis was begin on the Ada implementations. The goal is to supply the system designer with tools that will allow a rational decision to be made about whether a particular implementation can support a given application early in the design cycle. Primary activities were: analysis of the original approach to recovery in distributed Ada programs using the Advanced Transport Operating System (ATOPS) example; review and assessment of the original approach which was found to be capable of improvement; preparation and presentation of a paper at the 1987 Washington DC Ada Symposium; development of a refined approach to recovery that is presently being applied to the ATOPS example; and design and development of a performance assessment scheme for Ada programs based on a flexible user-driven benchmarking system.

Knight, J. C.↗
NASA NTRS1994

ARACHNID: A prototype object-oriented database tool for distributed systems

This paper discusses the results of a Phase 2 SBIR project sponsored by NASA and performed by MIMD Systems, Inc. A major objective of this project was to develop specific concepts for improved performance in accessing large databases. An object-oriented and distributed approach was used for the general design, while a geographical decomposition was used as a specific solution. The resulting software framework is called ARACHNID. The Faint Source Catalog developed by NASA was the initial database testbed. This is a database of many giga-bytes, where an order of magnitude improvement in query speed is being sought. This database contains faint infrared point sources obtained from telescope measurements of the sky. A geographical decomposition of this database is an attractive approach to dividing it into pieces. Each piece can then be searched on individual processors with only a weak data linkage between the processors being required. As a further demonstration of the concepts implemented in ARACHNID, a tourist information system is discussed. This version of ARACHNID is the commercial result of the project. It is a distributed, networked, database application where speed, maintenance, and reliability are important considerations. This paper focuses on the design concepts and technologies that form the basis for ARACHNID.

Younger, Herbert↗
NASA NTRS2003

Master clock and time distribution system for the NASA Deep Space Network

NASA's Deep Space Network (DSN) consists of more than 20 antennas located at three globally spaced ground communications facilities. Local generation and distribution of precise time and frequency reference signals comprise an essential and central component of each complex. Within each complex synchronized timing references are required by approximately 100 users located at distances up to 30 kilometers from the central control center and station Master Clock. In this paper a highly modular, hot-swappable, and expandable system design for generation, delivery, and synchronization of highly precise and stable timing signals over fiber optic cables is described.

Tjoelker, R. L.↗
NASA NTRS1992

A distributed system for visualizing and analyzing multivariate and multidisciplinary data

The Linked Windows Interactive Data System (Link Winds) is being developed with NASA support. The objective of this proposal is to adapt and apply that system in a complex network environment containing elements to be found by scientists working multidisciplinary teams on very large scale and distributed data sets. The proposed three year program will develop specific visualization and analysis tools, to be exercised locally and remotely in the Link Winds environment, to demonstrate visual data analysis, interdisciplinary data analysis and cooperative and interactive televisualization and analysis of data by geographically separated science teams. These demonstrations will involve at least two science disciplines with the aim of producing publishable results.

Jacobson, Allan S.↗