Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

A Survey of Collectives

Due to the increasing sophistication and miniaturization of computational components, complex, distributed systems of interacting agents are becoming ubiquitous. Such systems, where each agent aims to optimize its own performance, but where there is a well-defined set of system-level performance criteria, are called collectives. The fundamental problem in analyzing/designing such systems is in determining how the combined actions of self-interested agents leads to 'coordinated' behavior on a iarge scale. Examples of artificial systems which exhibit such behavior include packet routing across a data network, control of an array of communication satellites, coordination of multiple deployables, and dynamic job scheduling across a distributed computer grid. Examples of natural systems include ecosystems, economies, and the organelles within a living cell. No current scientific discipline provides a thorough understanding of the relation between the structure of collectives and how well they meet their overall performance criteria. Although still very young, research on collectives has resulted in successes both in understanding and designing such systems. It is eqected that as it matures and draws upon other disciplines related to collectives, this field will greatly expand the range of computationally addressable tasks. Moreover, in addition to drawing on them, such a fully developed field of collective intelligence may provide insight into already established scientific fields, such as mechanism design, economics, game theory, and population biology. This chapter provides a survey to the emerging science of collectives.

Tumer, Kagan↗

The SNS/HFIR Web Portal System for SANS

In a busy world, continuing with the status-quo, to do things the way we are already familiar, often seems to be the most efficient way to conduct our work. We look for the value-add to decide if investing in a new method is worth the effort. How shall we evaluate if we have reached this tipping point for change? For contemporary researchers, understanding the properties of the data is a good starting point. The new generation of neutron scattering instruments being built are higher resolution and produce one or more orders of magnitude larger data than the previous generation of instruments. For instance, we have grown out of being able to perform some important tasks with our laptops – the data are too big and the computations would simply take too long. These large datasets can be problematic as facility users now begin to grapple with many of the same issues faced by more established computing communities. These issues include data access, management, and movement, data format standards, distributed computing, and collaboration among others. The Neutron Science Portal has been architected, designed, and implemented to provide users with an easy-to-use interface for managing and processing data, while also keeping an eye on meeting modern cybersecurity requirements imposed on institutions. The cost of entry for users has been lowered by utilizing a web interface providing access to backend portal resources. Users can browse or search for data which they are allowed to see, data reduction applications can be run without having to load the software, sample activation calculations can be performed for SNS and HFIR beamlines, McStas simulations can be run on TeraGrid and ORNL computers, and advanced analysis applications such as those being produced by the DANSE project can be run. Behind the scenes is a “live cataloging” system which automatically catalogs and archives experiment data via the data management system, and provides proposal team members access to their experiment data. The complexity of data movement and utilizing distributed computing resources has been taken care on behalf of users. Collaboration is facilitated by providing users a read/writeable common area, shared across all experiment team members. To date, these shared areas are the fastest growing data spaces. The portal currently has over 370 registered users, almost 7TB of experiment and user data, approximately 1,000,000 files cataloged, and had almost 10,000 unique visits last year. Future directions for enhancing portal robustness include examining how to mirror data and portal services, better facilitation of collaborations via virtual organizations, enhancing disconnected service via “thick client” applications, and better inter-facility connectivity to support cross-cutting research. The portal has established itself in the SNS/HFIR user community, and the development team strives to continue to improve the quality of features and services provided in order to better serve the community.

data↗

The SNS/HFIR Web Portal System – How Can it Help Me?

Abstract. In a busy world, continuing with the status-quo, to do things the way we are already familiar, often seems to be the most efficient way to conduct our work. We look for the value-add to decide if investing in a new method is worth the effort. How shall we evaluate if we have reached this tipping point for change? For contemporary researchers, understanding the properties of the data is a good starting point. The new generation of neutron scattering instruments being built are higher resolution and produce one or more orders of magnitude larger data than the previous generation of instruments. For instance, we have grown out of being able to perform some important tasks with our laptops – the data are too big and the computations would simply take too long. These large datasets can be problematic as facility users now begin to grapple with many of the same issues faced by more established computing communities. These issues include data access, management, and movement, data format standards, distributed computing, and collaboration among others. The Neutron Science Portal has been architected, designed, and implemented to provide users with an easy-to-use interface for managing and processing data, while also keeping an eye on meeting modern cybersecurity requirements imposed on institutions. The cost of entry for users has been lowered by utilizing a web interface providing access to backend portal resources. Users can browse or search for data which they are allowed to see, data reduction applications can be run without having to load the software, sample activation calculations can be performed for SNS and HFIR beamlines, McStas simulations can be run on TeraGrid and ORNL computers, and advanced analysis applications such as those being produced by the DANSE project can be run. Behind the scenes is a “live cataloging” system which automatically catalogs and archives experiment data via the data management system, and provides proposal team members access to their experiment data. The complexity of data movement and utilizing distributed computing resources has been taken care on behalf of users. Collaboration is facilitated by providing users a read/writeable common area, shared across all experiment team members. To date, these shared areas are the fastest growing data spaces. The portal currently has over 370 registered users, almost 5TB of experiment and user data, approximately 660K files cataloged, and had almost 10,000 unique visits last year. Future directions for enhancing portal robustness include examining how to mirror data and portal services, better facilitation of collaborations via virtual organizations, enhancing disconnected service via “thick client” applications, and better inter-facility connectivity to support cross-cutting research. The portal has established itself in the SNS/HFIR user community, and the development team strives to continue to improve the quality of features and services provided in order to better serve the community.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Finite element analysis in a minicomputer/mainframe environment

Design considerations were evaluated for general purpose finite element systems to maximize performance when installed on distributed computer hardware/software systems. It is shown how the features of current minicomputers complement those of a modular implementation of the finite element method for increasing the control, speed, and visibility (interactive graphics) in solving structural problems at reduced cost. The approach used is to implement a finite element system in a distributed computer environment to solve structural problems and to explore alternatives in distributing finite element computations.

Storaasli, O. O.↗

Harnessing distributed GPU computing for generalizable graph convolutional networks in power grid reliability assessments

Although machine learning (ML) has emerged as a powerful tool for rapidly assessing grid contingencies, prior studies have largely considered a static grid topology in their analyses. This limits their application, since they need to be re-trained for every new topology. Here, this paper explores the development of generalizable graph convolutional network (GCN) models by pre-training them across a range of grid topologies and contingency types. We found that a GCN model with auto-regressive moving average (ARMA) layers with a line graph representation of the grid offered the best predictive performance in predicting voltage magnitudes (VM) and voltage angles (VA). We introduced the concept of phantom nodes to consider disparate grid topologies with a varying number of nodes and lines. For pre-training the GCN ARMA model across a variety of topologies, distributed graphics processing unit (GPU) computing afforded us significant training scalability. The predictive performance of this model on grid topologies that were part of the training data is substantially better than the direct current (DC) approximation. Although direct application of the pre-trained model to topologies that are not part of the grid is not particularly satisfactory, fine-tuning with small amounts of data from a specific topology of interest significantly improves predictive performance. In general, this paper highlights the feasibility of training large-scale GNN models to assess the reliability of power grids by considering a wide variety of grid topologies and contingency types. With the advent of foundational models in ML and the exponential increase in GPU computing clusters, generalizable ML models will significantly enhance how utilities manage power systems and make decisions in real-time or near-real-time.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Distributed-Memory Computing With the Langley Aerothermodynamic Upwind Relaxation Algorithm (LAURA)

The Langley Aerothermodynamic Upwind Relaxation Algorithm (LAURA), a Navier-Stokes solver, has been modified for use in a parallel, distributed-memory environment using the Message-Passing Interface (MPI) standard. A standard domain decomposition strategy is used in which the computational domain is divided into subdomains with each subdomain assigned to a processor. Performance is examined on dedicated parallel machines and a network of desktop workstations. The effect of domain decomposition and frequency of boundary updates on performance and convergence is also examined for several realistic configurations and conditions typical of large-scale computational fluid dynamic analysis.

Riley, Christopher J.↗

Staghorn: An Automated Large-Scale Distributed System Analysis Platform

Conducting experiments on large-scale distributed computing systems is becoming significantly easier with the assistance of emulation. Researchers can now create a model of a distributed computing environment and then generate a virtual, laboratory copy of the entire system composed of potentially thousands of virtual machines, switches, and software. The use of real software, running at clock rate in full virtual machines, allows experiments to produce meaningful results without necessitating a full understanding of all model components. However, the ability to inspect and modify elements within these models is bound by the limitation that such modifications must compete with the model, either running in or alongside it. This inhibits entire classes of analyses from being conducted upon these models. We developed a mechanism to snapshot an entire emulation-based model as it is running. This allows us to \freeze time" and subsequently fork execution, replay execution, modify arbitrary parts of the model, or deeply explore the model. This snapshot includes capturing packets in transit and other input/output state along with the running virtual machines. We were able to build this system in Linux using Open vSwitch and Kernel Virtual Machines on top of Sandia's emulation platform Firewheel. This primitive opens the door to numerous subsequent analyses on models, including state space exploration, debugging distributed systems, performance optimizations, improved training environments, and improved experiment repeatability.

97 MATHEMATICS AND COMPUTING↗

Deep Learning on Operational Facility Data Related to Large-Scale Distributed Area Scientific Workflows

Abstract—Distributed computing platforms provide a robust mechanism to perform large-scale computations by splitting the task and data among multiple locations, possibly located thousands of miles apart geographically. This distribution of resources can lead to benefits such as redundancy of data, engagement with scientific teams that involve a diverse group of domain experts distributed spatially around the globe. It allows scientists to share their expertise and data with everyone and achieve insights by deploying collective intelligence of entire team. However, this distributed computing comes with its associated problems such as rampant duplication of file transfers increasing congestion, long job completion times, unexpected site crashing, suboptimal data transfer rates, unpredictable reliability in a time range, overshooting congestion, suboptimal usage of storage elements. In addition, each sub-system becomes a potential failure node that can trigger system wide disruptions.

Machine Learning, Provenance, Time Series, Semanti↗

The globus compute dataset: An open function-as-a-service dataset from the edge to the cloud

Here we present a unique function-as-a-service (FaaS) dataset capturing the use of the Globus Compute (previously funcX) platform. Globus Compute implements a federated model via which users may deploy endpoints on arbitrary remote computers, from the edge to high performance computing (HPC) cluster, and they may then invoke Python functions on those endpoints via a reliable cloud -hosted service. The dataset covers 31 weeks and includes 2121472 task submissions from 252 users executed on 580 remote computing endpoints. It includes 277386 registered functions. We describe the dataset and various observations, some that are similar to other FaaS datasets, for example, that 74% of tasks run for less than 1 s, and some that are unique to Globus Compute, for example, that endpoints are used in different ways and that the majority of functions are related to scientific computing and machine learning. To the best of our knowledge, this dataset represents the first federated FaaS dataset that includes user workloads, distributed computing endpoints, and analysis of registered function bodies. We expect the dataset to be useful for researching FaaS architectures, workload modeling, container warming, and other distributed computing architectures.

97 MATHEMATICS AND COMPUTING↗

EINSTEIN@HOME DISCOVERY OF FOUR YOUNG GAMMA-RAY PULSARS IN FERMI LAT DATA

We report the discovery of four gamma-ray pulsars, detected in computing-intensive blind searches of data from the Fermi Large Area Telescope (LAT). The pulsars were found using a novel search approach, combining volunteer distributed computing via Einstein@Home and methods originally developed in gravitational-wave astronomy. The pulsars PSRs J0554+3107, J1422–6138, J1522–5735, and J1932+1916 are young and energetic, with characteristic ages between 35 and 56 kyr and spin-down powers in the range 6 × 10{sup 34}—10{sup 36} erg s{sup –1}. They are located in the Galactic plane and have rotation rates of less than 10 Hz, among which the 2.1 Hz spin frequency of PSR J0554+3107 is the slowest of any known gamma-ray pulsar. For two of the new pulsars, we find supernova remnants coincident on the sky and discuss the plausibility of such associations. Deep radio follow-up observations found no pulsations, suggesting that all four pulsars are radio-quiet as viewed from Earth. These discoveries, the first gamma-ray pulsars found by volunteer computing, motivate continued blind pulsar searches of the many other unidentified LAT gamma-ray sources.

79 ASTRONOMY AND ASTROPHYSICS↗

Architectures for fault-tolerant spacecraft computers

This paper summarizes the results of a long-term research program in fault-tolerant computing for spacecraft on-board processing. In response to changing device technology this program has progressed from the design of a fault-tolerant uniprocessor to the development of fault-tolerant distributed computer systems. The unusual requirements of spacecraft computing are described along with the resulting real-time computer architectures. The following aspects of these designs are discussed: (1) architectural features to minimize complexity in the distributed computer system, (2) fault-detection and recovery, (3) techniques to enhance reliability and testability, and (4) design approaches for LSI implementation.

Rennels, D. A.↗

Solid Oxide Fuel Cell - Multi-Physics and GUI

SOFC-MP is a simulation tool developed at PNNL to evaluate the tightly coupled multi-physical phenomena in SOFCs. The purpose of the tool is to allow SOFC manufacturers to numerically test changes in planar stack design to meet DOE technical targets. The SOFC-MP 2D module is designed for computational efficiency to enable rapid engineering evaluations for operation of tall symmetric stacks. It can quickly compute distributions for the current density, voltage, temperature, and species composition in tall stacks with co-flow or counter-flow orientations. The 3D module computes distributions in entire 3D domain and handles all planner configurations: co-flow, counter-flow, and cross-flow. The detailed data from 3D simulation can be used as input for structural analysis. SOFC-MP GUI integrates both 2D and 3D modules, and it provides user friendly pre-processing and post-processing capabilities.

Lai, Kevin↗

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

97 MATHEMATICS AND COMPUTING↗

MOLNs: A Cloud Platform for Interactive, Reproducible, and Scalable Spatial Stochastic Computational Experiments in Systems Biology Using PyURDME

Computational experiments using spatial stochastic simulations have led to important new biological insights, but they require specialized tools and a complex software stack, as well as large and scalable compute and data analysis resources due to the large computational cost associated with Monte Carlo computational workflows. The complexity of setting up and managing a large-scale distributed computation environment to support productive and reproducible modeling can be prohibitive for practitioners in systems biology. This results in a barrier to the adoption of spatial stochastic simulation tools, effectively limiting the type of biological questions addressed by quantitative modeling. In this paper, we present PyURDME, a new, user-friendly spatial modeling and simulation package, and MOLNs, a cloud computing appliance for distributed simulation of stochastic reaction-diffusion models. In conclusion, MOLNs is based on IPython and provides an interactive programming platform for development of sharable and reproducible distributed parallel computational experiments.

97 MATHEMATICS AND COMPUTING↗

Parallel processing for scientific computations

The scope of this project dealt with the investigation of the requirements to support distributed computing of scientific computations over a cluster of cooperative workstations. Various experiments on computations for the solution of simultaneous linear equations were performed in the early phase of the project to gain experience in the general nature and requirements of scientific applications. A specification of a distributed integrated computing environment, DICE, based on a distributed shared memory communication paradigm has been developed and evaluated. The distributed shared memory model facilitates porting existing parallel algorithms that have been designed for shared memory multiprocessor systems to the new environment. The potential of this new environment is to provide supercomputing capability through the utilization of the aggregate power of workstations cooperating in a cluster interconnected via a local area network. Workstations, generally, do not have the computing power to tackle complex scientific applications, making them primarily useful for visualization, data reduction, and filtering as far as complex scientific applications are concerned. There is a tremendous amount of computing power that is left unused in a network of workstations. Very often a workstation is simply sitting idle on a desk. A set of tools can be developed to take advantage of this potential computing power to create a platform suitable for large scientific computations. The integration of several workstations into a logical cluster of distributed, cooperative, computing stations presents an alternative to shared memory multiprocessor systems. In this project we designed and evaluated such a system.

Alkhatib, Hasan S.↗

A Science Driven Production Cyberinfrastructure: the Open Science Grid

This article describes the Open Science Grid, a large distributed computational infrastructure in the United States which supports many different high-throughput scientific applications, and partners (federates) with other infrastructures nationally and internationally to form multi-domain integrated distributed systems for science. The Open Science Grid consortium not only provides services and software to an increasingly diverse set of scientific communities, but also fosters a collaborative team of practitioners and researchers who use, support and advance the state of the art in large-scale distributed computing. The scale of the infrastructure can be expressed by the daily throughput of around seven hundred thousand jobs, just under a million hours of computing, a million file transfers, and half a petabyte of data movement. In this paper we introduce and reflect on some of the OSG capabilities, usage and activities.

Altunay, Mine↗

A Science Driven Production Cyberinfrastructure-the Open Science Grid

This article describes the Open Science Grid, a large distributed computational infrastructure in the United States which supports many different high-throughput scientific applications, and partners (federates) with other infrastructures nationally and internationally to form multi-domain integrated distributed systems for science. The Open Science Grid consortium not only provides services and software to an increasingly diverse set of scientific communities, but also fosters a collaborative team of practitioners and researchers who use, support and advance the state of the art in large-scale distributed computing. The scale of the infrastructure can be expressed by the daily throughput of around seven hundred thousand jobs, just under a million hours of computing, a million file transfers, and half a petabyte of data movement. In this paper we introduce and reflect on some of the OSG capabilities, usage and activities.

97 MATHEMATICS AND COMPUTING↗