Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel system”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Seamless Transition of Critical Infrastructures using Droop Controlled Grid-forming Inverters

Seamless recovery of power to critical infrastructures, after grid failure, is a crucial need arising in scenarios that are increasingly becoming more frequent. Here, this article proposes a seamless transition strategy using a single and unified mode-dependent droop controlled grid-forming inverters. The control strategy achieves the following objectives: 1) regulates the output active and reactive power by the droop-controlled inverters to a desired value while operating in on-grid mode; 2) seamless transition and recovery of power injections into the load after grid failure by inverters that operates in grid-forming mode all the time; 3) requires only a single bit of information on the grid/network status for the mode transition. A framework for assessing the stability of the system and to guide the choice of parameters for controllers is developed using control-oriented modeling. A controller hardware-in-the-loop-based real-time simulation study on a test system based on the realistic electrical network of a commercial-scale medical center is conducted for initial prototyping of the control strategy. A hardware experiment is conducted with two 3 - $\phi$, 480 -V, 125 -kVA grid-forming inverters, a 3 - $\phi$, 480 -V, 270 -kVA grid simulator, a physical grid switch, and a physical load bank. The experimental data establishes the effectiveness of the always grid-forming operation and control of inverters in meeting power delivery objectives when on-grid and off-grid under various kinds of loads and scenarios while minimizing transients during transitions. Furthermore, performance comparison with existing strategies showcases the advantage of the proposed strategy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Multiterminal High-Voltage dc Systems with Series-Parallel Valve Group-Based High-Voltage dc Substations

To transfer large amount of power over long distances, multiterminal direct current (MTdc) system based on bipole high-voltage direct current (HVdc) technology is a viable option. However, such system results in large dc transmission loss. The same can be reduced by increasing the dc voltage level. This paper introduces a new MTdc system architecture comprising of series (for increasing dc voltage level) and parallel (for increasing dc current capability) connected HVdc converters. The new architecture is compared with the bipole MTdc architecture in terms of equipment needed and dc transmission loss. The control modifications needed for the MTdc system are identified and the performance of the developed control is verified through electromagnetic transient (EMT) simulations.

Jaldanki, Sreenivasa↗
DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗
DOE OSTI2020

Embedding global barrier and collective in a torus network

Embodiments of the invention provide a method, system and computer program product for embedding a global barrier and global interrupt network in a parallel computer system organized as a torus network. The computer system includes a multitude of nodes. In one embodiment, the method comprises taking inputs from a set of receivers of the nodes, dividing the inputs from the receivers into a plurality of classes, combining the inputs of each of the classes to obtain a result, and sending said result to a set of senders of the nodes. Embodiments of the invention provide a method, system and computer program product for embedding a collective network in a parallel computer system organized as a torus network. In one embodiment, the method comprises adding to a torus network a central collective logic to route messages among at least a group of nodes in a tree structure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

The Portals 4.3 Network Programming Interface

This report presents a specification for the Portals 4 network programming interface. Portals 4 is intended to allow scalable, high-performance network communication between nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded systems. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platforms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is targeted to the next generation of machines employing advanced network interface architectures that support enhanced offload capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Pattern-aware prefetching using parallel log-structured file system

Techniques are provided for pattern-aware prefetching using a parallel log-structured file system. At least a portion of one or more files is accessed by detecting at least one pattern in a non-sequential access of the one or more files; and obtaining at least a portion of the one or more files based on the detected at least one pattern. The obtaining step comprises, for example, a prefetching or pre-allocation of the at least the portion of the one or more files. A prefetch cache can store the portion of the one or more obtained files. The cached portion of the one or more files can be provided from the prefetch cache to an application requesting the at least a portion of the one or more files.

Bent, John M.↗
DOE OSTI2026

T-FSM: A Scalable Distributed Task-Based System for Frequent Subgraph Pattern Mining from a Big Graph

Finding frequent subgraph patterns in a big graph is an important problem with many applications such as classifying chemical compounds and building indexes to speed up graph queries. Since this problem is NP-hard, some recent parallel and distributed systems have been developed to accelerate the mining. However, they often have a huge memory cost, very long running time, suboptimal load balancing, poor scale-out capability, and possibly inaccurate results. In this article, we propose an efficient system called T-FSM for parallel mining of frequent subgraph patterns in a big graph. T-FSM supports a new anti-monotonic frequentness measure called Fraction-Score, which is more accurate than the widely used MNI measure. The execution engine of T-FSM supports both intra-machine parallelism and inter-machine parallelism. For intra-machine parallelism, T-FSM adopts a novel task-based execution model to ensure high multithreading concurrency, bounded memory consumption, and effective load balancing. For inter-machine parallelism, T-FSM ensures good scale-out performance with a lightweight pattern rebalancing approach that reduces workload skewness of pattern evaluations among machines. To avoid recomputing the contexts for migrated patterns, we design a novel context cache table to support concurrent and asynchronous requesting and caching of remote context data, which can timely evict and garbage collect used pattern contexts that are no longer needed to keep memory consumption bounded. Extensive experiments show that T-FSM is orders of magnitude faster than existing state-of-the-art parallel systems (more than 10×, 51×, 131×, 55× speedup over ScaleMine, DistGraph, Pangolin and Peregrine, respectively) and distributed systems (more than 42× and 88× over ScaleMine and DistGraph, respectively) for frequent subgraph pattern mining, and it scales out satisfactorily to 512 CPU cores on the Polaris supercomputer at Argonne National Laboratory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Parallel computing for power system climate resiliency: Solving a large-scale stochastic capacity expansion problem with mpi-sppy

Here we propose a nodal stochastic generation and transmission expansion planning model that incorporates the output from high-resolution global climate models through load and generation availability scenarios. We implement our model in Pyomo and perform computational studies on a realistically-sized test case of the California electric grid in a high performance computing environment. We propose model reformulations and algorithm tuning to efficiently solve this large problem using a variant of the Progressive Hedging Algorithm. We utilize the parallelization capabilities and overall versatility of mpi-sppy, exploiting its hub-and-spoke architecture to concurrently obtain inner and outer bounds on an optimal expansion plan. Initial results show that instances with 360 representative days on a system with over 8,000 buses can be solved to within 5% of optimality in under 4 h of wall clock time, a first step towards solving a large-scale power system expansion planning problem across a wide range of climate-informed operational scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Adaptive multi-level checkpointing

In some examples, with respect to adaptive multi-level checkpointing, a transfer parameter associated with transfer of checkpoint data from a node-local storage to a parallel file system may be ascertained for the checkpoint data stored in the node-local storage. The transfer parameter may be compared to a specified transfer parameter threshold. A determination may be made, based on the comparison of the transfer parameter to the specified transfer parameter threshold, as to whether to transfer the checkpoint data from the node-local storage to the parallel file system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

I/O Bottleneck Detection and Tuning: Connecting the Dots using Interactive Log Analysis

Using parallel file systems efficiently is a tricky problem due to inter-dependencies among multiple layers of I/O software, including high-level I/O libraries (HDF5, netCDF, etc.), MPI-IO, POSIX, and file systems (GPFS, Lustre, etc.). Profiling tools such as Darshan collect traces to help understand the I/O performance behavior. However, there are significant gaps in analyzing the collected traces and then applying tuning options offered by various layers of I/O software. Seeking to connect the dots between I/O bottleneck detection and tuning, we propose DXT Explorer, an interactive log analysis tool. In this paper, we present a case study using our interactive log analysis tool to identify and apply various I/O optimizations. We report an evaluation of performance improvement achieved for four I/O kernels extracted from science applications.

Bez, Jean Luca↗
DOE OSTI2024

Continental rift evolution and drainage reorganization along the Dead Sea rift since the Miocene

The Dead Sea fault is a section of the Arabian-African plate boundary. Widespread field relations indicate that three major drainage systems (stages) occupied the landscape west of the Dead Sea fault since its initiation at ca. 20 Ma. Specifically, (1) an early to middle Miocene drainage system, only minorly reconfigured by the fault (all sediments of this system belong to the Hazeva Formation); (2) a late Miocene to early Pleistocene fault-parallel drainage system named Paran-Neqarot (all sediments of this system belong to the Arava and Zehiha Formations; and (3) the early Pleistocene to present drainage configuration. The temporal and spatial frameworks of drainage stage 1 are generally constrained by radiometric dating of interfingering volcanic units, and the onset and temporal and spatial frameworks of drainage stage 3 are well constrained by cosmogenic 10 Be surface exposure ages. The timing and longevity of the rift-parallel drainage (stage 2) have until now been evasive to direct dating. The overall time gap between stages 1 and 3 is ~12–13 million years. Thus, an early age (within this time gap) of stage 2 would imply an immediate response of drainage reorganization to rift tectonics, while a later age of this drainage system would imply a delayed response. We present 11 10 Be- 26 Al cosmogenic burial ages of alluvial and colluvial units related to the fault-parallel drainage system (stage 2), which collectively constrain the time of deposition of the Arava Formation sediments in the central Negev to ca. 8 Ma. The general lack of stratigraphic order, together with the large dispersion of ages both across and within the sampling sites, attests to significant recycling of sediments from drainage stage 1 into the Arava Formation deposits. The termination of Arava and Zehiha Formation sediment deposition at ca. 1.8 Ma was determined previously using cosmogenic exposure ages of desert pavements that cover the formations. Combining the previously published data with our new data, we established the longevity and character of the Paran-Neqarot drainage system. In conlcusion, this framework highlights the temporal aspect of drainage system build-up and collapse as it responded to transform and extensional plate boundary tectonics during the Neogene.

58 GEOSCIENCES↗
DOE OSTI2022

Large Scale Caching and Streaming of Training Data for Online Deep Learning

The training of deep neural network models on large data remains a difficult problem, despite progress towards scalable techniques. In particular, there is a mismatch between the random but predetermined order in which AI flows select training samples and the streaming I/O patterns for which traditional HPC data storage (e.g., parallel file systems) are designed. In addition, as more data are obtained, it is feasible neither simply to train learning models incrementally, due to catastrophic forgetting (i.e., bias towards new samples), nor to train frequently from scratch, due to prohibitive time and/or resource constraints. In this paper, we study data management techniques that combine caching and streaming with rehearsal support in order to enable efficient access to training samples in both offline training and continual learning. We revisit state-of-art streaming approaches based on data pipelines that transparently handle prefetching, caching, shuffling, and data augmentation, and discuss the challenges and opportunities that arise when combining these methods with data-parallel training techniques. We also report on preliminary experiments that evaluate the I/O overheads involved in accessing the training samples from a parallel file system (PFS) under several concurrency scenarios, highlighting the impact of the PFS on the design of the data pipelines.

data pipelines↗
DOE OSTI2020

Containers for Massive Ensemble of I/O Bound Hierarchical Coupled Simulations

We present our experience using containers to scale up a massive ensemble of coupled I/O bound workloads on the NERSC Cori supercomputer. We describe the design of a hierarchical simulation structure using the Integrated Plasma Simulator (IPS) that enables the flexible execution of coupled simulations at the system, node, and core level using the same coupling abstraction and API. The hierarchical design allows for the node-level execution to be efficiently executed using containers while not impacting the structure of the simulation at the system level. We demonstrate the viability of the approach by presenting experimental results from applications in coupled fusion plasma simulations that illustrate the performance impact of using containers to deploy the node-level workloads, in conjunction with the user mountable XFS file systems to ameliorate the load on the Lustre parallel file system. We also present results from production runs showing the ability of the ensemble simulations to scale to hundreds of Cori Haswell nodes, with little or no overhead.

Elwasif, Wael↗
DOE OSTI2024

System and method for parallel two-photon lithography using a metalens array

A metalens array is disclosed for controllably modifying a phase of a wavefront of an optical beam. The metalens array may have a substrate having at least first and second metalens unit cells, and forming a single integrated structure with no stitching being required of the first and second metalens unit cells. The first metalens unit cell has a first plurality of nanoscale features and is configured to modify a phase of a first portion of a wavefront of an optical signal incident thereon in accordance with a first predetermined phase pattern to create at least one first focal voxel within an image plane. The second metalens unit cell has a second plurality of nanoscale features configured to modify the phase of a second portion of the wavefront of the optical signal incident thereon, in accordance with a second predetermined phase pattern, to simultaneously create at least one second focal voxel within the image plane. Each metalens unit cell also has an overall diameter of no more than about 200 microns.

Xia, Xiaoxing↗