Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallel computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

Parallel processing for scientific computations

The main contribution of the effort in the last two years is the introduction of the MOPPS system. After doing extensive literature search, we introduced the system which is described next. MOPPS employs a new solution to the problem of managing programs which solve scientific and engineering applications on a distributed processing environment. Autonomous computers cooperate efficiently in solving large scientific problems with this solution. MOPPS has the advantage of not assuming the presence of any particular network topology or configuration, computer architecture, or operating system. It imposes little overhead on network and processor resources while efficiently managing programs concurrently. The core of MOPPS is an intelligent program manager that builds a knowledge base of the execution performance of the parallel programs it is managing under various conditions. The manager applies this knowledge to improve the performance of future runs. The program manager learns from experience.

Alkhatib, Hasan S.↗
DOE OSTI2024

Optimizing Distributed Training on Frontier for Large Language Models

Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Loss scaling studies have demonstrated the superior performance of larger LLMs compared to their smaller counterparts. Nevertheless, training LLMs with billions of parameters poses significant challenges and requires considerable computational resources. For example, training a one trillion parameter GPT-style model on 20 trillion tokens requires a staggering 120 million exaflops. This research explores efficient distributed training strategies to extract this computation from Frontier, the world's first exascale supercomputer. We enable and investigate various model and data parallel training techniques, such as tensor parallelism, pipeline parallelism, and sharded data parallelism, to facilitate training a trillion-parameter model on Frontier. We empirically assess these techniques and their associated parameters to determine their impact on memory footprint, communication latency, and GPU's computational efficiency. We analyze the complex interplay among these techniques and find a strategy to combine them to achieve high throughput through hyperparameter tuning. We have identified efficient strategies for training large LLMs of varying sizes through empirical analysis and hyperparameter tuning. For 22 Billion, 175 Billion, and 1 Trillion parameters, we achieved GPU throughputs of 38.38%, 36.14%, and 31.96%, respectively. For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved 100% weak scaling efficiency on 1024 and 3072 Mi250X GPUs, respectively. We also achieved strong scaling efficiencies of 89% and 87% for these two models. We trained these models only tens of iterations instead of training till completion.

Yin, Junqi↗
NASA NTRS1973

A study of selected radiation and propagation problems related to antennas and probes in magneto-ionic media

Research consisted of computations toward the solution of the problem of the current distribution on a cylindrical antenna in a magnetoplasma. The case of an antenna parallel to the applied magnetic field was investigated. A systematic method of asymptotic expansion was found which simplifies the solution in the general case by giving the field of a dipole even at relatively short range. Some useful properties of the dispersion surfaces in a lossy medium have also been found. A laboratory experiment was directed toward evaluating nonlinear effects, such as those due to power level, bias voltage and electron heating. The problem of reflection and transmission of waves in an electron heated plasma was treated theoretically. The profile inversion problem has been pursued. Some results are very encouraging, however, the general question of stability of the solution remains unsolved.

Source record↗
NASA NTRS1976

Relativistic electrons and whistlers in Jupiter's magnetosphere

The path-integrated gain of parallel propagating whistlers driven unstable by an anisotropic distribution of relativistic electrons in the stable trapping region of Jupiter's inner magnetosphere was computed. The requirement that a gain of 3 e-foldings of power balance the power lost by imperfect reflection along the flux tube sets a stably-trapped flux of electrons which is close to the non-relativistic result. Comparison with measurements shows that observed fluxes are near the stably-trapped limit, which suggests that whistler wave intensities may be high enough to cause significant diffusion of electrons accounting for the observed reduction of phase space densities. A crude estimate of the wave intensity necessary to diffuse electrons on a radial diffusion time scale yields a lower limit for the magnetic field fluctuation intensity.

Barbosa, D. D.↗
NASA NTRS1997

NAS Applications and Advanced Algorithms

This paper examines the applications most commonly run on the supercomputers at the Numerical Aerospace Simulation (NAS) facility. It analyzes the extent to which such applications are fundamentally oriented to vector computers, and whether or not they can be efficiently implemented on hierarchical memory machines, such as systems with cache memories and highly parallel, distributed memory systems.

Bailey, David H.↗
NASA NTRS1997

NAS Applications and Advanced Architectures

This paper examines the applications most commonly run on the supercomputers at the Numerical Aerospace Simulation (NAS) facility. It analyzes the extent to which such applications are fundamentally oriented to vector computers, and whether or not they can be efficiently implemented on hierarchical memory machines, such as systems with cache memories and highly parallel, distributed memory systems.

Bailey, David H.↗
NASA NTRS1993

Plasma heating by collisionless magnetic reconnection - Analysis and computation

Analytic and numerical results on particle acceleration in 2D collisionless magnetic reconnection are presented. The particles are followed until they reach an outgoing flux surface at the same distance from the origin as the starting surface. The magnetic moment is not conserved for particles passing through the unmagnetized region around the X line at the origin. Other particles cross the separatrix without passing near the X line. The magnetic moment of the first class of outgoing particles is randomized, whereas it can be considered for the second class. The analytic model is based upon the observation of the final kinetic energy as a function of the initial conditions. Analytic results are shown to predict a Maxwellian tail for the distribution function in the perpendicular kinetic energy, with this energy much greater than the parallel kinetic energy. Numerical results showing that the predicted tail temperature agrees with the numerically computed temperature to within 10 percent over 4 orders of magnitude in the electric field are presented.

Moses, R. W.↗
NASA NTRS1992

Concurrent file operations in a high performance FORTRAN

Distributed memory multiprocessor systems can provide the computing power necessary for large scale scientific applications. A critical performance issue for a number of these applications is the efficient transfer of data to secondary storage. Recently several research groups have proposed FORTRAN language extensions for exploiting the data parallelism of such scientific codes on distributed memory architectures. However, few of these high performance FORTRAN's provide appropriate constructs for controlling the use of the parallel I/O capabilities of modern multiprocessing machines. In this paper, we propose constructs to specify I/O operations for distributed data structures in the context of Vienna Fortran. These operations can be used by the programmer to provide information which can help the compiler and runtime environment make the most efficient use of the I/O subsystem.

Brezany, Peter↗
NASA NTRS1993

Parallel spatial direct numerical simulations on the Intel iPSC/860 hypercube

The implementation and performance of a parallel spatial direct numerical simulation (PSDNS) approach on the Intel iPSC/860 hypercube is documented. The direct numerical simulation approach is used to compute spatially evolving disturbances associated with the laminar-to-turbulent transition in boundary-layer flows. The feasibility of using the PSDNS on the hypercube to perform transition studies is examined. The results indicate that the direct numerical simulation approach can effectively be parallelized on a distributed-memory parallel machine. By increasing the number of processors nearly ideal linear speedups are achieved with nonoptimized routines; slower than linear speedups are achieved with optimized (machine dependent library) routines. This slower than linear speedup results because the Fast Fourier Transform (FFT) routine dominates the computational cost and because the routine indicates less than ideal speedups. However with the machine-dependent routines the total computational cost decreases by a factor of 4 to 5 compared with standard FORTRAN routines. The computational cost increases linearly with spanwise wall-normal and streamwise grid refinements. The hypercube with 32 processors was estimated to require approximately twice the amount of Cray supercomputer single processor time to complete a comparable simulation; however it is estimated that a subgrid-scale model which reduces the required number of grid points and becomes a large-eddy simulation (PSLES) would reduce the computational cost and memory requirements by a factor of 10 over the PSDNS. This PSLES implementation would enable transition simulations on the hypercube at a reasonable computational cost.

Joslin, Ronald D.↗
NASA NTRS1983

Integrated airframe/propulsion control system architectures (IAPSA) study

Three integrated airframe/propulsion control system (ACS) architectures have been developed and are discussed. The baseline is a state of the art design which integrates autonomous propusion control and airframe flight control systems. Two alternate ACSs for the 1990s using emerging technologies were developed and are compared with each other and the baseline. One of these, designated D/D, uses parallel distributed processing, while the other, designated C/D, centralizes the processing into two centrally located redundant electronics complex. The C/D system locates all redundant computing elements and interfacing electronics in two boxes with optical sensor and actuation devices optically connected to the two boxes. Evaluation reveals the C/D architecture to be better than the other two in five of ten categories; D/D is superior in only two of the ten.

Stern,, A. d.↗
NASA NTRS1994

Using PVM to host CLIPS in distributed environments

It is relatively easy to enhance CLIPS (C Language Integrated Production System) to support multiple expert systems running in a distributed environment with heterogeneous machines. The task is minimized by using the PVM (Parallel Virtual Machine) code from Oak Ridge Labs to provide the distributed utility. PVM is a library of C and FORTRAN subprograms that supports distributive computing on many different UNIX platforms. A PVM deamon is easily installed on each CPU that enters the virtual machine environment. Any user with rsh or rexec access to a machine can use the one PVM deamon to obtain a generous set of distributed facilities. The ready availability of both CLIPS and PVM makes the combination of software particularly attractive for budget conscious experimentation of heterogeneous distributive computing with multiple CLIPS executables. This paper presents a design that is sufficient to provide essential message passing functions in CLIPS and enable the full range of PVM facilities.

Myers, Leonard↗
NASA NTRS1936

Air Flow in a Separating Laminar Boundary Layer

The speed distribution in a laminar boundary layer on the surface of an elliptic cylinder, of major and minor axes 11.78 and 3.98 inches, respectively, has been determined by means of a hot-wire anemometer. The direction of the impinging air stream was parallel to the major axis. Special attention was given to the region of separation and to the exact location of the point of separation. An approximate method, developed by K. Pohlhausen for computing the speed distribution, the thickness of the layer, and the point of separation, is described in detail; and speed-distribution curves calculated by this method are presented for comparison with experiment.

Schubauer, G B↗
NASA NTRS1984

Dataflow computing approach in high-speed digital simulation

New computational tools and methodologies for the digital simulation of continuous systems were explored. Programmability, and cost effective performance in multiprocessor organizations for real time simulation was investigated. Approach is based on functional style languages and data flow computing principles, which allow for the natural representation of parallelism in algorithms and provides a suitable basis for the design of cost effective high performance distributed systems. The objectives of this research are to: (1) perform comparative evaluation of several existing data flow languages and develop an experimental data flow language suitable for real time simulation using multiprocessor systems; (2) investigate the main issues that arise in the architecture and organization of data flow multiprocessors for real time simulation; and (3) develop and apply performance evaluation models in typical applications.

Ercegovac, M. D.↗
NASA NTRS2006

A Concept for Run-Time Support of the Chapel Language

A document presents a concept for run-time implementation of other concepts embodied in the Chapel programming language. (Now undergoing development, Chapel is intended to become a standard language for parallel computing that would surpass older such languages in both computational performance in the efficiency with which pre-existing code can be reused and new code written.) The aforementioned other concepts are those of distributions, domains, allocations, and access, as defined in a separate document called "A Semantic Framework for Domains and Distributions in Chapel" and linked to a language specification defined in another separate document called "Chapel Specification 0.3." The concept presented in the instant report is recognition that a data domain that was invented for Chapel offers a novel approach to distributing and processing data in a massively parallel environment. The concept is offered as a starting point for development of working descriptions of functions and data structures that would be necessary to implement interfaces to a compiler for transforming the aforementioned other concepts from their representations in Chapel source code to their run-time implementations.

James, Mark↗
DOE OSTI2026

Scalable quantum computational science: A perspective from block-encodings and polynomial transformations

Significant developments made in quantum hardware and error correction recently have been driving quantum computing toward practical utility. However, gaps remain between abstract quantum algorithmic development and practical applications in computational sciences. In this perspective article, we propose several properties that scalable quantum computational science methods should possess. We further discuss how block-encodings and polynomial transformations can potentially serve as a unified framework with the desired properties. Recent advancements on these topics are presented, including the construction and assembly of block-encodings, and various generalizations of quantum signal processing (QSP) algorithms to perform polynomial transformations. The scalability of QSP methods on parallel and distributed quantum architectures is also highlighted. Promising applications in simulation and observable estimation in chemistry, physics, and optimization problems are presented. We hope this perspective serves as a gentle introduction to state-of-the-art quantum algorithms for the computational science community and inspires future development of scalable quantum computational science methodologies that bridge theory and practice.

Bayesian inference↗
DOE OSTI2026

FitCache: A Transparent Drop-In Framework for Multi-Tier Caching to Accelerate Distributed Deep Learning Workloads

Training in Deep learning (DL) remains highly compute- and data-intensive, with I/O becoming a critical bottleneck as models and datasets scale. Recent studies report that data loading can dominate training time, especially on large-scale HPC systems with shared parallel file systems (PFS). Existing caching approaches either rely on single-tier designs or require intrusive modifications to training pipelines, limiting their portability and effectiveness. In this work, we present FitCache, a transparent drop-in framework for multi-tier caching to accelerate distributed DL training by coordinating fast local memory (e.g., DRAM, Persistent Memory (PMem)) and NVMe as hierarchical caches atop PFS. Our design adapts to hardware diversity, i.e., if NVMe is missing, memory transparently acts as a caching tier, ensuring stable performance. FitCache transparently intercepts I/O requests and issues concurrent fetches across all tiers, returning data from the fastest responder without centralized metadata or static redirection paths. FitCache adapts to dynamic workloads and heterogeneous clusters while maintaining POSIX compatibility. Experiments on Frontier (2048 GPUs) and smaller research clusters show that FitCache reduces training time by up to 40% and per-batch I/O latency by up to 71.6% compared to Lustre Orion PFS, offering a drop-in solution for scalable DL training.

Hu, Guangxing [ORNL] (ORCID:0009000283203614)↗
NASA NTRS1981

RAMP - A fault tolerant distributed microcomputer structure for aircraft navigation and control

Design methodologies for realizing future high authority autoflight control systems are being investigated, taking into account also the study of distributed microcomputer architectures. Attention is given to the redundant asynchronous microprocessor (RAMP) structure. RAMP comprises a connected network of microcomputers which has as input command and sensor information, and which generates servo information to drive actuators, and thrust linkages. Tolerance to hardware failures is achieved by static redundancy. Results of a failed microcomputer are simply rejected. This is done in lieu of dynamic redundancy wherein the distributed computer system performs real time fault detection and reconfiguration of the system. Attention is given to the RAMP network structure and operation, flight control with parallel asynchronous computers, and intermittent fault tolerance.

Dunn, W. R.↗