Exploiting "time-domain" parallelism to accelerate neural network training and PDE constrained optimization.
Abstract not provided.
Cyr, Eric↗
SEARCH · Search NASA
Results for “Parallel in Time”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
TOWARD PARALLEL IN TIME FOR CHAOTIC DYNAMICAL SYSTEMS
Explore the source record for details and available documents.
Mathematics and Computing↗
Race conditions and data partitioning: risks posed by common errors to reproducible parallel simulations
When parallel algorithms for simulation were introduced in the 1970s, their development and use interested only experts in parallel computation. This circumstance changed as multi-core processors became commonplace, putting a parallel computer into the hands of every modeler. A natural outcome is growing interest in parallel simulation among persons not intimately familiar with parallel computing. At the same time, parallel simulation tools continue to be developed with the implicit assumption that the modeler is knowledgeable about parallel programming. The unintended consequence is a rapidly growing number of users of parallel simulation tools that are unlikely to recognize when the interaction of race conditions, partitioning strategies, and simultaneous action in their simulation models make results non-reproducible, thereby calling into question the validity of conclusions drawn from the simulation data. Here, we illustrate the potential dangers of exposing parallel algorithms to users who are not experts in parallel computation with example models constructed using existing parallel simulation tools. By doing so, we hope to refocus tool developers on usability, even if this new focus incurs loss of some performance.
97 MATHEMATICS AND COMPUTING↗
Synchronizing compute node time bases in a parallel computer
Synchronizing time bases in a parallel computer that includes compute nodes organized for data communications in a tree network, where one compute node is designated as a root, and, for each compute node: calculating data transmission latency from the root to the compute node; configuring a thread as a pulse waiter; initializing a wakeup unit; and performing a local barrier operation; upon each node completing the local barrier operation, entering, by all compute nodes, a global barrier operation; upon all nodes entering the global barrier operation, sending, to all the compute nodes, a pulse signal; and for each compute node upon receiving the pulse signal: waking, by the wakeup unit, the pulse waiter; setting a time base for the compute node equal to the data transmission latency between the root node and the compute node; and exiting the global barrier operation.
97 MATHEMATICS AND COMPUTING↗
Synchronizing compute node time bases in a parallel computer
Synchronizing time bases in a parallel computer that includes compute nodes organized for data communications in a tree network, where one compute node is designated as a root, and, for each compute node: calculating data transmission latency from the root to the compute node; configuring a thread as a pulse waiter; initializing a wakeup unit; and performing a local barrier operation; upon each node completing the local barrier operation, entering, by all compute nodes, a global barrier operation; upon all nodes entering the global barrier operation, sending, to all the compute nodes, a pulse signal; and for each compute node upon receiving the pulse signal: waking, by the wakeup unit, the pulse waiter; setting a time base for the compute node equal to the data transmission latency between the root node and the compute node; and exiting the global barrier operation.
97 MATHEMATICS AND COMPUTING↗
Parareal in time 3D numerical solver for the LWR Benchmark neutron diffusion transient model
In this paper we present a time-parallel algorithm for the 3D neutrons calculation of a transient model in a nuclear reactor core. The neutrons calculation consists in numerically solving the time dependent diffusion approximation equation, which is a simplified transport equation. The numerical resolution is done with finite elements method based on a tetrahedral meshing of the computational domain, representing the reactor core, and time discretization is achieved using a θ-scheme. The transient model presents moving control rods during the time of the reaction. Therefore, cross-sections (piecewise constants) are taken into account by interpolations with respect to the velocity of the control rods. The parallelism across the time is achieved by an adequate use of the parareal in time algorithm to the handled problem. This parallel method is a predictor corrector scheme that iteratively combines the use of two kinds of numerical propagators, one coarse and one fine. Our method is made efficient by means of a coarse solver defined with large time step and fixed position control rods model, while the fine propagator is assumed to be a high order numerical approximation of the full model. The parallel implementation of our method provides a good scalability of the algorithm. Numerical results show the efficiency of the parareal method on large light water reactor transient model corresponding to the Langenbuch–Maurer–Werner benchmark.
97 MATHEMATICS AND COMPUTING↗
libpfasst v1.0
This is a Fortran 95/2003 implementation of the Parallel Full Approximation Scheme in Space and Time (PFASST), which is a method for the time parallelization of ordinary and partial differential equations.
Minion, Michael↗
TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration
TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.
Layer-parallel↗
Parallel Multigrid in Time and Space for Extreme-Scale Computational Science: Chaotic and Hyperbolic Problems
The coming massive parallelism of exascale computing presents a pressing challenge for the many DOE simulations of time-dependent partial differential equations (PDEs), which typically use traditional sequential time stepping methods. Since this traditional approach is inherently serial, it presents a sequential bottleneck when moving to exascale computing, because future performance gains will come through greater concurrency, not faster clock speeds. Thus, the goal of this work is to research parallelism in time, i.e., methods that compute multiple time values simultaneously, not sequentially. The focus will be on hyperbolic and chaotic problems of interest to DOE, with the goal of enabling scalable simulations of time-dependent hyperbolic and chaotic problems on future architectures. The chosen methodology for solving these problems parallel-in-time is multigrid, because multigrid (when it works) is a powerful, optimal, and scalable solver for discretized PDEs. Multigrid is already commonly used in many DOE simulations for scalably and optimally solving space-only PDE problems. The areas of hyperbolic and chaotic problems are chosen because of their relevance to problems of programmatic interest to DOE. However, these problems are also well-known to be difficult for parallelin-time methods, with the most common method, parareal, diverging in many cases. The current stateof-the-art for parallel-in-time at LLNL is the multigrid reduction in time (MGRIT) XBraid package, which also struggles for such problems, while still showing some improvement over parareal. In summary, new methods are needed for an efficient parallel-in-time scheme for hyperbolic and chaotic problems, and this work shall research promising new multigrid methods in this area. In particular, we take inspiration from the Least Squares Shadowing (LSS by Wang) approach for solving chaotic problems. Here, an optimization approach is able to find “well-conditioned” shadow trajectories/solutions to the original “ill-conditioned” chaotic problem. Thus, the new multigrid methods researched here also arise in an optimization context.
97 MATHEMATICS AND COMPUTING↗
Non-iterative Voltage Stability
This report demonstrates promising capabilities and performance characteristics of the proposed method using several power systems models. The new method will help to develop a new generation of highly efficient tools suitable for real-time parallel implementation. The ultimate benefit obtained will be early detection of system instability and prevention of system blackouts in real time.
24 POWER TRANSMISSION AND DISTRIBUTION↗
A scalable framework for efficient coupling of thermal and microstructural simulations in additive manufacturing
Predicting microstructure evolution in metal additive manufacturing (AM) is important for process optimization, but spatiotemporal scale disparities between thermal transport and microstructure evolution create significant challenges for efficient data transfer between simulation codes. To address this, we present Stork, a scalable framework for coupling thermal and microstructural simulations. Stork uses a sparse data representation to identify and store active solidification sub-volumes, enabling highly parallel quad-linear interpolation from coarse thermal grids to fine microstructure grids without large intermediate storage. We demonstrate the framework by coupling the semi-analytic heat transfer code 3DThesis with the time-parallel cellular automata code Toucan. This approach achieves over two orders of magnitude reduction in data generation time and file size compared to prior workflows. Numerical studies show that quad-linear interpolation preserves grain morphology and crystallographic texture in laser powder bed fusion (LPBF) simulations for coarsening ratios up to 16. Overall, Stork provides a scalable pathway for high-throughput, component-scale AM simulations on modern high-performance computing systems.
36 MATERIALS SCIENCE↗
Time Series Discord Detection in Medical Data using a Parallel Relational Database [PowerPoint]
Recent advances in sensor technology have made continuous real-time health monitoring available in both hospital and non-hospital settings. Since data collected from high frequency medical sensors includes a huge amount of data, storing and processing continuous medical data is an emerging big data area. Especially detecting anomaly in real time is important for patients’ emergency detection and prevention. A time series discord indicates a subsequence that has the maximum difference to the rest of the time series subsequences, meaning that it has abnormal or unusual data trends. In this study, we implemented two versions of time series discord detection algorithms on a high performance parallel database management system (DBMS) and applied them to 240 Hz waveform data collected from 9,723 patients. The initial brute force version of the discord detection algorithm takes each possible subsequence and calculates a distance to the nearest non-self match to find the biggest discords in time series. For the heuristic version of the algorithm, a combination of an array and a trie structure was applied to order time series data for enhancing time efficiency. The study results showed efficient data loading, decoding and discord searches in a large amount of data, benefiting from the time series discord detection algorithm and the architectural characteristics of the parallel DBMS including data compression, data pipe-lining, and task scheduling.
47 OTHER INSTRUMENTATION↗
Parallel Multigrid in Time and Space for Extreme-Scale Computational Science
The coming massive parallelism of exascale computing presents a pressing challenge for the many DOE simulations of time-dependent partial differential equations, which typically use traditional sequential time stepping methods. Since this traditional approach is inherently serial, it presents a sequential bottleneck when moving to exascale computing, because future performance gains will come through greater concurrency, not faster clock speeds. Thus, the goal of this work is to research parallelism in time, i.e., methods that compute multiple time values simultaneously, not sequentially. The focus will be on hyperbolic and chaotic problems of programmatic interest to DOE, with the goal of enabling scalable simulations of time-dependent hyperbolic and chaotic problems on future architectures.
97 MATHEMATICS AND COMPUTING↗
RAPID
Parallel computer code for the simulator for dynamics of power systems which has the capability to initiate the system and create different faults for the dynamic analysis. The code is based on time-parallel method (Parareal) with Adaptive Method Reduction (AMR). The coarse solvers for the Parareal algorithm include several Semi Analytical Solution methods. Also, Integrated simulation of coupled transmission and distribution systems can be studied.
Simunovic, Srdjan [Oak Ridge National Lab. (ORNL),↗
Electrophysiological interrogation of asymmetric droplet interface bilayers reveals surface-bound alamethicin induces lipid flip-flop
The droplet interface bilayer (DIB) method offers simple control over initial leaflet compositions in model membranes, enabling an experimental path to filling gaps in our knowledge about the interplay between compositional lipid asymmetry, membrane properties, and the behaviors of membrane-active species. Yet, the stability of lipid leaflet asymmetry in DIBs has received very little attention, particularly in the presence of peptides and ion channels that are often studied in DIBs. Herein, we demonstrate for the first time parallel, capacitance-based measurements of intramembrane potential with arrays of asymmetric DIBs assembled in a microfluidic device to characterize the stability of leaflet asymmetry over many hours in the presence and absence of membrane-active peptides. DIBs assembled from opposing monolayers of the ester (DPhPC) and ether (DOPhPC) forms of diphytanoyl-phosphatidylcholine yielded asymmetric bilayers with leaflet compositions that were stable for at least 18 has indicated by a stable |137 mV| intramembrane potential. In contrast, the addition of surface-bound alamethicin peptides caused a gradual, concentration-dependent decrease in the magnitude of the dipole potential difference. Intermittent current-voltage measurements revealed that alamethicin in asymmetric DIBs also shifts the threshold voltage required to drive peptide insertion and ion channel formation. These outcomes take place over the course of 1 to 5 h after membrane formation, and suggest that alamethicin peptides promote lipid flip-flop, even in the un-inserted, surface-bound state, by disordering lipids in the monolayer to which they bind. Furthermore, this methodology establishes the use of parallel electrophysiology for efficiently studying membrane asymmetry in arrays of DIBs.
59 BASIC BIOLOGICAL SCIENCES↗
Temporal parallelization of edge plasma simulations using the parareal algorithm and the SOLPS code
We show that numerical modelling of edge plasma physics may be successfully parallelized in time. The parareal algorithm has been employed for this purpose and the SOLPS code package coupling the B2.5 finite-volume fluid plasma solver with the kinetic Monte-Carlo neutral code Eirene has been used as a test bed. The complex dynamics of the plasma and neutrals in the scrape-off layer (SOL) region makes this a unique application. It is demonstrated that a significant computational gain (more than an order of magnitude) may be obtained with this technique. The use of the IPS framework for event-based parareal implementation optimizes resource utilization and has been shown to significantly contribute to the computational gain.
97 MATHEMATICS AND COMPUTING↗
A biophysical approach to the performance diagnosis of human–building energy interaction: Information (bits) modeling, algorithm, and indicators of energy flow complexity
Highlights: • Energy-related human-building interaction is analyzed at the biophysical system scale. • Energy flow-networking model of environmental building performance is suggested. • Information is suggested as a holistic building performance metric. - Abstract: This article addresses the utmost upstream impacts of human–building energy interaction by proposing a network-based model, algorithms, and indicators. Hypothesizing that human behavior is a key factor for the symbiotic development of building and the geobiosphere system, the author seeks emergy (spelled with an “m”)-information integrated measures to indicate dynamic system-level performance of the interaction with building energy flow topology. To validate the hypothesis and methods, four representative building cases were tested on the building form (envelope) of (i) a building with no occupant intervention (baseline), (ii) a building controlled by responsive human behavior (bioclimatic adaptation), (iii) a building with reinforced insulation and behavior-dominated control (passive design), and (iv) a net-zero energy building (NZEB). The results demonstrate that adaptive human behavior in building operation increases the information content and complexity of energy-flow networking, improving performance and sustainability. Findings also reveal that increasing information, complexity, and power (energy availability over time) parallel the general energetic features of developing biophysical systems (greater feedback, internalization, and recycling of materials and energies). It becames clear that active behavioral response is a dominant agent of sustainable environments even on a far broader system scale.
54 ENVIRONMENTAL SCIENCES↗