Search NASA⌕ Search

SEARCH · Search NASA

Results for “Time parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

SU-F-J-146: Experimental Validation of 6 MV Photon PDD in Parallel Magnetic Field Calculated by EGSnrc

Purpose: Integrated linac-MR systems are uniquely suited for real time tumor tracking during radiation treatment. Understanding the magnetic field dose effects and incorporating them in treatment planning is paramount for linac-MR clinical implementation. We experimentally validated the EGSnrc dose calculations in the presence of a magnetic field parallel to the radiation beam travel. Methods: Two cylindrical bore electromagnets produced a 0.21 T magnetic field parallel to the central axis of a 6 MV photon beam. A parallel plate ion chamber was used to measure the PDD in a polystyrene phantom, placed inside the bore in two setups: phantom top surface coinciding with the magnet bore center (183 cm SSD), and with the magnet bore’s top surface (170 cm SSD). We measured the field of the magnet at several points and included the exact dimensions of the coils to generate a 3D magnetic field map in a finite element model. BEAMnrc and DOSXYZnrc simulated the PDD experiments in parallel magnetic field (i.e. 3D magnetic field included) and with no magnetic field. Results: With the phantom surface at the top of the electromagnet, the surface dose increased by 10% (compared to no-magnetic field), due to electrons being focused by the smaller fringe fields of the electromagnet. With the phantom surface at the bore center, the surface dose increased by 30% since extra 13 cm of air column was in relatively higher magnetic field (>0.13T) in the magnet bore. EGSnrc Monte Carlo code correctly calculated the radiation dose with and without the magnetic field, and all points passed the 2%, 2 mm Gamma criterion when the ion chamber’s entrance window and air cavity were included in the simulated phantom. Conclusion: A parallel magnetic field increases the surface and buildup dose during irradiation. The EGSnrc package can model these magnetic field dose effects accurately. Dr. Fallone is a co-founder and CEO of MagnetTx Oncology Solutions (under discussions to license Alberta bi-planar linac MR for commercialization).

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

Fit Fly: A Case Study of Interconnect Innovation through Parallel Simulation

To meet the demand for exascale-level performance from high-performance computing (HPC) interconnects, many system architects are turning to simulation results for accurate and reliable predictions of the performance of prospective technologies. Testing full-scale networks with a variety of benchmarking tools, including synthetic workloads and application traces, can give crucial insight into what ideas are most promising without needing to physically construct a test network. While flexible, however, this approach is extremely compute time intensive. We address this time complexity challenge through the use of large-scale, optimistic parallel simulation that ultimately leads to faster HPC network architecture innovations. In this paper we demonstrate this innovation capability through a real-world network design case study. Specifically, we have simulated and compared four extreme-scale interconnects: Dragonfly, Megafly, Slim Fly, and a new dual-rail-dual-plane variation of the Slim Fly network topology. We present this new variant of Slim Fly, dubbed Fit Fly, to show how interconnect innovation and evaluation-beyond what is possible through analytic methods-can be achieved through parallel simulation. We validate and compare the model with various network designs using the CODES interconnect simulation framework. By running large-scale simulations in a parallel environment, we are able to quickly generate reliable performance results that can help network designers break ground on the next generation of high-performance network designs.

High Performance Computing↗
DOE OSTI2018

Industrial Scale Demonstration of Smart Manufacturing, Achieving Transformational Energy Productivity Gains

While many U.S. manufacturing operations utilize optimization for individual unit processes, smart manufacturing (SM) systems that integrate manufacturing intelligence in real time across an entire production operation are rare in large companies and virtually nonexistent in smaller organizations. One example of an area where SM systems can be applied is in the management of waste heat. A smart system that not only seeks to recover waste heat, but also to use energy more efficiently, is a more cost-effective approach because energy use, waste energy loss and product output are optimized together. The overall objective of this project was to develop an industry-accepted SM Platform that enables smart systems by accelerating and lowering the cost of development and implementation scales for use in diverse sets of manufacturing industry sectors, manufacturing operations and company sizes. Specifically, the project (1) designed and demonstrated the application of a prototype SM Platform for two diverse commercial test beds, (2) demonstrated at one and estimated for the other test bed the potential reduction in waste heat generation, and (3) worked with leading automation vendors to catalyze low-cost commercialization of the technology developed. The Smart Manufacturing Application and Data Platform (SM Platform) developed during this project is an innovative approach that marries IT virtualization technologies with operational data and modeling system requirements to construct a ready-to-go platform for enabling sensor – data – modeling – actuation systems, driven by real-time plant data and performance metrics. This SM Platform allows manufacturing organizations, regardless of their industry or size, to assemble new operational systems at a much lower cost, optimizing process knowledge and improving energy productivity by integrating with existing process control and automation systems. Also developed during this project is a methodology for developing energy productivity metrics that use real-time process information and provide comparisons for potential, practical, and actual performance, as well as examples and the methodology for provisioning of an Energy Dashboard that displays alternatives to optimize energy use within specific business contexts aimed at helping improve energy productivity and reduce emissions in U.S. manufacturing. The SM Platform designed utilizes current manufacturing, operational and IT industry standards. The platform can be used to expedite sensor – data – model – actuation system development and deployment in a cloud-based environment using a novel end-to-end workflow as a service capability that is already integrated with data connection, ingestion, and contextualization services. In addition, the methodology for development of energy productivity metrics and reusable application templates called Toolkits have been defined so that they can be tailored to a specific business situation. The project’s two test beds, one in a hydrogen production plant and the other in a forging, heat treatment, and machining operation, utilize sensor-driven modeling, measurement, and simulation systems. In these two test beds, the implementation of these as a comprehensive system has made it possible for energy productivity to be managed in real-time throughout the plant and enterprise using an energy dashboard. The development pathway to direct machine actuation was articulated. During the first year, the project team developed the most highly instrumented steam-methane reformer (SMR) in the U.S. using infrared cameras and thermocouples. Real-time data are streamed to high fidelity and reduced-order models for analysis. A high fidelity CFD model, parallelized in the SM Platform cloud environment, was demonstrated for operational use. This is the first real-time operational productivity system of a steam methane reformer using both high fidelity and data-driven models together in a real time situation to balance natural gas flow to the reformer tubes. An infrared camera system was developed and shown to outperform thermocouples in coverage, accuracy, reliability, and cost effectiveness for use in a high temperature, harsh environment. A forging, heat-treating, machining line operation test bed includes new instrumentation for data capture analysis, modeling, and simulation and will be integrated into SM Platform workflows so that business performance can be managed in real-time. Composable metrics were developed using the SM Platform to drive improvements in energy productivity, environmental performance, safety, asset management, costs, and overall operations. A major focus of the SM systems developed was optimization of energy productivity at the enterprise level, which resulted in reductions in waste heat generation for the two project test beds. Reductions in the waste heat generated will result in greater energy savings than will waste heat recovery due to inefficiencies in the heat transfer process. For the SMR test bed an efficiency improvement of 1 – 2% with a corresponding waste heat reduction of 5% were demonstrated. It should be noted that the test bed SMR unit had an efficiency of more than 90% as compared to 70 to 80% efficiencies of most SMR units. So implementation of similar SM systems at typical SMR units could yield more than 15 to 25% waste heat reductions. Similarly the forging, heat-treating, machining line operation test bed is a DOE AMO Platinum Level Superior Energy Performance plant with a much higher than typical overall operating efficiency. For this test bed heat integration using recuperators is estimated to result in fuel savings of 15.9% due to waste heat recovery compared to the base case without recuperators with negligible changes in part exit conditions. This results in an 18.9% energy efficiency improvement. The project team estimates that broad multi-sector adoption of the developed technology could reduce waste heat generation by an amount equivalent to 1.3% of total U.S. energy use. This waste heat reduction is associated with 69 million tons reduction in annual carbon dioxide emissions. Other potential benefits include reducing process times, solid and liquid wastes, environmental impacts, and water use. It is estimated that this investment has an anticipated payback period of less than one year in energy-intensive applications. The SM Platform could also reduce the costs of deploying SM systems by 50% in manufacturing operations. Commercialization of the SM Platform will require significant cross-industry collaboration and technology provider market changes. Steps taken toward commercialization included 1) analyzing savings from deployment of the SM Platform and 2) holding SM Platform workshops, webinars, and conferences with targeted individual industry segments, with a focus on small and medium-sized organizations. An upgraded community website has been established as a portal for accessing and distributing information. Most importantly, the SM platform enabling technology developed during this project and its commercialization will continue, evolve and expand under the Clean Energy Smart Manufacturing Innovation Institute (CESMII).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Nonlinear multiobjective and dynamic real-time predictive optimization for optimal operation of baseload power plants under variable renewable energy

Considering the increase of disruptive variable renewable energy penetration into the power grid, this article focuses on the investigation of a multiobjective and dynamic real-time optimization framework to address the cycling of large-scale power plants under renewable penetration. In this framework, a parallelized particle swarm optimization step is first performed to generate feasible initial points. Then, a multiobjective and dynamic real-time optimization formulation generates optimal trajectories. Further, the benefit of predictive capability is investigated for the dynamic component, which introduces the novel nonlinear multiobjective and dynamic real-time predictive optimization approach. Two multiobjective formulations to obtain Pareto front optimal in real time are explored: the modified Tchebycheff-based weighted metric and ϵ-constraint methods. Economic and environmental objectives are considered in this study. A novel topical discussion on the intersection of dynamic real-time optimization with model predictive control is also presented. The developed framework is successfully applied to a baseload coal-fired power plant with postcombustion CO 2 capture. Results indicate that the approach can be deployed for a large-scale system if automatic differentiation, model reduction, and parallelization are adopted to improve computational tractability, with computational improvement up to 120-folds after performing these steps. Finally, market and carbon policies showed an impact on the optimal compromise between the objectives with an additional 63 ton of CO 2 captured under favorable market conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Parallelized domain decomposition for multi-dimensional Lagrangian random walk mass-transfer particle tracking schemes

Lagrangian particle tracking schemes allow a wide range of flow and transport processes to be simulated accurately, but a major challenge is numerically implementing the inter-particle interactions in an efficient manner. This article develops a multi-dimensional, parallelized domain decomposition (DDC) strategy for mass-transfer particle tracking (MTPT) methods in which particles exchange mass dynamically. We show that this can be efficiently parallelized by employing large numbers of CPU cores to accelerate run times. In order to validate the approach and our theoretical predictions we focus our efforts on a well-known benchmark problem with pure diffusion, where analytical solutions in any number of dimensions are well established. In this work, we investigate different procedures for “tiling” the domain in two and three dimensions (2-D and 3-D), as this type of formal DDC construction is currently limited to 1-D. An optimal tiling is prescribed based on physical problem parameters and the number of available CPU cores, as each tiling provides distinct results in both accuracy and run time. We further extend the most efficient technique to 3-D for comparison, leading to an analytical discussion of the effect of dimensionality on strategies for implementing DDC schemes. Increasing computational resources (cores) within the DDC method produces a trade-off between inter-node communication and on-node work. For an optimally subdivided diffusion problem, the 2-D parallelized algorithm achieves nearly perfect linear speedup in comparison with the serial run-up to around 2700 cores, reducing a 5 h simulation to 8 s, while the 3-D algorithm maintains appreciable speedup up to 1700 cores.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

FitCache: A Transparent Drop-In Framework for Multi-Tier Caching to Accelerate Distributed Deep Learning Workloads

Training in Deep learning (DL) remains highly compute- and data-intensive, with I/O becoming a critical bottleneck as models and datasets scale. Recent studies report that data loading can dominate training time, especially on large-scale HPC systems with shared parallel file systems (PFS). Existing caching approaches either rely on single-tier designs or require intrusive modifications to training pipelines, limiting their portability and effectiveness. In this work, we present FitCache, a transparent drop-in framework for multi-tier caching to accelerate distributed DL training by coordinating fast local memory (e.g., DRAM, Persistent Memory (PMem)) and NVMe as hierarchical caches atop PFS. Our design adapts to hardware diversity, i.e., if NVMe is missing, memory transparently acts as a caching tier, ensuring stable performance. FitCache transparently intercepts I/O requests and issues concurrent fetches across all tiers, returning data from the fastest responder without centralized metadata or static redirection paths. FitCache adapts to dynamic workloads and heterogeneous clusters while maintaining POSIX compatibility. Experiments on Frontier (2048 GPUs) and smaller research clusters show that FitCache reduces training time by up to 40% and per-batch I/O latency by up to 71.6% compared to Lustre Orion PFS, offering a drop-in solution for scalable DL training.

Hu, Guangxing [ORNL] (ORCID:0009000283203614)↗
DOE OSTI2017

The fusion code XGC: Enabling kinetic study of multi-scale edge turbulent transport in ITER [Book Chapter]

The XGC fusion gyrokinetic code combines state-of-the-art, portable computational and algorithmic technologies to enable complicated multiscale simulations of turbulence and transport dynamics in ITER edge plasma on the largest US open-science computer, the CRAY XK7 Titan, at its maximal heterogeneous capability, which have not been possible before due to a factor of over 10 shortage in the time-to-solution for less than 5 days of wall-clock time for one physics case. Frontier techniques such as nested OpenMP parallelism, adaptive parallel I/O, staging I/O and data reduction using dynamic and asynchronous applications interactions, dynamic repartitioning for balancing computational work in pushing particles and in grid related work, scalable and accurate discretization algorithms for non-linear Coulomb collisions, and communication-avoiding subcycling technology for pushing particles on both CPUs and GPUs are also utilized to dramatically improve the scalability and time-to-solution, hence enabling the difficult kinetic ITER edge simulation on a present-day leadership class computer.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2019

Reconstruction for Liquid Argon TPC Neutrino Detectors Using Parallel Architectures [Slides]

Neutrinos are particles that interact rarely, so identifying them requires large detectors which produce lots of data. Processing this data with the computing power available is becoming more difficult as the detectors increase in size to reach their physics goals. In liquid argon time projection chambers (TPCs) the charged particles from neutrino interactions produce ionization electrons which drift in an electric field towards a series of collection wires, and the signal on the wires is used to reconstruct the interaction. The MicroBooNE detector currently collecting data at Fermilab has 8000 wires, and planned future experiments like DUNE will have 100 times more, which means that the time required to reconstruct an event will scale accordingly. Modernization of liquid argon TPC reconstruction code, including vectorization, parallelization and code portability to GPUs, will help to mitigate these challenges. The liquid argon TPC hit finding algorithm within the L arSoft framework used across multiple experiments has been vectorized and parallelized. This increases the speed of the algorithm on the order of ten times within a stand alone version on Intel architectures. This new version has been incorporated back into LarSoft so that it can be generally used. These methods will also be applied to other low level reconstruction of the wire signals such as the deconvolution. The applications and performance of this modernized liquid argon TPC wire reconstruction will be presented.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2017

Longitudinal sound waves in a collisionless, quasineutral plasma

The time evolution of slow sound waves in a homogeneous, collisionless and quasineutral plasma, in particular their Landau damping, is investigated using the kinetic-magnetohydrodynamics formulation of Ramos ( J. Plasma Phys. vol. 81, 2015 p. 905810325; vol. 82, 2016 p. 905820607). In this approach, the electric field is eliminated from a closed, hybrid fluid-kinetic system that ensures automatically the fulfilment of the charge neutrality condition. Considering the time dependence of a spatial-Fourier-mode linear perturbation with wavevector parallel to the equilibrium magnetic field, this can be cast as a second-order self-adjoint problem with a continuum spectrum of real and positive squared frequencies. Therefore, a conventional resolution of the identity with a continuum basis of singular normal modes is guaranteed, which simplifies significantly a Van Kampen-like treatment of the Landau damping. The explicit form of such singular normal modes is obtained, along with their orthogonality relations. These are used to derive the damped time evolution of the fluid moments of solutions of initial-value problems, for the most general kinds of initial conditions. The non-zero parallel electric field is not used explicitly in this analysis, but it is calculated from any given solution after the later has been obtained.

Physics↗
DOE OSTI2018

Kinetic-magnetohydrodynamic description of longitudinal sound waves in a collisionless, quasineutral plasma

The time evolution of slow sound waves in a homogeneous, collisionless and quasineutral plasma, in particular their Landau damping, is investigated using the kinetic magnetohydrodynamics formulation of J.J. Ramos (J . Plasma Phys. vol. 81, 2015 p. 905810325; vol. 82, 2016 p. 905820607). In this approach, the electric field is eliminated from a closed, hybrid fluid-kinetic system that ensures automatically the fulfillment of the charge neutrality condition. Considering the time dependence of a spatial-Fourier-mode linear perturbation with wavevector parallel to the equilibrium magnetic field, this can be cast as a second-order self-adjoint problem with a continuum spectrum of real and positive squared frequencies. Therefore, a conventional resolution of the identity with a continuum basis of singular normal modes is guaranteed, which simplifies significantly a Van Kampen-like treatment of the Landau damping. The explicit form of such singular normal modes is obtained, along with their orthogonality relations. These are used to derive the damped time evolution of the fluid moments of solutions of initial-value problems, for the most general kinds of initial conditions. The non-zero parallel electric field is not used explicitly in this analysis, but it is calculated from any given solution after the later has been obtained.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

A case study on parallel HDF5 dataset concatenation for high energy physics data analysis

In High Energy Physics (HEP), experimentalists generate large volumes of data that, when analyzed, helps us better understand the fundamental particles and their interactions. This data is often captured in many files of small size, creating a data management challenge for scientists. In order to better facilitate data management, transfer, and analysis on large scale platforms, it is advantageous to aggregate data further into a smaller number of larger files. However, this translation process can consume significant time and resources, and if performed incorrectly the resulting aggregated files can be inefficient for highly parallel access during analysis on large scale platforms. In this paper, we present our case study on parallel I/O strategies and HDF5 features for reducing data aggregation time, making effective use of compression, and ensuring efficient access to the resulting data during analysis at scale. We focus on NOvA detector data in this case study, a large-scale HEP experiment generating many terabytes of data. Here, the lessons learned from our case study inform the handling of similar datasets, thus expanding community knowledge related to this common data management task.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2015

Shift-and-invert parallel spectral transformation eigensolver: Massively parallel performance for density-functional based tight-binding

The Shift-and-invert parallel spectral transformations (SIPs), a computational approach to solve sparse eigenvalue problems, is developed for massively parallel architectures with exceptional parallel scalability and robustness. The capabilities of SIPs are demonstrated by diagonalization of density-functional based tight-binding (DFTB) Hamiltonian and overlap matrices for single-wall metallic carbon nanotubes, diamond nanowires, and bulk diamond crystals. The largest (smallest) example studied is a 128,000 (2000) atom nanotube for which ~330,000 (~5600) eigenvalues and eigenfunctions are obtained in ~190 (~5) seconds when parallelized over 266,144 (16,384) Blue Gene/Q cores. Weak scaling and strong scaling of SIPs are analyzed and the performance of SIPs is compared with other novel methods. Different matrix ordering methods are investigated to reduce the cost of the factorization step, which dominates the time-to-solution at the strong scaling limit. As a result, a parallel implementation of assembling the density matrix from the distributed eigenvectors is demonstrated.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Neutron response of the TLYC scintillator

The response of the recently developed Tl 2 LiYCl 6 (Ce) scintillator to neutrons was measured and the 35 Cl(n,p) channel observed in the material for the first time. The scintillator was exposed to a 252 Cf source mounted within a parallel-plate avalanche counter for fission-fragment coincident measurements. Proton- and -like events are selected with the pulse-shape discrimination technique. Additionally, comparison of energy-deposition and time-of-flight allows the quenching of the different channels to be assessed. The 6 Li(n,t) channel is found to have its energy quenched to 37(1)% relative to -ray detection, while the proton energy deposition spectrum is quenched to 60(1)%.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2017

HPC formulations of optimization algorithms for tensor completion

Tensor completion is a powerful tool used to estimate or recover missing values in multi-way data. It has seen great success in domains such as product recommendation and healthcare. Tensor completion is most often accomplished via low-rank sparse tensor factorization, a computationally expensive non-convex optimization problem which has only recently been studied in the context of parallel computing. In this work, we study three optimization algorithms that have been successfully applied to tensor completion: alternating least squares (ALS), stochastic gradient descent (SGD), and coordinate descent (CCD++). We explore opportunities for parallelism on shared- and distributed-memory systems and address challenges such as memory- and operation-efficiency, load balance, cache locality, and communication. Among our advancements are a communication-efficient CCD++ algorithm, an ALS algorithm rich in level-3 BLAS routines, and an SGD algorithm which combines stratification with asynchronous communication. Furthermore, we show that introducing randomization during ALS and CCD++ can accelerate convergence. We evaluate our parallel formulations on a variety of real datasets on a modern supercomputer and demonstrate speedups through 16384 cores. Further, these improvements reduce time-to-solution from hours to seconds on real-world datasets. We show that after our optimizations, ALS is advantageous on parallel systems of small-to-moderate scale, while both ALS and CCD++ provide the lowest time-to-solution on large-scale distributed systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Fast Parallel Stochastic Subspace Algorithms for Large-Scale Ambient Oscillation Monitoring

With the installation of synchrophasors widely across the power grid, measurement-based oscillation monitoring algorithms are becoming increasingly useful in identifying the real-time oscillatory modal properties in power systems. When the number of phasor measurement unit (PMU) channels grows, the computational time of many PMU data based algorithms is dominated by the computational burden in processing large-scale dense matrices. In order to overcome this limitation, this paper presents new formulations and computational strategies for speeding up an ambient oscillation monitoring algorithm, namely, stochastic subspace identification (SSI). Based on previous work, two fast singular value decomposition (SVD) approaches are first applied to the SVD evaluation within the SSI algorithm. Next, block structures are exploited so that the large-scale dense matrix computations can be processed in parallel. This helps in memory savings as well as in overall computational time. Experimental results from three sets of archived data of the western interconnection demonstrate that the new approaches can provide significant speedups while retaining modal estimation accuracy. With proposed fast parallel algorithms, the real-time oscillation monitoring of the large-scale system using hundreds of PMU measurements becomes feasible.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Stable large-scale solver for Ginzburg–Landau equations for superconductors

Understanding the interaction of vortices with inclusions in type-II superconductors is a major outstanding challenge both for fundamental science and energy applications. At application-relevant scales, the long-range interactions between a dense configuration of vortices and the dependence of their behavior on external parameters, such as temperature and an applied magnetic field, are all important to the net response of the superconductor. Capturing these features, in general, precludes analytical description of vortex dynamics and has also made numerical simulation prohibitively expensive. Here we report on a highly optimized iterative implicit solver for the time-dependent Ginzburg-Landau equations suitable for investigations of type-II superconductors on massively parallel architectures. Its main purpose is to study vortex dynamics in disordered or geometrically confined mesoscopic systems. In this work, we present the discretization and time integration scheme in detail for two types of boundary conditions. We describe the necessary conditions for a stable and physically accurate integration of the equations of motion. Using an inclusion pattern generator, we can simulate complex pinning landscapes and the effect of geometric confinement. We show that our algorithm, implemented on a GPU, can provide static and dynamic solutions of the Ginzburg-Landau equations for mesoscopically large systems over thousands of time steps in a matter of hours. Using our formulation, studying scientifically-relevant problems is a computationally reasonable task.

Sadovskyy, I. A.↗
DOE OSTI2020

TuckerMPI: A Parallel C++/MPI Software Package for Large-scale Data Compression via the Tucker Tensor Decomposition

With this study, our goal is compression of massive-scale grid-structured data, such as the multi-terabyte output of a high-fidelity computational simulation. For such data sets, we have developed a new software package called TuckerMPI, a parallel C++/MPI software package for compressing distributed data. The approach is based on treating the data as a tensor, i.e., a multidimensional array, and computing its truncated Tucker decomposition, a higher-order analogue to the truncated singular value decomposition of a matrix. The result is a low-rank approximation of the original tensor-structured data. Compression efficiency is achieved by detecting latent global structure within the data, which we contrast to most compression methods that are focused on local structure. In this work, we describe TuckerMPI, our implementation of the truncated Tucker decomposition, including details of the data distribution and in-memory layouts, the parallel and serial implementations of the key kernels, and analysis of the storage, communication, and computational costs. We test the software on 4.5 and 6.7 terabyte data sets distributed across 100 s of nodes (1,000 s of MPI processes), achieving compression ratios between 100 and 200,000×, which equates to 99--99.999% compression (depending on the desired accuracy) in substantially less time than it would take to even read the same dataset from a parallel file system. Moreover, we show that our method also allows for reconstruction of partial or down-sampled data on a single node, without a parallel computer so long as the reconstructed portion is small enough to fit on a single machine, e.g., in the instance of reconstructing/visualizing a single down-sampled time step or computing summary statistics. The code is available at https://gitlab.com/tensors/TuckerMPI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Scaled ILU Smoothers for Navier-Stokes Pressure Projection

Incomplete LU (ILU) smoothers are effective in the algebraic multigrid (AMG) V-cycle for reducing high-frequency components of the error. However, the requisite direct triangular solves are comparatively slow on GPUs. Previous work has demonstrated the advantages of Jacobi iteration as an alternative to direct solution of these systems. Depending on the threshold and fill-level parameters chosen, the factors can be highly nonnormal and Jacobi is unlikely to converge in a low number of iterations. We demonstrate that row scaling can reduce the departure from normality, allowing us to replace the inherently sequential solve with a rapidly converging Richardson iteration. There are several advantages beyond the lower compute time. Scaling is performed locally for a diagonal block of the global matrix because it is applied directly to the factor. Further, an ILUT Schur complement smoother maintains a constant GMRES iteration count as the number of MPI ranks increases, and thus parallel strong-scaling is improved. Our algorithms have been incorporated into hypre, and we demonstrate improved time to solution for linear systems arising in the Nalu-Wind and PeleLM pressure solvers. For large problem sizes, GMRES+AMG executes at least five times faster when using iterative triangular solves compared with direct solves on massively parallel GPUs.

algebraic multigrid↗