Search NASA⌕ Search

SEARCH · Search NASA

Results for “Time parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis

X‐ray scattering experiments using free electron lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID‐19 viral proteins). XFEL experiments are a challenge to computing in two ways: (i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; (ii) data‐collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min—sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS‐NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis within 10 min is a first for XFEL experiments and an important milestone if we are to keep up with data‐collection trends.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A Comparison of Classical and Aggregation-Based Algebraic Multigrid Preconditioners for High-Fidelity Simulation of Wind Turbine Incompressible Flows

This paper presents a comparison of parallel strong scaling performance of classical and aggregation algebraic multigrid (AMG) preconditioners in the context of wind turbine simulations. Fluid motion is governed by the incompressible Navier--Stokes equations, discretized in space with control-volume finite elements and in time with an inexact projection scheme using an implicit integrator. A discontinuous-Galerkin sliding-mesh algorithm captures rotor motion. The momentum equations are solved with iterative Krylov methods, preconditioned by symmetric Gauss--Seidel (SGS) in Trilinos and $\ell_1$ SGS in hypre. The mass-continuity equation is solved with GMRES preconditioned by AMG and can account for the majority of simulation time. Reducing this continuity solve time is crucial. Wind turbine simulations present two unique challenges for AMG preconditioned solvers: the computational meshes include strongly anisotropic elements, and mesh motion requires matrix reinitialization and computation of preconditioners at each time step. Detailed timing profiles are presented and analyzed, and best practices are discussed for both classical and aggregation-based AMG. Results are presented for simulations of two different wind turbines with up to 6 billion grid points on two different computer architectures. For moving mesh problems that require linear-system reinitialization, the well-established strategy of amortizing preconditioner setup costs over a large number of time steps to reduce the solve time is no longer valid. Instead, results show that faster time to solution is achieved by reducing preconditioner setup costs at the expense of linear-system solve costs. Standard smoothed aggregation with Chebyshev relaxation was found to perform poorly when compared with classical AMG in terms of solve time and robustness. However, plain aggregation was comparable to classical AMG.

97 MATHEMATICS AND COMPUTING↗

Bayesian Analysis of TRISO Fuel: Quantifying Model Inadequacy, Incorporating Lower-Length-Scale Effects, and Developing Parallel Active Learning Capabilities

The U.S. Department of Energy (DOE)’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) program aims to develop predictive capabilities by applying computational methods to the analysis and design of advanced reactor and fuel-cycle systems. This program has been providing engineering-scale support for the continued development of BISON, a high-fidelity, high-resolution fuel performance tool. Fuel behavior in nuclear reactors is governed by a complex network of mechanisms that interact with various other physics aspects in the reactor system. Any model developed to represent fuel behavior will likely be idealized, resulting in uncertainties when comparing their predictions against the observed data. In Fiscal Year (FY)-23, we initiated the Uncertainty Quantification (UQ) work by using Bayesian methods to establish a level of model trustworthiness and further improve it, with a particular emphasis on TRI-Structural isOtropic (TRISO) nuclear fuel. This year, we further expanded on that UQ work by investigating an approach to quantifying model inadequacy and accounting for lower-length scale (LLS) effects in TRISO silver (Ag) release modeling. Furthermore, we are implementing parallel active learning capabilities to reduce the computational cost (i.e., required computational resources and elapsed time) of performing UQ. Specifically, we utilized The Kennedy O’Hagan framework for Bayesian uncertainty quantification (KOH) to account for model inadequacy in TRISO Ag release predictions made by BISON. The KOH framework represents an improvement over the standard Bayesian framework used in FY-23. Explicitly accounting for model inadequacy in the Bayesian framework helps establish the level of experimental noise uncertainty in the Advanced Gas Reactor (AGR) data. We compared the inverse UQ results obtained from both the standard Bayesian and KOH frameworks in light of the AGR-2/3/4 data, and also compared the predictive UQ results obtained from these two frameworks in light of the AGR-1 data. Next, we investigated the impact of considering LLS effects in the Ag release simulations. We developed an expanded database of LLS simulated effective diffusivities for Ag, covering a wide range of microstructures and temperatures. Using this database, we developed a framework for incorporating LLS effects into the engineering-scale Ag release UQ. We developed both parametric and non-parametric approaches for bridging the length scales. We then investigated the inverse UQ results in light of the AGR-2/3/4 data and the predictive UQ results in light of the AGR-1 data, and compared the LLS-informed approach and the Arrhenius equation, which does not include microstructure information. Finally, we discussed implementing parallel active learning capabilities in the Multiphysics Object Oriented Simulation Environment (MOOSE)/BISON to reduce the computational cost (i.e., computational resources and elapsed time) of Bayesian UQ. For verification purposes, we first tested these new capabil ities on a species interaction problem. We then demonstrated them on the TRISO Ag release application, showing that parallel active learning capabilities can enhance the accuracy of UQ while also substantially reducing the computational cost in comparison to the reference methods developed in FY-23.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Elevating SolTrace's Capabilities for the Next Generation of Concentrating Solar Analysis

SolTrace is an open-source Monte Carlo ray tracing software developed at NREL. SolTrace can characterize concentrating solar thermal (CST) collector optical performance and is CST technology agnostic. Shown in Fig. 1, SolTrace is a foundational tool in NREL's CST system and component modeling suite. SolTrace's generic surface elements can flexibly model novel collector and receiver designs to predict spatial and temporal flux distributions - critical to understand for CST component design, performance prediction, and system integration. Since its initial development, SolTrace has over 1,650 references on Google Scholar, over 9,800 downloads since 2017, and has served the CST research and development community as a benchmark of 3rd party verification. SolTrace provides users with many options for defining surface shape and boundaries. However, SolTrace provides limited documentation which can result in a steep learning curve for new users. Additionally, SolTrace lacks the computational performance required to evaluate optical performance of a CST system over the course of a year and/or iteratively over design parameters in a timely manner. To address this, we are working towards a new release of SolTrace that enables increased computational throughput by implementing ray tracing acceleration structures and enabling GPU parallelization. Additionally, we are working to improve SolTrace's usability, accessibility, and maintainability by (1) automating solar position time-dependent simulation processes, (2) creating general CST collector templates of grouped elements, (3) updating the user interface to better visualize model inputs and outputs, and (4) creating a user support network through forums, "how to" videos, and documentation.

14 SOLAR ENERGY↗

Module-Fluidics: Building Blocks for Spatio-Temporal Microenvironment Control

Generating the desired solute concentration signal in micro-environments is vital to many applications ranging from micromixing to analyzing cellular response to a dynamic microenvironment. We propose a new modular design to generate targeted temporally varying concentration signals in microfluidic systems while minimizing perturbations to the flow field. The modularized design, here referred to as module-fluidics, similar in principle to interlocking toy bricks, is constructed from a combination of two building blocks and allows one to achieve versatility and flexibility in dynamically controlling input concentration. The building blocks are an oscillator and an integrator, and their combination enables the creation of controlled and complex concentration signals, with different user-defined time-scales. We show two basic connection patterns, in-series and in-parallel, to test the generation, integration, sampling and superposition of temporally-varying signals. All such signals can be fully characterized by analytic functions, in analogy with electric circuits, and allow one to perform design and optimization before fabrication. Such modularization offers a versatile and promising platform that allows one to create highly customizable time-dependent concentration inputs which can be targeted to the specific application of interest.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Work-Efficient Parallel Sparse Matrix-Sparse Vector Multiplication Algorithm

We design and develop a work-efficient multithreaded algorithm for sparse matrix-sparse vector multiplication (SpMSpV) where the matrix, the input vector, and the output vector are all sparse. SpMSpV is an important primitive in the emerging GraphBLAS standard and is the workhorse of many graph algorithms including breadth-first search, bipartite graph matching, and maximal independent set. As thread counts increase, existing multithreaded SpMSpV algorithms can spend more time accessing the sparse matrix data structure than doing arithmetic. Our shared-memory parallel SpMSpV algorithm is work efficient in the sense that its total work is proportional to the number of arithmetic operations required. The key insight is to avoid each thread individually scan the list of matrix columns. Our algorithm is simple to implement and operates on existing column-based sparse matrix formats. It performs well on diverse matrices and vectors with heterogeneous sparsity patterns. A high-performance implementation of the algorithm attains up to 15x speedup on a 24-core Intel Ivy Bridge processor and up to 49x speedup on a 64-core Intel KNL manycore processor. In contrast to implementations of existing algorithms, the performance of our algorithm is sustained on a variety of different input types include matrices representing scale-free and high-diameter graphs.

97 MATHEMATICS AND COMPUTING↗

A comparative study of shear band tracking strategies in three-dimensional finite elements with embedded weak discontinuities

We present a computational framework for the treatment of shear localization in metallic materials under dynamic loading, based on the integration of a shear band tracking strategy into an explicit 3D finite element formulation with embedded weak discontinuities. Within this computational framework, an embedded shear band’s mid-surface is represented by an iso-surface of a level set function, which is obtained by solving a heatconduction type boundary value problem (BVP). The solution of this BVP is carried out either globally over the entire problem domain, or locally at the level of individual elements in the mesh. In this paper, we present a detailed comparison of these global and local algorithmic implementations of the shear band tracking strategy. Numerical results obtained using these two implementations, and using a simplified formulation without shear band tracking, are presented and compared. Moreover, we compare the computational efficiency and parallel scaling performance of the local and global implementations. This comparative study shows that both implementations can simulate the initiation of two independent shear bands and their propagation past each other without merging, whereas only the global implementation can successfully simulate the merging of two branches of a single shear band. This study also confirms that the global implementation is more computationally intensive, since it requires a global system of linear equations associated with the level set BVP to be solved at each time step. Both implementations exhibit very good scalability in domain decomposition-based parallel simulations.

36 MATERIALS SCIENCE↗

Monitoring the readout temperature in the ICARUS TPC (2023 Italian Students Program Final Report)

Being a Liquid Argon Time Projection Chamber (LAr-TPC) detector, ICARUS exploits a series of three parallel wire planes to measure and collect the ionization charge coming from an event inside the active volume. The wires are read out by A2795 custom boards, arranged in crates located on top of the two liquid Argon vessels. Each board is provided with two temperature sensors that allow temperature monitoring while the experiment is running: this is key in order to ensure data reliability and maintain hardware in safe conditions. Moreover, temperature-related problems may force unplanned shutdowns that result in losing valuable time for the experiment to run. In this work we are discussing the characterization of the ICARUS TPC board temperatures and the system we implemented in order to monitor them effectively. Among the tools used, Grafana results in a valid application for both displaying temperature data and sending alerts to the shifters whenever a pathological situation occurs.

42 ENGINEERING↗

A Robust Parallel Distributed State Estimation for Large Scale Distribution Systems

The growing need and interest in real-time monitoring of large distribution networks motivated by the rapid population of renewable sources, EVs and etc. demand a computationally efficient state estimation framework. Furthermore, this paper presents an improved computational framework for implementing a robust state estimator using a multi-core processor. The main contribution of the paper is the proposed computational framework along with two partitioning strategies which enable fast and robust state estimation for large scale radial and/or meshed distribution systems. Formulation of the proposed method and its implementation are described in detail. Performance of the estimator is tested by simulations first using a small 84-bus radial distribution system. Then the method’s scalability is demonstrated by simulations on two very large scale distribution networks one configured radially and the other meshed each containing over 12,500 buses.

42 ENGINEERING↗

Overlapping localized exponential time differencing methods for diffusion problems

The localized exponential time differencing (ETD) based on overlapping domain decomposition has been recently introduced for extreme-scale phase field simulations of coarsening dynamics, which displays excellent parallel scalability in supercomputers. This paper serves as the first step toward building a solid mathematical foundation for this approach. We study the overlapping localized ETD schemes for a model time-dependent diffusion equation discretized in space by the standard central difference. Two methods are proposed and analyzed for solving the fully discrete localized ETD systems: the first one is based on Schwarz iteration applied at each time step and involves solving stationary problems in the subdomains at each iteration, while the second one is based on the Schwarz waveform relaxation algorithm in which time-dependent subdomain problems are solved at each iteration. The convergences of the associated iterative solutions to the corresponding fully discrete localized ETD solution and to the exact semidiscrete solution are rigorously proved. Finally, numerical experiments are also carried out to confirm theoretical results and to compare the performance of the two methods.

58 GEOSCIENCES↗

Semi-implicit continuum kinetic modeling of weakly collisional parallel transport in a magnetic mirror

We present implicit-explicit (IMEX) kinetic simulations of weakly collisional parallel plasma transport in magnetic mirror configurations using the continuum code COGENT. The numerical scheme employs a Jacobian-free Newton–Krylov method with algebraic multigrid preconditioning to overcome the severe time step limitations imposed by strong mirror forces in fully explicit schemes. Applied to parameters relevant to the Wisconsin HTS Axisymmetric Mirror experiment, the IMEX approach enables time steps up to 2.5×10 4 times larger than those permitted by explicit methods, resulting in a 2500× speedup in 1D–2V simulations of parallel transport with kinetic ions and Boltzmann electrons. Additionally, a reduced bounce-averaged model for a square mirror is implemented to support the computationally intensive fully kinetic simulations. The bounce-averaged formulation is used to evaluate the numerical convergence of the velocity-space discretization algorithms and to assess the role of the collision model by comparing simulations employing the nonlinear Fokker–Planck and the simplified Lenard–Bernstein–Dougherty collision operators.

Collision theories↗

An efficient parallel simulation of unsteady blood flows in patient-specific pulmonary artery

Simulation of blood flows in the pulmonary artery provides some insight into certain diseases by examining the relationship between some continuum metrics, eg, the wall shear stress acting on the vascular endothelium, which responds to flow-induced mechanical forces by releasing vasodilators/constrictors. V. Kheyfets, in his previous work, studies numerically a patient-specific pulmonary circulation to show that decreasing wall shear stress is correlated with increasing pulmonary vascular impedance. We develop a scalable parallel algorithm based on domain decomposition methods to investigate an unsteady model with patient-specific pulsatile waveforms as the inlet boundary condition. The unsteady model offers tremendously more information about the dynamic behavior of the flow field, but computationally speaking, the simulation is a lot more expensive since a problem which is similar to the steady-state problem has to be solved many times, and therefore, the traditional sequential approach is not suitable anymore. We show computationally that simulations using the proposed parallel approach with up to 10 000 processor cores can be obtained with much reduced compute time. Finally, this makes the technology potentially usable for the routine study of the dynamic behavior of blood flows in the pulmonary artery, in particular, the changes of the blood flows and the wall shear stress in the spatial and temporal dimensions.

60 APPLIED LIFE SCIENCES↗

Ensemble Simulation Techniques and Fast Randomized Algorithms

The major goals of the project were to develop and analyze new ensemble simulation techniques, including trajectory stratification and preconditioned MCMC techniques, as well as develop fast numerical linear algebra techniques closely related to ensemble simulation ideas. The trajectory stratification techniques involve simulating in parallel short trajectory fragments of a Markov process confined to a specific region of space‐time and then patching together the statistics gathered to assemble estimates of very general dynamical properties. We have also developed this approach for rare event simulation and extended the techniques to applications requiring a more general framework (such as electronic structure calculations). The preconditioned MCMC techniques involve simulating multiple Markov chains in parallel and then using information from the ensemble to speed the mixing of each individual chain. The fast randomized linear algebra methods are motivated by the diffusion Monte Carlo technique, but are applicable to finding the dominant eigenvalue of (almost) general matrices. For most non‐negative matrices, the schemes result in an error (compared to the power method) that is constant in the dimension of the problem. For more general matrices, we see a very clear sublinear cost trend in computational tests.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

The Athena++ Adaptive Mesh Refinement Framework: Design and Magnetohydrodynamic Solvers

The design and implementation of a new framework for adaptive mesh refinement calculations are described. It is intended primarily for applications in astrophysical fluid dynamics, but its flexible and modular design enables its use for a wide variety of physics. The framework works with both uniform and nonuniform grids in Cartesian and curvilinear coordinate systems. It adopts a dynamic execution model based on a simple design called a "task list" that improves parallel performance by overlapping communication and computation, simplifies the inclusion of a diverse range of physics, and even enables multiphysics models involving different physics in different regions of the calculation. We describe physics modules implemented in this framework for both nonrelativistic and relativistic magnetohydrodynamics (MHD). These modules adopt mature and robust algorithms originally developed for the Athena MHD code and incorporate new extensions: support for curvilinear coordinates, higher-order time integrators, more realistic physics such as a general equation of state, and diffusion terms that can be integrated with super-time-stepping algorithms. The modules show excellent performance and scaling, with well over 80% parallel efficiency on over half a million threads. The source code has been made publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗

TOUGH3-FLAC3D: a modeling approach for parallel computing of fluid flow and geomechanics

The recent development of the TOUGH3 code allows for a faster and more reliable fluid flow simulator. At the same time, new versions of FLAC3D are released periodically, allowing for new features and faster execution. In this paper, we present the first implementation of the coupling between TOUGH3 and FLAC3Dv6/7, maintaining parallel computing capabilities for the coupled fluid flow and geomechanical codes. We compare the newly developed version with analytical solutions and with the previous approach, and provide some performance analysis on different meshes and varying the number of running processors. Finally, we present two case studies related to fault reactivation during CO 2 sequestration and nuclear waste disposal. The use of parallel computing allows for meshes with a larger number of elements, and hence more detailed understanding of thermo-hydro-mechanical processes occurring at depth.

58 GEOSCIENCES↗

A second-order distributed memory parallel fast sweeping method for the Eikonal equation

The Eikonal equation is used to calculate wave propagation and distance fields, and due to its complexity requires numerical treatment for its solution. In this work, we present a second-order distributed memory parallel fast sweeping method. The second-order solution switches on a two-point stencil when two upwind points are available, and reverts to first-order otherwise. In all examples, the second-order method improves the solution over the first-order, allowing for significant savings in memory while achieving the same accuracy. Parallelization over distributed memory saw good weak scaling with optimal convergence. The computational time for second-order was approximately 2.5 times slower than first-order, where the largest amount of mesh points ran on 144 cores (512 GB) was ≈20 billion. The savings in memory from the second-order method combined with the distributed memory algorithm result in the ability to solve problems much larger than are possible with the serial first-order method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The Effective Fragment Molecular Orbital Method: Achieving High Scalability and Accuracy for Large Systems

The effective fragment molecular orbital (EFMO) method has been developed to predict the total energy of a very large molecular system accurately (with respect to the underlying quantum mechanical method) and efficiently by taking advantage of the locality of strong chemical interactions and employing a two-level hierarchical parallelism. The accuracy of the EFMO method is partly attributed to the accurate and robust intermolecular interaction prediction between distant fragments, in particular, the many-body polarization and dispersion effects, which require the generation of static and dynamic polarizability tensors by solving the coupled perturbed Hartree–Fock (CPHF) and time-dependent HF (TDHF) equations, respectively. Solving the CPHF and TDHF equations is the main EFMO computational bottleneck due to the inefficient (serial) and I/O-intensive implementation of the CPHF and TDHF solvers. In this work, the efficiency and scalability of the EFMO method are significantly improved with a new CPU memory-based implementation for solving the CPHF and TDHF equations that are parallelized by either message passing interface (MPI) or hybrid MPI/OpenMP. Here, the accuracy of the EFMO method is demonstrated for both covalently bonded systems and noncovalently bound molecular clusters by systematically examining the effects of basis sets and a key distance-related cutoff parameter, R cut . R cut determines whether a fragment pair (dimer) is treated by the chosen ab initio method or calculated using the effective fragment potential (EFP) method (separated dimers). Decreasing the value of Rcut increases the number of separated (EFP) dimers, thereby decreasing the computational effort. It is demonstrated that excellent accuracy (<1 kcal/mol error per fragment) can be achieved when using a sufficiently large basis set with diffuse functions coupled with a small R cut value. With the new parallel implementation, the total EFMO wall time is substantially reduced, especially with a high number of MPI ranks. Given a sufficient workload, nearly ideal strong scaling is achieved for the CPHF and TDHF parts of the calculation. For the first time, EFMO calculations with the inclusion of long-range polarization and dispersion interactions on a hydrated mesoporous silica nanoparticle with explicit water solvent molecules (more than 15k atoms) are achieved on a massively parallel supercomputer using nearly 1000 physical nodes. In addition, EFMO calculations on the carbinolamine formation step of an amine-catalyzed aldol reaction at the nanoscale with explicit solvent effects are presented.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

kiloAMPS (Final Report)

This project aims to reduce the testing time of various electrochemical techniques for neural interface electrodes by developing an automated PCB design which would be capable of testing multiple interface electrodes in parallel. The system should be able to perform the full battery of electrochemical tests, though electrochemical impedance spectroscopy (EIS) will be of primary interest. Previous versions of this project have already improved upon traditional interface testing by automating many parts of the process, reducing the test time to around a day and requiring the technician to check up on the process once every six hours. A key bottleneck is that the electrode testing is still done sequentially. Testing time could be heavily reduced if electrodes could be tested in parallel.

42 ENGINEERING↗