Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,675 records · Page 93

Investigating the impact of the cielo cray XE6 architecture on scientific application codes.

Cielo, a Cray XE6, is the Department of Energy NNSA Advanced Simulation and Computing (ASC) campaign's newest capability machine. Rated at 1.37 PFLOPS, it consists of 8,944 dual-socket oct-core AMD Magny-Cours compute nodes, linked using Cray's Gemini interconnect. Its primary mission objective is to enable a suite of the ASC applications implemented using MPI to scale to tens of thousands of cores. Cielo is an evolutionary improvement to a successful architecture previously available to many of our codes, thus enabling a basis for understanding the capabilities of this new architecture. Using three codes strategically important to the ASC campaign, and supplemented with some micro-benchmarks that expose the fundamental capabilities of the XE6, we report on the performance characteristics and capabilities of Cielo.

99 GENERAL AND MISCELLANEOUS↗

Improving Uintah's Scalability Through the Use of Portable Kokkos-Based Data Parallel Tasks

The University of Utah’s Carbon Capture Multidisciplinary Simulation Center (CCMSC) is using the Uintah Computational Framework to predict performance of a 1000 MWe ultra-supercritical clean coal boiler. The center aims to utilize the Intel Xeon Phi-based DOE systems, Theta and Aurora, through the Aurora Early Science Program by using the Kokkos C++ library to enable node-level performance portability. This paper describes infrastructure advancements and portability improvements made possible by the integration of Kokkos within Uintah. This integration marks a step towards consolidating Uintah’s MPI+PThreads and MPI+CUDA hybrid parallelism approaches into a single MPI+Kokkos approach. Scalability results are presented that compare serial and data parallel task execution models for a challenging radiative heat transfer calculation, central to the center’s predictive boiler simulations. Here, these results demonstrate both good strong-scaling characteristics to 256 Knights Landing (KNL) processors on the NSF Stampede system, and show the KNL-based calculation to compete with prior GPU-based results for the same calculation.

97 MATHEMATICS AND COMPUTING↗

Layout-Aware I/O Scheduling for Terabits Data Movement

Many science facilities, such as the Department of Energy s Leadership Computing Facilities and experimental facilities including the Spallation Neutron Source, Stanford Linear Accelerator Center, and Advanced Photon Source, produce massive amounts of experimental and simulation data. These data are often shared among the facilities and with collaborating institutions. Moving large datasets over the wide- area network (WAN) is a major problem inhibiting collaboration. Next- generation, terabit-networks will help alleviate the problem, however, the parallel storage systems on the end-system hosts at these institutions can become a bottleneck for terabit data movement. The parallel storage system (PFS) is shared by simulation systems, experimental systems, analysis and visualization clusters, in addition to wide-area data movers. These competing uses often induce temporary, but significant, I/O load imbalances on the storage system, which impact the performance of all the users. The problem is a serious concern because some resources are more expensive (e.g. super computers) or have time-critical deadlines (e.g. experimental data from a light source), but parallel file systems handle all requests fairly even if some storage servers are under heavy load. This paper investigates the problem of competing workloads accessing the parallel file system and how the performance of wide-area data movement can be improved in these environments. First, we study the I/O load imbalance problems using actual I/O performance data collected from the Spider storage system at the Oak Ridge Leadership Computing Facility. Second, we present I/O optimization solutions with layout-awareness on end-system hosts for bulk data movement. With our evaluation, we show that our I/O optimization techniques can avoid the I/O congested disk groups, improving storage I/O times on parallel storage systems for terabit data movement.

I/O and Storage↗

Partnership for Edge Physics Simulation (EPSI)

We propose to develop advanced simulation codes, based upon an extreme parallelism, first principles kinetic approach, to address the challenges associated with the edge region of magnetically confined plasmas. This work is relevant to both existing magnetic fusion facilities and essential for next-generation burning plasma experiments, such as ITER where success is critically dependent upon H-mode operation achieving an edge pedestal of sufficient height for good core plasma performance without producing deleterious large scale edge localized instabilities. The plasma edge presents a well-known set of multi-physics, multi-scale problems involving complex 3D magnetic geometry. Perhaps the greatest computational challenge is the lack of scale separation – temporal scales for drift waves, Alfven waves, ELM dynamics for example have strong overlap. Similar overlap occurs on the spatial scales for the ion poloidal gyro-radius, drift wave and pedestal width. The traditional approach of separating fusion problems into weakly interacting spatial or temporal domains clearly breaks down in the edge. A full kinetic model (full-f model) must be solved to understand and predict the edge physics including non-equilibrium thermodynamic issues arising from the magnetic topology (the open field lines producing a spatially sensitive velocity hole), plasma wall interactions, neutral and atomic physics. The plan here is to model these phenomena within a comprehensive first principles set of equations without the need for the insurmountable multiple-codes coupling issues by building on the XGC1 code developed under the SciDAC Proto-FSP Center for Plasma Edge Simulation (CPES). This proposal includes the critical participants in the XGC1 development. We propose enhancing the capability of XGC1 by including all the important turbulence physics contained in kinetic ion and electron electromagnetic dynamics, by extending the PIC technology to incorporate several positive features found in the Eulerian grid technology, combining the coarse grained XGC0 code into it with a multiscale projective time integration technique to be developed in the project for experimental time scale simulation of the edge multi-physics, and by implementing modern computational technologies to move forward with the extreme scale heterogeneous hardware/software platform development. Ultra-fast edge localized instabilities, involving compressional Alven modes, will be studied using a proper MHD or two-fluid codes employing kinetic closure information from XGC. The proposed research integrates the most important physical processes on overlapping temporal and spatial scales, to study (i) transitions and thresholds for enhanced confinement regimes, (ii) the predictive understanding of the edge pedestal formation, structure and dynamics, (iii) the effect of the edge plasma on the core confinement, (iv) the physics of Edge Localized Modes and their suppression or mitigation via external control techniques, and (v) the heat load on the material wall. Our approach will implement physical models that are valid in their relevant collisionality regimes for current experiments and future burning plasma devices. Comparisons with experimental data will be made at multiple levels including characterization of turbulence dynamics, flux-gradient responses in the energy, particle and momentum channels, and observations of global phenomena such as L-H transitions, momentum generation, ELM dynamics, ELM-free regimes and the tokamak density limit. The proposal team has established strong connections, through joint membership, with other organizations that will be critical for success. These include FASTMath, SUPER and QUEST Institutes respectively for meshing tools and solvers; code performance tuning and uncertainty quantification as well as a proposed Data Management and Visualization Institute.. We will also be teaming with ongoing fusion SciDAC’s: CSWPI for RF-edge plasmas interactions; CEMM for the M3D MHD simulations and with PEPSC for incorporation of energetic particle physics. Finally, the team includes representatives from each of the major U.S. experimental facilities who will coordinate the validation program and the development of synthetic diagnostics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Experimental determination of transient strain in a thermally-cycled simulated turbine blade utilizing a non-contact technique

A type of noncontacting electro-optical extensometer was used to measure the displacement between parallel targets mounted on the leading edge of a simulated turbine blade throughout a complete heating and cooling cycle. The blade was cyclically heated and cooled by moving it into and out of a Mach 1 hot gas stream. The principle of operation and measurement procedure of the electro-optics extensometer are described.

Calfo, F. D.↗

Infrastructure for Multiphysics Software Integration in High Performance Computing-Aided Science and Engineering

The project described in this report constructed and exercised an innovative multiphysics coupling toolkit called the Illinois Rocstar MultiPhysics Application Coupling Toolkit (IMPACT). IMPACT is an open source, flexible, natively parallel infrastructure for coupling multiple uniphysics simulation codes into multiphysics computational systems. IMPACT works with codes written in several high-performance-computing (HPC) programming languages, and is designed from the beginning for HPC multiphysics code development. It is designed to be minimally invasive to the individual physics codes being integrated, and has few requirements on those physics codes for integration. The goal of IMPACT is to provide the support needed to enable coupling existing tools together in unique and innovative ways to produce powerful new multiphysics technologies without extensive modification and rewrite of the physics packages being integrated. There are three major outcomes from this project: 1) construction, testing, application, and open-source release of the IMPACT infrastructure, 2) production of example open-source multiphysics tools using IMPACT, and 3) identification and engagement of interested organizations in the tools and applications resulting from the project. This last outcome represents the incipient development of a user community and application echosystem being built using IMPACT. Multiphysics coupling standardization can only come from organizations working together to define needs and processes that span the space of necessary multiphysics outcomes, which Illinois Rocstar plans to continue driving toward. The IMPACT system, including source code, documentation, and test problems are all now available through the public gitHUB.org system to anyone interested in multiphysics code coupling. Many of the basic documents explaining use and architecture of IMPACT are also attached as appendices to this document. Online HTML documentation is available through the gitHUB site. There are over 100 unit tests provided that run through the Illinois Rocstar Application Development (IRAD) lightweight testing infrastructure that is also supplied along with IMPACT. The package as a whole provides an excellent base for developing high-quality multiphysics applications using modern software development practices. To facilitate understanding how to utilize IMPACT effectively, two multiphysics systems have been developed and are available open-source through gitHUB. The simpler of the two systems, named ElmerFoamFSI in the repository, is a multiphysics, fluid-structure-interaction (FSI) coupling of the solid mechanics package Elmer with a fluid dynamics module from OpenFOAM. This coupling illustrates how to combine software packages that are unrelated by either author or architecture and combine them into a robust, parallel multiphysics system. A more complex multiphysics tool is the Illinois Rocstar Rocstar Multiphysics code that was rebuilt during the project around IMPACT. Rocstar Multiphysics was already an HPC multiphysics tool, but now that it has been rearchitected around IMPACT, it can be readily expanded to capture new and different physics in the future. In fact, during this project, the Elmer and OpenFOAM tools were also coupled into Rocstar Multiphysics and demonstrated. The full Rocstar Multiphysics codebase is also available on gitHUB, and licensed for any organization to use as they wish. Finally, the new IMPACT product is already being used in several multiphysics code coupling projects for the Air Force, NASA and the Missile Defense Agency, and initial work on expansion of the IMPACT-enabled Rocstar Multiphysics has begun in support of a commercial company. These initiatives promise to expand the interest and reach of IMPACT and Rocstar Multiphysics, ultimately leading to the envisioned standardization and consortium of users that was one of the goals of this project.

42 ENGINEERING↗

Recent developments in DYNSUB: New models, code optimization and parallelization

DYNSUB is a high-fidelity coupled code system consisting of the reactor simulator DYN3D and the sub-channel code SUBCHANFLOW. It describes nuclear reactor core behavior with pin-by-pin resolution for both steady-state and transient scenarios. In the course of the coupled code system's active development, super-homogenization (SPH) and generalized equivalence theory (GET) discontinuity factors may be computed with and employed in DYNSUB to compensate pin level homogenization errors. Because of the largely increased numerical problem size for pin-by-pin simulations, DYNSUB has bene fitted from HPC techniques to improve its numerical performance. DYNSUB's coupling scheme has been structurally revised. Computational bottlenecks have been identified and parallelized for shared memory systems using OpenMP. Comparing the elapsed time for simulating a PWR core with one-eighth symmetry under hot zero power conditions applying the original and the optimized DYNSUB using 8 cores, overall speed up factors greater than 10 have been observed. The corresponding reduction in execution time enables a routine application of DYNSUB to study pin level safety parameters for engineering sized cases in a scientific environment. (authors)

97 MATHEMATICS AND COMPUTING↗

CoSyR: A novel beam dynamics code for the modeling of synchrotron radiation effects

The self-consistent nonlinear dynamics of a relativistic charged particle beam interacting with its complete self-fields is a fundamental problem underpinning many of the accelerator design issues in high brightness beam applications, as well as the development of advanced accelerators. Particularly, synchrotron radiation induced effects in a magnetic dispersive beamline element can lead to collective beam instabilities and emittance growth. A novel beam dynamics code is developed based on a Lagrangian method for the calculation of the particles’ radiation near-fields using wavefront/wavelet meshes via the Green’s function of the Maxwell equations. These fields are then interpolated onto a moving mesh for dynamic update of the beam. This method allows radiation co-propagation and self-consistent interaction with the beam in 2D/3D simulations at greatly reduced numerical errors. Multiple levels of parallelisms are inherent in this method and implemented in our code CoSyR to enable at-scale simulations of nonlinear beam dynamics on modern computing platforms using MPI, multi-threading, and GPUs. Here, the current 2D implementation of CoSyR has been used to evaluate the transverse and longitudinal coherent radiation effects on the beam and to investigate beam optics designs proposed for mitigation of beam brightness degradation in a magnetic bunch compressor. In this paper, the design of CoSyR, as well as the benchmark with other coherent synchrotron radiation models, are described and discussed. Extension of the core algorithms to 3D is possible and planned.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Comparative Study of Algorithms for the Numerical Simulation of Lattice QCD

Large-scale numerical simulations are the prime method for a nonperturbative study of QCD from first principles. Although the lattice simulation of the pure-gauge (or quenched-QCD) case may be performed very efficiently on parallel machines, there are several additional difficulties in the simulation of the full-QCD case, i.e. when dynamical quark effects are taken into account. We discuss the main aspects of full-QCD simulations, describing the most common algorithms. We present a comparative analysis of performance for two versions of the hybrid Monte Carlo method (the so-called R and RHMC algorithms), as provided in the MILC software package. We consider two degenerate flavors of light quarks in the staggered formulation, having in mind the case of finite-temperature QCD.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Mirror force induced wave dispersion in Alfvén waves

Recent hybrid MHD-kinetic electron simulations of global scale standing shear Alfvén waves along the Earth's closed dipolar magnetic field lines show that the upward parallel current region within these waves saturates and broadens perpendicular to the ambient magnetic field and that this broadening increases with the electron temperature. Using resistive MHD simulations, with a parallel Ohm's law derived from the linear Knight relation (which expresses the current-voltage relationship along an auroral field line), we explore the nature of this broadening in the context of the increased perpendicular Poynting flux resulting from the increased parallel electric field associated with mirror force effects. This increased Poynting flux facilitates wave energy dispersion across field lines which in-turn allows for electron acceleration to carry the field aligned current on adjacent field lines. This mirror force driven dispersion can dominate over that associated with electron inertial effects for global scale waves.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Layout-aware I/O Scheduling for terabits data movement

Many science facilities, such as the Department of Energy's Leadership Computing Facilities and experimental facilities including the Spallation Neutron Source, Stanford Linear Accelerator Center, and Advanced Photon Source, produce massive amounts of experimental and simulation data. These data are often shared among the facilities and with collaborating institutions. Moving large datasets over the wide-area network (WAN) is a major problem inhibiting collaboration. Next-generation, terabit-networks will help alleviate the problem, however, the parallel storage systems on the endsystem hosts at these institutions can become a bottleneck for terabit data movement. The parallel storage system (PFS) is shared by simulation systems, experimental systems, analysis and visualization clusters, in addition to wide-area data movers. These competing uses often induce temporary, but significant, I/O load imbalances on the storage system, which impact the performance of all the users. The problem is a serious concern because some resources are more expensive (e.g. super computers) or have time-critical deadlines (e.g. experimental data from a light source), but parallel file systems handle all requests fairly even if some storage servers are under heavy load. This paper investigates the problem of competing workloads accessing the parallel file system and how the performance of wide-area data movement can be improved in these environments. First, we study the I/O load imbalance problems using actual I/O performance data collected from the Spider storage system at the Oak Ridge Leadership Computing Facility. Second, we present I/O optimization solutions with layout-awareness on end-system hosts for bulk data movement. With our evaluation, we show that our I/O optimization techniques can avoid the I/O congested disk groups, improving storage I/O times on parallel storage systems for terabit data movement.

Kim, Youngjae↗

Designing the Cloud-based DOE Systems Biology Knowledgebase

Systems Biology research, even more than many other scientific domains, is becoming increasingly data-intensive. Not only have advances in experimental and computational technologies lead to an exponential increase in scientific data volumes and their complexity, but increasingly such databases themselves are providing the basis for new scientific discoveries. To engage effectively with these community resources, integrated analyses, synthesis and simulation software is needed, regularly supported by scientific workflows. In order to provide a more collaborative, community driven research environment for this heterogeneous setting, the Department of Energy (DOE) has decided to develop a federated, cloud based cyber infrastructure - the Systems Biology Knowledgebase (Kbase). Pacific Northwest National Laboratory (PNNL) with its long tradition in data intensive science lead two of the five initial pilot projects, these two focusing on defining and testing the basic federated cloud-based system architecture and develop a prototype implementation. Hereby the community wide accessibility of biological data and the capability to integrate and analyze this data within its changing research context were seen as key technical functionalities the Kbase needed to enable. In this paper we describe the results of our investigations into the design of a cloud based federated infrastructure for: (1) Semantics driven data discovery, access and integration; (2) Data annotation, publication and sharing; (3) Workflow enabled data analysis; and (4) Project based collaborative working. We describe our approach, exemplary use cases and our prototype implementation that demonstrates the feasibility of this approach.

99 GENERAL AND MISCELLANEOUS↗

Neural Network Molecular Dynamics at Scale

Neural network molecular dynamics (NNMD) simulations could revolutionize atomistic modeling of materials with quantum-mechanical accuracy at a fraction of computational cost. However, popular NNMD frameworks are generally implemented for a single computing node, and conventional energy-based NN models still suffer from large time-to-solution (T2S), prohibiting the application of NNMD to challenging materials simulations encompassing large spatiotemporal scales. Consequently, no leadership-scale NNMD simulation has thus far been reported. Here, we present a scalable parallel NNMD software (RXMD-NN) based on our scalable reactive molecular dynamics simulation engine named RXMD. RXMD-NN has achieved high scalability up to 786,432 IBM BlueGene/Q cores involving 1.7 billion atoms. Furthermore, we have achieved 4.6-fold reduction of T2S by using a novel network that directly predicts atomic forces from feature vectors. Reduced T2S has for the first time allowed the study of large-scale off-stoichiometry effect in a widely used phase change material, Ge 2 Se 2 Te 5 , thereby resolving its "firstsharp diffraction peak mystery".

fist sharp diffraction peak↗

Kinetic entropy-based measures of distribution function non-Maxwellianity: theory and simulations

We investigate kinetic entropy-based measures of the non-Maxwellianity of distribution functions in plasmas, i.e. entropy-based measures of the departure of a local distribution function from an associated Maxwellian distribution function with the same density, bulk flow and temperature as the local distribution. First, we consider a form previously employed by Kaufmann & Paterson (J. Geophys. Res., vol. 114, 2009, A00D04), assessing its properties and deriving equivalent forms. To provide a quantitative understanding of it, we derive analytical expressions for three common non-Maxwellian plasma distribution functions. We show that there are undesirable features of this non-Maxwellianity measure including that it can diverge in various physical limits and elucidate the reason for the divergence. We then introduce a new kinetic entropy-based non-Maxwellianity measure based on the velocity-space kinetic entropy density, which has a meaningful physical interpretation and does not diverge. We use collisionless particle-in-cell simulations of two-dimensional anti-parallel magnetic reconnection to assess the kinetic entropy-based non-Maxwellianity measures. We show that regions of non-zero non-Maxwellianity are linked to kinetic processes occurring during magnetic reconnection. We also show the simulated non-Maxwellianity agrees reasonably well with predictions for distributions resembling those calculated analytically. These results can be important for applications, as non-Maxwellianity can be used to identify regions of kinetic-scale physics or increased dissipation in plasmas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Self-Consistent Relativistic Electron Scattering using the Sherlock Scattering Model for X-ray Diagnostics

We present on a new, self-consistent, arbitrary-temperature Romberg integration scheme for modeling electron scattering in materials in a LANL Lagrangian Shock Hydro (LSH) code. Electron beam-target interactions are fundamental to a wide range of scientific and technological applications. When high-energy electron beams hit their target, they may scatter, deposit energy, or ionize the source. These processes govern the behavior and outcomes in nanotechnology manufacturing, electron microscopy, and modern X-ray diagnostics. Simulating these interactions is essential for interpreting experimental results, predicting material responses, and designing efficient tools and experiments. At Los Alamos, this is done using a LSH code, which is a multi-dimension, multi-material, massively parallel, multi-physics code used to simulate applications from asteroid impacts to electron beam interactions. By effectively and efficiently modeling the way that electrons scatter from the beam we can bolster these simulations and more accurately predict experimental outcomes. The model currently implemented in the LSH of interest is based on work by Papp and does not self-consistently preserve momentum in the slightly relativistic regime; here we adopt a model proposed by Braams and Karney and implement a Romberg integration scheme to compute the diffusion tensor. In this paper we will provide background on the Braams-Karney diffusion tensor as well as the Romberg integration scheme we employed to numerically solve for it. We will show that our integration scheme is accurate in solving for the set of scalar potentials used to re-express the diffusion tensor in differential form, and in solving for the diffusion coefficients in the larger LSH code. By using this diffusion tensor rather than the existing Papp one, and numerically integrating it with a Romberg method, we produce much more accurate, self-consistent results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Aperture-based antihydrogen gravity experiment: Parallel plate geometry

An analytical model and a Monte Carlo simulation are presented of an experiment that could be used to determine the direction of the acceleration of antihydrogen due to gravity. The experiment would rely on methods developed by existing antihydrogen research collaborations. The configuration consists of two circular, parallel plates that have an axis of symmetry directed away from the center of the earth. The plates are separated by a small vertical distance, and include one or more pairs of circular barriers that protrude from the upper and lower plates, thereby forming an aperture between the plates. Antihydrogen annihilations that occur just beyond each barrier, within a “shadow” region, are asymmetric on the upper plate relative to the lower plate. The probability for such annihilations is determined for a point, line and spheroidal source of antihydrogen. The production of 100,000 antiatoms is predicted to be necessary for the aperture-based experiment to indicate the direction of free fall acceleration of antimatter, provided that antihydrogen is produced within a sufficiently small antiproton plasma at a temperature of 4 K.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A piloted comparison of elastic and rigid blade-element rotor models using parallel processing technology

A piloted comparison of rigid and aeroelastic blade-element rotor models was conducted at the Crew Station Research and Development Facility (CSRDF) at Ames Research Center. FLIGHTLAB, a new simulation development and analysis tool, was used to implement these models in real time using parallel processing technology. Pilot comments and quantitative analysis performed both on-line and off-line confirmed that elastic degrees of freedom significantly affect perceived handling qualities. Trim comparisons show improved correlation with flight test data when elastic modes are modeled. The results demonstrate the efficiency with which the mathematical modeling sophistication of existing simulation facilities can be upgraded using parallel processing, and the importance of these upgrades to simulation fidelity.

Hill, Gary↗

The computation of supersonic combustor flows using multi-computers

An explicit CFD computer code with parallel processing is developed for the purpose of simulating internal high-speed reacting flows. The code solves the 3D Navier-Stokes equations for compressible flows. Preliminary results indicate that parallel processing should solve the resource and speed shortages presently encountered with sequential computers for problems of interest.

Underwood, Mark↗