Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel mesh generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Modeling the Structural Response from a Propagating High Explosive Using Smooth Particle Hydrodynamics

This report primarily concerns the use of two massively parallel finite element codes originally written and maintained at Lawrence Livermore National Laboratory. ALE3D is an explicit hydrodynamics code commonly employed to simulate wave propagation from high energy scenarios and the resulting interaction with nearby structures. This coupled response ensures that a structure is accurately applied with a blast loading varying both in space and time. Figure 1 illustrates the radial outward propagation of a pressure wave due to a center detonated spherical explosive originating from the lower left. The radial symmetry seen in this scenario is lost when instead a cylindrocal charge is detonated. Figure 2 indicates that a stronger, faster traveling pressure wave occurs in the direction of the normal axis to the cylinder. The ALE3D name is derived because of the use of arbitrary-Lagrange-Eulerian elements in which the mesh is allowed to advect; a process through which the mesh is modified to alleviate tanlging and general mesh distortion often cuased by high energy scenarios. The counterpart to an advecting element is a Lagrange element, whose mesh moves with the material. Ideally all structural components are kept Lagrange as long as possible to preserve accuracy of material variables and minimize advection related errors. Advection leads to mixed zoning, so using structural Lagrange elements also improves the visualization when post processing the results. A simplified representation of the advection process is shown in Figure 3. First the mesh is distorted due to material motion during the Lagrange step. The mesh is then shifted to an idealized and less distorted state to prevent irregular zones caused by the Lagrange motion. Lastly, the state variables are remapped to the elements of the newly constructed mesh. Note that Figure 3 represents a purely Eulerian mesh relaxation because the mesh is relocated back to the pre-Lagrange position. This is the case when the material flows through a still mesh. This is not typically done in an ALE3D analysis, especially if Lagrange elements exist. Deforming Lagrange elements would certainly tangle with a Eulerian mesh eventually. The best method in this case is to have an advecting mesh positioned as some relaxed version of the pre and post Lagrange step; this gives the best opportunity of modeling a high energy event with a combination of Lagrange and ALE elements. Dyne3D is another explicit dynamic analysis code, ParaDyn being the parallel version. ParaDyn is used for predicting the transient response of three dimensional structures using Lagrangian solid mechanics. Large deformation and mesh tangling is often resolved through the use of an element deletion scheme. This is useful to accommodate component failure, but if it is done purely as a means to preserve a useful mesh it can lead to problems because it does not maintain continuity of the material bulk response. Whatever medium exists between structural components is typically not modeled in ParaDyn. Instead, a structure either has a known loading profile applied or given initial conditions. The many included contact algorithms can calculate the loading response of materials if and when they collide. A recent implementation of an SPH module in which failed or deleted material nodes are converted to independent particles is currently being utilized for a variety of spall related problems and high velocity impact scenarios. Figure 4 shows an example of a projectile, given an initial velocity, and how it fails the first plate which generates SPH particles which then interact with and damage the second plate.

36 MATERIALS SCIENCE↗

Initial Investigation of Coupled Electron-Photon Transport with SCEPTRE and RAPTURE (Rev.1.1)

The SCEPTRE1 code models coupled photon and electron transport in material media using discrete ordinates methods. The code can use different solvers in 1D and 3D in parallel, promising users the ability to run problems that are significant both in physics and computational scale. One capability that is perhaps unique at this time in the NNSA complex is SCEPTRE’s coupled photon– electron transport physics in a discrete ordinates approach. For example, incident x–rays can generate a significant electron population, whose transport can be simulated within the same code. In this context, fully–coupled means that photons can produce electrons and electrons can produce photons; uncoupled or no–coupling means photons do not produce electrons. Partial–coupling, an intermediate option in which photons can produce electrons but electrons do not produce photons, is not considered in the present work. The MCNP code can treat this physics with a Monte Carlo approach. MCNP should be viewed as complementary to SCEPTRE in both computational method (Monte Carlo versus discrete ordinates) as well as data libraries. SCEPTRE and MCNP should provide important checks on each other, especially from complementary modeling frameworks of a S n (discrete ordinates) mesh based method versus Monte Carlo particle based method. In order to learn more about SCEPTRE, and how it might inform engineering work at the Laboratory, we used it to carry out a set of initial studies. This was a series of 1D simulations of photon beams hitting Aluminum slabs. Our studies mainly used SCEPTRE directly, and made little use of the closely associated RAPTURE code (more on this below). The goals of these studies were first, to familiarize ourselves with operating the code, and second, to begin to assess the importance of coupled electron–photon transport. To support these studies, we began to develop a set of scripts to initialize inputs, run tools, process outputs, and plot results. An initial version of these scripts has been committed to the Lightning workflow repository. This memo relates the results of those studies.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Development of a Robust and Efficient Parallel Solver for Unsteady Turbomachinery Flows

The traditional design and analysis practice for advanced propulsion systems relies heavily on expensive full-scale prototype development and testing. Over the past decade, use of high-fidelity analysis and design tools such as CFD early in the product development cycle has been identified as one way to alleviate testing costs and to develop these devices better, faster and cheaper. In the design of advanced propulsion systems, CFD plays a major role in defining the required performance over the entire flight regime, as well as in testing the sensitivity of the design to the different modes of operation. Increased emphasis is being placed on developing and applying CFD models to simulate the flow field environments and performance of advanced propulsion systems. This necessitates the development of next generation computational tools which can be used effectively and reliably in a design environment. The turbomachinery simulation capability presented here is being developed in a computational tool called Loci-STREAM [1]. It integrates proven numerical methods for generalized grids and state-of-the-art physical models in a novel rule-based programming framework called Loci [2] which allows: (a) seamless integration of multidisciplinary physics in a unified manner, and (b) automatic handling of massively parallel computing. The objective is to be able to routinely simulate problems involving complex geometries requiring large unstructured grids and complex multidisciplinary physics. An immediate application of interest is simulation of unsteady flows in rocket turbopumps, particularly in cryogenic liquid rocket engines. The key components of the overall methodology presented in this paper are the following: (a) high fidelity unsteady simulation capability based on Detached Eddy Simulation (DES) in conjunction with second-order temporal discretization, (b) compliance with Geometric Conservation Law (GCL) in order to maintain conservative property on moving meshes for second-order time-stepping scheme, (c) a novel cloud-of-points interpolation method (based on a fast parallel kd-tree search algorithm) for interfaces between turbomachinery components in relative motion which is demonstrated to be highly scalable, and (d) demonstrated accuracy and parallel scalability on large grids (approx 250 million cells) in full turbomachinery geometries.

West, Jeff↗

Developing Uintah's Runtime System for Forthcoming Architectures

The need to solve ever-more complex science and engineering problems on computer architectures that are rapidly evolving poses considerable challenges for modern simulation frameworks and packages. While the decomposition of software into a programming model that generates a task-graph for execution by a runtime system make portability and performance possible, it also imposes considerable demands on that runtime system. Here, the challenge of meeting those demands for near-term and longer term systems in the context of the Uintah framework is considered in key areas such as support for data structures, tasks on heterogeneous architectures, performance portability, power management and designing for resilience by the use of replication based upon adaptive mesh refinement processes is addressed. Examples of techniques in these areas are used to illustrate performance improvements obtained for present large-scale systems and the needs of proposed systems considered.

97 MATHEMATICS AND COMPUTING↗

TRIPOLI-4{sup R} version 9 Shielding for the Shielding Community: Overview, Relevant Benchmarking for Shielding and Licensing Policy - Paper 49

This is a review paper of the Shielding only version of TRIPOLI-4. TRIPOLI-4 and TRIPOLI-4 S: TRIPOLI{sup R} represents a family of radiation transport codes using the Monte Carlo method (1,2). It has been continuously developed at CEA since the mid 60's. TRIPOLI-4{sup R}, being the fourth generation, has been developed starting from the mid 90's in C++. It tracks neutrons from 20 MeV down to 10{sup -5} eV, photons, electrons and positrons from 100 MeV to 1 keV, in 3D arbitrary geometries either in TRIPOLI{sup R} proprietary format or ROOT(4) format. It reads its continuous energy nuclear data from any evaluation in ENDF format, including but not limited to JEFF-3.1.1 and ENDF/BVII.0. It has various simulation modes, fixed-source (with or without fission), criticality, or both of them chained. A variety of tallies are available: averaged flux in a volume (track length and collision estimators), surface flux and current, point flux, reaction rates, deposited energy, dpa, pka, gamma spectroscopy, mesh tallies, tallies for criticality (k{sub eff}, beta eff, kinetic parameters). It provides the user with embedded response functions from IRDF and EAF as well. Several variance reduction techniques (3), including automated ones, make it particularly suitable to deep penetration problems (implicit capture, Russian roulette, exponential biasing with automatic estimation of the importance map). It may run in parallel, on single multi-core machines as well as heterogeneous networks of stations or massively parallel machines. It comes with productivity tools designed to speed-up the elaboration and verification of the input deck as well as the output analysis: 2D interactive display of geometry; 2D display of mesh tallies and collision sites; 3D input deck modeler; compatibility with ROOT tools to post-process TRIPOLI{sup R} outputs. Its V and V test base is made of more than 1000 internationally accepted benchmarks from ICSBEP and SINBAD (REPLICA, ASPIS, NESDIP) and covers all its application domains, including several shielding cases which will be detailed in the talk. TRIPOLI-4{sup R} is the reference Monte Carlo code for CEA, EDF, and also for AREVA for a number of applications. A 'Shielding-only' version has been designed to specifically answer the needs of the shielding community. It is distributed through the NEA Databank with a license for R and D, covering the fusion applications, whose terms are detailed in the paper.

61 RADIATION PROTECTION AND DOSIMETRY↗

MOOSE framework enhancements for meshing reactor geometries

MOOSE is an open-source, parallel finite element framework designed to permit rapid development of robust multi-physics modeling capabilities. Under the DOE-NEAMS program, numerous solvers have been developed utilizing the open-source MOOSE framework for multiphysics reactor analysis. These solvers require input finite element meshes representing the discretized geometry. Typically, reactor analysts turn to licensed external tools for creation of reactor geometry meshes. Recently, enhancements have been added to the MOOSE framework to mesh common reactor geometries and improve MOOSE-based application user workflows. Support for hexagonal pins, assemblies, and cores has been added, and Cartesian support has been extended. Options for modeling static and rotating control drums within a hexagonal assembly are now available. Pin, assembly, and plane regions can be identified through automatically applied tags on the mesh called 'reporting IDs' for easier post- processing of physics results. An external open-source triangle routine has been leveraged within MOOSE to mesh core periphery zones. A set of reactor geometry builder routines further streamline the construction of hexagonal and Cartesian cores and include the ability to assign materials to regions during mesh generation. The new meshing routines are available through the MOOSE framework in the open-source 'Reactor' module, and the resulting directly within MOOSE-based applications or exported as Exodus II files for use in other finite element solvers. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Reference guide to WPP version 2.0

WPP is a computer program for simulating seismic wave propagation on parallel machines. WPP solves the governing equations in second order formulation using a node-based finite difference approach. The basic numerical method is described in [9]. WPP implements substantial capabilities for 3-D seismic modeling, with a free surface condition on the top boundary, non-reflecting far-field boundary conditions on the other boundaries, point force and point moment tensor source terms with many predefined time dependencies, fully 3-D heterogeneous material model specification, output of synthetic seismograms in the SAC [4] format, output of GMT [11] scripts for laying out simulation information on a map, and output of 2-D slices of (derived quantites of) the solution field as well as the material model. Version 2.0 of WPP allows the free surface boundary condition to be imposed on a curved topography. For this purpose a curvilinear mesh is used near the free surface, extending into the computational domain down to a user specified level. The elastic wave equations and the free surface boundary conditions are discretized on the curvilinear mesh using the energy conserving technique described in [2]. A curvilinear mesh generator is built into WPP and the curvilinear mesh is automatically generated from the topography. Below the curvilinear grid, the elastic wave equation is discretized on Cartesian meshes, which leads to a more computationally efficient algorithm. In version 2.0 of WPP, Cartesian local mesh refinement can be used to make the computational mesh finer near the free surface, where more resolution often is needed to resolve short wave lenghts in the solution, for example in sedimentary basins. The mesh refinement is performed in the vertical direction and each Cartesian grid is constructed from user specified refinement levels. In this approach, the grid size in all three spatial directions is doubled across each mesh refinement interface, leading to substantial savings in memory and computational effort. The energy conserving mesh refinement coupling method described in [10] is used to handle the hanging nodes along the refinement interface. The examples subdirectory of the WPP source distribution contains several examples and validation tests. Many Matlab/octave scripts are provided in the tools directory.

58 GEOSCIENCES↗

Analyzing and visualizing next generation climate data (Final Report)

The project "Analyzing and visualizing next generation climate data" adds block-structured (mosaic) grid support, parallel processing, and 2D/3D curvilinear interpolation to the open-source UV-CDAT climate data analysis tool. Block structured grid support complies to the Gridspec extension submitted to the Climate and Forecast metadata conventions. It contains two parts: aggregation of data spread over multiple mosaic tiles (M-SPEC) and aggregation of temporal data stored in different files (F-SPEC). Together, M-SPEC and F-SPEC allow users to interact with data stored in multiple files as if the data were in a single file. For computational expensive tasks, a flexible, multi-dimensional, multi-type distributed array class allows users to process data in parallel using remote memory access. Both nodal and cell based interpolation is supported; users can choose between different interpolation libraries including ESMF and LibCF depending on the their particular needs.

54 ENVIRONMENTAL SCIENCES↗

Efficient exascale discretizations: High-order finite element methods

Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on unstructured grids is to use matrix-free/partially assembled high-order finite element methods, since these methods can increase the accuracy and/or lower the computational time due to reduced data motion. In this paper we provide an overview of the research and development activities in the Center for Efficient Exascale Discretizations (CEED), a co-design center in the Exascale Computing Project that is focused on the development of next-generation discretization software and algorithms to enable a wide range of finite element applications to run efficiently on future hardware. CEED is a research partnership involving more than 30 computational scientists from two US national labs and five universities, including members of the Nek5000, MFEM, MAGMA and PETSc projects. We discuss the CEED co-design activities based on targeted benchmarks, miniapps and discretization libraries and our work on performance optimizations for large-scale GPU architectures. We also provide a broad overview of research and development activities in areas such as unstructured adaptive mesh refinement algorithms, matrix-free linear solvers, high-order data visualization, and list examples of collaborations with several ECP and external applications.

97 MATHEMATICS AND COMPUTING↗

Generation of gear tooth surfaces by application of CNC machines

This study will demonstrate the importance of application of computer numerically controlled (CNC) machines in generation of gear tooth surfaces with new topology. This topology decreases gear vibration and will extend the gear capacity and service life. A preliminary investigation by a tooth contact analysis (TCA) program has shown that gear tooth surfaces in line contact (for instance, involute helical gears with parallel axes, worm gear drives with cylindrical worms, etc.) are very sensitive to angular errors of misalignment that cause edge contact and an unfavorable shape of transmission errors and vibration. The new topology of gear tooth surfaces is based on the localization of bearing contact, and the synthesis of a predesigned parabolic function of transmission errors that is able to absorb a piecewise linear function of transmission errors caused by gear misalignment. The report will describe the following topics: description of kinematics of CNC machines with six degrees of freedom that can be applied for generation of gear tooth surfaces with new topology. A new method for grinding of gear tooth surfaces by a cone surface or surface of revolution based on application of CNC machines is described. This method provides an optimal approximation of the ground surface to the given one. This method is especially beneficial when undeveloped ruled surfaces are to be ground. Execution of motions of the CNC machine is also described. The solution to this problem can be applied as well for the transfer of machine tool settings from a conventional generator to the CNC machine. The developed theory required the derivation of a modified equation of meshing based on application of the concept of space curves, space curves represented on surfaces, geodesic curvature, surface torsion, etc. Condensed information on these topics of differential geometry is provided as well.

Litvin, F. L.↗

Mapping of orthogonal 2D flux coordinates for two nearby magnetic X-points to logically rectangular domains

The behavior of plasmas in configurations with magnetic null points is of potential interest for a number of systems. Such nulls, sometimes referred to as X-points, a name strictly valid only for wellseparated nulls where magnetic separatrices cross at a right angle, may occur in only one component of the magnetic field, such as the poloidal field in a tokamak. Numerical simulations are often used to study and predict turbulence and transport in such systems. For cases with strong magnetic field such that the ion and electron gyro-frequencies are much larger that particle collision frequencies, there is a strong anisotropy in the plasma transport along the magnetic field compared to across the field. Because of the strong anisotropy, it is often advantageous for simulations to use a coordinate system based on magnetic flux surfaces together with a coordinate orthogonal to the flux surfaces. This report presents the topological considerations needed to construct the mapping on an orthogonal flux-surface mesh onto a logically rectangular domain often used in simulation codes of the divertor region of tokamaks having two nearby X-points, thus extending the mapping now common single-null or well-separated double nullpoint cases with one null located at the top and bottom of the device. This extension is especially relevant to study the capability of snowflake divertors in tokamaks for reducing the peak heat flux of exhaust plasma on the target plates [1-3]. The analysis presented here follows the analytic description of the poloidal magnetic field, B pol , near two closely-spaced X-points given by Ryutov [1-3]. This analysis is applicable to experimental devices where the two X-points are generated by distributed poloidal field coils and the plasma core current. This analytic description is used to identify topological features of an orthogonal mesh. A key parameter that specifies the various topology changes is the poloidal angle between the primary X-point and the second X-point, and the changes are associated with a set of critical angles. Understanding these changes leads to the final step in the mesh construction by identifying the locations of various regions or patches in configuration space and in a corresponding logically rectangular index space. UEDGE [4] and other 2-D fluid plasma simulation codes [5-7] describe the edge plasma of toroidally symmetric tokamak configurations with one or more X-points in a divertor region. The 2-D spatial mesh is usually based on a magnetic configuration produced by an MHD equilibrium code such as EFIT and written out as one or more “eqdsk” files. As illustrated in Fig. 1, the domain of the simulated plasma consists of several flux tubes bounded by poloidal magnetic flux surfaces. The number and spacing of the flux surfaces defines the “radial” resolution of the mesh. The “radial” direction is locally orthogonal to flux surfaces; the “poloidal” direction is locally parallel to flux surfaces. The “poloidal” resolution of the mesh is defined by the number of cells in the poloidal direction between adjacent flux surfaces. For boundary plasma transport codes used for tokamak configurations such as UEDGE it is convenient, though not necessary, to assume that all flux tubes have the same number of poloidal cells so the mesh is logically rectangular in index space, but does possess internal boundaries linking different sub-rectangles in the computational domain. The following sections of this report describe indexing schemes that map tokamak magnetic configurations with one or more x-points in the divertor region(s) onto a rectangular index space. Simulation results from the UEDGE code illustrate the schemes for a few configurations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

TH-E-18A-01: Developments in Monte Carlo Methods for Medical Imaging

Monte Carlo simulation methods are widely used in medical physics research and are starting to be implemented in clinical applications such as radiation therapy planning systems. Monte Carlo simulations offer the capability to accurately estimate quantities of interest that are challenging to measure experimentally while taking into account the realistic anatomy of an individual patient. Traditionally, practical application of Monte Carlo simulation codes in diagnostic imaging was limited by the need for large computational resources or long execution times. However, recent advancements in high-performance computing hardware, combined with a new generation of Monte Carlo simulation algorithms and novel postprocessing methods, are allowing for the computation of relevant imaging parameters of interest such as patient organ doses and scatter-to-primaryratios in radiographic projections in just a few seconds using affordable computational resources. Programmable Graphics Processing Units (GPUs), for example, provide a convenient, affordable platform for parallelized Monte Carlo executions that yield simulation times on the order of 10{sup 7} xray/ s. Even with GPU acceleration, however, Monte Carlo simulation times can be prohibitive for routine clinical practice. To reduce simulation times further, variance reduction techniques can be used to alter the probabilistic models underlying the x-ray tracking process, resulting in lower variance in the results without biasing the estimates. Other complementary strategies for further reductions in computation time are denoising of the Monte Carlo estimates and estimating (scoring) the quantity of interest at a sparse set of sampling locations (e.g. at a small number of detector pixels in a scatter simulation) followed by interpolation. Beyond reduction of the computational resources required for performing Monte Carlo simulations in medical imaging, the use of accurate representations of patient anatomy is crucial to the virtual generation of medical images and accurate estimation of radiation dose and other imaging parameters. For this, detailed computational phantoms of the patient anatomy must be utilized and implemented within the radiation transport code. Computational phantoms presently come in one of three format types, and in one of four morphometric categories. Format types include stylized (mathematical equation-based), voxel (segmented CT/MR images), and hybrid (NURBS and polygon mesh surfaces). Morphometric categories include reference (small library of phantoms by age at 50th height/weight percentile), patient-dependent (larger library of phantoms at various combinations of height/weight percentiles), patient-sculpted (phantoms altered to match the patient's unique outer body contour), and finally, patient-specific (an exact representation of the patient with respect to both body contour and internal anatomy). The existence and availability of these phantoms represents a very important advance for the simulation of realistic medical imaging applications using Monte Carlo methods. New Monte Carlo simulation codes need to be thoroughly validated before they can be used to perform novel research. Ideally, the validation process would involve comparison of results with those of an experimental measurement, but accurate replication of experimental conditions can be very challenging. It is very common to validate new Monte Carlo simulations by replicating previously published simulation results of similar experiments. This process, however, is commonly problematic due to the lack of sufficient information in the published reports of previous work so as to be able to replicate the simulation in detail. To aid in this process, the AAPM Task Group 195 prepared a report in which six different imaging research experiments commonly performed using Monte Carlo simulations are described and their results provided. The simulation conditions of all six cases are provided in full detail, with all necessary data on material composition, source, geometry, scoring and other parameters provided. The results of these simulations when performed with the four most common publicly available Monte Carlo packages are also provided in tabular form. The Task Group 195 Report will be useful for researchers needing to validate their Monte Carlo work, and for trainees needing to learn Monte Carlo simulation methods. In this symposium we will review the recent advancements in highperformance computing hardware enabling the reduction in computational resources needed for Monte Carlo simulations in medical imaging. We will review variance reduction techniques commonly applied in Monte Carlo simulations of medical imaging systems and present implementation strategies for efficient combination of these techniques with GPU acceleration. Trade-offs involved in Monte Carlo acceleration by means of denoising and “sparse sampling” will be discussed. A method for rapid scatter correction in cone-beam CT (<5 min/scan) will be presented as an illustration of the simulation speeds achievable with optimized Monte Carlo simulations. We will also discuss the development, availability, and capability of the various combinations of computational phantoms for Monte Carlo simulation of medical imaging systems. Finally, we will review some examples of experimental validation of Monte Carlo simulations and will present the AAPM Task Group 195 Report. Learning Objectives: Describe the advances in hardware available for performing Monte Carlo simulations in high performance computing environments. Explain variance reduction, denoising and sparse sampling techniques available for reduction of computational time needed for Monte Carlo simulations of medical imaging. List and compare the computational anthropomorphic phantoms currently available for more accurate assessment of medical imaging parameters in Monte Carlo simulations. Describe experimental methods used for validation of Monte Carlo simulations in medical imaging. Describe the AAPM Task Group 195 Report and its use for validation and teaching of Monte Carlo simulations in medical imaging.

60 APPLIED LIFE SCIENCES↗

Improving Fidelity of Launch Vehicle Liftoff Acoustic Simulations

Launch vehicles experience high acoustic loads during ignition and liftoff affected by the interaction of rocket plume generated acoustic waves with launch pad structures. Application of highly parallelized Computational Fluid Dynamics (CFD) analysis tools optimized for application on the NAS computer systems such as the Loci/CHEM program now enable simulation of time-accurate, turbulent, multi-species plume formation and interaction with launch pad geometry and capture the generation of acoustic noise at the source regions in the plume shear layers and impingement regions. These CFD solvers are robust in capturing the acoustic fluctuations, but they are too dissipative to accurately resolve the propagation of the acoustic waves throughout the launch environment domain along the vehicle. A hybrid Computational Fluid Dynamics and Computational Aero-Acoustics (CFD/CAA) modeling framework has been developed to improve such liftoff acoustic environment predictions. The framework combines the existing highly-scalable NASA production CFD code, Loci/CHEM, with a high-order accurate discontinuous Galerkin (DG) solver, Loci/THRUST, developed in the same computational framework. Loci/THRUST employs a low dissipation, high-order, unstructured DG method to accurately propagate acoustic waves away from the source regions across large distances. The DG solver is currently capable of solving up to 4th order solutions for non-linear, conservative acoustic field propagation. Higher order boundary conditions are implemented to accurately model the reflection and refraction of acoustic waves on launch pad components. The DG solver accepts generalized unstructured meshes, enabling efficient application of common mesh generation tools for CHEM and THRUST simulations. The DG solution is coupled with the CFD solution at interface boundaries placed near the CFD acoustic source regions. Both simulations are executed simultaneously with coordinated boundary condition data exchange.

Liever, Peter↗

Efficacy of the Cell Perturbation Method in Large-Eddy Simulations of Boundary Layer Flow over Complex Terrain

A challenge to simulating turbulent flow in multiscale atmospheric applications is the efficient generation of resolved turbulence motions over an area of interest. One approach is to apply small perturbations to flow variables near the inflow planes of turbulence-resolving simulation domains nested within larger mesoscale domains. While this approach has been examined in numerous idealized and simple terrain cases, its efficacy in complex terrain environments has not yet been fully explored. Here, we examine the benefits of the stochastic cell perturbation method (CPM) over real complex terrain using data from the 2017 Perdigão field campaign, conducted in an approximately 2-km wide valley situated between two nearly parallel ridges. Following a typical configuration for multiscale simulation using nested domains within the Weather Research and Forecasting (WRF) model to downscale from the mesoscale to a large-eddy simulation (LES), we apply the CPM on a domain with horizontal grid spacing of 150 m. At this resolution, spurious coherent structures are often observed under unstable atmospheric conditions with moderate mean wind speeds. Results from such an intermediate resolution grid are often nested down for finer, more detailed LES, where these spurious structures adversely affect the development of turbulence on the subsequent finer grid nest. We therefore examine the impacts of the CPM on the representation of turbulence within the nested LES domain under moderate mean flow conditions in three different stability regimes: weakly convective, strongly convective, and weakly stable. In addition, two different resolutions of the underlying terrain are used to explore the role of the complex topography itself in generating turbulent structures. We demonstrate that the CPM improves the representation of turbulence within the LES domain, relative to the use of high-resolution complex terrain alone. During the convective conditions, the CPM improves the rate at which smaller-scales of turbulence form, while also accelerating the attenuation of the spurious numerically generated roll structures near the inflow boundary. During stable conditions, the coarse mesh spacing of the intermediate LES domain used herein was insufficient to maintain resolved turbulence using CPM as the flow develops downstream, highlighting the need for yet higher resolution under even weakly stable conditions, and the importance of accurate representation of flow on intermediate LES grids.

17 WIND ENERGY↗

Langmuir turbulence and filament frontogenesis in the oceanic surface boundary layer

Submesoscale currents, small-scale turbulence and surface gravity waves co-exist in the upper ocean and interact in complex ways. To expose the couplings, the frontogenetic life cycle of an idealized cold dense submesoscale filament interacting with upper ocean Langmuir turbulence is investigated in large-eddy simulations (LESs) based on the incompressible wave-averaged equations. The simulations utilize large domains and fine meshes with $6.4\times 10^{9}$ grid points. Case studies are made with surface winds or surface cooling with waves oriented in across-filament (perpendicular) or down-filament (parallel) directions relative to the two-dimensional filament axis. The currents $u$ , $v$ and $w$ are aligned with the across-filament, down-filament and vertical directions, respectively. Frontogenesis is induced by across-filament Lagrangian secondary circulations in the boundary layer, and it is shown to be strongly impacted by surface waves, in particular the propagation direction relative to the filament axis. In a horizontally heterogeneous boundary layer, surface waves induce both mean and fluctuating Stokes-drift vortex forces that modify a linear, hydrostatic turbulent thermal wind (TTW) approximation for momentum. Down-filament winds and waves are found to be especially impactful, they significantly reduce the peak level of frontogenesis by fragmenting the filament into primary and secondary down-welling sites in a broad frontal zone over a width ${\sim}500~\text{m}$ . At peak frontogenesis, opposing down-filament jets $\langle v\rangle$ overlie each other resulting in a vigorous vertical shear layer $\unicode[STIX]{x2202}_{z}\langle v\rangle$ with large vertical momentum flux $\langle v^{\prime }w^{\prime }\rangle$ . Filament arrest is induced by a lateral shear instability that generates horizontal momentum flux $\langle u^{\prime }v^{\prime }\rangle$ at low wavenumbers. The turbulent vertical velocity patterns, indicative of coherent Langmuir cells, change markedly across the horizontal domain with both across-filament and down-filament winds under the action of submesoscale currents.

Mechanics↗

A Scalable Algorithm for Radiative Heat Transfer Using Reverse Monte Carlo Ray Tracing

Radiative heat transfer is an important mechanism in a class of challenging engineering and research problems. A direct all-to-all treatment of these problems is prohibitively expensive on large core counts due to pervasive all- to-all MPI communication. The massive heat transfer problem arising from the next generation of clean coal boilers being modeled by the Uintah framework has radiation as a dominant heat transfer mode. Reverse Monte Carlo ray tracing (RMCRT) can be used to solve for the radiative-flux divergence while accounting for the effects of participating media. The ray tracing approach used here replicates the geometry of the boiler on a multi-core node and then uses an all-to-all communication phase to distribute the results globally. The cost of this all-to-all is reduced by using an adaptive mesh approach in which a fine mesh is only used locally, and a coarse mesh is used elsewhere. Here, a model for communication and computation complexity is used to predict performance of this new method. We show this model is consistent with observed results and demonstrate excellent strong scaling to 262K cores on the DOE Titan system on problem sizes that were previously computationally intractable.

97 MATHEMATICS AND COMPUTING↗

Parametric binary dissection

Binary dissection is widely used to partition non-uniform domains over parallel computers. This algorithm does not consider the perimeter, surface area, or aspect ratio of the regions being generated and can yield decompositions that have poor communication to computation ratio. Parametric Binary Dissection (PBD) is a new algorithm in which each cut is chosen to minimize load + lambda x(shape). In a 2 (or 3) dimensional problem, load is the amount of computation to be performed in a subregion and shape could refer to the perimeter (respectively surface) of that subregion. Shape is a measure of communication overhead and the parameter permits us to trade off load imbalance against communication overhead. When A is zero, the algorithm reduces to plain binary dissection. This algorithm can be used to partition graphs embedded in 2 or 3-d. Load is the number of nodes in a subregion, shape the number of edges that leave that subregion, and lambda the ratio of time to communicate over an edge to the time to compute at a node. An algorithm is presented that finds the depth d parametric dissection of an embedded graph with n vertices and e edges in O(max(n log n, de)) time, which is an improvement over the O(dn log n) time of plain binary dissection. Parallel versions of this algorithm are also presented; the best of these requires O((n/p) log(sup 3)p) time on a p processor hypercube, assuming graphs of bounded degree. How PBD is applied to 3-d unstructured meshes and yields partitions that are better than those obtained by plain dissection is described. Its application to the color image quantization problem is also discussed, in which samples in a high-resolution color space are mapped onto a lower resolution space in a way that minimizes the color error.

Bokhari, Shahid H.↗

Development Status of PROTEUS-MOC

The PROTEUS-MOC solver is intended to resolve the outstanding computational challenges that the PROTEUS-SN is limited by and bridge the gap between PROTEUS-SN and existing homogenized structured grid solvers. The primary push behind all of this work is to carry out coupled thermal-mechanical, fluid dynamics, and neutronics physics problems. The PROTEUS-MOC development has taken advantage of the combined experience with PROTEUS-SN and DeCART with the goal of producing a tool which combines the best features of both tools. Taking the DeCART tool as the model, it merges a subgroup cross section interpolation technique with a 2D method of characteristics approach and a 1D SN formulation. DeCART has been specifically demonstrated to be accurate on PWR and VHTR problems, but it has displayed several issues with regard to accuracy on thin axial mesh sizes which are attributable to the 2D1D scheme it is built upon. There are also notable problems with performance and scalability all of which we hope to resolve in PROTEUS-MOC. PROTEUS-MOC is at its core, based upon the success achieved with MOCFE which demonstrated excellent parallelization with respect to angle and good parallelization with respect to space. More recent efforts have indicated parallelization with respect to energy is also feasible. The work completed yielded a novel methodology where the axial basis functions are treated separately from the radial ones, but the geometry is extruded. This eliminates the need to carry out and store 3D ray tracing data and simultaneously allows the coarse axial node sizes typical with DeCART. To rapidly build a prototype, a standard power iteration algorithm was constructed using an unpreconditioned GMRES solution algorithm for solving the flux vector in each fission source iteration. The new PROTEUS-MOC solver was deployed on two test problems: Takeda and a heterogeneous cylindrical heavy water system. The Takeda benchmark problem is a standard homogeneous test problem which was previously found to be too difficult for the 3D MOCFE code to solve. Using the PROTEUS-MOC we were able to show space-angle convergence although full convergence requires further improvements in performance. The second benchmark problem was constructed more for future work than this year, but the results generated are promising indicating that it will be very useful in highlighting the weaknesses and strengths of the PROTEUS-MOC tool. While we are far from a production tool, the results included in this report give us confidence that we are on the right path to constructing an interceding code between the plethora of structured grid homogenized assembly solvers and the supercomputer consuming PROTEUS-SN code. With time, the creation of a suitable preconditioner along with typical optimization should reduce the performance shortcomings that are observable today.

Marin-Lafleche, A.↗