SEARCH · Search NASA

Results for “data repartitioning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2018

Dynamic Data Repartitioning for Load-Balanced Parallel Particle Tracing

We present a novel dynamic load-balancing algorithm based on data repartitioning for parallel particle tracing in flow visualization. Instead of static data assignment, we dynamically repartition the data into blocks and reassign the blocks to processes to balance the workload distribution among the processes. Block repartitioning is performed based on a dynamic workload estimation method that predicts the workload in the flow field on the fly as the input. In our approach, we allow data duplication in the repartitioning, enabling the same data blocks to be assigned to multiple processes. Load balance is achieved by regularly exchanging the blocks (together with the particles in the blocks) among processes according to the output of the data repartitioning. Compared with other load-balancing algorithms, our approach does not need any preprocessing on the raw data and does not require any dedicated process for work scheduling, while it has the capability to balance uneven workload efficiently. Results show improved load balance and high efficiency of our method on tracing particles in both steady and unsteady flow.

data repartitioning↗

NASA NTRS1997

Efficient Load Balancing and Data Remapping for Adaptive Grid Calculations

Mesh adaption is a powerful tool for efficient unstructured- grid computations but causes load imbalance among processors on a parallel machine. We present a novel method to dynamically balance the processor workloads with a global view. This paper presents, for the first time, the implementation and integration of all major components within our dynamic load balancing strategy for adaptive grid calculations. Mesh adaption, repartitioning, processor assignment, and remapping are critical components of the framework that must be accomplished rapidly and efficiently so as not to cause a significant overhead to the numerical simulation. Previous results indicated that mesh repartitioning and data remapping are potential bottlenecks for performing large-scale scientific calculations. We resolve these issues and demonstrate that our framework remains viable on a large number of processors.

Oliker, Leonid↗

DOE OSTI2021

Multichannel Analysis of Surface Waves Accelerated (MASWAccelerated): Software for efficient surface wave inversion using MPI and GPUs

Multichannel Analysis of Surface Waves (MASW) is a technique frequently used in geotechnical engineering and engineering geophysics to infer 1D layered models of seismic shear wave velocities in the top tens to hundreds of meters of the subsurface. We aim to accelerate MASW calculations by capitalizing on modern computer hardware available in the workstations of most engineers: multiple cores and graphics processing units (GPUs). We propose new parallel and GPU accelerated algorithms for computing 1D MASW inversion, and provide software implementations in C using Message Passing Interface (MPI) and CUDA. These algorithms take advantage of sparsity that arises in the problem, and the work balance between processes considers typical data trends. We compare our methods to an existing open source Matlab MASW tool. Our serial C implementation achieves a 2x speedup over the Matlab software, and we continue to see improvements by parallelizing the problem with MPI. Here we see nearly perfect strong and weak scaling for uniform data, and improve strong scaling for realistic data by repartitioning the problem to process mapping. By utilizing GPUs available on most modern workstations, we observe an additional 1.3x speedup over the serial C implementation on the first use of the method. We typically repeatedly evaluate theoretical dispersion curves as part of an optimization procedure, and on the GPU the kernel can be cached for faster reuse on later runs. We observe a 3.2x speedup on the cached GPU runs compared to the serial C runs. This work is the first open-source parallel or GPU-accelerated software tool for MASW imaging, and should enable geotechnical engineers to fully utilize all computer hardware at their disposal.

58 GEOSCIENCES↗

DOE OSTI2014

TU-A-12A-08: Computing Longitudinal Material Changes in Bone Metastases Using Dual Energy Computed Tomography

Purpose: This study demonstrates a methodology for tracking changes in metastatic bone disease using trajectories in material basis space in serial dual energy computed tomography (DECT) studies. Methods: This study includes patients with bone metastases from breast cancer that had clinical surveillance CT scans using a General Electric CT750HD in dual energy mode. A radiologist defined regions-of-interested (ROI) for bone metastasis, normal bone, and marrow across the serial DECT scans. Our approach employs a Radon transform to forward-projection the basis images, namely, water and iodine, into sinogram space. This data is then repartitioned into fat/bone and effective density/Z image pairs using assumed energy spectrums for the x-ray energies. This approach both helps remove negative material densities and avoids adding spectrum-hardening artifacts. These new basis data sets were then reconstructed via filtered back-projection to create new material basis pair images. The trajectories of these pairs were then plotted in the new basis space providing a means to both visualize and quantitatively measure changes in the material properties of the tumors. Results: ROI containing radiologist defined metastatic bone disease showed well-defined trajectories in both fat/bone and effective density/Z space. ROI that contained radiologist defined normal bone and marrow did not exhibit any discernible trajectories and were stable from scan to scan. Conclusions: The preliminary results show that changes in material composition and effective density/Z image pairs were seen primarily in metastasis and not in normal tissue. This study indicates that by using routine clinical DECT it may be possible to monitor therapy response of bone metastases because healing or worsening bone metastases change material composition of bone. Additional studies are needed to further validate these results and to test for their correlation with outcome.

60 APPLIED LIFE SCIENCES↗

NASA NTRS1994

Convective flow effects on protein crystal growth

A high-resolution microscopic interferometric setup for the monitoring of protein morphologies has been developed. Growth or dissolution of a crystal can be resolved with a long-term depth resolution of 200 A and a lateral resolution of 2 microns. This capability of simultaneously monitoring the interfacial displacement with high local depth resolution has yielded several novel results. We have found with lysozyme that (1) the normal growth rate is oscillatory, and (2) depending on the impurity content of the solution, the growth step density is either greater or lower at the periphery of a facet than in its center. The repartitioning of Na plus and Cl minus ions between lysozyme solutions and crystals was studied for a wide range of crystallization conditions. A nucleation-growth-repartitioning model was developed, to interpret the large body of data in unified way. The results strongly suggest that (1) the ion to lysozyne ratio in the crystal depends mostly on kinetic rather than crystallographic parameters, and (2) lysozyme crystals possess a salt-rich core with a diameter electron microscopy results appear to confirm this finding, which could have far-reaching consequences for x-ray diffraction studies. A computational model for diffusive-convective transport in protein crystallization has been applied to a realistic growth cell geometry, taking into account the findings of the above repartitioning studies and our kinetics data for the growth of lysozyme. The results show that even in the small cell employed, protein concentration nonuniformities and gravity-driven solutal convection can be significant. The calculated convection velocities are of the same order to magnitude as those found in earlier experiments. As expected, convective transport, i.e., at Og, lysozyme crystal growth remains kinetically limited. The salt distribution in the crystal is predicted to be non-uniform at both 1g and 0g, as a consequence of protein depletion in the solution. Static and dynamic light scattering studies in undersaturated and supersaturated solutions have been performed. Diffusivities in undersaturated solutions, were found to vary with lysozyme concentrations. Depending on the salt concentration, the diffusivities either increase or decrease. Interestingly, the corresponding static scattering intensities behave oppositely, Our current analysis indicates that these changes are inconsistent with aggregation in undersaturated solutions. However, the data are compatible with concentration-dependent changes of the interactions between protein and salt.

Rosenberger, Franz↗

NASA NTRS1996

Repartitioning of NaCl and Protein Impurities in Lysozyme Crystallization

Nonuniform precipitant and impurity incorporation in protein crystals can cause lattice strain and, thus, possibly decrease the X-ray diffraction resolution. To address this issue, a series of crystallization experiments were carried out, in which initial supersaturation, NaCl concentration, protein purity level and crystallized fraction were varied. Lysozyme and protein impurities, as well as sodium and chloride were independently determined in the initial solution, supernatant and crystals. The segregation coefficients for Na(+) and Cl(-) were found to be independent of supersaturation and NaCl concentration, and decreased with crystallized fraction/crystal size. Numerical evaluation of the extensive body of data, based on a nucleation-growth- repartitioning model, suggests a core of approx. 40 microns in which salt is incorporated in much greater concentrations than during later growth. Small crystals containing higher amounts of incorporated NaCl also had higher protein impurity contents. This suggests that the excess salt is associated with the protein impurities in the core. X-ray topography revealed strain fields in the center of the crystals comparable in size to the inferred core. The growth rates of crystals smaller than 30-40 pm in size were consistently 1.5-2 times lower than those of larger crystals, presumably due to higher chemical potentials in the core.

Vekilov, Peter G.↗

NASA NTRS1994

Convective flow effects on protein crystal growth

The long-term stability of the interferometric setup for the monitoring of protein morphologies has been improved. Growth or dissolution of a crystal on a 100 A scale can now be clearly distinguished from dimensional changes occurring within the optical path of the interferometer. This capability of simultaneously monitoring the local interfacial displacement at several widely-spaced positions on the crystal surface with high local depth resolution, has already yielded novel results. We found with lysozyme that (1) the normal growth rate is oscillatory, and (2) the mean growth step density is greater at the periphery of a facet than in its center. The repartitioning of Na(+) and Cl(-) ions between lysozyme solutions and crystals was studied for a wide range of crystallization conditions. A nucleation-growth-repartitioning model was developed to interpret the large body of data in a unified way. The results strongly suggests that (1) the ion to lysozyme ratio in the crystal depends mostly on kinetic rather than crystallographic parameters, and (2) lysozyme crystals possess a salt-rich core with a diameter on the order of 10 microns. The computational model for diffusive-convective transport in protein crystallization (see the First Report) has been applied to a realistic growth cell geometry, taking into account the findings of the above repartitioning studies. These results show that some elements of a moving boundary problem must be incorporated into the model in order to obtain a more realistic description. Our experimental setup for light scattering investigations of aggregation and nucleation in protein solutions has been extensively tested. Scattering intensity measurements with a true Rayleigh scatterer produced systematically increased forward scattering, indicating problems with glare. These have been resolved. Preliminary measurements with supersaturated lysozyme solutions revealed that the scatterers grow with time. Work has begun on a computer program for the unified evaluation of simultaneously obtained, multi-angle static and dynamic light scattering data.

Rosenberger, Franz↗

DOE OSTI2017

The fusion code XGC: Enabling kinetic study of multi-scale edge turbulent transport in ITER

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2017

The fusion code XGC: Enabling kinetic study of multi-scale edge turbulent transport in ITER [Book Chapter]

The XGC fusion gyrokinetic code combines state-of-the-art, portable computational and algorithmic technologies to enable complicated multiscale simulations of turbulence and transport dynamics in ITER edge plasma on the largest US open-science computer, the CRAY XK7 Titan, at its maximal heterogeneous capability, which have not been possible before due to a factor of over 10 shortage in the time-to-solution for less than 5 days of wall-clock time for one physics case. Frontier techniques such as nested OpenMP parallelism, adaptive parallel I/O, staging I/O and data reduction using dynamic and asynchronous applications interactions, dynamic repartitioning for balancing computational work in pushing particles and in grid related work, scalable and accurate discretization algorithms for non-linear Coulomb collisions, and communication-avoiding subcycling technology for pushing particles on both CPUs and GPUs are also utilized to dramatically improve the scalability and time-to-solution, hence enabling the difficult kinetic ITER edge simulation on a present-day leadership class computer.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2016

Field-gradient partitioning for fracture and frictional contact in the material point method: Field-gradient partitioning for fracture and frictional contact in the material point method [Fracture and frictional contact in material point method using damage-field gradients for velocity-field partitioning]

Contact and fracture in the material point method require grid-scale enrichment or partitioning of material into distinct velocity fields to allow for displacement or velocity discontinuities at a material interface. We present a new method which a kernel-based damage field is constructed from the particle data. The gradient of this field is used to dynamically repartition the material into contact pairs at each node. Our approach avoids the need to construct and evolve explicit cracks or contact surfaces and is therefore well suited to problems involving complex 3-D fracture with crack branching and coalescence. A straightforward extension of this approach permits frictional ‘self-contact’ between surfaces that are initially part of a single velocity field, enabling more accurate simulation of granular flow, porous compaction, fragmentation, and comminution of brittle materials. Finally, numerical simulations of self contact and dynamic crack propagation are presented to demonstrate the accuracy of the approach.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2016

Evaluating the Impact of Data Placement to Spark and SciDB with an Earth Science Use Case

We investigate the impact of data placement for two Big Data technologies, Spark and SciDB, with a use case from Earth Science where data arrays are multidimensional. Simultaneously, this investigation provides an opportunity to evaluate the performance of the technologies involved. Two datastores, HDFS and Cassandra, are used with Spark for our comparison. It is found that Spark with Cassandra performs better than with HDFS, but SciDB performs better yet than Spark with either datastore. The investigation also underscores the value of having data aligned for the most common analysis scenarios in advance on a shared nothing architecture. Otherwise, repartitioning needs to be carried out on the fly, degrading overall performance.

Spark↗

DOE OSTI2025

Spotlight: efficient automated global optimization in rietveld analysis of diffraction data

Performing reliable Rietveld analysis on tens or hundreds of powder diffraction datasets from parametric or time-resolved experiments often poses a bottleneck in extracting meaningful results from the data. While automated analysis of data has recently been demonstrated, high temperature annealing studies, during which phase transformations occur and lattice parameters may change due to repartitioning of elements, are prime examples where automation by a simple phase identification from a database of room temperature structures or automation by sequential refinements is likely to fail. To enable reliable, efficient, automated Rietveld analysis, we present a Python package named Spotlight , building on established Rietveld packages such as MAUD, GSAS , or GSAS-II , which extends the refinement of best fit parameters to a global optimization using an ensemble of optimizers leveraging hierarchical parallel execution on high-performance computing clusters. Spotlight further enables the efficient design of refinement plans through the iterative automated machine-learning of a surrogate for the refinement on which the global optimizations are performed until results from the surrogate converge to the response surface data. We demonstrate Spotlight with the analysis of uranium molybdenum and Ti–6Al–4V datasets, as well as in two open-source tutorials analyzing aluminium oxide and lead sulphate.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Energy Flux Densities near the Electron Dissipation Region in Asymmetric Magnetopause Reconnection

Magnetic reconnection is of fundamental importance to plasmas because of its role in releasing and repartitioning stored magnetic energy. Previous results suggest that this energy is predominantly released as ion enthalpy flux along the reconnection outflow. Using Magnetospheric Multiscale data we find the existence of very significant electron energy flux densities in the vicinity of the magnetopause electron dissipation region, orthogonal to the ion energy outflow. Finally, these may significantly impact models of electron transport, wave generation, and particle acceleration.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2016

Tunable allosteric library of caspase-3 identifies coupling between conserved water molecules and conformational selection

The native ensemble of caspases is described globally by a complex energy landscape where the binding of substrate selects for the active conformation, whereas targeting an allosteric site in the dimer interface selects an inactive conformation that contains disordered active-site loops. Mutations and posttranslational modifications stabilize high-energy inactive conformations, with mostly formed, but distorted, active sites. To examine the interconversion of active and inactive states in the ensemble, we used detection of related solvent positions to analyze 4,995 waters in 15 high-resolution (<2.0 Å) structures of wild-type caspase-3, resulting in 450 clusters with the most highly conserved set containing 145 water molecules. The data show that regions of the protein that contact the conserved waters also correspond to sites of posttranslational modifications, suggesting that the conserved waters are an integral part of allosteric mechanisms. To test this hypothesis, we created a library of 19 caspase-3 variants through saturation mutagenesis in a single position of the allosteric site of the dimer interface, and we show that the enzyme activity varies by more than four orders of magnitude. Altogether, our database consists of 37 high-resolution structures of caspase-3 variants, and we demonstrate that the decrease in activity correlates with a loss of conserved water molecules. Furthermore, the data show that the activity of caspase-3 can be fine-tuned through globally desolvating the active conformation within the native ensemble, providing a mechanism for cells to repartition the ensemble and thus fine-tune activity through conformational selection.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2015

Parallel adaptive wavelet collocation method for PDEs

A parallel adaptive wavelet collocation method for solving a large class of Partial Differential Equations is presented. The parallelization is achieved by developing an asynchronous parallel wavelet transform, which allows one to perform parallel wavelet transform and derivative calculations with only one data synchronization at the highest level of resolution. The data are stored using tree-like structure with tree roots starting at a priori defined level of resolution. Both static and dynamic domain partitioning approaches are developed. For the dynamic domain partitioning, trees are considered to be the minimum quanta of data to be migrated between the processes. This allows fully automated and efficient handling of non-simply connected partitioning of a computational domain. Dynamic load balancing is achieved via domain repartitioning during the grid adaptation step and reassigning trees to the appropriate processes to ensure approximately the same number of grid points on each process. The parallel efficiency of the approach is discussed based on parallel adaptive wavelet-based Coherent Vortex Simulations of homogeneous turbulence with linear forcing at effective non-adaptive resolutions up to 2048{sup 3} using as many as 2048 CPU cores.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1998

PLUM: Parallel Load Balancing for Adaptive Unstructured Meshes

Mesh adaption is a powerful tool for efficient unstructured-grid computations but causes load imbalance among processors on a parallel machine. We present a novel method called PLUM to dynamically balance the processor workloads with a global view. This paper presents the implementation and integration of all major components within our dynamic load balancing strategy for adaptive grid calculations. Mesh adaption, repartitioning, processor assignment, and remapping are critical components of the framework that must be accomplished rapidly and efficiently so as not to cause a significant overhead to the numerical simulation. A data redistribution model is also presented that predicts the remapping cost on the SP2. This model is required to determine whether the gain from a balanced workload distribution offsets the cost of data movement. Results presented in this paper demonstrate that PLUM is an effective dynamic load balancing strategy which remains viable on a large number of processors.

Oliker, Leonid↗

NASA NTRS1998

Parallel Load Balancing for Adaptive Unstructured Meshes

Mesh adaption is a powerful tool for efficient unstructured-grid computations but causes load imbalance among processors on a parallel machine. We describe a novel method to dynamically balance the processor workloads with a global view. Mesh question, repartitioning, processor assignment, and remapping are critical components of the framework that must be accomplished rapidly and efficiently so as not to cause a significant overhead to the numerical simulation. A data redistribution model will also be presented that predicts the remapping cost. This model is required to determine whether the gain from a balanced workload distribution offsets the cost of data movement. Results presented will demonstrate that this is an effective dynamic load balancing strategy which remains viable on a large number of processors.

Biswas, Rupak↗

NASA NTRS2010

Towards Global Simulation of Irrigation in a Land Surface Model: Multiple Cropping and Rice Paddy in Southeast Asia

Agricultural land use significantly influences the surface water and energy balances. Effects of irrigation on land surface states and fluxes include repartitioning of latent and sensible heat fluxes, an increase in net radiation, and an increase in soil moisture and runoff. We are working on representing irrigation practices in continental- to global-scale land surface simulation in NASA's Global Land Data Assimilation System (GLDAS). Because agricultural practices across the nations are diverse, and complex, we are attempting to capture the first-order reality of the regional practices before achieving a global implementation. This study focuses on two issues in Southeast Asia: multiple cropping and rice paddy irrigation systems. We first characterize agricultural practices in the region (i.e., crop types, growing seasons, and irrigation) using the Global data set of monthly irrigated and rainfed crop areas around the year 2000 (MIRCA2000) dataset. Rice paddy extent is identified using remote sensing products. Whether irrigated or rainfed, flooded fields need to be represented and treated explicitly. By incorporating these properties and processes into a physically based land surface model, we are able to quantify the impacts on the simulated states and fluxes.

Beaudoing, Hiroko Kato↗