Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

A Dynamical and Statistical Characterization of U.S. Extreme Precipitation Events and Their Associated Large-Scale Meteorological Patterns

Abstract Regional patterns of extreme precipitation events occurring over the continental United States are identified via hierarchical cluster analysis of observed daily precipitation for the period 1950–2005. Six canonical extreme precipitation patterns (EPPs) are isolated for the boreal warm season and five for the cool season. The large-scale meteorological pattern (LMP) inducing each EPP is identified and used to create a “base function” for evaluating a climate model’s potential for accurately representing the different patterns of precipitation extremes. A parallel analysis of the Community Climate System Model, version 4 (CCSM4), reveals that the CCSM4 successfully captures the main U.S. EPPs for both the warm and cool seasons, albeit with varying degrees of accuracy. The model’s skill in simulating each EPP tends to be positively correlated with its capability in representing the associated LMP. Model bias in the occurrence frequency of a governing LMP is directly related to the frequency bias in the corresponding EPP. In addition, however, discrepancies are found between the CCSM4’s representation of LMPs and EPPs over regions such as the western United States and Midwest, where topographic precipitation influences and organized convection are prominent, respectively. In these cases, the model representation of finer-scale physical processes appears to be at least equally important compared to the LMPs in driving the occurrence of extreme precipitation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2011

A Semi-Quantitative, Synteny-Based Method to Improve Functional Predictions for Hypothetical and Poorly Annotated Bacterial and Archaeal Genes

During microbial evolution, genome rearrangement increases with increasing sequence divergence. If the relationship between synteny and sequence divergence can be modeled, gene clusters in genomes of distantly related organisms exhibiting anomalous synteny can be identified and used to infer functional conservation. We applied the phylogenetic pairwise comparison method to establish and model a strong correlation between synteny and sequence divergence in all 634 available Archaeal and Bacterial genomes from the NCBI database and four newly assembled genomes of uncultivated Archaea from an acid mine drainage (AMD) community. In parallel, we established and modeled the trend between synteny and functional relatedness in the 118 genomes available in the STRING database. By combining these models, we developed a gene functional annotation method that weights evolutionary distance to estimate the probability of functional associations of syntenous proteins between genome pairs. The method was applied to the hypothetical proteins and poorly annotated genes in newly assembled acid mine drainage Archaeal genomes to add or improve gene annotations. This is the first method to assign possible functions to poorly annotated genes through quantification of the probability of gene functional relationships based on synteny at a significant evolutionary distance, and has the potential for broad application.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2012

Applied Parallel Metadata Indexing

The GPFS Archive is parallel archive is a parallel archive used by hundreds of users in the Turquoise collaboration network. It houses 4+ petabytes of data in more than 170 million files. Currently, users must navigate the file system to retrieve their data, requiring them to remember file paths and names. A better solution might allow users to tag data with meaningful labels and searach the archive using standard and user-defined metadata, while maintaining security. last summer, I developed the backend to a tool that adheres to these design goals. The backend works by importing GPFS metadata into a MongoDB cluster, which is then indexed on each attribute. This summer, the author implemented security and developed the user interfae for the search tool. To meet security requirements, each database table is associated with a single user, which only stores records that the user may read, and requires a set of credentials to access. The interface to the search tool is implemented using FUSE (Filesystem in USErspace). FUSE is an intermediate layer that intercepts file system calls and allows the developer to redefine how those calls behave. In the case of this tool, FUSE interfaces with MongoDB to issue queries and populate output. A FUSE implementation is desirable because it allows users to interact with the search tool using commands they are already familiar with. These security and interface additions are essential for a usable product.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

A quantitative model of application slow-down in multi-resource shared systems

Scheduling multiple jobs onto a platform enhances system utilization by sharing resources. The benefits from higher resource utilization include reduced cost to construct, operate, and maintain a system, which often include energy consumption. Maximizing these benefits comes at a price-resource contention among jobs increases job completion time. In this study, we analyze slow-downs of jobs due to contention for multiple resources in a system; referred to as dilation factor. We observe that multiple-resource contention creates non-linear dilation factors of jobs. From this observation, we establish a general quantitative model for dilation factors of jobs in multi-resource systems. A job is characterized by a vector-valued loading statistics and dilation factors of a job set are given by a quadratic function of their loading vectors. We demonstrate how to systematically characterize a job, maintain the data structure to calculate the dilation factor (loading matrix), and calculate the dilation factor of each job. We validate the accuracy of the model with multiple processes running on a native Linux server, virtualized servers, and with multiple MapReduce workloads co-scheduled in a cluster. Evaluation with measured data shows that the D-factor model has an error margin of less than 16%. We extended the D-factor model to capture the slow-down of applications when multiple identical resources exist such as multi-core environments and multi-disks environments. Finally, validation results of the extended D-factor model with HPC checkpoint applications on the parallel file systems show that D-factor accurately captures the slow down of concurrent applications in such environments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Porting fragmentation methods to GPUs using an OpenMP API: Offloading the resolution-of-the-identity second-order Møller–Plesset perturbation method

Here, using an OpenMP Application Programming Interface, the resolution-of-the-identity second-order Møller–Plesset perturbation (RI-MP2) method has been off-loaded onto graphical processing units (GPUs), both as a standalone method in the GAMESS electronic structure program and as an electron correlation energy component in the effective fragment molecular orbital (EFMO) framework. First, a new scheme has been proposed to maximize data digestion on GPUs that subsequently linearizes data transfer from central processing units (CPUs) to GPUs. Second, the GAMESS Fortran code has been interfaced with GPU numerical libraries (e.g., NVIDIA cuBLAS and cuSOLVER) for efficient matrix operations (e.g., matrix multiplication, matrix decomposition, and matrix inversion). The standalone GPU RI-MP2 code shows an increasing speedup of up to 7.5× using one NVIDIA V100 GPU with one IBM 42-core P9 CPU for calculations on fullerenes of increasing size from 40 to 260 carbon atoms using the 6-31G(d)/cc-pVDZ-RI basis sets. A single Summit node with six V100s can compute the RI-MP2 correlation energy of a cluster of 175 water molecules using the correlation consistent basis sets cc-pVDZ/cc-pVDZ-RI containing 4375 atomic orbitals and 14 700 auxiliary basis functions in ~0.85 h. In the EFMO framework, the GPU RI-MP2 component shows near linear scaling for a large number of V100s when computing the energy of an 1800-atom mesoporous silica nanoparticle in a bath of 4000 water molecules. The parallel efficiencies of the GPU RI-MP2 component with 2304 and 4608 V100s are 98.0% and 96.1%, respectively.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2014

The PAndAS field of streams: Stellar structures in the milky way halo toward Andromeda and Triangulum

We reveal the highly structured nature of the Milky Way (MW) stellar halo within the footprint of the Pan-Andromeda Archaeological Survey (PAndAS) photometric survey from blue main sequence (MS) and MS turn-off stars. We map no fewer than five stellar structures within a heliocentric range of ∼5-30 kpc. Some of these are known (the Monoceros Ring, the Pisces/Triangulum globular cluster stream), but we also uncover three well-defined stellar structures that could be, at least partly, responsible for the so-called Triangulum/Andromeda and Triangulum/Andromeda 2 features. In particular, we trace a new faint stellar stream located at a heliocentric distance of ∼17 kpc. With a surface brightness of Σ {sub V} ∼ 32-32.5 mag arcsec{sup –2}, it follows an orbit that is almost parallel to the Galactic plane north of M31 and has so far eluded surveys of the MW halo as these tend to steer away from regions dominated by the Galactic disk. Investigating our follow-up spectroscopic observations of PAndAS, we serendipitously uncover a radial velocity signature from stars that have colors and magnitudes compatible with the stream. From the velocity of eight likely member stars, we show that this stellar structure is dynamically cold, with an unresolved velocity dispersion that is lower than 7.1 km s{sup –1} at the 90% confidence level. Along with the width of the stream (300-650 pc), its dynamics point to a dwarf-galaxy-accretion origin. The numerous stellar structures we can map in the MW stellar halo between 5 and 30 kpc and their varying morphology is a testament to the complex nature of the stellar halo at these intermediate distances.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Asynchronous domain decomposition methods for nonlinear PDEs

One- and two-level parallel asynchronous methods for the numerical solution of nonlinear systems of equations, especially those arising from (nonlinear) partial differential equations, are studied. The proposed methods are based on domain decomposition techniques. Local convergence theorems are presented in several cases, with appropriate hypotheses. Computational results on a shared memory multiprocessor machine for various problems exhibiting nonlinearities are reported, illustrating the potential of these asynchronous methods, especially for heterogeneous clusters.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

OpenSHMEM Implementation of IOR Benchmark

The benchmarking effort within the Computational Research & Development Programs at the Oak Ridge National Laboratory (ORNL) seeks to design and enable High Performance Computing (HPC) benchmarks and test suites. The work described in this report is a part of the effort focusing on the comparison and analysis of OpenSHMEM implementations using the Interleave Or Random (IOR) software for benchmarking parallel file system using POSIX, MPIIO, or HDF5 interfaces. We describe the effort to emulate the MPIIO parallel collective capabilities in the IOR benchmark using OpenSHMEM communication. One development effort was in emulating the MPI derived datatype used in the read/write operations and in setting the file view. Another effort was in implementing an internal cache in OpenSHMEM distributed shared memory to facilitate global collective I/O operations. Experiments comparing collective I/O in MPIIO implementations with the OpenSHMEM implementations were performed on the SGI Turing Cluster and the Cray XK7 Titan supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). The preliminary results suggest that on the Cray XK7 Titan, the MPIIO implementations obtained higher write performance and the OpenSHMEM version obtained slightly higher read performance. On the SGI Turing Cluster, the MPIIO implementations obtained slightly higher performance over the OpenSHMEM implementations on large files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Modeling Large-Scale Slim Fly Networks Using Parallel Discrete-Event Simulation

As supercomputers approach exascale performance, the increased number of processors translates to an increased demand on the underlying network interconnect. We present that the slim fly network topology, a new low-diameter, low-latency, and low-cost interconnection network, is gaining interest as one possible solution for next-generation supercomputing interconnect systems. In this article, we present a high-fidelity slim fly packet-level model leveraging the Rensselaer Optimistic Simulation System (ROSS) and Co-Design of Exascale Storage (CODES) frameworks. We validate the model with published work before scaling the network size up to an unprecedented 1 million compute nodes and confirming that the slim fly observes peak network throughput at extreme scale. In addition to synthetic workloads, we evaluate large-scale slim fly models with real communication workloads from applications in the Design Forward program with over 110,000 MPI processes. We show strong scaling of the slim fly model on an Intel cluster achieving a peak network packet transfer rate of 2.3 million packets per second and processing over 7 billion discrete events using 128 MPI tasks. Enabled by the strong performance capabilities of the model, we perform a detailed application trace and routing protocol performance study. Lastly, through analysis of metrics such as packet latency, hop count, and congestion, we find that the slim fly network is able to leverage simple minimal routing and achieve the same performance as more complex adaptive routing for tested DOE benchmark applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Modular performance prediction for scientific workflows using Machine Learning

Scientific workflows provide an opportunity for declarative computational experiment design in an intuitive and efficient way. A distributed workflow is typically executed on a variety of resources, and it uses a variety of computational algorithms or tools to achieve the desired outcomes. Such a variety imposes additional complexity in scheduling these workflows on large scale computers. As computation becomes more distributed, insights into expected workload that a workflow presents become critical for effective resource allocation. In this paper, we present a modular framework that leverages Machine Learning for creating precise performance predictions of a workflow. The central idea is to partition a workflow in such a way that makes the task of forecasting each atomic unit manageable and gives us a way to combine the individual predictions efficiently. We recognize a combination of an executable and a specific physical resource as a single module. This gives us a handle to characterize workload and machine power as a single unit of prediction. Overall, our modular technique of creating atomic modules and deployment of longest-path approach to estimate workflow performance, allows the framework to adapt to highly complex nested directed acyclic workflows and scale to new scenarios, since it does not make assumptions of underlying workflow structure. We present performance estimation results of independent workflow modules executed on the XSEDE SDSC Comet cluster using various Machine Learning algorithms. The results provide insights into the behavior and effectiveness of different algorithms in the context of scientific workflow performance prediction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

An X-ray magnetic circular dichroism (XMCD) study of Fe ordering in a synthetic MgAl2O4 – Fe3O4 (spinel – magnetite) solid solution series; implications for magnetic properties and cation site ordering

Fe L2,3-edge XAS and XMCD studies have been used to unravel structural trends in the MgAl2O4–Fe3O4, solid solution where thermodynamic modelling has represented a challenge due to the complex ordering arrangements of the end-members. Partitioning of Fe3+ and Fe2+ between tetrahedral (Td) and octahedral (Oh) sites has been established. In the most Fe-rich samples, despite rapid quenching from a disordered state, Fe2+Td is not present which matches the ordered, inverse spinel nature of end-member magnetite (Mgt) at room-T. However, in intermediate compositions Al and Mg substantially replace Fe and small amounts of Fe2+Td are found, stabilized or trapped by decreasing occurrence of the continuous nearest neighbour Fe – Fe interactions which facilitate charge redistribution by electron transfer. Furthermore, in the composition range ~Mgt0.4-0.9, XAS and XMCD bonding and site occupancy data suggest that nano-scale, magnetite-like Fe clusters are present. By contrast, at the spinel-rich end of the series, Mgt0.17 and Mgt0.23 have a homogeneous long-range distribution of Fe, Mg and Al. These relationships are consistent with the intermediate and Fe-rich samples falling within a wide solvus in this system such that the Fe-clusters occur as proto-nuclei for phases which would exsolve following development of long-range crystalline order during slow cooling. Calculated saturation magnetic moments (Ms) for the Fe-rich samples show excellent agreement with measured values but for the most Mg-rich samples are displaced to slightly higher values; this displacement is due to the presence of abundant Mg and Al disrupting the anti-parallel alignment of electron spins for Fe atoms.

Henderson, C. Michael B.↗
DOE OSTI2014

Ordering Traces Logically to Identify Lateness in Parallel Programs

Event traces are a valuable tool for understanding the behavior of parallel programs. Automatically analyzing a large trace, however, especially without a specific objective, is difficult. We aid this process by extracting a trace’s logical structure, an ordering of events derived from their happened before relationships. From this perspective, we can calculate delays relative to peers rather than duration and determine where they begin and how they propagate. The logical structure also acts as a platform for comparing and clustering processes as well as showing communication patterns in a trace visualization. We present an algorithm for determining this idealized logical structure from a trace and develop metrics to quantify delays and differences among processes. We implement this in our novel trace visualizer, Ravel, and apply our approach to several applications, demonstrating the accuracy of our extracted structure and its utility in analyzing these codes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

SUPPRESSION OF ELECTRON THERMAL CONDUCTION IN THE HIGH β INTRACLUSTER MEDIUM OF GALAXY CLUSTERS

Understanding the thermodynamic state of the hot intracluster medium (ICM) in a galaxy cluster requires knowledge of the plasma transport processes, especially thermal conduction. The basic physics of thermal conduction in plasmas with ICM-like conditions has yet to be elucidated, however. We use particle-in-cell simulations and analytic models to explore the dynamics of an ICM-like plasma (with small gyroradius, large mean free path, and strongly sub-dominant magnetic pressure) driven by the diffusive heat flux associated with thermal conduction. Linear theory reveals that whistler waves are driven unstable by electron heat flux, even when the heat flux is weak. The resonant interaction of electrons with these waves then plays a critical role in scattering electrons and suppressing the heat flux. In a 1D model where only whistler modes that are parallel to the magnetic field are captured, the only resonant electrons are moving in the opposite direction to the heat flux, and the electron heat flux suppression is small. In 2D or more, oblique whistler modes also resonate with electrons moving in the direction of the heat flux. The overlap of resonances leads to effective symmetrization of the electron distribution function and a strong suppression of heat flux. The results suggest that thermal conduction in the ICM might be strongly suppressed, possibly to negligible levels.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Enhanced critical current in superconducting FeSe 0.5 Te 0.5 films at all magnetic field orientations by scalable gold ion irradiation

The loss-less electrical current-carrying capability of type II superconductors, measured by the critical current density J c , can be increased by engineering desirable defects in superconductors to pin the magnetic vortices. Here, we demonstrate that such desirable defects can be created in superconducting FeSe 0.5 Te 0.5 films by 6 MeV Au-ions irradiations that produce cluster-like defects with sizes of 10–15 nm over the entire film. The pristine FeSe 0.5 Te 0.5 film exhibits a low anisotropy in the angular dependence of J c . A clear improvement in the J c is observed upon Au-ion irradiation for all field orientations at 4.2 K. Furthermore, a nearly 70% increase in J c is observed at a magnetic field of 9 T applied parallel to the crystallographic c-axis at 10 K with little reduction of the superconducting transition temperature T c . Our studies show that a dose of 1×10 12 Au cm –2 irradiation at a few MeV is sufficient in order to provide a strong isotropic pinning defect landscape in iron-based superconducting films.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗
DOE OSTI2022

Theoretical study of the CO 2 –O 2 van der Waals complex: potential energy surface and applications

A four-dimensional-potential energy surface (4D-PES) of the atmospherically relevant carbon dioxide–oxygen molecule (CO 2 –O 2 ) van der Waals complex is mapped using the ab initio explicitly correlated coupled cluster method with single, double, and perturbative triple excitations (UCCSD(T)-F12b), and extrapolation to the complete basis set (CBS) limit using the cc-pVTZ-F12/cc-pVQZ-F12 bases and the l –3 formula. An analytic representation of the 4D-PES was fitted using the method of interpolating moving least squares (IMLS). These calculations predict that the most stable configuration of CO 2 –O 2 complex corresponds to a planar slipped-parallel structure with a binding energy of V ~ –243 cm –1 . Another isomer is found on the PES, corresponding to a non-planar cross-shaped structure, with V ~ –218 cm –1 . The transition structure connecting the two minima is found at V ~ –211 cm –1 . We also performed comparisons with some CO 2 –X van der Waals complexes. Moreover, we provide a SAPT analysis of this molecular system. Then, we discuss the complexation induced shifts of CO 2 and O 2 . Afterwards, this new 4D-PES is employed to compute the second virial coefficient including temperature dependence. A comparison between quantities obtained in our calculations and those from experiments found close agreement attesting to the high quality of the PES and to the importance of considering a full description of the anisotropic potential for the derivation of thermophysical properties of CO 2 –O 2 mixtures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2017

Aces4: A Platform for Computational Chemistry Calculations with Extremely Large Block-Sparse Arrays

Aces4 is a parallel programming platform comprising a DSL for Computational Chemistry and its runtime system. It offers a convenient way to express parallelism together with extensive support for extremely large, possibly sparse, distributed arrays. It aids scientists in the creation of performant, scalable, massively parallel programs that can effectively take advantage of leadership class computing systems to address important scientific questions. Aces4 has enabled the development and implementation of new methods in electronic structure theory which are breaking new ground in their ability to perform highly accurate calculations on ever larger molecular systems. In this paper the design of Aces4, which is based on the the Super Instruction Architecture approach, is described. Experimental scaling results for Molecular Cluster Perturbation Theory, a new method enabled by Aces4, and CCSD, a widely used computational chemistry method are given.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Comparing the Performance of Julia on CPUs versus GPUs and Julia-MPI versus Fortran-MPI: a case study with MPAS-Ocean (Version 7.1)

Abstract. Some programming languages are easy to develop at the cost of slow execution, while others are fast at runtime but much more difficult to write. Julia is a programming language that aims to be the best of both worlds – a development and production language at the same time. To test Julia's utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, the Model for Prediction Across Scales–Ocean (MPAS-Ocean), as well as a Python shallow water code. Three versions of the Julia shallow water code were created: for single-core CPU, graphics processing unit (GPU), and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the Julia model was 13 times faster than Python using NumPy, where both used an unthreaded single-core CPU. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20× speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code was almost identical in terms of performance to the MPI parallelized code on 64 processes, an unexpected result for such different architectures. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts and ranges from 2× faster to 2× slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

54 ENVIRONMENTAL SCIENCES↗