SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Ad Hoc File Systems for High-Performance Computing

Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include nodelocal SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 “Challenges and Opportunities of User-Level File Systems for HPC” and discusses application scenarios as well as design strategies for ad hoc file systems using node-local storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Advances in Patch-Based Adaptive Mesh Refinement Scalability

Patch-based structured adaptive mesh refinement (SAMR) is widely used for high-resolution simu- lations. Combined with modern supercomputers, it could provide simulations of unprecedented size and resolution. A persistent challenge for this com- bination has been managing dynamically adaptive meshes on more and more MPI tasks. The dis- tributed mesh management scheme in SAMRAI has made some progress SAMR scalability, but early al- gorithms still had trouble scaling past the regime of 105 MPI tasks. This work provides two critical SAMR regridding algorithms, which are integrated into that scheme to ensure efficiency of the whole. The clustering algorithm is an extension of the tile- clustering approach, making it more flexible and efficient in both clustering and parallelism. The partitioner is a new algorithm designed to prevent the network congestion experienced by its prede- cessor. We evaluated performance using weak- and strong-scaling benchmarks designed to be difficult for dynamic adaptivity. Results show good scaling on up to 1.5M cores and 2M MPI tasks. Detailed timing diagnostics suggest scaling would continue well past that.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

The Importance of Three-Body Interactions in Molecular Dynamics Simulations of Water with the Fragment Molecular Orbital Method

The analytic first derivative with respect to nuclear coordinates is formulated and implemented in the framework of the three-body fragment molecular orbital (FMO) method. The gradient has been derived and implemented for restricted Hartree-Fock, second-order Møller-Plesset perturbation, and density functional theories. The importance of the three-body fully analytic gradient is illustrated through the failure of the two-body FMO method during molecular dynamics simulations of a small water cluster. The parallel implementation of the fragment molecular orbital method, its parallel efficiency, and its scalability on the Blue Gene/Q architecture up to 262,144 CPU cores, are also discussed.

Pruitt, Spencer R.↗

DOE OSTI2024

Accelerating science: The usage of commercial clouds in ATLAS Distributed Computing

The ATLAS experiment at CERN is one of the largest scientific machines built to date and will have ever growing computing needs as the Large Hadron Collider collects an increasingly larger volume of data over the next 20 years. ATLAS is conducting R&D projects on Amazon Web Services and Google Cloud as complementary resources for distributed computing, focusing on some of the key features of commercial clouds: lightweight operation, elasticity and availability of multiple chip architectures. The proof of concept phases have concluded with the cloud-native, vendoragnostic integration with the experiment’s data and workload management frameworks. Google Cloud has been used to evaluate elastic batch computing, ramping up ephemeral clusters of up to O(100k) cores to process tasks requiring quick turnaround. Amazon Web Services has been exploited for the successful physics validation of the Athena simulation software on ARM processors. We have also set up an interactive facility for physics analysis allowing endusers to spin up private, on-demand clusters for parallel computing with up to 4 000 cores, or run GPU enabled notebooks and jobs for machine learning applications. The success of the proof of concept phases has led to the extension of the Google Cloud project, where ATLAS will study the total cost of ownership of a production cloud site during 15 months with 10k cores on average, fully integrated with distributed grid computing resources and continue the R&D projects.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2015

HUBBLE FRONTIER FIELDS FIRST COMPLETE CLUSTER DATA: FAINT GALAXIES AT z ∼ 5-10 FOR UV LUMINOSITY FUNCTIONS AND COSMIC REIONIZATION

We present comprehensive analyses of faint dropout galaxies up to z ∼ 10 with the first full-depth data set of the A2744 lensing cluster and parallel fields observed by the Hubble Frontier Fields (HFF) program. We identify 54 dropouts at z ∼ 5-10 in the HFF fields and enlarge the size of the z ∼ 9 galaxy sample obtained to date. Although the number of highly magnified (μ ∼ 10) galaxies is small because of the tiny survey volume of strong lensing, our study reaches the galaxies' intrinsic luminosities comparable to the deepest-field HUDF studies. We derive UV luminosity functions with these faint dropouts, carefully evaluating by intensive simulations the combination of observational incompleteness and lensing effects in the image plane, including magnification, distortion, and multiplication of images, with the evaluation of mass model dependencies. Our results confirm that the faint-end slope, α, is as steep as –2 at z ∼ 6-8 and strengthen the evidence for the rapid decrease of UV luminosity densities, ρ{sub UV}, at z > 8 from the large z ∼ 9 sample. We examine whether the rapid ρ{sub UV} decrease trend can be reconciled with the large Thomson scattering optical depth, τ{sub e}, measured by cosmic microwave background experiments, allowing a large space of free parameters, such as an average ionizing photon escape fraction and a stellar-population-dependent conversion factor. No parameter set can reproduce both the rapid ρ{sub UV} decrease and the large τ {sub e}. It is possible that the ρ{sub UV} decrease moderates at z ≳ 11, that the free parameters significantly evolve toward high z, or that there exist additional sources of reionization such as X-ray binaries and faint active galactic nuclei.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

The sizes of z ∼ 6–8 lensed galaxies from the Hubble Frontier Fields Abell 2744 data

We investigate the sizes of z ∼ 6–8 dropout galaxies using the complete data of the Abell 2744 cluster and parallel fields in the Hubble Frontier Fields program. By directly fitting light profiles of observed galaxies with lensing-distorted Sérsic profiles on the image plane with the glafic software, we accurately measure intrinsic sizes of 31 z ∼ 6–7 and 8 z∼8 galaxies, including those as faint as M{sub UV}≃−16.6. We find that half-light radii r{sub e} positively correlates with UV luminosity at each redshift, although the correlation is not very tight. The largest (r{sub e}>0.8 kpc) galaxies are mostly red in UV color while the smallest (r{sub e}<0.08 kpc) ones tend to be blue. We also find that galaxies with multiple cores tend to be brighter. Combined with previous results at 2.5≲z≲12, our result confirms that the average r{sub e} of bright ((0.3–1)L{sub z=3}{sup ∗}) galaxies scales as r{sub e}∝(1+z){sup −m} with m=1.24±0.1. We find that the ratio of r{sub e} to virial radius is virtually constant at 3.3 ± 0.1% over a wide redshift range, where the virial radii of hosting dark matter halos are derived based on the abundance matching. This constant ratio is consistent with the disk formation model by Mo et al. with j{sub d}∼m{sub d}, where j{sub d} and m{sub d} are the fractions of the angular momentum and mass within halos confined in the disks. A comparison with various types of local galaxies indicates that our galaxies are most similar to circumnuclear star-forming regions of barred galaxies in the sense that a sizable amount of stars are forming in a very small area.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2016

psst: An open-source power system simulation toolbox in Python

This paper is an overview of Power System Simulation Toolbox (psst). psst is an open-source Python application for the simulation and analysis of power system models. psst simulates the wholesale market operation by solving a DC Optimal Power Flow (DCOPF), Security Constrained Unit Commitment (SCUC) and a Security Constrained Economic Dispatch (SCED). psst also includes models for the various entities in a power system such as Generator Companies (GenCos), Load Serving Entities (LSEs) and an Independent System Operator (ISO). psst features an open modular object oriented architecture that will make it useful for researchers to customize, expand, experiment beyond solving traditional problems. psst also includes a web based Graphical User Interface (GUI) that allows for user friendly interaction and for implementation on remote High Performance Computing (HPCs) clusters for parallelized operations. This paper also provides an illustrative application of psst and benchmarks with standard IEEE test cases to show the advanced features and the performance of toolbox.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Characterizing Parallel Scientific Applications on Commodity Clusters: An Empirical Study of a Tapered Fat Tree

Abstract not provided

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

A Global Climate Model Agent for High Spatial and Temporal Resolution Data

Fine cell granularity in modern climate models can produce terabytes of data in each snapshot, causing significant I/O overhead. To address this issue, a method of reducing the I/O latency of high-resolution climate models by identifying and selectively outputting regions of interest is presented. Working with a Global Cloud Resolving Model and running with up to 10240 processors on a Cray XE6, this method provides significant I/O bandwidth reduction depending on the frequency of writes and size of the region of interest. The implementation challenges of determining global parameters in a strictly core-localized model and properly formatting output files that only contain subsections of the global grid are addressed, as well as the overall bandwidth impact and benefits of the method. The gains in I/O throughput provided by this method allow dual output rates for high-resolution climate models: a low-frequency global snapshot as well as a high-frequency regional snapshot when events of particular interest occur.

global cloud resolving model↗

NASA NTRS2001

Aircraft Engine Systems

This report outlines the detailed simulation of Aircraft Turbofan Engine. The objectives were to develop a detailed flow model of a full turbofan engine that runs on parallel workstation clusters overnight and to develop an integrated system of codes for combustor design and analysis to enable significant reduction in design time and cost. The model will initially simulate the 3-D flow in the primary flow path including the flow and chemistry in the combustor, and ultimately result in a multidisciplinary model of the engine. The overnight 3-D simulation capability of the primary flow path in a complete engine will enable significant reduction in the design and development time of gas turbine engines. In addition, the NPSS (Numerical Propulsion System Simulation) multidisciplinary integration and analysis are discussed.

Veres, Joseph↗

NASA NTRS1996

NASA Approach to HPCCP Support Software and Tools

The NASA HPCC Program, together with other agencies participating in the Federal HPCC Program, intends to advance technologies to enable the execution of grand challenge applications at sustained rates up to TeraFLOPS. During 1995-6 NASA undertook two major systems software efforts to improve the state of high performance support software and tools. The first of these activities was a replanning of support software and tools activities internal to the Agency. In replanning the software activities emphasis was placed on Meeting the needs of Grand Challenge Uses Few projects Near term useful results. The revised NASA plan calls for support software and tools activities in four areas: Application Creation Process Support Application Usage/Operations Support Advanced Support Software and Tools Concepts Metrics Based Monitoring and Management The second major activity undertaken was participation in a multiagency Task Force resulting from the Second Pasadena Workshop on System Software and Tools. The task force developed the Guidelines for Writing System Software and Tools Requirements for Parallel and Clustered Computers.

Blaylock, Bruce↗

NASA NTRS2001

Arthropod phylogeny based on eight molecular loci and morphology

The interrelationships of major clades within the Arthropoda remain one of the most contentious issues in systematics, which has traditionally been the domain of morphologists. A growing body of DNA sequences and other types of molecular data has revitalized study of arthropod phylogeny and has inspired new considerations of character evolution. Novel hypotheses such as a crustacean-hexapod affinity were based on analyses of single or few genes and limited taxon sampling, but have received recent support from mitochondrial gene order, and eye and brain ultrastructure and neurogenesis. Here we assess relationships within Arthropoda based on a synthesis of all well sampled molecular loci together with a comprehensive data set of morphological, developmental, ultrastructural and gene-order characters. The molecular data include sequences of three nuclear ribosomal genes, three nuclear protein-coding genes, and two mitochondrial genes (one protein coding, one ribosomal). We devised new optimization procedures and constructed a parallel computer cluster with 256 central processing units to analyse molecular data on a scale not previously possible. The optimal 'total evidence' cladogram supports the crustacean-hexapod clade, recognizes pycnogonids as sister to other euarthropods, and indicates monophyly of Myriapoda and Mandibulata.

Non-NASA Center↗

NASA NTRS2001

Validation of a High-Order Compact Code for Nonlinear Flows About Complex Geometric

The slides review computational requirements for nozzle exhaust flow and noise calculations and the current numerical method, validation of prefactored compact scheme on CAA benchmark problems, a curvilinear grid performance test of gust response of a Joukowski airfoil, airfoil surface RMS pressure distribution and far field noise radiation results for Joukowski airfoil in a vortical gust, boundary distance study for Joukowski airfoil problem, and performance of ICOMP parallel Macintosh cluster.

Hixon, Ray↗

NASA NTRS2004

Accessing and visualizing scientific spatiotemporal data

rendering↗

NASA NTRS2004

Accessing and Visualizing scientific spatiotemporal data

This paper discusses work done by JPL 's Parallel Applications Technologies Group in helping scientists access and visualize very large data sets through the use of multiple computing resources, such as parallel supercomputers, clusters, and grids These tools do one or more of the following tasks visualize local data sets for local users, visualize local data sets for remote users, and access and visualize remote data sets The tools are used for various types of data, including remotely sensed image data, digital elevation models, astronomical surveys, etc The paper attempts to pull some common elements out of these tools that may be useful for others who have to work with similarly large data sets.

data sets↗

NASA NTRS2014

Terminal Area Simulation System User's Guide - Version 10.0

The Terminal Area Simulation System (TASS) is a three-dimensional, time-dependent, large eddy simulation model that has been developed for studies of wake vortex and weather hazards to aviation, along with other atmospheric turbulence, and cloud-scale weather phenomenology. This document describes the source code for TASS version 10.0 and provides users with needed documentation to run the model. The source code is programed in Fortran language and is formulated to take advantage of vector and efficient multi-processor scaling for execution on massively-parallel supercomputer clusters. The code contains different initialization modules allowing the study of aircraft wake vortex interaction with the atmosphere and ground, atmospheric turbulence, atmospheric boundary layers, precipitating convective clouds, hail storms, gust fronts, microburst windshear, supercell and mesoscale convective systems, tornadic storms, and ring vortices. The model is able to operate in either two- or three-dimensions with equations numerically formulated on a Cartesian grid. The primary output from the TASS is time-dependent domain fields generated by the prognostic equations and diagnosed variables. This document will enable a user to understand the general logic of TASS, and will show how to configure and initialize the model domain. Also described are the formats of the input and output files, as well as the parameters that control the input and output.

Switzer, George F.↗

NASA NTRS2015

Advances in Parallelization for Large Scale Oct-Tree Mesh Generation

Despite great advancements in the parallelization of numerical simulation codes over the last 20 years, it is still common to perform grid generation in serial. Generating large scale grids in serial often requires using special "grid generation" compute machines that can have more than ten times the memory of average machines. While some parallel mesh generation techniques have been proposed, generating very large meshes for LES or aeroacoustic simulations is still a challenging problem. An automated method for the parallel generation of very large scale off-body hierarchical meshes is presented here. This work enables large scale parallel generation of off-body meshes by using a novel combination of parallel grid generation techniques and a hybrid "top down" and "bottom up" oct-tree method. Meshes are generated using hardware commonly found in parallel compute clusters. The capability to generate very large meshes is demonstrated by the generation of off-body meshes surrounding complex aerospace geometries. Results are shown including a one billion cell mesh generated around a Predator Unmanned Aerial Vehicle geometry, which was generated on 64 processors in under 45 minutes.

O'Connell, Matthew↗

DOE OSTI2012

Performance monitoring for new phase dynamic optimization of instruction dispatch cluster configuration

In a processor having multiple clusters which operate in parallel, the number of clusters in use can be varied dynamically. At the start of each program phase, the configuration option for an interval is run to determine the optimal configuration, which is used until the next phase change is detected. The optimum instruction interval is determined by starting with a minimum interval and doubling it until a low stability factor is reached.

97 MATHEMATICS AND COMPUTING↗