Compiling on Linux Clusters
This is a presentation for LANL's Parallel Computing Summer Research Internship that describes the process of compiling Linux clusters.
97 MATHEMATICS AND COMPUTING↗
SEARCH · Search NASA
Results for “parallel clustering”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Profiling the BLAST bioinformatics application for load balancing on high-performance computing clusters
Abstract Background The Basic Local Alignment Search Tool (BLAST) is a suite of commonly used algorithms for identifying matches between biological sequences. The user supplies a database file and query file of sequences for BLAST to find identical sequences between the two. The typical millions of database and query sequences make BLAST computationally challenging but also well suited for parallelization on high-performance computing clusters. The efficacy of parallelization depends on the data partitioning, where the optimal data partitioning relies on an accurate performance model. In previous studies, a BLAST job was sped up by 27 times by partitioning the database and query among thousands of processor nodes. However, the optimality of the partitioning method was not studied. Unlike BLAST performance models proposed in the literature that usually have problem size and hardware configuration as the only variables, the execution time of a BLAST job is a function of database size, query size, and hardware capability. In this work, the nucleotide BLAST application BLASTN was profiled using three methods: shell-level profiling with the Unix “time” command, code-level profiling with the built-in “profiler” module, and system-level profiling with the Unix “gprof” program. The runtimes were measured for six node types, using six different database files and 15 query files, on a heterogeneous HPC cluster with 500+ nodes. The empirical measurement data were fitted with quadratic functions to develop performance models that were used to guide the data parallelization for BLASTN jobs. Results Profiling results showed that BLASTN contains more than 34,500 different functions, but a single function, RunMTBySplitDB, takes 99.12% of the total runtime. Among its 53 child functions, five core functions were identified to make up 92.12% of the overall BLASTN runtime. Based on the performance models, static load balancing algorithms can be applied to the BLASTN input data to minimize the runtime of the longest job on an HPC cluster. Four test cases being run on homogeneous and heterogeneous clusters were tested. Experiment results showed that the runtime can be reduced by 81% on a homogeneous cluster and by 20% on a heterogeneous cluster by re-distributing the workload. Discussion Optimal data partitioning can improve BLASTN’s overall runtime 5.4-fold in comparison with dividing the database and query into the same number of fragments. The proposed methodology can be used in the other applications in the BLAST+ suite or any other application as long as source code is available.
59 BASIC BIOLOGICAL SCIENCES↗
Field Observation of the Green Ocean Amazon. Neutral Cluster Air Ion Spectrometer (NAIS) Final Campaign Summary
The neutral cluster and air ion spectrometer (NAIS) was deployed to the T3 site for Intensive Operations Periods 1 and 2 (IOP1 and IOP2). The NAIS is an instrument that measures aerosol particle and ion number size distributions in the mobility diameter range of 0.8 to 42 nm, corresponding to electrical mobility range between 3.2 and 0.0013 cm 2 V -1 s -1 . New particle formation (NPF) events were detected using the NAIS at the T3 field site during IOP1 and IOP2. Secondary NPF is a globally important source of aerosol number. To fully explain atmospheric NPF and subsequent growth, we need to directly measure the initial steps of the formation processes in different environments, including rain forest. Particle formation characteristics, such as formation and growth rates, were used as indicators of the relevant processes and participating compounds in the initial formation. In a case of parallel ion and neutral cluster measurements, we estimated the relative contribution of ion-induced and neutral nucleation to the total particle formation.
47 OTHER INSTRUMENTATION↗
NWChem: Quantum Chemistry Simulations at Scale
Methods based on quantum mechanics equations have been developed since the 1930's with the purpose of accurately studying the electronic structure of molecules. However, it is only during the last two decades that intense development of new computational algorithms has opened the possibility of performing accurate simulations of challenging molecular processes with high-order many-body methods. A wealth of evidence indicates that the proper inclusion of instantaneous interactions between electrons (or the so-called electron correlation effects) is indispensable for the accurate characterization of chemical reactivity, molecular properties, and interactions of light with matter. The availability of reliable methods for benchmarking of medium-size molecular systems provides also a unique chance to propagate high-level accuracy across spatial scales through the multiscale methodologies. Some of these methods have potential to utilize computational resources in an effi*cient way since they are characterized by high numerical complexity and appropriate level of data granularity, which can be effi*ciently distributed over multi-processor architectures. The broad spectrum of coupled cluster (CC) methods falls into this class of methodologies. Several recent CC implementations clearly demonstrated the scalability of CC formalisms on architectures composed of hundreds thousand computational cores. In this context NWChem provides a collection of Tensor Contraction Engine (TCE) generated parallel implementations of various coupled cluster methods capable of taking advantage of many thousand of cores on leadership class parallel architectures.
chemistry↗
Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis
X‐ray scattering experiments using free electron lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID‐19 viral proteins). XFEL experiments are a challenge to computing in two ways: (i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; (ii) data‐collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min—sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS‐NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis within 10 min is a first for XFEL experiments and an important milestone if we are to keep up with data‐collection trends.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
TATB Thermal Decomposition: An Improved Kinetic Model for Explosive Safety Analysis
Explore the source record for details and available documents.
Materials science, Chemistry - Chemical explosives↗
Progressive Hedging Decomposition for Solutions of Large-Scale Process Family Design Problems
Rapid, wide-scale deployment of green process systems, such as carbon capture or water desalination systems, is essential for combatting climate change. Methods relying on traditional design or modularity fail to capture the benefits of both economies of numbers and economies of scale. We have proposed process family design, which designs a family of processes simultaneously exploiting opportunities for common elements. In previous work, we explored different optimization formulations to solve this problem. In this work, we develop a decomposition approach to tackle larger problems efficiently. We solve a water desalination case study, which is too large to solve within a reasonable timeframe with the discretization formulation. We exploit the block angular structure of the discretization problem to decompose and solve using Progressive Hedging (PH). We use the open-source Python package mpi-sppy to execute PH which allows us to leverage parallelization and a HPC cluster to further improve solution time.
Stinchfield, Georgia↗
Unraveling the functional dark matter through global metagenomics
Metagenomes encode an enormous diversity of proteins, reflecting a multiplicity of functions and activities1,2. Exploration of this vast sequence space has been limited to a comparative analysis against reference microbial genomes and protein families derived from those genomes. Here, to examine the scale of yet untapped functional diversity beyond what is currently possible through the lens of reference genomes, we develop a computational approach to generate reference-free protein families from the sequence space in metagenomes. We analyse 26,931 metagenomes and identify 1.17 billion protein sequences longer than 35 amino acids with no similarity to any sequences from 102,491 reference genomes or the Pfam database3. Using massively parallel graph-based clustering, we group these proteins into 106,198 novel sequence clusters with more than 100 members, doubling the number of protein families obtained from the reference genomes clustered using the same approach. We annotate these families on the basis of their taxonomic, habitat, geographical and gene neighbourhood distributions and, where sufficient sequence diversity is available, predict protein three-dimensional models, revealing novel structures. Overall, our results uncover an enormously diverse functional space, highlighting the importance of further exploring the microbial functional dark matter.
54 ENVIRONMENTAL SCIENCES↗
Tau-REx I: A next generation retrieval code for exoplanetary atmospheres
Spectroscopy of exoplanetary atmospheres has become a well established method for the characterization of extrasolar planets. We here present a novel inverse retrieval code for exoplanetary atmospheres. T-REx(Tau Retrieval for Exoplanets) is a line-by-line radiative transfer fully Bayesian retrieval framework. T-REx includes the following features:(1) the optimized use of molecular line lists from the ExoMol project; (2) an unbiased atmospheric composition prior selection, through custom built pattern recognition software; (3) the use of two independent algorithms to fully sample the Bayesian likelihood space: nested sampling as well as a more classical Markov Chain Monte Carlo approach; (4) iterative Bayesian parameter and model selection using the full Bayesian Evidence as well as the Savage–Dickey Ratio for nested models; and (5) the ability to fully map very large parameter spaces through optimal code parallelization and scalability to cluster computing. In this publication we outline the T-REx framework and demonstrate, using a theoretical hot-Jupiter transmission spectrum, the parameter retrieval and model selection. We investigate the impact of signal-to-noise ratio and spectral resolution on the retrievability of individual model parameters, both in terms of error bars on the temperature and molecular mixing ratios as well as its effect on the model’s global Bayesian evidence.
79 ASTRONOMY AND ASTROPHYSICS↗
Poplar: a phylogenomics pipeline
Motivation Generating phylogenomic trees from the genomic data is essential in understanding biological systems. Each step of this complex process has received extensive attention and has been significantly streamlined over the years. Given the public availability of data, obtaining genomes for a wide selection of species is straightforward. However, analyzing that data to generate a phylogenomic tree is a multistep process with legitimate scientific and technical challenges, often requiring a significant input from a domain-area scientist. Results We present Poplar, a new, streamlined computational pipeline, to address the computational logistical issues that arise when constructing the phylogenomic trees. It provides a framework that runs state-of-the-art software for essential steps in the phylogenomic pipeline, beginning from a genome with or without an annotation, and resulting in a species tree. Running Poplar requires no external databases. In the execution, it enables parallelism for execution for clusters and cloud computing. The trees generated by Poplar match closely with state-of-the-art published trees. The usage and performance of Poplar is far simpler and quicker than manually running a phylogenomic pipeline. Availability and implementation Freely available on GitHub at https://github.com/sandialabs/poplar. Implemented using Python and supported on Linux.
Koning, Elizabeth [Sandia National Laboratories (S↗
Implementation of a practical Markov chain Monte Carlo sampling algorithm in PyBioNetFit
Abstract Summary Bayesian inference in biological modeling commonly relies on Markov chain Monte Carlo (MCMC) sampling of a multidimensional and non-Gaussian posterior distribution that is not analytically tractable. Here, we present the implementation of a practical MCMC method in the open-source software package PyBioNetFit (PyBNF), which is designed to support parameterization of mathematical models for biological systems. The new MCMC method, am, incorporates an adaptive move proposal distribution. For warm starts, sampling can be initiated at a specified location in parameter space and with a multivariate Gaussian proposal distribution defined initially by a specified covariance matrix. Multiple chains can be generated in parallel using a computer cluster. We demonstrate that am can be used to successfully solve real-world Bayesian inference problems, including forecasting of new Coronavirus Disease 2019 case detection with Bayesian quantification of forecast uncertainty. Availability and implementation PyBNF version 1.1.9, the first stable release with am, is available at PyPI and can be installed using the pip package-management system on platforms that have a working installation of Python 3. PyBNF relies on libRoadRunner and BioNetGen for simulations (e.g. numerical integration of ordinary differential equations defined in SBML or BNGL files) and Dask.Distributed for task scheduling on Linux computer clusters. The Python source code can be freely downloaded/cloned from GitHub and used and modified under terms of the BSD-3 license (https://github.com/lanl/pybnf). Online documentation covering installation/usage is available (https://pybnf.readthedocs.io/en/latest/). A tutorial video is available on YouTube (https://www.youtube.com/watch?v=2aRqpqFOiS4&t=63s). Supplementary information Supplementary data are available at Bioinformatics online.
59 BASIC BIOLOGICAL SCIENCES↗
TH-AB-207A-05: A Fully-Automated Pipeline for Generating CT Images Across a Range of Doses and Reconstruction Methods
Purpose: To evaluate the robustness of CAD or Quantitative Imaging methods, they should be tested on a variety of cases and under a variety of image acquisition and reconstruction conditions that represent the heterogeneity encountered in clinical practice. The purpose of this work was to develop a fully-automated pipeline for generating CT images that represent a wide range of dose and reconstruction conditions. Methods: The pipeline consists of three main modules: reduced-dose simulation, image reconstruction, and quantitative analysis. The first two modules of the pipeline can be operated in a completely automated fashion, using configuration files and running the modules in a batch queue. The input to the pipeline is raw projection CT data; this data is used to simulate different levels of dose reduction using a previously-published algorithm. Filtered-backprojection reconstructions are then performed using FreeCT-wFBP, a freely-available reconstruction software for helical CT. We also added support for an in-house, model-based iterative reconstruction algorithm using iterative coordinate-descent optimization, which may be run in tandem with the more conventional recon methods. The reduced-dose simulations and image reconstructions are controlled automatically by a single script, and they can be run in parallel on our research cluster. The pipeline was tested on phantom and lung screening datasets from a clinical scanner (Definition AS, Siemens Healthcare). Results: The images generated from our test datasets appeared to represent a realistic range of acquisition and reconstruction conditions that we would expect to find clinically. The time to generate images was approximately 30 minutes per dose/reconstruction combination on a hybrid CPU/GPU architecture. Conclusion: The automated research pipeline promises to be a useful tool for either training or evaluating performance of quantitative imaging software such as classifiers and CAD algorithms across the range of acquisition and reconstruction parameters present in the clinical environment. Funding support: NIH U01 CA181156; Disclosures (McNitt-Gray): Institutional research agreement, Siemens Healthcare; Past recipient, research grant support, Siemens Healthcare; Consultant, Toshiba America Medical Systems; Consultant, Samsung Electronics.
60 APPLIED LIFE SCIENCES↗
Scalable deep text comprehension for Cancer surveillance on high-performance computing
Background: Deep Learning (DL) has advanced the state-of-the-art capabilities in bioinformatics applications which has resulted in trends of increasingly sophisticated and computationally demanding models trained by larger and larger data sets. This vastly increased computational demand challenges the feasibility of conducting cutting-edge research. One solution is to distribute the vast computational workload across multiple computing cluster nodes with data parallelism algorithms. In this study, we used a High-Performance Computing environment and implemented the Downpour Stochastic Gradient Descent algorithm for data parallelism to train a Convolutional Neural Network (CNN) for the natural language processing task of information extraction from a massive dataset of cancer pathology reports. We evaluated the scalability improvements using data parallelism training and the Titan supercomputer at Oak Ridge Leadership Computing Facility. To evaluate scalability, we used different numbers of worker nodes and performed a set of experiments comparing the effects of different training batch sizes and optimizer functions.Results: We found that Adadelta would consistently converge at a lower validation loss, though requiring over twice as many training epochs as the fastest converging optimizer, RMSProp. The Adam optimizer consistently achieved a close 2nd place minimum validation loss significantly faster; using a batch size of 16 and 32 allowed the network to converge in only 4.5 training epochs.Conclusions: We demonstrated that the networked training process is scalable across multiple compute nodes communicating with message passing interface while achieving higher classification accuracy compared to a traditional machine learning algorithm.
Qiu, John X.↗
Mode-multiplexed photonic integrated vector dot-product core from inverse design
Photonic computing has the potential to harness the full degrees of freedom (DOFs) of the light field, including the wavelength, spatial mode, spatial location, phase quadrature, and polarization, to achieve a higher level of computing parallelism and scalability than digital electronic processors. While multiplexing using the wavelength and other DOFs can be readily integrated on silicon photonics platforms with compact footprints, conventional mode-division multiplexed (MDM) photonic designs occupy areas exceeding tens to hundreds of microns for a few spatial modes, significantly limiting their scalability. Here, we utilize inverse design to demonstrate an ultracompact photonic computing core that calculates vector dot products based on MDM coherent mixing. Our dot-product core integrates the functionalities of two-mode multiplexers and one multimode coherent mixer within a nominal footprint of 5 μm x 3 μm . We have experimentally demonstrated computing examples on the fabricated dot-product core, including complex number multiplication and motion estimation using optical flow. The compact dot-product core design enables large-scale on-chip integration in a parallel photonic computing primitive cluster for high-throughput scientific computing and computer vision tasks.
97 MATHEMATICS AND COMPUTING↗
Advanced Artificial Science. The development of an artificial science and engineering research infrastructure to facilitate innovative computational modeling, analysis, and application to interdisciplinary areas of scientific investigation.
This is a final report of the DOE award DE-SC0001132, Advanced Artificial Science. The development of an artificial science and engineering research infrastructure to facilitate innovative computational modeling, analysis, and application to interdisciplinary areas of scientific investigation. This document describes the achievements of the goals, and resulting research made possible by this award.
97 MATHEMATICS AND COMPUTING↗
Charon User Manual (V.2.1) (Rev.01)
This manual gives usage information for the Charon semiconductor device simulator. Charon was developed to meet the modeling needs of Sandia National Laboratories and to improve on the capabilities of the commercial TCAD simulators; in particular, the additional capabilities are running very large simulations on parallel computers and modeling displacement damage and other radiation effects in significant detail. The parallel capabilities are based around the MPI interface which allows the code to be ported to a large number of parallel systems, including linux clusters and proprietary "big iron" systems found at the national laboratories and in large industrial settings.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Charon User Manual: v. 2.2 (revision1)
This manual gives usage information for the Charon semiconductor device simulator. Charon was developed to meet the modeling needs of Sandia National Laboratories and to improve on the capabilities of the commercial TCAD simulators; in particular, the additional capabilities are running very large simulations on parallel computers and modeling displacement damage and other radiation effects in significant detail. The parallel capabilities are based around the MPI interface which allows the code to be ported to a large number of parallel systems, including linux clusters and proprietary “big iron” systems found at the national laboratories and in large industrial settings.
42 ENGINEERING↗
PV Hosting Capacity Estimation: Experiences with Scalable Framework; Preprint
Hosting capacity is an indication of the amount of photovoltaics (PV) can be hosted in a distribution system. This paper presents a framework for estimating distributed PV hosting capacity at scale. We first analyze the key challenges of performing relevant large scale simulation including computational and modeling challenges. Then, we develop two python-based software tools in order to conduct repeatable distribution analyses: Distribution Integration Solution Cost Options (DISCO) for configuring and analyzing simulations, and JADE for parallelizing jobs on HPC clusters. A case study of hosting capacity estimation for SMART-DS SFO 2000+ synthetic feeders is used to demonstrate the capability of the developed framework and tools. The framework and tools can help utilities assess the overall hosting capacity of their service territory, through which the overall upgrade cost can be better planned in order to integrate more PV in the future.
distributed energy resources↗