Search NASA⌕ Search

SEARCH · Search NASA

Results for “massive data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Efficient cosmological analysis of the SDSS/BOSS data from the Effective Field Theory of Large-Scale Structure

The precision of the cosmological data allows us to accurately approximate the predictions for cosmological observables by Taylor expanding up to a low order the dependence on the cosmological parameters around a reference cosmology. By applying this observation to the redshift-space one-loop galaxy power spectrum of the Effective Field Theory of Large-Scale Structure, we analyze the BOSS DR12 data by scanning over all the parameters of ΛCDM cosmology with massive neutrinos. Furthermore, we impose several sets of priors, the widest of which is just a Big Bang Nucleosynthesis prior on the current fractional energy density of baryons, Ω b h 2 , and a bound on the sum of neutrino masses to be less than 0.9 eV. In this case we measure the primordial amplitude of the power spectrum, A s , the abundance of matter, Ω m , the Hubble parameter, H 0 , and the tilt of the primordial power spectrum, n s , to about 19%, 5.7%, 2.2% and 7.3% respectively, obtaining ln (10 10 A s )=2.91± 0.19, Ω m =0.314± 0.018, H 0 =68.7± 1.5 km/(s Mpc) and n s =0.979± 0.071 at 68% confidence level.

79 ASTRONOMY AND ASTROPHYSICS↗

Towards the Interoperability of Web, Database, and Mass Storage Technologies for Petabyte Archives

At the San Diego Supercomputer Center, a massive data analysis system (MDAS) is being developed to support data-intensive applications that manipulate terabyte sized data sets. The objective is to support scientific application access to data whether it is located at a Web site, stored as an object in a database, and/or storage in an archival storage system. We are developing a suite of demonstration programs which illustrate how Web, database (DBMS), and archival storage (mass storage) technologies can be integrated. An application presentation interface is being designed that integrates data access to all of these sources. We have developed a data movement interface between the Illustra object-relational database and the NSL UniTree archival storage system running in a production mode at the San Diego Supercomputer Center. With this interface, an Illustra client can transparently access data on UniTree under the control of the Illustr DBMS server. The current implementation is based on the creation of a new DBMS storage manager class, and a set of library functions that allow the manipulation and migration of data stored as Illustra 'large objects'. We have extended this interface to allow a Web client application to control data movement between its local disk, the Web server, the DBMS Illustra server, and the UniTree mass storage environment. This paper describes some of the current approaches successfully integrating these technologies. This framework is measured against a representative sample of environmental data extracted from the San Diego Ba Environmental Data Repository. Practical lessons are drawn and critical research areas are highlighted.

Moore, Reagan↗

A massive open cluster hiding in full sight

ABSTRACT Obscuration and confusion conspire to limit our knowledge of the inner Milky Way. Even at moderate distances, the identification of stellar systems becomes compounded by the extremely high density of background sources. Here, we provide a very revealing example of these complications by unveiling a large, massive, young cluster in the Sagittarius arm that has escaped detection until now despite containing more than 30 stars brighter than G = 13. By combining Gaia DR2 astrometry, Gaia and 2MASS photometry, and optical spectroscopy, we find that the new cluster, which we name Valparaiso 1, located at $\sim 2.3\,$ kpc, is about 75 Ma old and includes a large complement of evolved stars, among which we highlight the 4 d classical Cepheid CM Sct and an M-type giant that probably represents the first detection of an asymptotic giant branch star in a Galactic young open cluster. Although strong differential reddening renders accurate parameter determination unfeasible with the current data set, direct comparison to clusters of similar age suggests that Valparaiso 1 was born as one of the most massive clusters in the solar neighbourhood, with an initial mass close to $10^{4}\, \mathrm{M}_{\odot }$.

Negueruela, I.↗

A Parametric, Data-Driven, Non-Intrusive Reduced-Order Model Framework for Crystal Plasticity Simulations of Voids

The influence of the internal structure at micrometer length scales on the deformation of polycrystalline materials can be effectively captured using crystal plasticity finite element methods (CPFEM). However, the complexity and nonlinearity of the deformation equations CPFEM solves demand significant computational power and resources to achieve accurate predictions, limiting its broader application. To address this challenge, we have identified a reduced-order representation of the complex data in order to establish a computationally efficient reduced-order models (ROM) and drastically reduce the computational expense of CPFEM. Specifically, in this work, we developed a parametric, data-driven, and non-intrusive ROM framework for CPFEM using proper orthogonal decomposition (POD) and sparse variational Gaussian process (SVGP) regression for single-crystal microstructures under tensile loading conditions. The developed protocol enables one to compress field into a latent/low-dimensional space described by principal component analysis (PCA) via the singular value decomposition (SVD) algorithm. As a result, the high-dimensional data are reduced to a significantly smaller amount of dimensions with POD bases and POD coefficients. Furthermore, we deployed an ensemble of SVGPs—extended from the classical Gaussian process (GP) regression for scalability and handling big data—in a massively parallel manner to train and predict latent POD coefficients using known POD bases from a set of previously obtained simulations results. Lastly, using the predicted POD coefficients, we reconstructed the full-field results and showed reasonable agreement compared with the true values obtained from running CPFEM. The developed framework is validated with a set of CPFEM simulations of a single embedded void in single-crystal aluminum alloy. While the framework is broadly applicable, this work specifically focuses on single-crystal microstructures, a single load case (e.g., tensile), and a specific void geometry (spherical).

Anisotropy↗

The Exploration of Local VolumE Satellites (ELVES) Survey: A Nearly Volume-limited Sample of Nearby Dwarf Satellite Systems

We present the final sample of the Exploration of Local VolumE Satellites (ELVES) survey, a survey of the dwarf satellites of a nearly volume-limited sample of Milky Way (MW)-like hosts in the Local Volume. Hosts are selected simply via a cut in luminosity (${M}_{{K}_{s}}\lt -22.1$ mag) and distance (D < 12 Mpc). We cataloged the satellites of 25 of the 31 such hosts, with another five taken from the literature. All hosts are surveyed out to at least 150 projected kpc ( ~ R vir /2), with the majority surveyed to 300 kpc ( ~ R vir ). Satellites are detected using a consistent semiautomated algorithm specialized for low surface brightness dwarfs. As shown through extensive tests with injected galaxies, the catalogs are complete to M V ~ -9 mag and μ 0,V ~ 26.5 mag arcsec -2 . Candidates are confirmed to be real satellites through distance measurements including redshift, tip of the red giant branch, and surface brightness fluctuations. Across all 30 surveyed hosts, there are 338 confirmed satellites with M V < -9 mag, with a further 106 candidates awaiting distance measurement. For the vast majority of these, we provide consistent multiband Sérsic photometry. We show that satellite abundance correlates with host mass, with the MW being quite typical among comparable systems, and that satellite quenched fraction rises steeply with decreasing satellite mass, mirroring the quenched fraction for the MW and M31. The ELVES survey represents a massive increase in the statistics of surveyed systems with known completeness, and the provided catalogs are a unique data set to explore various aspects of small-scale structure and dwarf galaxy evolution.

79 ASTRONOMY AND ASTROPHYSICS↗

Bayesian Inference of the Symmetry Energy of Superdense Neutron-rich Matter from Future Radius Measurements of Massive Neutron Stars

In this work, using as references the posterior probability distribution functions of the equation of state (EOS) parameters inferred from the radii of canonical neutron stars (NSs) reported by the LIGO/VIRGO and NICER Collaborations based on their observations of GW170817 and PSR J0030+0451, we investigate how future radius measurements of more massive NSs will improve our current knowledge about the EOS of superdense neutron-rich nuclear matter, especially its symmetry energy term. Within the Bayesian statistical approach using an explicitly isospin-dependent parametric EOS for the core of NSs, we infer the EOS parameters of superdense neutron-rich nuclear matter from three sets of imagined mass–radius correlation data representing typical predictions by various nuclear many-body theories, that is, the radius stays the same, decreases, or increases with increasing NS mass within ±15% between 1.4 and 2.0 M$_\odot$. The corresponding NS average density increases quickly or slowly or slightly decreases as the NS mass increases from 1.4 to 2.0 M$_\odot$. While the EOSs of symmetric nuclear matter (SNM) inferred from the three data sets are approximately the same, the corresponding symmetry energies above about twice the saturation density of nuclear matter are very different, indicating that the radii of massive NSs carry important information about the high-density behavior of nuclear symmetry energy with little influence from the remaining uncertainties of the SNM EOS at suprasaturation densities.

79 ASTRONOMY AND ASTROPHYSICS↗

The influence of environment on the chemical evolution in low-mass galaxies

The mean alpha-to-iron abundance ratio ([α/Fe]) of galaxies is sensitive to the chemical evolution processes at early time, and it is an indicator of star formation timescale (τ{sub SF}). Although the physical reason remains ambiguous, there is a tight relation between [α/Fe] and stellar velocity dispersion (σ) among massive early-type galaxies (ETGs). However, no work has shown convincing results as to how this relation behaves at low masses. We assemble 15 data sets from the literature and build a large sample that includes 192 nearby low-mass (18<σ<80 km s{sup −1}) ETGs. We find that the [α/Fe]–σ relation generally holds for low-mass ETGs, except in extreme environments. Specifically, in normal galaxy cluster environments, the [α/Fe]–σ relation and its intrinsic scatter are, within uncertainties, similar for low-mass and high-mass ETGs. However, in the most massive relaxed galaxy cluster in our sample, the zero point of the relation is higher and the intrinsic scatter is significantly larger. By contrast, in galaxy groups the zero point of the relation offsets in the opposite direction, again with substantial intrinsic scatter. The elevated [α/Fe] of low-mass ETGs in the densest environments suggests that their star formation was quenched earlier. For the low-mass ETGs in the lowest-density environments, we suggest that their more extended star formation histories suppressed their average [α/Fe]. The large scatter in [α/Fe] may reflect stochasticity in the chemical evolution of low-mass galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗

SHαDE: survey description and mass–kinematics scaling relations for dwarf galaxies

ABSTRACT The Study of H α from Dwarf Emissions (SHαDE) is a high spectral resolution (R = 13 500) H α integral field survey of 69 dwarf galaxies with stellar masses 106 < M⋆ < 109 M⊙. The survey used FLAMES on the ESO Very Large Telescope. SHαDE is designed to study the kinematics and stellar populations of dwarf galaxies using consistent methods applied to massive galaxies and at matching level of detail, connecting these mass ranges in an unbiased way. In this paper, we set out the science goals of SHαDE, describe the sample properties, outline the data reduction, and analysis processes. We investigate the log M⋆–log S0.5 mass–kinematics scaling relation, which has previously shown potential for combining galaxies of all morphologies in a single scaling relation. We extend the scaling relation from massive galaxies to dwarf galaxies, demonstrating this relation is linear down to a stellar mass of M⋆ ∼ 108.6 M⊙. Below this limit, the kinematics of galaxies inside one effective radius appears to be dominated by the internal velocity dispersion limit of the H α-emitting gas, giving a bend in the log M⋆–log S0.5 relation. Replacing stellar mass with total baryonic mass using gas mass estimate reduces the severity but does not remove the linearity limit of the scaling relation. An extrapolation to estimate the galaxies’ dark matter halo masses, yields a log Mh–log S0.5 scaling relation that is free of any bend, has reduced curvature over the whole mass range, and brings galaxies of all masses and morphologies on to the virial relation.

Barat, Dilyar↗

The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample

Here we present cosmological results from the final galaxy clustering data set of the Baryon Oscillation Spectroscopic Survey, part of the Sloan Digital Sky Survey III. Our combined galaxy sample comprises 1.2 million massive galaxies over an effective area of 9329 deg 2 and volume of 18.7 Gpc 3 , divided into three partially overlapping redshift slices centred at effective redshifts 0.38, 0.51 and 0.61. We measure the angular diameter distance DM and Hubble parameter H from the baryon acoustic oscillation (BAO) method, in combination with a cosmic microwave background prior on the sound horizon scale, after applying reconstruction to reduce non-linear effects on the BAO feature. Using the anisotropic clustering of the pre-reconstruction density field, we measure the product DMH from the Alcock–Paczynski (AP) effect and the growth of structure, quantified by fσ8(z), from redshift-space distortions (RSD). We combine individual measurements presented in seven companion papers into a set of consensus values and likelihoods, obtaining constraints that are tighter and more robust than those from any one method; in particular, the AP measurement from sub-BAO scales sharpens constraints from post-reconstruction BAOs by breaking degeneracy between DM and H. Combined with Planck 2016 cosmic microwave background measurements, our distance scale measurements simultaneously imply curvature ΩK = 0.0003 ± 0.0026 and a dark energy equation-of-state parameter w = -1.01 ± 0.06, in strong affirmation of the spatially flat cold dark matter (CDM) model with a cosmological constant (ΛCDM). Our RSD measurements of fσ8, at 6 per cent precision, are similarly consistent with this model. When combined with supernova Ia data, we find H0 = 67.3 ± 1.0 km s -1 Mpc -1 even for our most general dark energy model, in tension with some direct measurements. Adding extra relativistic species as a degree of freedom loosens the constraint only slightly, to H0 = 67.8 ± 1.2km s -1 Mpc -1 . Assuming flat ΛCDM, we find Ωm = 0.310 ± 0.005 and H0 = 67.6 ± 0.5 km s -1 Mpc -1 , and we find a 95 per cent upper limit of 0.16 eV c -2 on the neutrino mass sum.

79 ASTRONOMY AND ASTROPHYSICS↗

Search for pair production of vector-like T and B quarks in single-lepton final states using boosted jet substructure in proton-proton collisions at $\sqrt{s}=13$ TeV

A search for pair production of massive vector-like T and B quarks in proton-proton collisions at $ \sqrt{s}=13 $ TeV is presented. The data set was collected in 2015 by the CMS experiment at the LHC and corresponds to an integrated luminosity of up to 2.6 fb$^{−1}$. The T and B quarks are assumed to decay through three possible channels into a heavy boson (either a W, Z or Higgs boson) and a third generation quark. This search is performed in final states with one charged lepton and several jets, exploiting techniques to identify W or Higgs bosons decaying hadronically with large transverse momenta. No excess over the predicted standard model background is observed. Upper limits at 95% confidence level on the T quark pair production cross section are set that exclude T quark masses below 860 GeV in the singlet, and below 830 GeV in the doublet branching fraction scenario. For other branching fraction combinations with ℬ(T → tH) + ℬ(T → bW) ≥ 0.4, lower limits on the T quark range from 790 to 940 GeV. Limits are also set on pair production of singlet vector-like B quarks, which can be excluded up to a mass of 730 GeV. The techniques showcased here for understanding highly-boosted final states are important as the sensitivity to new particles is extended to higher masses.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

CMB distance priors revisited: effects of dark energy dynamics, spatial curvature, primordial power spectrum, and neutrino parameters

As a physical and sufficient compression of the full CMB data, the CMB distance priors, or shift parameters, have been widely used and provide a convenient way to include CMB data when obtaining cosmological constraints. In this paper, we revisit this data vector and examine its stability under different cosmological models. We find that the CMB distance priors are an accurate substitute for the full CMB data when probing dark energy dynamics. This is true when the primordial power spectrum model is directly generalized from the power spectrum of the model used in the derivation of the distance priors from the CMB data. We discover a difference when a non-flat model with the untilted primordial inflation power spectrum is used to measure the distance priors. This power spectrum is a radical change from the more conventional tilted primordial power spectrum and violates fundamental assumptions for the reliability of the CMB shift parameters. We also investigate the performance of CMB distance priors when the sum of neutrino masses Σm ν and the effective number of relativistic species N eff are allowed to vary. Our findings are consistent with earlier results: the neutrino parameters can change the measurement of the sound horizon from CMB data, and thus the CMB distance priors. We find that when the neutrino model is allowed to vary, the cold dark matter density ω c and N eff need to be included in the set of parameters that summarize CMB data, in order to reproduce the constraints from the full CMB data. As a result, we present an updated and expanded set of CMB distance priors which can reproduce constraints from the full CMB data within 1σ, and are applicable to models with massive neutrinos, as well as non-standard cosmologies.

79 ASTRONOMY AND ASTROPHYSICS↗

TuckerMPI: A Parallel C++/MPI Software Package for Large-scale Data Compression via the Tucker Tensor Decomposition

With this study, our goal is compression of massive-scale grid-structured data, such as the multi-terabyte output of a high-fidelity computational simulation. For such data sets, we have developed a new software package called TuckerMPI, a parallel C++/MPI software package for compressing distributed data. The approach is based on treating the data as a tensor, i.e., a multidimensional array, and computing its truncated Tucker decomposition, a higher-order analogue to the truncated singular value decomposition of a matrix. The result is a low-rank approximation of the original tensor-structured data. Compression efficiency is achieved by detecting latent global structure within the data, which we contrast to most compression methods that are focused on local structure. In this work, we describe TuckerMPI, our implementation of the truncated Tucker decomposition, including details of the data distribution and in-memory layouts, the parallel and serial implementations of the key kernels, and analysis of the storage, communication, and computational costs. We test the software on 4.5 and 6.7 terabyte data sets distributed across 100 s of nodes (1,000 s of MPI processes), achieving compression ratios between 100 and 200,000×, which equates to 99--99.999% compression (depending on the desired accuracy) in substantially less time than it would take to even read the same dataset from a parallel file system. Moreover, we show that our method also allows for reconstruction of partial or down-sampled data on a single node, without a parallel computer so long as the reconstructed portion is small enough to fit on a single machine, e.g., in the instance of reconstructing/visualizing a single down-sampled time step or computing summary statistics. The code is available at https://gitlab.com/tensors/TuckerMPI.

97 MATHEMATICS AND COMPUTING↗

Fast and Flexible Multivariate Time Series Subsequence Search

Multivariate Time-Series (MTS) are ubiquitous, and are generated in areas as disparate as sensor recordings in aerospace systems, music and video streams, medical monitoring, and financial systems. Domain experts are often interested in searching for interesting multivariate patterns from these MTS databases which often contain several gigabytes of data. Surprisingly, research on MTS search is very limited. Most of the existing work only supports queries with the same length of data, or queries on a fixed set of variables. In this paper, we propose an efficient and flexible subsequence search framework for massive MTS databases, that, for the first time, enables querying on any subset of variables with arbitrary time delays between them. We propose two algorithms to solve this problem (1) a List Based Search (LBS) algorithm which uses sorted lists for indexing, and (2) a R*-tree Based Search (RBS) which uses Minimum Bounding Rectangles (MBR) to organize the subsequences. Both algorithms guarantee that all matching patterns within the specified thresholds will be returned (no false dismissals). The very few false alarms can be removed by a post-processing step. Since our framework is also capable of Univariate Time-Series (UTS) subsequence search, we first demonstrate the efficiency of our algorithms on several UTS datasets previously used in the literature. We follow this up with experiments using two large MTS databases from the aviation domain, each containing several millions of observations. Both these tests show that our algorithms have very high prune rates (>99%) thus needing actual disk access for only less than 1% of the observations. To the best of our knowledge, MTS subsequence search has never been attempted on datasets of the size we have used in this paper.

Bhaduri, Kanishka↗

Fast Multivariate Search on Large Aviation Datasets

Multivariate Time-Series (MTS) are ubiquitous, and are generated in areas as disparate as sensor recordings in aerospace systems, music and video streams, medical monitoring, and financial systems. Domain experts are often interested in searching for interesting multivariate patterns from these MTS databases which can contain up to several gigabytes of data. Surprisingly, research on MTS search is very limited. Most existing work only supports queries with the same length of data, or queries on a fixed set of variables. In this paper, we propose an efficient and flexible subsequence search framework for massive MTS databases, that, for the first time, enables querying on any subset of variables with arbitrary time delays between them. We propose two provably correct algorithms to solve this problem (1) an R-tree Based Search (RBS) which uses Minimum Bounding Rectangles (MBR) to organize the subsequences, and (2) a List Based Search (LBS) algorithm which uses sorted lists for indexing. We demonstrate the performance of these algorithms using two large MTS databases from the aviation domain, each containing several millions of observations Both these tests show that our algorithms have very high prune rates (>95%) thus needing actual

Bhaduri, Kanishka↗

An Innovative Infrastructure with a Universal Geo-Spatiotemporal Data Representation Supporting Cost-Effective Integration of Diverse Earth Science Data

The SpatioTemporal Adaptive Resolution Encoding (STARE) is a unifying scheme encoding geospatial and temporal information for organizing data on scalable computing/storage resources, minimizing expensive data transfers. STARE provides a compact representation that turns set-logic functions into integer operations, e.g. conditional sub-setting, taking into account representative spatiotemporal resolutions of the data in the datasets. STARE geo-spatiotemporally aligns data placements of diverse data on massive parallel resources to maximize performance. Automating important scientific functions (e.g. regridding) and computational functions (e.g. data placement) allows scientists to focus on domain-specific questions instead of expending their efforts and expertise on data processing. With STARE-enabled automation, SciDB (Scientific Database) plus STARE provides a database interface, reducing costly data preparation, increasing the volume and variety of interoperable data, and easing result sharing. Using SciDB plus STARE as part of an integrated analysis infrastructure dramatically eases combining diametrically different datasets.

Rilee, Michael Lee↗

Level-set Modeling Simulations of Chemical Vapor Infiltration for Ceramic Matrix Composites Manufacturing

Silicon-carbide (SiC) reinforced ceramic matrix composites (CMCs) are a key enabling technology to reduce fuel consumption and emissions of gas turbine engines. In one manufacturing approach, chemical vapor infiltration (CVI) is limited to only coating SiC fibers. The preform is then fabricated using a lay-up of basic plys or 2-D woven sheets composed of the precoated fibers. At the other extreme, CVI is used to completely densify a 3-D woven preform shaped almost like the gas turbine component itself. The latter approach is more suitable for highly engineered components which sit directly in the gas path of the engine, for example, a high pressure turbine blade. In this case, the geometry is necessarily complex for aerodynamic, stress, and lifing (multi-physics) requirements. Presently, optimizing the CVI-dominated manufacturing approach is largely by trial-and-error. In this work, a first-principles modeling of CVI is performed to realize optimization of SiC/SiC CMC manufacturing. The modeling is based on a level-set framework to describe the interface between the vapor and solid phases. A finite-difference numerical scheme using an immersed boundary method is developed for fixed, structured meshes. Massively parallel direct numerical simulations (DNS) of CVI through fiber-woven geometries are performed using one-step chemistry, and over a range of Thiele moduli. Illustrative applications of the resulting large DNS data sets are given, including the development of fiber-weave specific infiltration models and structure functions for mean-field (porous media) Computational Fluid Dynamics (CFD) simulations of CVI.

36 MATERIALS SCIENCE↗

The HectoMAP Cluster Survey: Spectroscopically Identified Clusters and their Brightest Cluster Galaxies (BCGs)

We apply a friends-of-friends (FoF) algorithm to identify galaxy clusters and we use the catalog to explore the evolutionary synergy between brightest cluster galaxies (BCGs) and their host clusters. We base the cluster catalog on the dense HectoMAP redshift survey (2000 redshifts deg -2 ). The HectoMAP FoF catalog includes 346 clusters with 10 or more spectroscopic members within the range 0.05 < z < 0.55 and with a median z = 0.29. We list these clusters and their members. We also include central velocity dispersions (σ*, BCG ) for the FoF cluster BCGs, a distinctive feature of the HectoMAP FoF catalog. HectoMAP clusters with higher galaxy number density (80 systems) are all genuine clusters with a strong concentration and a prominent BCG in Subaru/Hyper Suprime-Cam images. The phase-space diagrams show the expected elongation along the line of sight. Lower-density systems include some low reliability systems. We establish a connection between BCGs and their host clusters by demonstrating that σ*, BCG /σ cl decreases as a function of cluster velocity dispersion (σ cl ), in contrast, numerical simulations predict a constant σ*, BCG /σ cl . Sets of clusters at two different redshifts show that BCG evolution in massive systems is slow over the redshift range z < 0.4. The data strongly suggest that minor mergers may play an important role in BCG evolution in clusters with σ cl ≳ 300 km s -1 . For lower mass systems (σ cl < 300 km s -1 ), major mergers may play a significant role. The coordinated evolution of BCGs and their host clusters provides an interesting test of simulations in high-density regions of the universe.

79 ASTRONOMY AND ASTROPHYSICS↗

CoreCruncher : Fast and Robust Construction of Core Genomes in Large Prokaryotic Data Sets

The core genome represents the set of genes shared by all, or nearly all, strains of a given population or species of prokaryotes. Inferring the core genome is integral to many genomic analyses, however, most methods rely on the comparison of all the pairs of genomes; a step that is becoming increasingly difficult given the massive accumulation of genomic data. Here, we present CoreCruncher; a program that robustly and rapidly constructs core genomes across hundreds or thousands of genomes. CoreCruncher does not compute all pairwise genome comparisons and uses a heuristic based on the distributions of identity scores to classify sequences as orthologs or paralogs/xenologs. Although it is much faster than current methods, our results indicate that our approach is more conservative than other tools and less sensitive to the presence of paralogs and xenologs. CoreCruncher is freely available from: https://github.com/lbobay/CoreCruncher. CoreCruncher is written in Python 3.7 and can also run on Python 2.7 without modification. It requires the python library Numpy and either Usearch or Blast. Certain options require the programs muscle or mafft.

59 BASIC BIOLOGICAL SCIENCES↗