Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

Role of Interactions and Correlations on Collective Dynamics of Molecular Motors Along Parallel Filaments

Cytoskeletal motors known as motor proteins are molecules that drive cellular transport along several parallel cytoskeletal filaments and support many biological processes. Experimental evidences suggest that they interact with the nearest molecules of their filament while performing any mechanical work. These interactions modify the microscopic level properties of motor proteins. In this work, a new version of two-channel totally asymmetric simple exclusion process, that incorporates the intra-channel interactions in a thermodynamically consistent way, is proposed. As the existing approaches for multi-channel systems deviate from analyzing the combined effect of inter and intra-channel interactions, a new approach known as modified vertical cluster mean field is developed. The approach along with Monte Carlo simulations successfully encounters some correlations and computes the complex dynamic properties of the system. Role of symmetry of interactions and inter-channel coupling is observed on the phase diagrams, maximal particle current and its corresponding optimal interaction strength. Surprisingly, for all values of coupling rate and most of the interaction splittings, the optimal interaction strength corresponding to maximal current belongs to the case of weak repulsive interactions. Moreover, for weak interaction splittings and with an increase in the coupling rate, the optimal interaction strength tends towards the known experimental results. The effect of coupling as well as interaction energy is also measured for correlations. They are found to be short-range and weaker for repulsive and weak attractive interactions while they are long-range and stronger for large attractions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The Generalized Green’s function Cluster Expansion: A Python package for simulating polarons

We present an efficient implementation of the Generalized Green’s function Cluster Expansion (GGCE), which is a new method for computing the ground-state properties and dynamics of polarons (single electrons coupled to lattice vibrations) in model electron-phonon systems. The GGCE works at arbitrary temperature and is well suited for a variety of electron-phonon couplings, including, but not limited to, site and bond Holstein and Peierls (Su-SchriefferHeeger) couplings, and couplings to multiple phonon modes with different energy scales and coupling strengths. Quick calculations can be performed efficiently on a laptop using solvers from NumPy and SciPy, or in parallel at scale using the PETSc sparse linear solver engine.

36 MATERIALS SCIENCE↗

Active Flash: Out-of-core Data Analytics on Flash Storage

Next generation science will increasingly come to rely on the ability to perform efficient, on-the-fly analytics of data generated by high-performance computing (HPC) simulations, modeling complex physical phenomena. Scientific computing workflows are stymied by the traditional chaining of simulation and data analysis, creating multiple rounds of redundant reads and writes to the storage system, which grows in cost with the ever-increasing gap between compute and storage speeds in HPC clusters. Recent HPC acquisitions have introduced compute node-local flash storage as a means to alleviate this I/O bottleneck. We propose a novel approach, Active Flash, to expedite data analysis pipelines by migrating to the location of the data, the flash device itself. We argue that Active Flash has the potential to enable true out-of-core data analytics by freeing up both the compute core and the associated main memory. By performing analysis locally, dependence on limited bandwidth to a central storage system is reduced, while allowing this analysis to proceed in parallel with the main application. In addition, offloading work from the host to the more power-efficient controller reduces peak system power usage, which is already in the megawatt range and poses a major barrier to HPC system scalability. We propose an architecture for Active Flash, explore energy and performance trade-offs in moving computation from host to storage, demonstrate the ability of appropriate embedded controllers to perform data analysis and reduction tasks at speeds sufficient for this application, and present a simulation study of Active Flash scheduling policies. These results show the viability of the Active Flash model, and its capability to potentially have a transformative impact on scientific data analysis.

99 GENERAL AND MISCELLANEOUS↗

A statistical study of EMIC waves observed by Cluster: 2. Associated plasma conditions

This is the second in a pair of papers discussing a statistical study of electromagnetic ion cyclotron (EMIC) waves detected during 10 years (2001–2010) of Cluster observations. In the first paper, an analysis of EMIC wave properties (i.e., wave power, polarization, normal angle, and wave propagation angle) is presented in both the magnetic latitude (MLAT)-distance as well as magnetic local time (MLT)-L frames. In addition, this paper focuses on the distribution of EMIC wave-associated plasma conditions as well as two EMIC wave generation proxies (the electron plasma frequency to gyrofrequency ratio proxy and the linear theory proxy) in these same frames. Based on the distributions of hot H + anisotropy, electron and hot H+ density measurements, hot H + parallel plasma beta, and the calculated wave generation proxies, three source regions of EMIC waves appear to exist: (1) the well-known overlap between cold plasmaspheric or plume populations with hot anisotropic ring current populations in the postnoon to dusk MLT region; (2) regions all along the dayside magnetosphere at high L shells related to dayside magnetospheric compression and drift shell splitting; and (3) off-equator regions possibly associated with the Shabansky orbits in the dayside magnetosphere.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Evaluation of a pair-based, joint-likelihood association approach for regional infrasound event identification

SUMMARY A Bayesian framework for the association of infrasonic detections is presented and evaluated for analysis at regional propagation scales. A pair-based, joint-likelihood association approach is developed that identifies events by computing the probability that individual detection pairs are attributable to a hypothetical common source and applying hierarchical clustering to identify events from the pair-based analysis. The framework is based on a Bayesian formulation introduced for infrasonic source localization and utilizes the propagation models developed for that application with modifications to improve the numerical efficiency of the analysis. Clustering analysis is completed using hierarchical analysis via weighted linkage for a non-Euclidean distance matrix defined by the negative log-joint-likelihood values. The method is evaluated using regional synthetic data with propagation distances of hundreds of kilometres in order to study the sensitivity of the method to uncertainties and errors in backazimuth and time of arrival. The method is found to be robust and stable for typical uncertainties, able to effectively distinguish noise detections within the data set from those in events, and can be made numerically efficient due to its ease of parallelization.

58 GEOSCIENCES↗

Elemental: a new framework for distributed memory dense matrix computations.

Parallelizing dense matrix computations to distributed memory architectures is a well-studied subject and generally considered to be among the best understood domains of parallel computing. Two packages, developed in the mid 1990s, still enjoy regular use: ScaLAPACK and PLAPACK. With the advent of many-core architectures, which may very well take the shape of distributed memory architectures within a single processor, these packages must be revisited since the traditional MPI-based approaches will likely need to be extended. Thus, this is a good time to review lessons learned since the introduction of these two packages and to propose a simple yet effective alternative. Preliminary performance results show the new solution achieves competitive, if not superior, performance on large clusters.

99 GENERAL AND MISCELLANEOUS↗

Flexible User-Defined Domain Decomposition in Kilometer-Scale E3SM Land Model Simulation

The Energy Exascale Earth System Model (E3SM) Land Model (ELM) has been extended to kilometer-scale (km-ELM) resolutions, enabling high-fidelity simulations of terrestrial processes at 1 km x 1 km grid spacing. In ELM, domain decomposition partitions the computational domain across processors, ensuring efficient parallel execution. Currently, round-robin decomposition is applied, providing a straightforward way to distribute computational workload. As ELM continues evolving at the kilometer-scale (km-scale), particularly with integrating lateral flow modeling, decomposition strategies must also account for the increased workload and data movement. This paper introduces a flexible user-defined domain decomposition framework, allowing users to customize domain partitioning based on application requirements. The impact of different decomposition strategies is evaluated across various applications concerning computation, communication, and I/O. Results demonstrate that while 1D partitioning yields superior I/O performance, k-nearest neighbors (KNN) clustering effectively reduces inter-process communication overhead. This study lays the groundwork for scalable partitioning in large-scale land surface simulations, enhancing next-generation Earth system modeling.

Wang, Dali [ORNL] (ORCID:0000000168065108)↗

Comparison of DeePMD, MTP, GAP, ACE and MACE Machine‐Learned Potentials for Radiation‐Damage Simulations: A User Perspective

Accurate and efficient interatomic potentials are essential for molecular dynamics (MD) simulations of radiation damage, gas diffusion, and phase stability in complex ceramics such as LiAlO 2 , especially under extreme conditions relevant to tritium production. Here, we evaluate the performance of six machine-learned interatomic potentials (MLIPs), moment tensor potential (MTP), Gaussian approximation potential, deep potential (DeePMD), atomic cluster expansion (ACE), message-passing ACE (multilayer atomic cluster expansion (MACE) pretrained) and MACE (trained from-scratch), all trained on the same density functional theory dataset with inclusion of tritium. The MLIPs are benchmarked against traditional Buckingham and ReaxFF potentials in terms of energy accuracy, density predictions, thermal equilibration behavior, threshold displacement energy (E d ), tritium diffusivity, and computational cost. Among the models, MTP shows the best overall balance between efficiency and accuracy, with low force and energy errors and realistic E d values for Li and Al. The ACE and MACE (pretrained and trained from scratch) models exhibit high E d (>200 eV) and unphysical pair interactions. DeePMD underestimates Ed due to overly repulsive behavior even at equilibrium distances. All models over-estimate tritium diffusion but the pretrained MACE model behaves well during tritium-diffusion simulations up to 500 K, maintaining diffusivities in the physically consistent 10 −11 m 2 /s range. Finally, we quantify the computational cost of each potential in large-scale atomic/molecular massively parallel simulator, finding that only MTP is more efficient than traditional empirical potentials, while others are significantly more expensive. These findings explain the trade-offs between accuracy and computational cost in MLIP development and provide essential guidance for use in high-throughput radiation damage and gas diffusion simulations in nuclear ceramics.

74 ATOMIC AND MOLECULAR PHYSICS↗

HydraGNN v4.0

The new version of HydraGNN v4.0 provides additional core capabilities, such as: Inclusion of multi-body atomistic cluster expansion MACE, polarizable atom interaction neural network PAINN, and equivariant principal neighborhood aggregation (PNAEq) among the message passing layers supported -Inclusion of graph transformers to directly model long-range interactions between nodes that are distant in the graph topology Integration of graph transformers with message passing layers by combining the graph embedding generated by the two mechanisms, which allows for an improved expressivity of the HydraGNN architecture Improved re-implementation of multi-task learning (MTL) to allow its use for stabilized training across imbalanced, multi-source, multi-fidelity data Introduction of multi-task parallelism, a newly proposed type of model parallelism specifically for MTL architectures, which allows to dispatch different output decoding heads to different GPU devices Integration of multi-task parallelism with pre-existing distributed data parallelism to enable a 2D parallelization for distributed training Improved portability of the distributed training across Intel GPUs, which has been testes on ALCF exascale supercomputer Aurora Inclusion of 2-level fine-grained energy profilers portable across NVIDIA, AMD, and Intel GPUs to monitor the power and energy consumption associated with different functions executed by the HydraGNN code during data pre-load and training Restructuring of previous examples and inclusion of new sets of examples to illustrate the download, preprocess, and training of HydraGNN models on new large-scale open-source datasets for atomistic materials modeling (e.g., Alexandria, Transition1x, OMat24, OMol25)

Lupo Pasini, Massimiliano [Oak Ridge National Labo↗

Hot Chips and Hot Interconnects for High End Computing Systems

I will discuss several processors: 1. The Cray proprietary processor used in the Cray X1; 2. The IBM Power 3 and Power 4 used in an IBM SP 3 and IBM SP 4 systems; 3. The Intel Itanium and Xeon, used in the SGI Altix systems and clusters respectively; 4. IBM System-on-a-Chip used in IBM BlueGene/L; 5. HP Alpha EV68 processor used in DOE ASCI Q cluster; 6. SPARC64 V processor, which is used in the Fujitsu PRIMEPOWER HPC2500; 7. An NEC proprietary processor, which is used in NEC SX-6/7; 8. Power 4+ processor, which is used in Hitachi SR11000; 9. NEC proprietary processor, which is used in Earth Simulator. The IBM POWER5 and Red Storm Computing Systems will also be discussed. The architectures of these processors will first be presented, followed by interconnection networks and a description of high-end computer systems based on these processors and networks. The performance of various hardware/programming model combinations will then be compared, based on latest NAS Parallel Benchmark results (MPI, OpenMP/HPF and hybrid (MPI + OpenMP). The tutorial will conclude with a discussion of general trends in the field of high performance computing, (quantum computing, DNA computing, cellular engineering, and neural networks).

Saini, Subhash↗

ROOT RNTuple and EOS: The Next Generation of Event Data I/O

For several years, the ROOT team is developing the new RNTuple I/O subsystem in preparation of the next generation of collider experiments. Both HL-LHC and DUNE are expected to start data taking by the end of this decade. They pose unprecedented challenges to event data I/O in terms of data rates, event sizes, and event complexity. At the same time, the I/O landscape is becoming more diverse. HPC cluster file systems and object stores, NVMe disk cache layers in analysis facilities, and S3 storage on cloud resources are mixing with traditional XRootD-managed spinning disk pools.The ROOT team will finalize a first production version of the RNTuple binary format by the end of 2024. After this point, ROOT will provide backward compatibility for RNTuple data. This contribution provides an overview of the RNTuple feature set, the related R&D activities and the long-term vision for RNTuple. We report on performance, interface design, tooling, robustness, integration with experiment frameworks, and validation results, as well as recent R&D on parallel reading and writing and exploitation of modern hardware and storage systems. We will give an outlook on possible future features after a first production release.Collaboratively, the IT and EP departments at CERN have launched a formal project within the Research and Computing sector to evaluate the novel data format for physics analysis data utilized in LHC experiments and other fields. This part of the project focuses on validating the scalability of the EOS storage backend during the transition from the over 25 years old TTree production format to the newly developed RNTuple format, using both replicated and erasure-coded storage profiles.

Blomer, Jakob [CERN]↗

Parallelization of a Six Degree of Freedom Entry Vehicle Trajectory Simulation Using OpenMP and OpenACC

The art and science of writing parallelized software, using methods such as Open Multi-Processing (OpenMP) and Open Accelerators (OpenACC), is dominated by computer scientists. Engineers and non-computer scientists looking to apply these techniques to their project applications face a steep learning curve, especially when looking to adapt their original single threaded software to run multi-threaded on graphics processing units (GPUs). There are significant changes in mindset that must occur; such as how to manage memory, the organization of instructions, and the use of if statements (also known as branching). The purpose of this work is twofold: 1) to demonstrate the applicability of parallelized coding methodologies, OpenMP and OpenACC, to tasks outside of the typical large scale matrix mathematics; and 2) to discuss, from an engineer’s perspective, the lessons learned from parallelizing software using these computer science techniques. This work applies OpenMP, on both multi-core central processing units (CPUs) and Intel® Xeon Phi™ 7210, and OpenACC on GPUs. These parallelization techniques are used to tackle the simulation of thousands of entry vehicle trajectories through the integration of six degree of freedom (DoF) equations of motion (EoM). The forces and moments acting on the entry vehicle, and used by the EoM, are estimated using multiple models of varying levels of complexity. Several benchmark comparisons are made on the execution of six DoF trajectory simulation: single thread Intel® Xeon® E5-2670 CPU, multi-thread CPU using OpenMP, multi-thread Xeon Phi™ 7210 using OpenMP, and multi-thread NVIDIA® Tesla® K40 GPU using OpenACC. These benchmarks are run on the Pleiades Supercomputer Cluster at the National Aeronautics and Space Administration (NASA) Ames Research Center (ARC), and a Xeon Phi™ 7210 node at NASA Langley Research Center (LaRC).

Green, Justin S.↗

Appointing silver and bronze standards for noncovalent interactions: A comparison of spin-component-scaled (SCS), explicitly correlated (F12), and specialized wavefunction approaches

A systematic examination of noncovalent interactions as modeled by wavefunction theory is presented in comparison to gold-standard quality benchmarks available for 345 interaction energies of 49 bimolecular complexes. Quantum chemical techniques examined include spin-component-scaling (SCS) variations on second-order perturbation theory (MP2) [SCS, SCS(N), SCS(MI)] and coupled cluster singles and doubles (CCSD) [SCS, SCS(MI)]; also, method combinations designed to improve dispersion contacts [DW-MP2, MP2C, MP2.5, DW-CCSD(T)-F12]; where available, explicitly correlated (F12) counterparts are also considered. Dunning basis sets augmented by diffuse functions are employed for all accessible ζ-levels; truncations of the diffuse space are also considered. After examination of both accuracy and performance for 394 model chemistries, SCS(MI)-MP2/cc-pVQZ can be recommended for general use, having good accuracy at low cost and no ill-effects such as imbalance between hydrogen-bonding and dispersion-dominated systems or non-parallelity across dissociation curves. Moreover, when benchmarking accuracy is desirable but gold-standard computations are unaffordable, this work recommends silver-standard [DW-CCSD(T**)-F12/aug-cc-pVDZ] and bronze-standard [MP2C-F12/aug-cc-pVDZ] model chemistries, which support accuracies of 0.05 and 0.16 kcal/mol and efficiencies of 97.3 and 5.5 h for adenine·thymine, respectively. Choice comparisons of wavefunction results with the best symmetry-adapted perturbation theory [T. M. Parker, L. A. Burns, R. M. Parrish, A. G. Ryno, and C. D. Sherrill, J. Chem. Phys. 140, 094106 (2014)] and density functional theory [L. A. Burns, Á. Vázquez-Mayagoitia, B. G. Sumpter, and C. D. Sherrill, J. Chem. Phys. 134, 084107 (2011)] methods previously studied for these databases are provided for readers' guidance.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

The impact of the fiducial cosmology assumption on BAO distance scale measurements

ABSTRACT Standard analysis pipelines for measurements of Baryon Acoustic Oscillations (BAO) in galaxy surveys make use of a fiducial cosmological model to guide the data compression required to transform from observed redshifts and angles to the measured angular and radial BAO peak positions. In order to remove any dependence on the fiducial cosmology from the results, all models compared to the data should mimic the compression and its dependence on the fiducial model. In practice, approximations are made when testing models: (1) There is assumed to be no residual dependence on the fiducial cosmology after reconstruction, (2) differences in the distance–redshift relationship are assumed to match a linear scaling, and (3) differences in clustering between true and fiducial models are assumed to be removed by the free parameters used to null the non-BAO signal. We test these approximations using the current standard measurement procedure with a set of halo catalogues from the aemulus suite of N-body simulations, which span a range of wCDM cosmological models. We focus on reconstruction of the primordial BAO and locating the BAO. For the range of wCDM cosmologies covered by the aemulus suite, we find no evidence for systematic errors in the measured BAO shift parameters α∥ and α⊥ to $\lt 0.1\%$. However, the measured errors $\sigma _{\alpha _{\parallel }}$ and $\sigma _{\alpha _{\bot }}$ show a notable absolute increase by up to +0.001 and +0.002, respectively, in the case that the fiducial cosmology does not match the truth. These effects on the inferred BAO scale will be important, given the precision of measurements expected from future surveys including DESI, Euclid, and WFIRST.

79 ASTRONOMY AND ASTROPHYSICS↗

Towards Superior Software Portability with SHAD and HPX C++ Libraries

As hardware architectures and software stacks complexity grows, development productivity, performance and software portability, quickly evolve from desirable features to actual needs. SHAD, the Scalable High-performance Algorithms and Data-structures C++ library is designed to mitigate these issues: it provides general purpose building blocks as well as high-level custom utilities, and offers a shared-memory programming abstraction which facilitates the programming of complex systems, scaling up to High Performance Computing clusters. SHAD’s portability is achieved through an abstract runtime interface, which decouples the upper layers of the library and hides the low level details of the underlying architecture. This layer enables SHAD to interface with different runtime/threading systems, e.g. Intel TBB and Global Memory and Threading (GMT). However, current backends targeting distributed systems, rely on a centralized controller which may possibly limit scalability up to hundreds of nodes and creates a network hot spot due to all to one communication for synchronization, and possibly resulting in degraded performance at high process counts. In this research, we explore HPX, the C++ standard library for parallelism and concurrency, as an additional backend in support of the SHAD library, and present the methodologies in support of local and remote task executions in SHAD with respect to HPX. Finally, we evaluate the proposed system by comparing against existing backends of SHAD and analyzing their performance on C++ Standard Template Library algorithms.

Wu, Nanmiao↗

An Extended Lagrangian Method

A unique formulation of describing fluid motion is presented. The method, referred to as 'extended Lagrangian method,' is interesting from both theoretical and numerical points of view. The formulation offers accuracy in numerical solution by avoiding numerical diffusion resulting from mixing of fluxes in the Eulerian description. The present method and the Arbitrary Lagrangian-Eulerian (ALE) method have a similarity in spirit-eliminating the cross-streamline numerical diffusion. For this purpose, we suggest a simple grid constraint condition and utilize an accurate discretization procedure. This grid constraint is only applied to the transverse cell face parallel to the local stream velocity, and hence our method for the steady state problems naturally reduces to the streamline-curvature method, without explicitly solving the steady stream-coordinate equations formulated a priori. Unlike the Lagrangian method proposed by Loh and Hui which is valid only for steady supersonic flows, the present method is general and capable of treating subsonic flows and supersonic flows as well as unsteady flows, simply by invoking in the same code an appropriate grid constraint suggested in this paper. The approach is found to be robust and stable. It automatically adapts to flow features without resorting to clustering, thereby maintaining rather uniform grid spacing throughout and large time step. Moreover, the method is shown to resolve multi-dimensional discontinuities with a high level of accuracy, similar to that found in one-dimensional problems.

Liou, Meng-Sing↗

New Insight into Short-Wavelength Solar Wind Fluctuations from Vlasov Theory

The nature of solar wind (SW) turbulence below the proton gyroscale is a topic that is being investigated extensively nowadays, both theoretically and observationally. Although recent observations gave evidence of the dominance of kinetic Alfven waves (KAWs) at sub-ion scales with omega < omega(sub ci), other studies suggest that the KAW mode cannot carry the turbulence cascade down to electron scales and that the whistler mode (i.e., omega > omega (sub ci)) is more relevant. Here, we study key properties of the short-wavelength plasma modes under limited, but realistic, SW conditions, Typically Beta(sub i) approx. > Beta (sub e) ∼ 1 and for high oblique angles of propagation 80 deg <= Theta (sub kB) < 90 deg as observed from the Cluster spacecraft data. The linear properties of the plasma modes under these conditions are poorly known, which contrasts with the well-documented cold plasma limit and/or moderate oblique angles of propagation (Theta (sub kB) < 80 deg). Based on linear solutions of the Vlasov kinetic theory, we discuss the relevance of each plasma mode (fast, Bernstein, KAW, whistler) in carrying the energy cascade down to electron scales. We show, in particular, that the shear Alfven mode (known in the magnetohydrodynamic limit) extends at scales kappa rho (sub i) approx. > 1 to frequencies either larger or smaller than omega (sub ci), depending on the anisotropy kappa (parallel )/ kappa(perpendicular). This extension into small scales is more readily called whistler (omega > omega (sub ci)) or KAW (omega < omega (sub ci)) although the mode is essentially the same. This contrasts with the well-accepted idea that the whistler branch always develops as a continuation at high frequencies of the fast magnetosonic mode. We show, furthermore, that the whistler branch is more damped than the KAW one, which makes the latter the more relevant candidate to carry the energy cascade down to electron scales. We discuss how these new findings may facilitate resolution of the controversy concerning the nature of the small-scale turbulence, and we discuss the implications for present and future spacecraft wave measurements in the SW.

waves↗

On the Origin of Reverse Transcriptase-Using CRISPR-Cas Systems and Their Hyperdiverse, Enigmatic Spacer Repertoires

Cas1 integrase is the key enzyme of the clustered regularly interspaced short palindromic repeat (CRISPR)-Cas adaptation module that mediates acquisition of spacers derived from foreign DNA by CRISPR arrays. In diverse bacteria, the cas1 gene is fused (or adjacent) to a gene encoding a reverse transcriptase (RT) related to group II intron RTs. An RT-Cas1 fusion protein has been recently shown to enable acquisition of CRISPR spacers from RNA. Phylogenetic analysis of the CRISPR-associated RTs demonstrates monophyly of the RT-Cas1 fusion, and coevolution of the RT and Cas1 domains. Nearly all such RTs are present within type III CRISPR-Cas loci, but their phylogeny does not parallel the CRISPR-Cas type classification, indicating that RT-Cas1 is an autonomous functional module that is disseminated by horizontal gene transfer and can function with diverse type III systems. To compare the sequence pools sampled by RT-Cas1-associated and RT-lacking CRISPR-Cas systems, we obtained samples of a commercially grown cyanobacterium— Arthrospira platensis . Sequencing of the CRISPR arrays uncovered a highly diverse population of spacers. Spacer diversity was particularly striking for the RT-Cas1-containing type III-B system, where no saturation was evident even with millions of sequences analyzed. In contrast, analysis of the RT-lacking type III-D system yielded a highly diverse pool but reached a point where fewer novel spacers were recovered as sequencing depth was increased. Matches could be identified for a small fraction of the non-RT-Cas1-associated spacers, and for only a single RT-Cas1-associated spacer. Thus, the principal source(s) of the spacers, particularly the hypervariable spacer repertoire of the RT-associated arrays, remains unknown. IMPORTANCE While the majority of CRISPR-Cas immune systems adapt to foreign genetic elements by capturing segments of invasive DNA, some systems carry reverse transcriptases (RTs) that enable adaptation to RNA molecules. From analysis of available bacterial sequence data, we find evidence that RT-based RNA adaptation machinery has been able to join with CRISPR-Cas immune systems in many, diverse bacterial species. To investigate whether the abilities to adapt to DNA and RNA molecules are utilized for defense against distinct classes of invaders in nature, we sequenced CRISPR arrays from samples of commercial-scale open-air cultures of Arthrospira platensis , a cyanobacterium that contains both RT-lacking and RT-containing CRISPR-Cas systems. We uncovered a diverse pool of naturally occurring immune memories, with the RT-lacking locus acquiring a number of segments matching known viral or bacterial genes, while the RT-containing locus has acquired spacers from a distinct sequence pool for which the source remains enigmatic. While the majority of CRISPR-Cas immune systems adapt to foreign genetic elements by capturing segments of invasive DNA, some systems carry reverse transcriptases (RTs) that enable adaptation to RNA molecules. From analysis of available bacterial sequence data, we find evidence that RT-based RNA adaptation machinery has been able to join with CRISPR-Cas immune systems in many, diverse bacterial species. To investigate whether the abilities to adapt to DNA and RNA molecules are utilized for defense against distinct classes of invaders in nature, we sequenced CRISPR arrays from samples of commercial-scale open-air cultures of Arthrospira platensis , a cyanobacterium that contains both RT-lacking and RT-containing CRISPR-Cas systems. We uncovered a diverse pool of naturally occurring immune memories, with the RT-lacking locus acquiring a number of segments matching known viral or bacterial genes, while the RT-containing locus has acquired spacers from a distinct sequence pool for which the source remains enigmatic.

horizontal gene transfer↗