Search NASA⌕ Search

SEARCH · Search NASA

Results for “patterns in complex systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Formation of a constructed microbial community in a nutrient-rich environment indicates bacterial interspecific competition

ABSTRACT Understanding the organizational principles of microbial communities is essential for interpreting ecosystem stability. Previous studies have investigated the formation of bacterial communities under nutrient-poor conditions or obligate relationships to observe cooperative interactions among different species. How microorganisms form stabilized communities in nutrient-rich environments, without obligate metabolic interdependency for growth, is still not fully disclosed. In this study, three bacterial strains isolated from the Populus deltoides rhizosphere were co-cultured in complex medium, and their growth behavior was tracked. These strains co-exist in mixed culture over serial transfer for multiple growth-dilution cycles. Competition is proposed as an emergent interaction relationship among the three bacteria based on their significantly decreased growth levels. The effects of different initial inoculum ratios, up to three orders of magnitude, on community structure were investigated, and the final compositions of the mixed communities with various starting composition indicate that community structure is not dependent on the initial inoculum ratio. Furthermore, the competitive relationships within the community were not altered by different initial inoculum ratios. The community structure was simulated by generalized Lotka-Volterra and dynamic flux balance analysis to provide mechanistic predictions into emergence of community structure under a nutrient-rich environment. Metaproteomic analyses provide support for the metabolite exchanges predicted by computational modeling and for highly altered physiologies when microbes are grown in co-culture. These findings broaden our understanding of bacterial community dynamics and metabolic diversity in higher-order interactions and could be significant in the management of rhizospheric bacterial communities. IMPORTANCE Bacteria naturally co-exist in multispecies consortia, and the ability to engineer such systems can be useful in biotechnology. Despite this, few studies have been performed to understand how bacteria form a stable community and interact with each other under nutrient-rich conditions. In this study, we investigated the effects of initial inoculum ratios on bacterial community structure using a complex medium and found that the initial inoculum ratio has no significant impact on resultant community structure or on interaction patterns between community members. The microbial population profiles were simulated using computational tools in order to understand intermicrobial relationships and to identify potential metabolic exchanges that occur during stabilization of the bacterial community. Studying microbial community assembly processes is essential for understanding fundamental ecological principles in microbial ecosystems and can be critical in predicting microbial community structure and function.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Visualization for Insight and Data Analysis in Energy Research

This talk explores how advanced visualization technologies are transforming analytical reasoning and knowledge discovery in energy research, drawing on recent work at the National Laboratory of the Rockies' Computational Science Center. Through a series of scientific case studies, we demonstrate how immersive and high-resolution visualization environments enable scientists and engineers to identify previously unseen patterns and features - insights that often remain hidden in traditional desktop-based analysis. By embedding richer information into interactive analytics tools, these approaches support the exploration of complex, multivariate parameter spaces, where interaction itself catalyzes understanding. Beyond capability, we emphasize the critical role of visualization design grounded in perception and cognition, showing how visual encodings directly influence analytical outcomes. Spanning applications from materials science to integrated energy systems, these visualization approaches accelerate innovation and improve decision-making by enabling deeper, more reliable insight into increasingly complex energy data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1979

Combined pressure and temperature distortion effects on internal flow of a turbofan engine

An additional data base for improving and verifying a computer simulation developed by an engine manufacturer was obtained. The multisegment parallel compressor simulation was designed to predict the effects of steady-state circumferential inlet total-pressure and total-temperature distortions on the flows into and through a turbofan compression system. It also predicts the degree of distortion that will result in surge of the compressor. The effect of combined 180 deg square-wave distortion patterns of total pressure and total temperature in various relative positions is reported. The observed effects of the combined distortion on a unitary bypass ratio turbofan engine are presented in terms of total and static pressure profiles and total temperature profiles at stations ahead of the inlet guide vanes as well as through the fan-compressor system. These observed profiles are compared with those predicted by the complex multisegment model. The effects of relative position of the two components comprising the combined distortion on the degree resulting in surge are discussed. Certain relative positions required less combined distortion than either a temperature or pressure distortion by itself.

Braithwaite, W. M.↗
DOE OSTI2025

BlueCRAB Domain Overlapping Coupling: Theory & Verification

Coupling low- and high-fidelity codes is a useful way to model complex engineering systems. Thanks to the high-fidelity code, complex phenomena can be resolved in areas of the system where this is required, yet the efficiency of the lower-fidelity code is still retained in modeling the rest of the system. This document details the theory and implementation of coupling two different thermal-hydraulic codes: the system thermal-hydraulics (STH) code System Analysis Module (SAM) and the coarse-mesh computational fluid dynamics (CFD) code Pronghorn. Both applications are included in the Comprehensive Reactor Analysis Bundle (BlueCRAB) code suite and are based on the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework. The domain overlapping (DO) coupling approach was adopted, as it offers proven advantages over more conventional domain decomposition methods. In the DO coupling, SAM provides Pronghorn with boundary conditions that depend on the system-level simulation of the entire plant. In return, the overlapping coupled SAM components–termed “surrogate components”–receive friction factors and source terms computed online based on the Pronghorn simulation. The framework is designed to be generic and enable coupling regardless of geometry and the number of inlet/outlet boundaries in the DO coupled domain. The developed method leads to consistent pressure drops, enthalpies, and scalar concentrations when comparing the coupled SAM and Pronghorn simulations. This document presents the DO coupling approach, along with two verification and two demonstration cases. The proposed problems explore different physical aspects relevant to nuclear reactor analysis, including buoyancy-driven flows, complex flow patterns, and multiple inlets and outlets. Periodically, new versions of this “BlueCRAB Domain Overlapping Coupling: Theory & Verification” report will be issued to reflect future developments and verification tests.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2002

Purification of the small mechanosensitive channel of Escherichia coli (MscS): the subunit structure, conduction, and gating characteristics in liposomes

The small mechanosensitive channel, MscS, is a part of the turgor-driven solute efflux system that protects bacteria from lysis in the event of osmotic downshift. It has been identified in Escherichia coli as a product of the orphan yggB gene, now called mscS (Levina et al., 1999, EMBO J. 18:1730). Here I show that that the isolated 31-kDa MscS protein is sufficient to form a functional mechanosensitive channel gated directly by tension in the lipid bilayer. MscS-6His complexes purified in the presence of octylglucoside and lipids migrate in a high-resolution gel-filtration column as particles of approximately 200 kDa. Consistent with that, the protein cross-linking patterns predict a hexamer. The channel reconstituted in soybean asolectin liposomes was activated by pressures of 20-60 mm Hg and displayed the same asymmetric I-V curve and slight anionic preference as in situ. At the same time, the single-channel conductance is proportional to the buffer conductivity in a wide range of salt concentrations. The rate of channel activation in response to increasing pressure gradient across the patch was slower than the rate of closure in response to decreasing steps of pressure gradient. Therefore, the open probability curves were recorded with descending series of pressures. Determination of the curvature of patches by video imaging permitted measurements of the channel activity as a function of membrane tension (gamma). Po(gamma) curves had the midpoint at 5.5 +/- 0.1 dyne/cm and gave estimates for the energy of opening DeltaG = 11.4 +/- 0.5 kT, and the transition-related area change DeltaA = 8.4 +/- 0.4 nm(2) when fitted with a two-state Boltzmann model. The correspondence between channel properties in the native and reconstituted systems is discussed.

NASA Discipline Cell Biology↗
DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

42 ENGINEERING↗
DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

actuated signal control↗
DOE OSTI2024

NeuroSEM: A hybrid framework for simulating multiphysics problems by coupling PINNs and spectral elements

Multiphysics problems that are characterized by complex interactions among fluid dynamics, heat transfer, structural mechanics, and electromagnetics, are inherently challenging due to their coupled nature. While experimental data on certain state variables may be available, integrating these data with numerical solvers remains a significant challenge. Physics-informed neural networks (PINNs) have shown promising results in various engineering disciplines, particularly in handling noisy data and solving inverse problems in partial differential equations (PDEs). However, their effectiveness in forecasting nonlinear phenomena in multiphysics regimes, particularly involving turbulence, is yet to be fully established. Here, this study introduces NeuroSEM, a hybrid framework integrating PINNs with the highfidelity Spectral Element Method (SEM) solver, Nektar++. NeuroSEM leverages the strengths of both PINNs and SEM, providing robust solutions for multiphysics problems. PINNs are trained to assimilate data and model physical phenomena in specific subdomains, which are then integrated into the Nektar++ solver. We demonstrate the efficiency and accuracy of NeuroSEM for thermal convection in cavity flow and flow past a cylinder. The framework effectively handles data assimilation by addressing those subdomains and state variables where the data is available. We applied NeuroSEM to the Rayleigh-B´enard convection system, including cases with missing thermal boundary conditions and noisy datasets. Finally, we applied the proposed NeuroSEM framework to real particle image velocimetry (PIV) data to capture flow patterns characterized by horseshoe vortical structures. Our results indicate that NeuroSEM accurately models the physical phenomena and assimilates the data within the specified subdomains. The framework’s plug-and-play nature facilitates its extension to other multiphysics or multiscale problems. Furthermore, NeuroSEM is optimized for efficient execution on emerging integrated GPU-CPU architectures. This hybrid approach enhances the accuracy and efficiency of simulations, making it a powerful tool for tackling complex engineering challenges in various scientific domains.

42 ENGINEERING↗
DOE OSTI2025

Invariant discovery of features across multiple length scales: Applications in microscopy and autonomous materials characterization

Physical imaging is a foundational characterization method in areas from condensed matter physics and chemistry to astronomy and spans length scales from atomic to universe. Images encapsulate crucial data regarding atomic bonding, materials microstructures, and dynamic phenomena such as microstructural evolution and turbulence, among other phenomena. The challenge lies in effectively extracting and interpreting this information. Variational Autoencoders (VAEs) have emerged as powerful tools for identifying the underlying factors of variation in image data, providing a systematic approach to distilling meaningful patterns from complex data sets. However, a significant hurdle in their application is the definition and selection of appropriate descriptors reflecting local structures. Here, we introduce the scale-invariant VAE approach (SI-VAE) based on the progressive training of the VAE with the descriptors sampled at different length scales. The SI-VAE allows the discovery of the length scale-dependent factors of variation in the system. Here, we illustrate this approach using the ferroelectric domain images and generalize it to the movies of the electron-beam induced phenomena in graphene and topography evolution across combinatorial libraries. This approach can further be used to initialize the decision making in automated experiments including structure–property discovery and can be applied across a broad range of imaging methods. This approach is universal and can be applied to any spatially resolved data including both experimental imaging studies and simulations, and can be particularly useful for exploration of phenomena such as turbulence and scale-invariant transformation fronts.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Geometric frustration in the myosin superlattice of vertebrate muscle

Geometric frustration results from an incompatibility between minimum energy arrangements and the geometry of a system, and gives rise to interesting and novel phenomena. Here, we report geometric frustration in a native biological macromolecular system---vertebrate muscle. We analyse the disorder in the myosin filament rotations in the myofibrils of vertebrate striated (skeletal and cardiac) muscle, as seen in thin-section electron micrographs, and show that the distribution of rotations corresponds to an archetypical geometrically frustrated system---the triangular Ising antiferromagnet. Spatial correlations are evident out to at least six lattice spacings. The results demonstrate that geometric frustration can drive the development of structure in complex biological systems, and may have implications for the nature of the actin--myosin interactions involved in muscle contraction. Identification of the distribution of myosin filament rotations with an Ising model allows the extensive results on the latter to be applied to this system. It shows how local interactions (between adjacent myosin filaments) can determine long-range order and, conversely, how observations of long-range order (such as patterns seen in electron micrographs) can be used to estimate the energetics of these local interactions. Furthermore, since diffraction by a disordered system is a function of the second-order statistics, the derived correlations allow more accurate diffraction calculations, which can aid in interpretation of X-ray diffraction data from muscle specimens for structural analysis.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1990

Two-dimensional simulation of a two-phase, regenerative pumped radiator loop utilizing direct contact heat transfer with phase change

An innovative pumped loop concept for 600 K space power system radiators utilizing direct contact heat transfer, which facilitates repeated startup/shutdown of the power system without complex and time-consuming coolant thawing during power startup, is under development. The heat transfer process with melting/freezing of Li in an NaK flow was studied through two-dimensional time-dependent numerical simulations to characterize and predict the Li/NaK radiator performance during startup (thawing) and shutdown (cold-trapping). Effects of system parameters and the criteria for the plugging domain are presented together with temperature distribution patterns in solid Li and subsequent melting surface profile variations in time.

Rhee, Hyop S.↗
NASA NTRS1985

Subtle alternating electrocardiographic morphology as an indicator of decreased cardiac electrical stability

Observations from finite-element computer models, together with analytic developments based on percolation theory have suggested that subtle fluctuations of ECG morphology might serve as an indicator diminished cardiac electrical stability. With fixed-rate atrial pacing in canines, we have previously observed a pattern of alternation in T wave energy which correlated with cardiac electrical stability. We report here on a series of 20 canine experiments in which cardiac electrical stability (measured via Ventricular Fibrillation Threshold determination) was compared to a non-degenerate, multidimensional measurement of the degree of alternating activity present in the ECG complex morphology. The decrease in cardiac electrical stability brought on by both coronary artery occlusion and systemic hypothermia was consistently accompanied by subtle alternation in ECG morphology, with the absolute degree of alternating activity being significantly (negatively) correlated with cardiac electrical stability.

Ventricular Fibrillation↗
DOE OSTI2024

Custom Accessors: Enabling Scalable Data Ingestion, (Re-)Organization, and Analysis on Distributed Systems

The emerging class of high velocity and high volume data analytic workflows comprise interwoven data ingestion, organization, and processing stages, with ingestion and organization steps often contributing comparable or even higher computational costs than actual processing steps. Since complex workflows consist of a variety of phases that view and use data differently, being able to construct efficient, scalable, distributed data structures (arrays, vectors, sets, maps, and multi-maps) is essential and requires custom methods to extend and shrink containers, analyze and position data, and, maintain globallyconsistent meta-data. In this paper, we propose a novel datastructure access paradigm based on the concept of Accessors. At a high level, accessors are customizable callable objects that can modify the behavior of insert, read, update, and delete operations for distributed containers while preserving atomicity guarantees. Accessors provide a very clean and natural way to implement a variety of programming patterns, e.g., conditional insertion/deletion and cascading computations, which would be otherwise hard (or even impossible) to express in parallel and distributed settings without using locks. We demonstrate the practicality and usefulness of our approach with two representative use cases and study the performance of these applications on a distributed High-Performance Computing system. Our analysis highlights that our proposed abstraction allows for an effective overlapping and concurrent execution of different workflow steps (e.g., data ingestion and analysis), which in a conventional analytics pipeline would execute sequentially, contributing cumulatively to the overall latency.

Castellana, Vito G. [BATTELLE (PACIFIC NW LAB)] (O↗
NASA NTRS2000

Vestibular stimulation leads to distinct hemodynamic patterning

Previous studies demonstrated that responses of a particular sympathetic nerve to vestibular stimulation depend on the type of tissue the nerve innervates as well as its anatomic location. In the present study, we sought to determine whether such precise patterning of vestibulosympathetic reflexes could lead to specific hemodynamic alterations in response to vestibular afferent activation. We simultaneously measured changes in systemic blood pressure and blood flow (with the use of Doppler flowmetry) to the hindlimb (femoral artery), forelimb (brachial artery), and kidney (renal artery) in chloralose-urethane-anesthetized, baroreceptor-denervated cats. Electrical vestibular stimulation led to depressor responses, 8 +/- 2 mmHg (mean +/- SE) in magnitude, that were accompanied by decreases in femoral vasoconstriction (23 +/- 4% decrease in vascular resistance or 36 +/- 7% increase in vascular conductance) and increases in brachial vascular tone (resistance increase of 10 +/- 6% and conductance decrease of 11 +/- 4%). Relatively small changes (<5%) in renal vascular tone were observed. In contrast, electrical stimulation of muscle and cutaneous afferents produced pressor responses (20 +/- 6 mmHg) that were accompanied by vasoconstriction in all three beds. These data suggest that vestibular inputs lead to a complex pattern of cardiovascular changes that is distinct from that which occurs in response to activation of other types of somatic afferents.

NASA Discipline Neuroscience↗
DOE OSTI2024

Soft interface instability and gas flow channeling in low-permeability deformable media

Understanding gas percolation through a clay layer or a shale formation is of great importance for the development of a geologic repository for nuclear waste disposal, a subsurface system for gas storage, and an engineering approach for hydrocarbon extraction from unconventional reservoirs. Gas injection experiments have revealed complex dynamic behaviours of gas percolation through water saturated compacted bentonite, characterized by a high breakthrough pressure, rapid breakthrough, a pressure/stress decay after the breakthrough, a relatively high migration rate, high-frequency periodic/nonperiodic variations in flow rate, stepwise rate reductions during relaxation, and low gas saturation over the whole process, all indicating channelling nature of the processes. Using linear stability analyses, we show that this channelling can autonomously emerge from the instability of the deformable interface between the injected gas and the compacted bentonite matrix driven by local stress concentration, pore dilation, and hydrologic gradient. Channel patterns formed would possess a fractal geometry. We further show that, once a percolating channel is established, the gas injected would percolate through the channel in a chain of gas bubbles, also due to the interface instability, resulting in periodic/chaotic variations in gas flow rate. Our work provides a unified explanation for key features observed for gas percolation in low-permeability deformable media. The work also suggests a possibility of designing an engineered barrier system for a nuclear waste repository that can have controllable gas release while limit water transport.

Bentonite↗
DOE OSTI2024

The GFDL Variable-Resolution Global Chemistry-Climate Model for Research at the Nexus of US Climate and Air Quality Extremes

We present a variable-resolution global chemistry-climate model (AM4VR) developed at NOAA's Geophysical Fluid Dynamics Laboratory (GFDL) for research at the nexus of US climate and air quality extremes. AM4VR has a horizontal resolution of 13 km over the US, allowing it to resolve urban-to-rural chemical regimes, mesoscale convective systems, and land-surface heterogeneity. With the resolution gradually reducing to 100 km over the Indian Ocean, we achieve multi-decadal simulations driven by observed sea surface temperatures at 50% of the computational cost for a 25-km uniform-resolution grid. In contrast with GFDL's AM4.1 contributing to the sixth Coupled Model Intercomparison Project at 100 km resolution, AM4VR features much improved US climate mean patterns and variability. In particular, AM4VR shows improved representation of: precipitation seasonal-to-diurnal cycles and extremes, notably reducing the central US dry-and-warm bias; western US snowpack and summer drought, with implications for wildfires; and the North American monsoon, affecting dust storms. AM4VR exhibits excellent representation of winter precipitation, summer drought, and air pollution meteorology in California with complex terrain, enabling skillful prediction of both extreme summer ozone pollution and winter haze events in the Central Valley. AM4VR also provides vast improvements in the process-level representations of biogenic volatile organic compound emissions, interactive dust emissions from land, and removal of air pollutants by terrestrial ecosystems. We highlight the value of increased model resolution in representing climate–air quality interactions through land-biosphere feedbacks. AM4VR offers a novel opportunity to study global dimensions to US air quality, especially the role of Earth system feedbacks in a changing climate.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

RISE: Reducing I/O Contention in Staging-based Extreme-Scale In-situ Workflows

While in-situ workflow formulations have addressed some of the data-related challenges associated with extreme-scale scientific workflows, these workflows involve complex interactions and different modes of data exchange. In the context of increasing system complexity, such workflows present significant resource management challenges, requiring complex cost-performance tradeoffs. This paper presents RISE, an intelligent staging-based data management middleware, which builds on the DataSpaces framework and performs intelligent scheduling of data management operations to reduce I/O contention. In RISE, data are always written immediately to local buffers to reduce the effect of the transfer impact upon application performance. RISE identifies applications’ data access patterns and moves data towards data consumers only when the network is expected to be idle, reducing the impact of asynchronous background data movement upon critical data read/write requests. Here, we experimentally demonstrate that RISE can take advantage of staging nodes to offload data during writes without degrading application data movement performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

RingX: Scalable Parallel Attention for Long-Context Learning on HPC

The attention mechanism has become foundational for remarkable AI breakthroughs since the introduction of the Transformer, driving the demand for increasingly longer context to power frontier models such as large-scale reasoning language models and high-resolution image/video generators. However, its quadratic computational and memory complexities present substantial challenges. Current state-of-the-art parallel attention methods, such as ring attention, are widely adopted for long-context training but utilize a point-to-point communication strategy that fails to fully exploit the capabilities of modern HPC network architectures. In this work, we propose ringX, a scalable family of parallel attention methods optimized explicitly for HPC systems. By enhancing workload partitioning, refining communication patterns, and improving load balancing, ringX achieves up to 3.4 × speedup compared to conventional ring attention on the Frontier supercomputer. Optimized for both bi-directional and causal attention mechanisms, ringX demonstrates its effectiveness through training benchmarks of a Vision Transformer (ViT) on a climate dataset and a Generative Pre-Trained Transformer (GPT) model, Llama3 8B. Our method attains an end-to-end training speedup of approximately 1.5 × in both scenarios. To our knowledge, the achieved 38% model FLOPs utilization (MFU) for training Llama3 8B with a 1M-token sequence length on 4,096 GPUs represents one of the highest training efficiencies reported for long-context learning on HPC systems. Our code implementation is available at https://github.com/jqyin/ringX-attention.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗