Search NASA⌕ Search

SEARCH · Search NASA

Results for “Parallelization efficiency”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,657 records · Page 92

University of Lodz an electron spectrometer—A new conversion-electron spectrometer for “in-beam” measurements

The designed and constructed at the University of Lodz an electron spectrometer is devoted to “in-beam” measurements. The apparatus is characterized by high efficiency up to 9%, good energy resolution (FWHM = 5 keV at 482 keV) and, what is very important good suppression of delta electrons, positrons, and photons emitted by the targets. This achievement was obtained using a combination of magnetic field in two different layouts: perpendicular and parallel to the axis of the spectrometer being orthogonal to the beamline. The conversion-electron spectrometer coupled to the EAGLE array was successfully tested in an “in-beam” measurement.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Algorithms for Efficient Reproducible Floating Point Summation

We define “reproducibility” as getting bitwise identical results from multiple runs of the same program, perhaps with different hardware resources or other changes that should not affect the answer. Many users depend on reproducibility for debugging or correctness. However, dynamic scheduling of parallel computing resources, combined with nonassociative floating point addition, makes reproducibility challenging even for summation, or operations like the BLAS. We describe a “reproducible accumulator” data structure (the “binned number”) and associated algorithms to reproducibly sum binary floating point numbers, independent of summation order. We use a subset of the IEEE Floating Point Standard 754-2008 and bitwise operations on the standard representations in memory. Our approach requires only one read-only pass over the data, and one reduction in parallel, using a 6-word reproducible accumulator (more words can be used for higher accuracy), enabling standard tiling optimization techniques. Summing n words with a 6-word reproducible accumulator requires approximately 9 n floating point operations (arithmetic, comparison, and absolute value) and approximately 3 n bitwise operations. The final error bound with a 6-word reproducible accumulator and our default settings can be up to 2 29 times smaller than the error bound for conventional (recursive) summation on ill-conditioned double-precision inputs.

Computer Science↗

Microchannel-based Membrane-less Extraction of Li from Unconventional Lithium Sources & the Separation of REE

This final report provides an overview of the Project's entire duration, covering July 1, 2021 to December 31, 2023. It primarily focuses on the achievements, technological developments, and unique challenges the team faced while working on separating and extracting Lithium from produced waters. The project's primary aim was to create an integrated, high-throughput, membrane-less, and modular microfluidic platform that could extract Lithium from unconventional sources. We have successfully met all goals and milestones envisioned in the SOPO document. The most critical primary milestones, including the Go-No-Go milestone (refer to the Gantt chart in the Appendices), were successfully accomplished. We demonstrated phase separation (>90%) and extraction (>85%) performance in the MPSE using synthetic, and representative produced water composition feed at 50 ml/min total flow through MPSE 36. We have also performed a parametric study of the MPSE operations, beyond the scope of SOPO, exploring operating conditions of current and broader interest. The extended investigation of operational parameters is concurrent with our efforts to seek further development of the MPSE technology beyond the scope of the Project. Along these lines of development, we have made efforts to be responsive to DOE calls for technological developments of other types of resources (beyond PW) for the recovery of Critical Materials and higher TRL development (beyond TRL 4). During the work on this Project, we developed and implemented three innovative technical approaches that emerged from our efforts to successfully meet the Project milestones. The innovative & original technical approaches developed and implemented in this Project are now the contributions to process engineering that could be clearly credited to the Project. First, Convergent Design Approach is a comprehensive feedforward & feedback loop of four design phases: i) design for functionality, ii) design for manufacturing, iii) design for sustainability, and iv) design for market. Next was Process Intensification. A major aim of this Project was to create an innovative phase separation & extraction microscale-based technology for Li separation – thus the words microchannel-based in the Project title. A microscale-based technology is intrinsically in the center of the Process Intensification domain as defined by its unique principles. Therefore, Process Intensification was implicitly envisioned in the Project’s SOPO. Lastly, Time Scale Analysis is a novel tool for discovering the needs and directions of Process Intensification implementations in any process technology. This Project is fully credited for developing and implementing the three novel technical approaches mentioned above. These are general contributions to process engineering that emerged from this Project. Beyond the original SOPO scope, the OSU-U.Pitt research group utilized a Convergent Design methodology, integrating first-principles mathematical modeling with experimental validation on the Minimum Development Vehicle. By creating these Digital Twins, the team rapidly assessed manufacturing iterations to support TEA analysis. This framework further enabled the development of advanced Surface Modification Techniques, where hydrophobic and oleophobic coating strategies were optimized via Digital Twin tools and validated through rigorous 100-hour longevity testing. TEA Analysis: The closing efforts of this Project were focused on the TEA analysis. TEA analysis had two primary functions: i) enabling critical assessments of design variations withing 10 the Concurrent Design Approach, thus enabling evolution of the MPSE design to reach faster- better-cheaper alternatives; and ii) to create a bridge between the accomplishments of this Project and future projects of higher TRL, beyond TRL 6 level. It is important to note that the TEA model created in the Project stirred the technological solutions for the recovery of critical materials toward a vision of a very profitable modular plant that has unique zero-waste water discharge signature. More importantly, thanks to our experimental performance data and conservative assumptions, the TEA model predicts minimal technological and investment risks. Low cost of a modular unit of a nominal capacity of [1000 tons of Li 2 CO 3 /year] positions the MPSE based technology within the reach of community investors, thus offering a paradigm shift in the development of critical technologies. The project successfully navigated two primary challenges: solvent selection and manufacturing adaptation. Restricted by the SOPO to existing literature for lithium recovery, the team identified a critical need for a "material excellence program" to develop next-generation solvents, eventually concluding with a preliminary investigation into promising Ionic Liquids (ILs). Simultaneously, COVID-19 supply chain disruptions forced a pivot from traditional manufacturing to advanced additive methods at ATAMI-OSU. By transitioning from stainless steel to 3D-printed polymer substrates, the team achieved a transformative three-order-of- magnitude reduction in manufacturing costs and compressed prototyping timelines from several months to just two days. The MPSE technology offers significant energy, environmental, and economic advantages by overcoming the traditional bottlenecks of phase-separation hardware and contactor size. Unlike conventional mixer-settlers or membrane-based systems, MPSE operates without moving parts or fouling-prone membranes, achieving robust performance even with challenging, viscous, or particulate-heavy feeds. Key performance metrics include an energy intensity reduction of 5–50x (3–40 kJ/m 3 ) compared to incumbent technologies and a dramatic reduction of processing time to under 60 seconds, which drastically reduces the physical plant footprint. These technical efficiencies translate into superior economic outcomes; for a 100 t/year Li 2 CO 3 facility, implementing MPSE is projected to nearly halve contactor CAPEX (from $\$$6.08M to $\$$3.01M) and significantly increase the project's Net Present Value (NPV), derisking new investment and enabling distributed critical-mineral processing configurations. The commercialization of MPSE technology is being spearheaded by Vigsur Dynamics Inc., which has adopted a structured, parallel approach to technical and business development since its formation in January 2026. Following extensive customer discovery and engagement with the Oregon State University accelerator, Vigsur Dynamics is working to establish a business model that transitions from pilot demonstrations to modular hardware sales, ultimately aiming for a "build-own-operate" service strategy. Current technical milestones—including 100 hours of continuous operation, superior energy efficiency, and successful 6-unit modular scale-up— provide a foundation for this transition. Backed by ongoing IP licensing and a growing network of industrial and venture advisors, the company is actively de-risking the platform to replace conventional mixer-settler systems in the critical minerals market.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Developing an ELM Ecosystem Dynamics Model on GPU with OpenACC

Porting a complex scientific code, such as the E3SM land model (ELM), onto a new computing architecture is challenging. The paper presents design strategies and technical approaches to develop an ELM ecosystem dynamics model with compiler directives (OpenACC) on NVIDIA GPUs. The code has been refactored with advanced OpenACC features (such as deepcopy and routine directives) to reduce memory consumption and to increase the levels of parallelism through parallel loop reconstruction and new data structures. As a result, the optimized parallel implementation achieved more than a 140-time speedup (50 ms vs 7600 ms), compared to a naive implementation that uses OpenACC routine directive and parallelizes the code across existing loops on a single NVIDIA V100. On a fully loaded computing node with 44 CPUs and 6 GPUs, the code achieved over a 3.0-times speedup, compared to the original code on the CPU. Furthermore, the memory footprint of the optimized parallel implementation is 300 MB, which is around 15% of the 2.15 GB of memory consumed by a naive implementation. This study is the first effort to develop the ELM component on GPUs efficiently to support ultra-high-resolution land simulations at continental scales.

Schwartz, Peter↗

A high accuracy/resolution spectral element/Fourier–Galerkin method for the simulation of shoaling non-linear internal waves and turbulence in long domains with variable bathymetry

A high-order hybrid continuous-Galerkin numerical method, designed for the simulation of non-linear, non -hydrostatic internal waves and turbulence in long computational domains with complex bathymetry, is presented. The spatial discretization in the non-periodic wave-propagating directions, utilizes the nodal spectral element method. Such a high-order element-based discretization allows the highly accurate representation of complex domain geometry along with the flexibility of concentrating resolution in areas of interest. Under the assumption of the normal-to-isobath propagation of non-linear internal waves, a third periodic direction is incorporated via a Fourier-Galerkin discretization. The distinct non-hydrostatic nature of non-linear internal waves and, any instabilities and turbulence therein, necessitates the numerically challenging solution of the pressure Poisson problem. A defining feature of this work is the application of a domain decomposition approach, combined with block-Jacobi/deflation-based preconditioning to the pressure Poisson problem. Such a combined approach is particularly suitable for the long high aspect-ratio complex domains of interest and enables the efficient high-accuracy reproduction of the non-hydrostatic dynamics of non-linear internal waves. Implementation details are also described in the context of the stability of the solver and its parallelization strategy. A series of benchmarks of increasing complexity demonstrate the robustness of the flow solver. The benchmarks culminate with the three-dimensional simulation of a convectively breaking mode-one non-linear internal wave over a realistic South-China-Sea bathymetric transect and background current/stratification profiles.

Deflation↗

Assembly and Analysis of Cell-Scale Membrane Envelopes

The march toward exascale computing will enable routine molecular simulation of larger and more complex systems, for example, simulation of entire viral particles, on the scale of approximately billions of atoms-a simulation size commensurate with a small bacterial cell. Anticipating the future hardware capabilities that will enable this type of research and paralleling advances in experimental structural biology, efforts are currently underway to develop software tools, procedures, and workflows for constructing cell-scale structures. Herein, we describe our efforts in developing and implementing an efficient and robust workflow for construction of cell-scale membrane envelopes and embedding membrane proteins into them. A new approach for construction of massive membrane structures that are stable during the simulations is built on implementing a subtractive assembly technique coupled with the development of a structure concatenation tool (fastmerge), which eliminates overlapping elements based on volumetric criteria rather than adding successive molecules to the simulation system. Using this approach, we have constructed two "protocells" consisting of MARTINI coarse-grained beads to represent cellular membranes, one the size of a cellular organelle and another the size of a small bacterial cell. The membrane envelopes constructed here remain whole during the molecular dynamics simulations performed and exhibit water flux only through specific proteins, demonstrating the success of our methodology in creating tight cell-like membrane compartments. Extended simulations of these cell-scale structures highlight the propensity for nonspecific interactions between adjacent membrane proteins leading to the formation of protein microclusters on the cell surface, an insight uniquely enabled by the scale of the simulations. We anticipate that the experiences and best practices presented here will form the basis for the next generation of cell-scale models, which will begin to address the addition of soluble proteins, nucleic acids, and small molecules essential to the function of a cell.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Resonator-quantum well infrared photodetectors

We applied a recent electromagnetic model to design the resonator-quantum well infrared photodetector (R-QWIP). In this design, we used an array of rings as diffractive elements to diffract normal incident light into parallel propagation and used the pixel volume as a resonator to intensify the diffracted light. With a proper pixel size, the detector resonates at certain optical wavelengths and thus yields a high quantum efficiency (QE). To test this detector concept, we fabricated a number of R-QWIPs with different quantum well materials and detector geometries. The experimental result agrees satisfactorily with the prediction, and the highest QE achieved is 71%.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Broadband turbulent spectra in gamma-ray burst light curves

Broadband power density spectra offer a window to understanding turbulent behavior in the emission mechanism and, at the highest frequencies, in the putative inner engines powering long gamma-ray bursts (GRBs). We describe a chirp search method alongside Fourier analysis for signal detection in the Poisson noise-dominated, 2 kHz sampled, BeppoSAX light curves. An efficient numerical implementation is described in O(Nnlog n) operations, where N is the number of chirp templates and n is the length of the light-curve time series, suited for embarrassingly parallel processing. For the detection of individual chirps over a 1 s duration, the method is one order of magnitude more sensitive in signal-to-noise ratio than Fourier analysis. The Fourier-chirp spectra of GRB 010408 and GRB 970816 show a continuation of the spectral slope with up to 1 kHz of turbulence identified in low-frequency Fourier analysis. The same continuation is observed in an average spectrum of 42 bright, long GRBs. An outlook on a similar analysis of upcoming gravitational wave data is included.

79 ASTRONOMY AND ASTROPHYSICS↗

SHAPING THE DUST MASS-STAR-FORMATION RATE RELATION

There is a remarkably tight relation between the observationally inferred dust masses and star-formation rates (SFRs) of Sloan Digital Sky Survey galaxies, M {sub dust} ∝ SFR{sup 1.11}. Here we extend the M {sub dust}-SFR relation to the high end and show that it bends over at very large SFRs (i.e., dust masses are lower than predicted for a given SFR). We identify several distinct evolutionary processes in the diagram: (1) a star-bursting phase in which dust builds up rapidly at early times. The maximum attainable dust mass in this process is the cause of the bend-over of the relation. A high dust-formation efficiency, a bottom-light initial mass function, and negligible supernova shock dust destruction are required to produce sufficiently high dust masses. (2) A quiescent star-forming phase in which the subsequent parallel decline in dust mass and SFR gives rise to the M {sub dust}-SFR relation, through astration and dust destruction. The dust-to-gas ratio is approximately constant along the relation. We show that the power-law slope of the M {sub dust}-SFR relation is inversely proportional to the global Schmidt-Kennicutt law exponent (i.e., ∼0.9) in simple chemical evolution models. (3) A quenching phase which causes star formation to drop while the dust mass stays roughly constant or drops proportionally. Combined with merging, these processes, as well as the range in total baryonic mass, give rise to a complex population of the diagram which adds significant scatter to the original M {sub dust}-SFR relation. (4) At very high redshifts, a population of galaxies located significantly below the local relation is predicted.

79 ASTRONOMY AND ASTROPHYSICS↗

Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections

File transfers over dedicated connections, supported by large parallel file systems, have become increasingly important in high-performance computing and big data workflows. It remains a challenge to achieve peak rates for such transfers due to the complexities of file I/O, host, and network transport subsystems, and equally importantly, their interactions. We present extensive measurements of disk-to-disk file transfers using Lustre and XFS file systems mounted on multi-core servers over a suite of 10 Gbps emulated connections with 0-366 ms round trip times. Our results indicate that large buffer sizes and many parallel flows do not always guarantee high transfer rates. Furthermore, large variations in the measured rates necessitate repeated measurements to ensure confidence in inferences based on them. We propose a new method to efficiently identify the optimal joint file I/O and network transport parameters using a small number of measurements. We show that for XFS and Lustre with direct I/O, this method identifies configurations achieving 97% of the peak transfer rate while probing only 12% of the parameter space.

Rao, Nageswara S.↗

Distributing Power Grid State Estimation on HPC Clusters A System Architecture Prototype

The future power grid is expected to further expand with highly distributed energy sources and smart loads. The increased size and complexity lead to increased burden on existing computational resources in energy control centers. Thus the need to perform real-time assessment on such systems entails efficient means to distribute centralized functions such as state estimation in the power system. In this paper, we present our early prototype of a system architecture that connects distributed state estimators individually running parallel programs to solve non-linear estimation procedure. The prototype consists of a middleware and data processing toolkits that allows data exchange in the distributed state estimation. We build a test case based on the IEEE 118 bus system and partition the state estimation of the whole system model to available HPC clusters. The measurement from the testbed demonstrates the low overhead of our solution.

Distributed architecture↗

A Multiobjective Optimization Framework for Online Stochastic Optimal Control in Hybrid Electric Vehicles

The increasing urgency to extract additional efficiency from hybrid propulsion systems has led to the development of advanced power management control algorithms. In this paper we address the problem of online optimization of the supervisory power management control in parallel hybrid electric vehicles (HEVs). We model HEV operation as a controlled Markov chain and we show that the control policy yielding the Pareto optimal solution minimizes online the long-run expected average cost per unit time criterion. The effectiveness of the proposed solution is validated through simulation and compared to the solution derived with dynamic programming using the average cost criterion. Both solutions achieved the same cumulative fuel consumption demonstrating that the online Pareto control policy is an optimal control policy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A New Class of AMG Interpolation Methods Based on Matrix-Matrix Multiplications

A new class of distance-two interpolation methods for algebraic multigrid (AMG) that can be formulated in terms of sparse matrix-matrix multiplications is presented and analyzed. Compared with similar distance-two prolongation operators, the proposed algorithms exhibit improved efficiency and portability to various computing platforms, since they allow one to easily exploit existing high-performance sparse matrix kernels. The new interpolation methods have been implemented in hypre, a widely used parallel multigrid solver library. With the proposed interpolations, the overall time of hypre's BoomerAMG setup can be considerably reduced, while sustaining equivalent, sometimes improved, convergence rates. Numerical results for a variety of test problems on parallel machines are presented that support the superiority of the proposed interpolation operators over the existing ones in hypre.

97 MATHEMATICS AND COMPUTING↗

Additive Polynomial Time Integrators, Part I: Framework and Fully Implicit-Explicit Collocation Methods

In this paper we generalize the polynomial time integration framework to additively partitioned initial value problems. The framework we present is general and enables the construction of many new families of additive integrators with arbitrary order-of-accuracy and varying degree of implicitness. In this first work, we focus on a new class of implicit-explicit polynomial block methods that are based on fully implicit Runge–Kutta methods with Radau nodes and that possess high stage order. Here, we show that the new fully implicit-explicit (FIMEX) integrators have improved stability compared to existing IMEX Runge–Kutta methods, while also being more computationally efficient due to recent developments in preconditioning techniques for solving the associated systems of nonlinear equations. For PDEs on periodic domains where the implicit component is trivial to invert, we will show how parallelization of the right-hand side evaluations can be exploited to obtain significant speedup compared to existing serial IMEX Runge–Kutta methods. For parallel (in space) finite element discretizations, the new methods can achieve orders of magnitude better accuracy than existing IMEX Runge–Kutta methods and/or achieve a given accuracy several times times faster in terms of computational runtime.

97 MATHEMATICS AND COMPUTING↗

Analysis of Potential Energy Corridors Proposed by the Western Electricity Coordinating Council

This report, Analysis of Potential Energy Corridors Proposed by the Western Electricity Coordinating Council (WECC), was prepared by the Environmental Science Division of Argonne National Laboratory (Argonne). The intent of WECC’s work was to identify planning-level energy corridors that the Department of Energy (DOE) and its affiliates could study in greater detail. Argonne was tasked by DOE to analyze the WECC Proposed Energy Corridors in five topic areas for use in reviewing and revising existing corridors, as well as designating additional energy corridors in the 11 western states. In compliance with Section 368 of the Energy Policy Act of 2005 (EPAct), the Secretaries of Energy, Agriculture, and the Interior (Secretaries) published a Programmatic Environmental Impact Statement in 2008 to address the proposed designation of energy transport corridors on federal lands in the 11 western states. Subsequently, Records of Decision designating the corridors were issued in 2009 by the Bureau of Land Management (BLM) and the U.S. Forest Service (USFS). The 2012 settlement of a lawsuit, brought by The Wilderness Society and others against the United States, which identified environmental concerns for many of the corridors requires, among other things, periodic reviews of the corridors to assess the need for revisions, deletions, or additions. A 2013 Presidential Memorandum requires the Secretaries to undertake a continuing effort to identify and designate energy corridors. The WECC Proposed Energy Corridors and their analyses in this report provide key information for reviewing and revising existing corridors, as well as designating additional energy corridors in the 11 western states. Load centers and generation hubs identified in the WECC analysis, particularly as they reflect renewable energy development, would be useful in reviewing and potentially updating the designated Section 368 corridor network. Argonne used Geographic Information System (GIS) technology to analyze the proposed energy corridors in the WECC report in five topic areas: • Federal land jurisdiction, • Existing Section 368 corridors, • Existing transmission lines, • Previously studied corridor locations, and • Protected areas. Analysis methods are explained and tables and maps are provided to describe the results of the analyses in all five topic areas. WECC used a rational approach to connecting the hubs it identified, although there may be opportunities for adapting some of the proposed WECC routes to previously designated Section 368 corridors, for example: • The WECC proposed energy corridors are in fact centerlines of proposed routes connecting hubs of various descriptions related to electric energy transmission. Although the centerlines were sited to avoid sensitive areas, infrastructure proposed within actual pathways or corridors defined by the centerlines would sometimes affect lands where such development would not normally be allowed, such as National Parks and Monuments, National Wildlife Refuges, and Wilderness Areas. • Many WECC proposed energy corridors are sited along centerlines of existing roads, including Interstate Highways, where in some cases additional width to accommodate energy transmission infrastructure may not be available. Examples include the WECC Proposed Corridor along Interstate 70 through Glenwood Canyon in Colorado, and along U.S. Highway 89 across Glen Canyon Dam in Arizona. • Several WECC proposed energy corridors are parallel to designated Section 368 corridors that have already cleared the preliminary steps to right-of-way approval. In many of these cases, the WECC hub connection objectives can be met more efficiently by routing on the designated Section 368 corridors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Computational Fluid Dynamics Using the Adaptive Wavelet-Collocation Method

Advancements to the adaptive wavelet-collocation method over the last decade have opened up a number of new possible areas for active research. Volume penalization techniques allow complex immersed boundary conditions to be used with high efficiency for both internal and external flows. Anisotropic methods make it possible to use body-fitted meshes while still taking advantage of the dynamic adaptability properties wavelet-based methods provide. The parallelization of the approach has made it possible to perform large high-resolution simulations of detonation initiation and fluid instabilities to uncover new physical insights that would otherwise be difficult to discover. Other developments include space-time adaptive methods and nonreflecting boundary conditions. This article summarizes the work performed using the adaptive wavelet-collocation method developed by Vasilyev and coworkers over the past decade.

42 ENGINEERING↗

Space charge effect of the high intensity proton beam during the resonance extraction for the Mu2e experiment at Fermilab

The proposed Mu2e experiment to search for direct {mu} {yields} e conversion at Fermilab plans slow, resonant extraction of a beam with 3 x 10{sup 12} protons from the Debuncher ring. Space charge of this high intensity beam is a critical factor, since it induces significant betatron tune spread and consequently affects resonance extraction processes, such as spill uniformity and beam losses. This study shows the multi-particle simulation results in the early stages of resonance extraction and spill uniformity in the presence of 2D and 3D space charge effects. We have presented the results of the third-integer resonance extraction in early stage for the Mu2e experiment in the presence of space charge effects. In order to track particles and to calculate self-consistent space charge effects, Synergia2 was used, which is capable of parallel computing. The space charge tune shift was computed and was reasonable value compared with the analytical calculation. Locations of the septum and Lambertson were chosen so that particles are kicked and extracted efficiently. The spill rates for with and without space charge effects were uniform, but should be improved for the early stage after the sextupole field ramping.

43 PARTICLE ACCELERATORS↗

HARMONY: Large-Scale Architecture Search for Efficient Hybrid Language Models

As large language models scale to trillions of parameters, their computational and memory requirements present critical challenges for efficient training and deployment. While Mixture of Experts (MoE) architectures enable efficient scaling through sparse parameter activation, and state-space models like Mamba offer linear-time complexity, principled methods for combining these paradigms remain undeveloped. We introduce HARMONY (Hybrid Architecture Research for Mamba, Optimized with Neural efficiencY), a multi-objective evolutionary neural architecture search framework for discovering efficient hybrid language models that integrate Transformer attention mechanisms, Mixture-of-Experts routing, and Mamba state-space components. Through large-scale distributed search using 16,384 MI250X GPUs on the Frontier supercomputer, HARMONY explores a comprehensive design space encompassing six attention variants (MHA, MQA, GQA, MLA, SWA, and Mamba-2), variable MoE configurations with both routed and shared experts, and extensive Mamba hyperparameters. Our framework discovers heterogeneous architectures that balance training performance with computational efficiency through multi-objective optimization incorporating latency penalties and fitness-based selection. Analysis of discovered architectures reveals that optimal hybrid designs favor heterogeneous component mixing rather than homogeneous patterns, with Mamba-2 and Multi-Head Latent Attention (MLA) emerging as preferred mechanisms. Discovered architectures demonstrate superior training efficiency: our best configuration achieves a final perplexity of 1.0874 with 2.38B parameters while processing 4,320 tokens/second, outperforming significantly larger manually designed models. Full-scale evaluation shows HARMONY's top architectures achieve better loss trajectories than equivalently-sized models using state-of-the-art configurations including Mixtral, Jamba, and Samba. Additionally, we demonstrate 91% weak scaling efficiency when training discovered 36B-parameter models across 1,024 GPUs. HARMONY is released as an open framework with comprehensive tools for building and training hybrid models using expert-data-pipeline parallelism, democratizing access to automated architecture design for next-generation language models.

Herron, Emily [ORNL] (ORCID:0000000273008172)↗