Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Unraveling the Determinant Mechanisms in Flow-Mediated Crystal Growth and Phase Behaviors

To uncover the critical mechanisms responsible for mesoscopic level development during flow-mediated crystal growth, we develop a semi-two-way hydrodynamic coupled structural phase-field crystal formalism (HXPFC-s2). The new formalism, inspired by previous attempts at coupling hydrodynamic and phase-field crystal (PFC) equations, allows for studying mesoscopic flow-mediated crystallization at diffusive timescales pertinent to industrial applications. Unlike previous efforts, the devised coupling to the structural PFC (XPFC) equations allows generalization to more complex crystal structures through explicit parameterization of the direct correlation function (DCF). Utilizing the HXPFC-s2 formalism, we seek to uncover the determinant physical mechanisms in crystallization under simple shear flows by comparing temperature-driven crystallization to flow-mediated crystallization under varying flow-strengths. Parallels and deviations of under-cooling and flow-strength effects on crystal growth are drawn using the crystal cluster-size and system ordering time evolutions. In doing so, we identify scaling behaviors with a Peclet-like number, Pe∼, a critical Peclet-like number, Pe∼*, and flow-field-crystal plane-dependent interactions. Our findings may be relevant for controlling crystal growth and phase behaviors in flow applications.

Willis, L. Connor (ORCID:0009000961321848)↗
DOE OSTI2024

Drift kinetic electrostatic simulations of the edge localized mode heat pulse

In the present work, electrostatic drift kinetic simulations of parallel plasma transport within the tokamak scrape-off layer (SOL) are conducted using the COGENT code. The SOL configuration is represented in one-dimensional slab geometry, incorporating a heat source localized in the midplane. The heat source parameters correspond to those characterizing edge-localized modes observed in the Joint European Torus (JET) tokamak. The numerical model includes kinetic treatment of both ions and electrons, a simplified model for the gyrokinetic Poisson equation that allows one to step over short time scales associated with fast electrostatic shear Alfvèn waves, and the logical sheath boundary condition (LSBC) that enforces global system quasineutrality. A third-order accurate LSBC is derived to be consistent with the third-order accurate upwind advection scheme utilized in the code, and it was shown to noticeably impact the simulation results, especially parallel heat flux at the target plate. The findings of this study are in agreement with results from preceding fluid and kinetic simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

OpenSn: A massively parallel, open-source simulation environment for discrete ordinates radiation transport

OpenSn is an open-source, massively parallel deterministic radiation transport code for solving the discrete-ordinates ( S N ) form of the Boltzmann transport equation on unstructured, arbitrary polyhedral meshes. It supports high-fidelity simulations involving steady-state, eigenvalue, and adjoint problems for neutral particles (e.g., neutrons, photons, multi-particles), using the multigroup approximation in energy. OpenSn combines angular discretization via discrete ordinates with a discontinuous Galerkin finite element method (DGFEM) in space, enabling accurate resolution of transport physics on arbitrary polyhedral cells, included locally refined spatial grids. It includes multiple angular quadrature types, including locally refined angular quadratures. Written in modern C++ with a Python API, OpenSn runs efficiently on platforms ranging from laptops to supercomputers. The transport sweep algorithm is implemented using a task-based, directed-acyclic-graph (DAG) approach for each angle and supports asynchronous parallelism across thousands of MPI ranks. Group-set aggregation improves compute intensity, and synthetic acceleration techniques (e.g., diffusion synthetic acceleration, second-moment method) enhance solver convergence. OpenSn has been verified on reactor physics problems and demonstrated excellent weak and strong scaling performance on more than 32,768 processes, making it a versatile and robust platform for large-scale transport simulations in complex geometries.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Scalability of high-performance PDE solvers

Performance tests and analyses are critical to effective high-performance computing software development and are central components in the design and implementation of computational algorithms for achieving faster simulations on existing and future computing architectures for large-scale application problems. In this article, we explore performance and space-time trade-offs for important compute-intensive kernels of large-scale numerical solvers for partial differential equations (PDEs) that govern a wide range of physical applications. We consider a sequence of PDE-motivated bake-off problems designed to establish best practices for efficient high-order simulations across a variety of codes and platforms. We measure peak performance (degrees of freedom per second) on a fixed number of nodes and identify effective code optimization strategies for each architecture. In addition to peak performance, we identify the minimum time to solution at 80% parallel efficiency. The performance analysis is based on spectral and p-type finite elements but is equally applicable to a broad spectrum of numerical PDE discretizations, including finite difference, finite volume, and h-type finite elements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Final Technical Report

Statement of the problem or situation that is being addressed in your application. The DOE and its national laboratories developed the Home Energy Score™ (HES) to encourage homeowners to improve their energy performance, lower costs and to share energy information through the MLS listing, appraisal, and financing channels. While the HES is an instrumental tool, it is currently underutilized and consists of technical, structural and sector barriers which need to be addressed in order to scale and many energy efficiency contractors are understandably overwhelmed by the added time and effort and lack of incentive to sell and deliver deep retrofit projects while simultaneously meeting the DOE HES program requirements; consequently, contractors may decide to forgo participation. Home Energy Rating System (HERS) Raters have the opportunity to play the critical Assessor role in producing a Home Energy Score (HES); this role has immense potential but currently is unfulfilled. Lastly, while utilities are interested in their customer base achieving greater energy efficiency, especially to help offset growing residential loads in states like California that are accelerating electrification, utilities do not have access to the market actors who are on the front line of influence to homeowners or review and approve their permits: HERS Raters, assessors, contractors and building departments. General statement of how this problem is being addressed: ConSol will integrate the Home Energy Score™ (HES) to its State of California, approved home energy rating services (HERS) platform (CHEERS) to develop a single tool for contractors nationwide to assess, record and install recommended cost, energy, and emissions saving measures to the 140 million single-family homes throughout the U.S. and 14 million homes in California (CHEERS+HES). The CHEERS high fidelity energy code permitting data will be integrated with HES for simple, accurate, easy-to-use home energy estimation and analysis and will directly gain access to the retrofit and renovations markets with the same upgraded platform. This innovative project will assist the utilities in supporting existing homes in their jurisdictions with HES and develop measures to improve energy efficiency and reduce emissions. How is this problem being addressed? What is the overall project approach? In effort to expand the Home Energy Score™ (HES) by increasing the use of aggregable home energy asset data, ConSol proposes to integrate the DOE HES via Application Programming Interface (API) to its State of California approved home energy rating services platform (CHEERS). Once the CHEERS platform and HES are integrated (CHEERS+HES), this enhanced platform will be instantly available and actively deployed via Phase 1 pilot to HERS Raters, assessors and contractors in California to market-test the solution, understand the rate of adoption and identify opportunities for improvement prior to scaling nationally. The CHEERS high fidelity energy code permitting data will be integrated with HES for simple, accurate, easy-to-use home energy estimation and analysis and will directly gain access to the retrofit and renovations markets with the same upgraded platform. This innovative project will assist the building industry and homeowners with an easy-to-use assessment if energy and carbon impacts of existing homes, and assist the utilities in supporting existing homes in their jurisdictions with HES to improve energy efficiency and reduce emissions. What is to be done in Phase I? During Phase I of this proposed project, ConSol will (1) design software architecture that links CHEERS to the Home Energy ScoreTM via API, (2) solicit partnership from one or more California utilities for a regional pilot, (3) test the new software with its HERS Raters and contractor network in the partnership utility jurisdiction, (4) launch a pilot version of the newly developed software with HERS Raters and contractors in the utility territory, and (5) explore California’s GoGreen energy efficiency homeowner lending program in parallel with the pilot. Commercial Applications and Other Benefits. Summarize the future applications or public benefits if the project is carried over into Phase II or Phase III and beyond. The CHEERS+HES commercialized product will be ready for national market scale following a successful Phase 1 performance. The CHEERS+HES adoption is estimated to reach a 5% adoption growth rate versus the 110,000 baseline, starting in Year 1 after Phase I completion, and continuing each year. As a direct benefit to the DOE, CHEERS will set a goal of 100,000 Home Energy Score assessments for existing home alterations within the first 10 years following Phase 1 performance. The technical benefits of this proposed project include the harmonized, automated, and seamless integration of the DOE HES into the widely used and market leading California energy registry, CHEERS. The social benefits include the aggregate energy, cost and GHG savings by allowing the broader public streamlined access to the CHEERS+HES measurement and the energy efficiency recommended measures that may result. Key Words: Home Energy ScoreTM (HES); Application Programming Interface (API); Home Energy Rating Services (HERS); HERS Raters; contractors; assessors; existing homes, energy asset data; cost, energy, and emissions saving measures; energy code (Title 24) compliance; document repository; utilities; pilot; newly developed software; energy efficiency; homeowner. Summary for Members of Congress: The DOE Home Energy Score™ (HES) is a tool to encourage homeowners to improve their energy performance, lower costs and share energy information but is underutilized and consists of barriers which need to be addressed in order to scale. In effort to expand the HES, CHEERS, Inc. will integrate the HES to its State of California, approved home energy rating services (HERS) platform (CHEERS) to develop a single tool for contractors nationwide to assess, record and install recommended cost, energy, and emissions saving measures to the 140 million single-family homes throughout the U.S. and 14 million homes in California.

Application Programming Interface (API)↗
DOE OSTI2020

Neuromorphic scaling advantages for energy-efficient random walk computations

Computing stands to be radically improved by neuromorphic computing (NMC) approaches inspired by the brain's incredible efficiency and capabilities. Most NMC research, which aims to replicate the brain's computational structure and architecture in man-made hardware, has focused on artificial intelligence; however, less explored is whether this brain-inspired hardware can provide value beyond cognitive tasks. We demonstrate that high-degree parallelism and configurability of spiking neuromorphic architectures makes them well-suited to implement random walks via discrete time Markov chains. Such random walks are useful in Monte Carlo methods, which represent a fundamental computational tool for solving a wide range of numerical computing tasks. Additionally, we show how the mathematical basis for a probabilistic solution involving a class of stochastic differential equations can leverage those simulations to provide solutions for a range of broadly applicable computational tasks. Despite being in an early development stage, we find that NMC platforms, at a sufficient scale, can drastically reduce the energy demands of high-performance computing platforms.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Parallel magnetic field suppresses dissipation in superconducting nanostrips

The motion of Abrikosov vortices in type-II superconductors results in a finite resistance in the presence of an applied electric current. Elimination or reduction of the resistance via immobilization of vortices is the "holy grail" of superconductivity research. Common wisdom dictates that an increase in the magnetic field escalates the loss of energy since the number of vortices increases. Here we show that this is no longer true if the magnetic field and the current are applied parallel to each other. Our experimental studies on the resistive behavior of a superconducting Mo0.79Ge0.21 nanostrip reveal the emergence of a dissipative state with increasing magnetic field, followed by a pronounced resistance drop, signifying a reentrance to the superconducting state. Large-scale simulations of the 3D time-dependent Ginzburg-Landau model indicate that the intermediate resistive state is due to an unwinding of twisted vortices. When the magnetic field increases, this instability is suppressed due to a better accommodation of the vortex lattice to the pinning configuration. Our findings show that magnetic field and geometrical confinement can suppress the dissipation induced by vortex motion and thus radically improve the performance of superconducting materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2019

Parsl: Pervasive Parallel Programming in Python

High-level programming languages such as Python are increasingly used to provide intuitive interfaces to libraries written in lower-level languages and for assembling applications from various components. This migration towards orchestration rather than implementation, coupled with the growing need for parallel computing (e.g., due to big data and the end of Moore's law), necessitates rethinking how parallelism is expressed in programs. Here, we present Parsl, a parallel scripting library that augments Python with simple, scalable, and flexible constructs for encoding parallelism. These constructs allow Parsl to construct a dynamic dependency graph of components that it can then execute efficiently on one or many processors. Parsl is designed for scalability, with an extensible set of executors tailored to different use cases, such as low-latency, high-throughput, or extreme-scale execution. We show, via experiments on the Blue Waters supercomputer, that Parsl executors can allow Python scripts to execute components with as little as 5 ms of overhead, scale to more than 250000 workers across more than 8000 nodes, and process upward of 1200 tasks per second. Other Parsl features simplify the construction and execution of composite programs by supporting elastic provisioning and scaling of infrastructure, fault-tolerant execution, and integrated wide-area data management. We show that these capabilities satisfy the needs of many-task, interactive, online, and machine learning applications in fields such as biology, cosmology, and materials science.

parallel programming↗
DOE OSTI2020

Understanding resistances in capacitive deionization devices

The desalination characteristics of capacitive deionization (CDI) are significantly impacted by the cell resistance. Here we apply electrochemical impedance spectroscopy (EIS) to analyze the resistive properties of a CDI device, which include EIS measurements on a complete assembled cell (in two wire mode) and EIS measurements with additional contacts to decouple electrode/electrolyte and contact impedances (four wire mode). These measurements shed light on the interface between current collector and electrode as well as the internal capacitive and resistive elements of the cell. With two-wire and four-wire EIS measurements, we find that the often observed high-frequency arc(s) in the impedance spectra can be due to contact effects and/or an internal ionic-transfer resistance that is locally in parallel with macropore wall capacitance. In some cases, the two associated arcs in the Nyquist plot are overlapping and present simultaneously, making their identification in the EIS spectra difficult unless both two-wire and four-wire measurements are made. Despite causing a large high-frequency arc in the Nyquist plot, we find that the apparent internal ionic-transfer resistance is negligible compared to the resistances of the electrode, separator, and external electronic components. Here, an investigation of the contact impedance between the electrode and current collector reveals a parallel RC arc that can grow over time if an improper material is used (e.g., titanium). In our system, the electrode ionic resistance dominates the total Ohmic power dissipation, which has implications for scale-up of CDI systems and future improvements of cell design.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Time-dependent study of anisotropy in Rayleigh-Taylor instability induced turbulent flows with a variety of density ratios

This study focuses on understanding the time-dependent anisotropy, mixing, scaling of flows induced by Rayleigh-Taylor instability, and complementing the late-time snapshots reported by Cabot and Zhou [“Statistical measurements of scaling and anisotropy of turbulent flows induced by Rayleigh-Taylor instability,” Phys. Fluids 25, 015107 (2013)]. In particular, we utilize three large datasets with different Atwood numbers (density ratios) from well resolved direct numerical simulations at a moderate Reynolds number with the goal of determining the degree of departure of this inhomogeneous flow from that of homogeneous, isotropic turbulence. Three key time-dependent statistical measurements are considered in detail to delineate the role played by the acceleration. First, a number of directional length scales in this anisotropic turbulence are inspected. Second, the relationship among the outer-scale, the turbulence length, and the Taylor-microscale based Reynolds numbers is also clarified. Lastly, the normalized dissipation rate is employed to inspect the distinctive features of the flow in the inhomogeneous direction parallel to gravity and in the homogeneous perpendicular directions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Phasing of seven-channel fibre laser radiation with dynamic turbulent phase distortions using a stochastic parallel gradient algorithm at a bandwidth of 450 kHz

We have demonstrated an experimental setup for the coherent phasing of a seven-channel fibre laser system ( λ = 1064 nm) in a scheme comprising a master oscillator and a set of parallel amplifiers with lithium niobate-based fibre-optic phase modulators. Using a stochastic parallel gradient algorithm, an instrumental phase modulator control unit ensures a bandwidth of the system up to 450 kHz. The effectiveness of phasing of light transmitted through a turbulent medium with a characteristic time scale τ{sub turb} has been studied experimentally as a function of phasing time τ{sub ph}. The results demonstrate that the average Strehl ratio begins to rise at τ{sub turb}/τ{sub ph} ⩾ 2 and that the effectiveness of compensation for dynamic phase distortions in the beam propagation path rises sharply at τ{sub turb}/τ{sub ph} ≈ 20. For τ{sub turb}/τ{sub ph} ⩾ 30 – 40, the average Strehl ratio remains constant at the level reached. (control of laser radiation parameters)

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Exploring MPI Communication Models for Graph Applications Using Graph Matching as a Case Study

Traditional implementations of parallel graph operations on distributed memory platforms are written using Message Passing Interface (MPI) point to point communication primitives such as Send-Recv (blocking and nonblocking). Apart from this classical model, the MPI community has over the years added other communication models; however, their suitability for handling the irregular traffic workloads typical of graph operations remain comparatively less explored. Our aim in this paper is to study these relatively underutilized communication models of MPI for graph applications. More specifically, we evaluate MPI’s one-sided programming, or Remote Memory Access (RMA), and nearest neighborhood collectives using a process graph topology. There are features in these newer models that are intended to better map to irregular communication patterns, as exemplified in graph algorithms. As a concrete application for our case study, we use distributed memory implementations of a weighted graph matching algorithm to investigate performances of MPI-3 RMA and neighborhood collective operations compared to nonblocking Send- Recv. A matching in a graph is a subset of edges such that no two matched edges are incident on the same vertex. A maximum weight matching is a matching of maximum weight computed as the sum of the weights of matched edges. Algorithms for optimal matching have quadratic complexity and are difficult to parallelize. More importantly, execution of graph matching is dominated by high volume of irregular memory accesses, making it an ideal candidate for studying the effects of various MPI communication models on graph applications at scale. Our neighborhood collectives and RMA implementations yields up to 6× speedup over traditional optimized nonblocking Send-Recv implementations, on thousands of cores of the NERSC Cori supercomputer. We believe the lessons learned from this case study can be adopted to benefit a wider range of graph application codes.

Ghosh, Sayan↗
DOE OSTI2020

Optimizing High Performance Markov Clustering for Pre-Exascale Architectures

HipMCL is a high-performance distributed memory implementation of the popular Markov Cluster Algorithm (MCL) and can cluster large-scale networks within hours using a few thousand CPU-equipped nodes. It relies on sparse matrix computations and heavily makes use of the sparse matrix-sparse matrix multiplication kernel (SpGEMM). The existing parallel algorithms in HipMCL are not scalable to Exascale architectures, both due to their communication costs dominating the runtime at large concurrencies and also due to their inability to take advantage of accelerators that are increasingly popular. In this work, we systematically remove scalability and performance bottlenecks of HipMCL. We enable GPUs by performing the expensive expansion phase of the MCL algorithm on GPU. Additionally, we propose a CPU-GPU joint distributed SpGEMM algorithm called pipelined Sparse SUMMA and integrate a probabilistic memory requirement estimator that is fast and accurate. Furthermore, we develop a new merging algorithm for the incremental processing of partial results produced by the GPUs, which improves the overlap efficiency and the peak memory usage. We also integrate a recent and faster algorithm for performing SpGEMM on CPUs. We validate our new algorithms and optimizations with extensive evaluations. With the enabling of the GPUs and integration of new algorithms, HipMCL is up to 12.4x faster, being able to cluster a network with 70 million proteins and 68 billion connections just under 15 minutes using 1024 nodes of ORNL's Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A new HVdc Substation Architecture with Higher Power Transfer Capability

The power transferred through high voltage direct current transmission (HVdc) systems is increasing with increased resilience and reliability needs as well as large scale deployment of renewable energy systems. In this paper, a new HVdc substation (SS) architecture with higher power transfer capability is proposed. The HVdc converters or valve groups (VG) are connected in series and parallel fashion such that the total power rating of the SS is increased. The control challenges associated with such SS architecture are identified and appropriate control modifications are proposed. The same is verified through PSCAD based simulations.

Jaldanki, Sreenivasa↗
DOE OSTI2017

Integrated Predictive Tools for Customizing Microstructure and Material Properties of Additively Manufactured Aerospace Components

Additive Manufacturing (AM) refers to a process by which digital three-dimensional (3-D) design data is converted to build up a component by depositing material layer-by-layer. United Technologies Corporation (UTC) is currently involved in fabrication and certification of several AM aerospace structural components made from aerospace materials. This is accomplished by using optimized process parameters determined through numerous design-of-experiments (DOE)-based studies. Certification of these components is broadly recognized as a significant challenge, with long lead times, very expensive new product development cycles and very high energy consumption. Because of these challenges, United Technologies Research Center (UTRC), together with UTC business units have been developing and validating an advanced physics-based process model. The specific goal is to develop a physics-based framework of an AM process and reliably predict fatigue properties of built-up structures as based on detailed solidification microstructures. Microstructures are predicted using process control parameters including energy source power, scan velocity, deposition pattern, and powder properties. The multi-scale multi-physics model requires solution and coupling of governing physics that will allow prediction of the thermal field and enable solution at the microstructural scale. The state-of-the-art approach to solve these problems requires a huge computational framework and this kind of resource is only available within academia and national laboratories. The project utilized the parallel phase-fields codes at Oak Ridge National Laboratory (ORNL) and Lawrence Livermore National Laboratory (LLNL), along with the high-performance computing (HPC) capabilities existing at the two labs to demonstrate the simulation of multiple dendrite growth in threedimensions (3-D). The LLNL code AMPE was used to implement the UTRC phase field model that was previously developed for a model binary alloy, and the simulation results were compared against the UTRC simulation results, followed by extension of the UTRC model to simulate multiple dendrite growth in 3-D. The ORNL MEUMAPPS code was used to simulate dendritic growth in a model ternary alloy with the same equilibrium solidification range as the Ni-base alloy 718 using realistic model parameters, including thermodynamic integration with a Calphad based model for the ternary alloy. Implementation of the UTRC model in AMPE met with several numerical and parametric issues that were resolved and good comparison between the simulation results obtained by the two codes was demonstrated for two dimensional (2-D) dendrites. 3-D dendrite growth was then demonstrated with the AMPE code using nondimensional parameters obtained in 2-D simulations. Multiple dendrite growth in 2-D and 3-D were demonstrated using ORNL’s MEUMAPPS code using simple thermal boundary conditions. MEUMAPPS was then modified to incorporate the complex, time-dependent thermal boundary conditions obtained by UTRC’s thermal modeling of single track AM experiments to drive the phase field simulations. The results were in good agreement with UTRC’s experimental measurements.

36 MATERIALS SCIENCE↗
DOE OSTI2023

BOTTLE 2 - Analysis

The use of analysis is foundational to BOTTLE. In particular, we conduct techno-economic analysis (TEA), life cycle assessment (LCA), and supply chain modeling for each new BOTTLE innovation, spanning new deconstruction and upcycling technologies to redesigned polymers. These critical analysis tools are used in parallel with laboratory research to ensure that BOTTLE technologies can ultimately meet our key metrics. The BOTTLE team used analysis early in the current 3-year project cycle to identify research priorities from a scale, energy, and economics perspective (Nicholson et al., Joule 2021), and recently we presented a comprehensive framework for comparing new circularity-focused approaches to linear, incumbent practices (Nicholson et al., Ann. Rev. Chem. Biomolec. Eng. 2022). This presentation will also review case studies in analysis for closed-loop PET recycling across a range of recycling methods, baseline studies that we have conducted for plastics pyrolysis and gasification as another comparator for BOTTLE technologies, and a comprehensive study of existing closed-loop recycling technologies from a full suite of environmental impacts and feedstock quality metrics. We have undertaken studies of circular polymers being developed in the Redesign task, and we will present an exemplary case for a bio-based acrylic polymer that can replace polymethyl methacrylate. Overall, analysis is critical to enable the BOTTLE Consortium to focus on impactful, realistic technologies.

BIOMASS FUELS↗
DOE OSTI2026

HARMONY: Large-Scale Architecture Search for Efficient Hybrid Language Models

As large language models scale to trillions of parameters, their computational and memory requirements present critical challenges for efficient training and deployment. While Mixture of Experts (MoE) architectures enable efficient scaling through sparse parameter activation, and state-space models like Mamba offer linear-time complexity, principled methods for combining these paradigms remain undeveloped. We introduce HARMONY (Hybrid Architecture Research for Mamba, Optimized with Neural efficiencY), a multi-objective evolutionary neural architecture search framework for discovering efficient hybrid language models that integrate Transformer attention mechanisms, Mixture-of-Experts routing, and Mamba state-space components. Through large-scale distributed search using 16,384 MI250X GPUs on the Frontier supercomputer, HARMONY explores a comprehensive design space encompassing six attention variants (MHA, MQA, GQA, MLA, SWA, and Mamba-2), variable MoE configurations with both routed and shared experts, and extensive Mamba hyperparameters. Our framework discovers heterogeneous architectures that balance training performance with computational efficiency through multi-objective optimization incorporating latency penalties and fitness-based selection. Analysis of discovered architectures reveals that optimal hybrid designs favor heterogeneous component mixing rather than homogeneous patterns, with Mamba-2 and Multi-Head Latent Attention (MLA) emerging as preferred mechanisms. Discovered architectures demonstrate superior training efficiency: our best configuration achieves a final perplexity of 1.0874 with 2.38B parameters while processing 4,320 tokens/second, outperforming significantly larger manually designed models. Full-scale evaluation shows HARMONY's top architectures achieve better loss trajectories than equivalently-sized models using state-of-the-art configurations including Mixtral, Jamba, and Samba. Additionally, we demonstrate 91% weak scaling efficiency when training discovered 36B-parameter models across 1,024 GPUs. HARMONY is released as an open framework with comprehensive tools for building and training hybrid models using expert-data-pipeline parallelism, democratizing access to automated architecture design for next-generation language models.

Herron, Emily [ORNL] (ORCID:0000000273008172)↗
DOE OSTI2018

Asynchronously parallel optimization solver for finding multiple minima

This paper proposes and analyzes an asynchronously parallel optimization algorithm for finding multiple, high-quality minima of nonlinear optimization problems. Our multistart algorithm considers all previously evaluated points when determining where to start or continue a local optimization run. Theoretical results show that, under certain assumptions, the algorithm almost surely starts a finite number of local optimization runs and identifies, or has a single local optimization run converging to, every minimum. The algorithm is applicable to general optimization settings, but our numerical results focus on the case when derivatives are unavailable. In numerical tests, a PYTHON implementation of the algorithm is shown to yield good approximations of many minima (including a global minimum), and this ability scales well with additional resources. Our implementation’s time to solution is shown also to scale well even when the time to evaluate the function evaluation is highly variable.

97 MATHEMATICS AND COMPUTING↗