Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61

A Subcritical Assembly for Training and Education Use at the Oak Ridge National Laboratory [Abstract]

A near-final design has been completed for a new subcritical assembly at Oak Ridge National Laboratory (ORNL). This subcritical assembly will support the Nuclear Criticality Safety Program (NCSP) training and education program to augment current subcritical assembly capabilities at Sandia National Laboratories (SNL) and at the National Criticality Experiments Research Center (NCERC). Other collaborations have been proposed with the Sunrise Consortium and others. This proposed subcritical assembly uses legacy AGN-201M research reactor fuel plates that are available from the Y-12 National Security Complex. This subcritical assembly contains approximately 620 grams of 235 U as UO 2 particulates distributed homogeneously in radiation-stabilized polyethylene. The fuel plates will have a graphite neutron reflector to obtain a core multiplication, M, from 10 to 50, corresponding to a k eff of 0.9 to 0.95, respectively. The subcritical assembly will be able to support at least four experiments for the training courses: (1) the addition of fissile material to increase neutron multiplication (mass), (2) a core separation experiment (interaction), (3) the effect of adding moderators to the core (moderation), and (4) the effect of adding neutron absorbers to the core (poison/absorption). The proposed ORNL subcritical assembly will provide additional student bandwidth to support NCSP training courses and to provide for some backup capacity if nuclear facility operations are disrupted at SNL or NCERC due to some unforeseen issue. The subcritical assembly will be designed to be an inherently safe subcritical assembly—subcritical under all normal and abnormal conditions—and it will provide the capability to conduct hands-on training to support NCSP and general nuclear criticality safety (NCS) staff training and qualification goals.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Hybrid geological modeling: Combining machine learning and multiple-point statistics

Accurately modeling and constructing a geologically realistic subsurface model remains an outstanding problem as the morphology controls the flow behaviors. Particularly, one of the pattern-based methods, namely cross-correlation based simulation, has been proved to be an effective way to reconstruct a realistic model, at both small and large scales. However, conditioning to point data in the large-scale problems is still a crucial issue in these algorithms, since there is always a trade-off between the quality of the realizations and the degree of point data reproduction. Specifically, it is not practical to build a training image (TI) which includes all the possibilities and variabilities. Therefore, finding a pattern that can represent the point data and, at the same time, preserving the connectivities is difficult. This leads to producing highly-connected realizations with a significant mismatch or poor models with a reasonable degree of point data reproduction. To accurately reproduce the densely distributed hard data, pixel-based methods can also produce some unrealistic artifacts around the hard data. In this paper, to overcome this challenge, however, we use pattern-based methods as they often produce more disconnected geobodies when dealing with dense hard data, and proposed a hybrid algorithm using the pattern-based methods and convolutional neural network (CNN). The trained CNN model is utilized to improve the quality of conditioning to point data for the original realizations generated by the pattern-based algorithm. As such, the mismatch locations are identified, and the same regions are used in the training of CNN to mimic the procedure through which a missing region can be filled. To evaluate the performance of the proposed hybrid algorithm, it is tested on cases with different dimensions and different numbers of facies. Then, the newly improved realizations are compared with the initial realizations generated by the pattern-based algorithm. The comparison is also conducted by the flow simulation test. And it indicates that the proposed hybrid algorithm can better reproduce the point data, while the connectivities are better preserved.

58 GEOSCIENCES↗

Technical note: Using long short-term memory models to fill data gaps in hydrological monitoring networks

Abstract. Quantifying the spatiotemporal dynamics in subsurface hydrological flows over a long time window usually employs a network of monitoring wells. However, such observations are often spatially sparse with potential temporal gaps due to poor quality or instrument failure. In this study, we explore the ability of recurrent neural networks to fill gaps in a spatially distributed time-series dataset. We use a well network that monitors the dynamic and heterogeneous hydrologic exchanges between the Columbia River and its adjacent groundwater aquifer at the U.S. Department of Energy's Hanford site. This 10-year-long dataset contains hourly temperature, specific conductance, and groundwater table elevation measurements from 42 wells with gaps of various lengths. We employ a long short-term memory (LSTM) model to capture the temporal variations in the observed system behaviors needed for gap filling. The performance of the LSTM-based gap-filling method was evaluated against a traditional autoregressive integrated moving average (ARIMA) method in terms of error statistics and accuracy in capturing the temporal patterns of river corridor wells with various dynamics signatures. Our study demonstrates that the ARIMA models yield better average error statistics, although they tend to have larger errors during time windows with abrupt changes or high-frequency (daily and subdaily) variations. The LSTM-based models excel in capturing both high-frequency and low-frequency (monthly and seasonal) dynamics. However, the inclusion of high-frequency fluctuations may also lead to overly dynamic predictions in time windows that lack such fluctuations. The LSTM can take advantage of the spatial information from neighboring wells to improve the gap-filling accuracy, especially for long gaps in system states that vary at subdaily scales. While LSTM models require substantial training data and have limited extrapolation power beyond the conditions represented in the training data, they afford great flexibility to account for the spatial correlations, temporal correlations, and nonlinearity in data without a priori assumptions. Thus, LSTMs provide effective alternatives to fill in data gaps in spatially distributed time-series observations characterized by multiple dominant frequencies of variability, which are essential for advancing our understanding of dynamic complex systems.

54 ENVIRONMENTAL SCIENCES↗

Inter-radiologist agreement using Society of Abdominal Radiology-American Gastroenterological Association (SAR-AGA) consensus nomenclature for reporting CT and MR enterography in children and young adults with small bowel Crohn disease

Purpose: To assess inter-radiologist agreement using the Society of Abdominal Radiology-American Gastroenterological Association (SAR-AGA) consensus recommendations for reporting CT/MR enterography exams in pediatric and young adult small bowel Crohn disease (CD). Methods: Institutional review board approval was obtained for this HIPAA-compliant retrospective investigation; the requirement for informed consent was waived. 25 CT and 25 MR enterography exams performed in children and young adults (age range: 6–23 years) between January 2015 and April 2017 with a distribution of ileal CD severity (phenotype) were identified: normal or chronic CD without active inflammation (40%), active inflammatory CD (20%), stricturing CD (20%), and penetrating CD (20%). Five fellowship-trained pediatric radiologists, blinded to one another, documented key imaging findings and standardized impressions based on SAR-AGA consensus recommendations. Inter-radiologist agreement was evaluated using Fleiss’ multi-rater kappa statistic (κ) with 95% confidence intervals (CI). Results: Inter-radiologist agreement was moderate for all key imaging findings except presence of ulcerations (κ 0.37 [95% CI 0.28–0.46]) and sacculations (κ 0.31 [95% CI 0.23–0.40]). Agreement for standardized impressions was substantial for stricturing disease (κ 0.79 [95% CI 0.70–0.87]) and moderate for presence of inflammation (κ 0.49 [95% CI 0.44–0.56]) and penetrating disease (κ 0.58 [95% CI 0.49–0.67]). No significant difference in agreement was found between CT and MRI. Conclusions: Agreement among five pediatric radiologists was moderate to substantial for SAR-AGA standardized impressions and fair to moderate for key imaging findings of pediatric and young adult CD.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Development of a Computer-Aided Differential Diagnosis System to Distinguish Between Usual Interstitial Pneumonia and Non-specific Interstitial Pneumonia Using Texture- and Shape-Based Hierarchical Classifiers on HRCT Images

A computer-aided differential diagnosis (CADD) system that distinguishes between usual interstitial pneumonia (UIP) and non-specific interstitial pneumonia (NSIP) using high-resolution computed tomography (HRCT) images was developed, and its results compared against the decision of a radiologist. Six local interstitial lung disease patterns in the images were determined, and 900 typical regions of interest were marked by an experienced radiologist. A support vector machine classifier was used to train and label the regions of interest of the lung parenchyma based on the texture and shape characteristics. Based on the regional classifications of the entire lung using HRCT, the distributions and extents of the six regional patterns were characterized through their CADD features. The disease division index of every area fraction combination and the asymmetric index between the left and right lungs were also evaluated. A second SVM classifier was employed to classify the UIP and NSIP, and features were selected through sequential-forward floating feature selection. For the evaluation, 54 HRCT images of UIP (n = 26) and NSIP (n = 28) patients clinically diagnosed by a pulmonologist were included and evaluated. The classification accuracy was measured based on a fivefold cross-validation with 20 repetitions using random shuffling. For comparison, thoracic radiologists assessed each case using HRCT images without clinical information or diagnosis. The accuracies of the radiologists’ decisions were 75 and 87%. The accuracies of the CADD system using different features ranged from 70 to 81%. Finally, the accuracy of the proposed CADD system after sequential-forward feature selection was 91%.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy

Austenitic 347H stainless steel offers superior mechanical properties and corrosion resistance required for extreme operating conditions such as high temperature. The change in microstructure due to composition and process variations is expected to impact material properties. Identifying microstructural features such as grain boundaries thus becomes an important task in the process-microstructure-properties loop. Applying convolutional neural network (CNN)-based deep learning models is a powerful technique to detect features from material micrographs in an automated manner. In contrast to microstructural classification, supervised CNN models for segmentation tasks require pixel-wise annotation labels. However, manual labeling of the images for the segmentation task poses a major bottleneck for generating training data and labels in a reliable and reproducible way within a reasonable timeframe. Microstructural characterization especially needs to be expedited for faster material discovery by changing alloy compositions. Here, in this study, we attempt to overcome such limitations by utilizing multimodal microscopy to generate labels directly instead of manual labeling. We combine scanning electron microscopy images of 347H stainless steel as training data and electron backscatter diffraction micrographs as pixel-wise labels for grain boundary detection as a semantic segmentation task. The viability of our method is evaluated by considering a set of deep CNN architectures. We demonstrate that despite producing instrumentation drift during data collection between two modes of microscopy, this method performs comparably to similar segmentation tasks that used manual labeling. Additionally, we find that naïve pixel-wise segmentation results in small gaps and missing boundaries in the predicted grain boundary map. By incorporating topological information during model training, the connectivity of the grain boundary network and segmentation performance is improved. Finally, our approach is validated by accurate computation on downstream tasks of predicting the underlying grain morphology distributions which are the ultimate quantities of interest for microstructural characterization.

36 MATERIALS SCIENCE↗

A differentiable approach to the maximum independent set problem using dataless neural networks

The success of machine learning solutions for reasoning about discrete structures has brought attention to its adoption within combinatorial optimization algorithms. Such approaches generally rely on supervised learning by leveraging datasets of the combinatorial structures of interest drawn from some distribution of problem instances. Reinforcement learning has also been employed to find such structures. Here, in this paper, we propose a different approach in that no data is required for training the neural networks that produce the solution. In this sense, what we present is not a machine learning solution, but rather one that is dependent on neural networks and where backpropagation is applied to a loss function defined by the structure of the neural network architecture as opposed to a training dataset. In particular, we reduce the popular combinatorial optimization problem of finding a maximum independent set to a neural network and employ a dataless training scheme to refine the parameters of the network such that those parameters yield the structure of interest. Additionally, we propose a universal graph reduction procedure to handle large-scale graphs. The reduction exploits community detection for graph partitioning and is applicable to any graph type and/or density. Experimental results on both real and synthetic graphs demonstrate that our proposed method performs on par or outperforms state-of-the-art learning-based methods in terms of the size of the found set without requiring any training data.

97 MATHEMATICS AND COMPUTING↗

Neural Network‐Based Methods for Ocean Surface Wave Measurement Using Submarine Distributed Acoustic Sensing (DAS)

Two new data-driven models for estimating ocean surface waves from distributed acoustic sensing (DAS) submarine cable strain rate are developed using supervised machine learning on a 10-day data set collected offshore of Oliktok Point, Alaska. The new models were trained on target data from seafloor pressure moorings at three sites spaced evenly along 27.1 km of cable and were benchmarked against an empirical transfer function method previously used to estimate waves from DAS. A model which uses convolutional neural networks to transform 2-km frequency-wavenumber strain spectra to seafloor pressure spectra outperforms the benchmark in wave height prediction (RMSE of 0.15 vs. 0.41 m) and period prediction (0.29 vs. 0.37 s) when evaluated on a held-out test data set. When applied to a DAS data set collected on the same cable 2 years prior, the CNN-based model maintained similar significant wave height performance (RMSE = 0.23 m) relative to available satellite altimetry data. A two-hidden-layer, fully connected neural network which transforms 1-D strain spectra to seafloor pressure spectra also outperforms the benchmark in wave height prediction (RMSE of 0.19 vs. 0.41 m), but does not generalize as well to the prior data. Regression-based machine learning is useful for estimating waves from DAS data when the pressure-strain relationship varies temporally and spatially across different wave conditions. Models can be applied to DAS data to measure waves with higher spatial resolution and longer temporal coverage than traditional methods, which often measure waves only at a single point.

Davis, Jacob R. [Univ. of Washington, Seattle, WA ↗

All-optical phase conjugation using diffractive wavefront processing

Abstract Optical phase conjugation (OPC) is a nonlinear technique used for counteracting wavefront distortions, with applications ranging from imaging to beam focusing. Here, we present a diffractive wavefront processor to approximate all-optical phase conjugation. Leveraging deep learning, a set of diffractive layers was optimized to all-optically process an arbitrary phase-aberrated input field, producing an output field with a phase distribution that is the conjugate of the input wave. We experimentally validated this wavefront processor by 3D-fabricating diffractive layers and performing OPC on phase distortions never seen during training. Employing terahertz radiation, our diffractive processor successfully performed OPC through a shallow volume that axially spans tens of wavelengths. We also created a diffractive phase-conjugate mirror by combining deep learning-optimized diffractive layers with a standard mirror. Given its compact, passive and multi-wavelength nature, this diffractive wavefront processor can be used for various applications, e.g., turbidity suppression and aberration correction across different spectral bands.

42 ENGINEERING↗

Deep Learning for Intelligent Bubble Size Detection in the Spallation Neutron Source Visual Target

The Spallation Neutron Source (SNS) at Oak Ridge National Laboratory (ORNL) will undergo proton power upgrade (PPU), increasing the proton beam power from 1.4 MW to 2.8 MW. From 2.8 MW, 2.0 MW will go to the current First Target Station and the rest will go to the future Second Target Station (STS). The First Target Station uses a liquid mercury target that is contained in a 316L stainless steel vessel. The proton beam is pulsed at 60 Hz, with a pulse of about 0.7μs. When the proton beam hits the target, the intense energy deposition leads to a rapid rise in temperature in the mercury. This temperature rise creates pressure waves that propagate through the mercury and cause cavitation erosion. The power upgrade will cause stronger pressure waves that will further increase damage because of cavitation. Injecting small helium bubbles in the mercury has been an efficient method of mitigating the pressure wave at 1.4 MW. However, at higher power, additional mitigation is necessary. Therefore, the 2 MW target vessel will be equipped with swirl bubblers and an additional gas injection port near the nose to inject more gas in the target. To develop a gas injection strategy and design, flow visualization in water with a transparent prototypical target (“visual target”) was performed. Bubble sizes and their spatial distribution in the flow loop are crucial to understanding the effectiveness of the bubbles in mitigating pressure waves. Bubbles were generated in the visual target under varied conditions of input pressures with helium and air. Images were captured using a high-speed camera at varied frame rates at different positions away from the swirl bubbler and different depths in the flow loop under varying lighting conditions. Initially, methods such as circular Hough transforms were applied after a series of images processing to obtain a general distribution of bubble sizes. Bubbles smaller than 500 μm are preferred to effectively mitigate the effect of pressure waves, which demands an accurate bubble detection and sizing system. Intelligent detection and identification of bubble sizes alleviate misdetection and improves accuracies. Employing neural networks, intelligent detection of bubble sizes and their distribution was developed and provides a robust alternative to traditional techniques. Human intervention was employed to label in-focus and out-of-focus bubbles in the set of training images. An object detection network using a pretrained convolutional neural network was created that extracted the features from the training images. Data augmentation was used to improve network accuracy through a random transformation of the original data.

Rasheed, Fayaz↗

NREL’s Cyber-Energy Emulation Platform for Research and System Visualization

This paper presents NREL’s Cyber Energy Emulation (CEE) Platform. This is a novel emulation platform for achieving real-time visualization of large-scale environments involving cyber-physical devices. It allows for the environment to include real, physical hardware, along with emulated devices communicating with each other as part of the same system. The CEE Platform is also capable of streaming, collecting, storing, transporting, and visualizing all data within the emulated environment. By providing this capability, it enables high-fidelity visual analysis of events to be performed in real time as well as the use of historical data for forensic analysis. This paper presents the design of the CEE Platform as well as several potential use cases and applications. It also aims to highlight why this type of visualization tool has potential for research and education about cyber-physical systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Inference-Optimized AI and High Performance Computing for Gravitational Wave Detection at Scale

We introduce an ensemble of artificial intelligence models for gravitational wave detection that we trained in the Summit supercomputer using 32 nodes, equivalent to 192 NVIDIA V100 GPUs, within 2 h. Once fully trained, we optimized these models for accelerated inference using NVIDIA TensorRT. We deployed our inference-optimized AI ensemble in the ThetaGPU supercomputer at Argonne Leadership Computer Facility to conduct distributed inference. Using the entire ThetaGPU supercomputer, consisting of 20 nodes each of which has 8 NVIDIA A100 Tensor Core GPUs and 2 AMD Rome CPUs, our NVIDIA TensorRT-optimized AI ensemble processed an entire month of advanced LIGO data (including Hanford and Livingston data streams) within 50 s. Our inference-optimized AI ensemble retains the same sensitivity of traditional AI models, namely, it identifies all known binary black hole mergers previously identified in this advanced LIGO dataset and reports no misclassifications, while also providing a 3X inference speedup compared to traditional artificial intelligence models. We used time slides to quantify the performance of our AI ensemble to process up to 5 years worth of advanced LIGO data. In this synthetically enhanced dataset, our AI ensemble reports an average of one misclassification for every month of searched advanced LIGO data. We also present the receiver operating characteristic curve of our AI ensemble using this 5 year long advanced LIGO dataset. This approach provides the required tools to conduct accelerated, AI-driven gravitational wave detection at scale.

97 MATHEMATICS AND COMPUTING↗

ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning

The resilience of safety-critical systems is gaining importance due to the rise in cyber and physical threats, especially within critical infrastructure. Traditional static resilience metrics may not capture dynamic system states, leading to inaccurate assessments and ineffective responses to cyber threats. This work aims to develop a data-driven, adaptive method for resilience metric learning. We propose a data-driven approach using inverse reinforcement learning (IRL) to learn a single, adaptive resilience metric. The method infers a reward function from expert control actions. Unlike previous approaches using static weights or fuzzy logic, this work applies adversarial inverse reinforcement learning (AIRL), training a generator and discriminator in parallel to learn the reward structure and derive an optimal policy. The proposed approach is evaluated on multiple scenarios: optimal communication network rerouting, power distribution network reconfiguration, and cyber–physical restoration of critical loads using the IEEE 123-bus system. The adaptive, learned resilience metric enables faster critical load restoration in comparison to conventional RL approaches.

97 MATHEMATICS AND COMPUTING↗

Restoring Critical Loads In Resilient Distribution Systems using A Curriculum Learned Controller

In this paper, we propose a curriculum learned reinforcement learning (RL) controller to facilitate distribution system critical load restoration (CLR), leveraging RL's fast online response and its outstanding optimal sequential control capability. Like many grid control problems, CLR is complicated due to the large control action space and renewable uncertainty in a heavily constrained non-linear environment with strong intertemporal dependency. The nature of the problem oftentimes causes the RL policy to converge to a poor-performing local optimum if learned directly. To overcome this, we design a two-stage curriculum in which the RL agent will learn generation control and load restoration decision under different scenarios progressively. Via curriculum learning, the trained RL controller is expected to achieve a better control performance, with critical loads restored as rapidly and reliably as possible. Using the IEEE 13-bus test system, we illustrate the performance of the RL controller trained by the proposed curriculum-based method.

curriculum learning↗

Learning Spectral Templates for Photometric Redshift Estimation from Broadband Photometry

Estimating redshifts from broadband photometry is often limited by how accurately we can map the colors of galaxies to an underlying spectral template. Current techniques utilize spectrophotometric samples of galaxies or spectra derived from spectral synthesis models. Both of these approaches have their limitations: either the sample sizes are small and often not representative of the diversity of galaxy colors, or the model colors can be biased (often as a function of wavelength), which introduces systematics in the derived redshifts. In this paper, we learn the underlying spectral energy distributions from an ensemble of ∼100 K galaxies with measured redshifts and colors. We show that we are able to reconstruct emission and absorption lines at a significantly higher resolution than the broadband filters used to measure the photometry for a sample of 20 spectral templates. We find that our training algorithm reduces the fraction of outliers in the derived photometric redshifts by up to 28%, bias up to 91%, and scatter up to 25%, when compared to estimates using a standard set of spectral templates. We discuss the current limitations of this approach and its applicability for recovering the underlying properties of galaxies. Our derived templates and the code used to produce these results are publicly available in a dedicated Github repository: https://github.com/dirac-institute/photoz-template-learning.

79 ASTRONOMY AND ASTROPHYSICS↗

Winter QPF Sensitivities to Snow Parameterizations and Comparisons to NASA CloudSat Observations

Steady increases in computing power have allowed for numerical weather prediction models to be initialized and run at high spatial resolution, permitting a transition from larger scale parameterizations of the effects of clouds and precipitation to the simulation of specific microphysical processes and hydrometeor size distributions. Although still relatively coarse in comparison to true cloud resolving models, these high resolution forecasts (on the order of 4 km or less) have demonstrated value in the prediction of severe storm mode and evolution and are being explored for use in winter weather events . Several single-moment bulk water microphysics schemes are available within the latest release of the Weather Research and Forecast (WRF) model suite, including the NASA Goddard Cumulus Ensemble, which incorporate some assumptions in the size distribution of a small number of hydrometeor classes in order to predict their evolution, advection and precipitation within the forecast domain. Although many of these schemes produce similar forecasts of events on the synoptic scale, there are often significant details regarding precipitation and cloud cover, as well as the distribution of water mass among the constituent hydrometeor classes. Unfortunately, validating data for cloud resolving model simulations are sparse. Field campaigns require in-cloud measurements of hydrometeors from aircraft in coordination with extensive and coincident ground based measurements. Radar remote sensing is utilized to detect the spatial coverage and structure of precipitation. Here, two radar systems characterize the structure of winter precipitation for comparison to equivalent features within a forecast model: a 3 GHz, Weather Surveillance Radar-1988 Doppler (WSR-88D) based in Omaha, Nebraska, and the 94 GHz NASA CloudSat Cloud Profiling Radar, a spaceborne instrument and member of the afternoon or "A-Train" of polar orbiting satellites tasked with cataloguing global cloud characteristics. Each system provides a unique perspective. The WSR-88D operates in a surveillance mode, sampling cloud volumes of Rayleigh scatterers where reflectivity is proportional to the sixth moment of the size distribution of equivalent spheres. The CloudSat radar provides enhanced sensitivity to smaller cloud ice crystals aloft, as well as consistent vertical profiles along each orbit. However, CloudSat reflectivity signatures are complicated somewhat by resonant Mie scattering effects and significant attenuation in the presence of cloud or rain water. Here, both radar systems are applied to a case of light to moderate snowfall within the warm frontal zone of a cold season, synoptic scale storm. Radars allow for an evaluation of the accuracy of a single-moment scheme in replicating precipitation structures, based on the bulk statistical properties of precipitation as suggested by reflectivity signatures.

Molthan, Andrew↗

Ion Distribution and Cation Exchange at Mica–Electrolyte Interfaces Probed with Deep Potential Molecular Dynamics

Here, we investigate the Stern layer structure and cation exchange mechanism at muscovite mica-electrolyte interfaces using nanosecond timescale molecular dynamics simulations based on deep neural network interatomic potentials trained on Density Functional Theory (DFT) data. Focusing on mica with exposed surface K + interfaced with aqueous NaCl and mica with surface Na + interfaced with KCl solution, we find that K + remains predominantly in inner-sphere configurations, while Na + exhibits notable populations in outer-sphere states. Most importantly, our simulations show that contact with an electrolyte solution results in the co-adsorption of multiple cation species, making the mica surface locally overcharged and thus reshaping the cation speciation in a manner that enhances the tendency of neighboring surface cations to desorb. These findings are consistent with recent experimental observations that co-adsorption of different cation species induces changes in cation speciation and slow kinetics of cation exchange at the muscovite-water interface, providing a basis for their detailed understanding.

36 MATERIALS SCIENCE↗

HARMONY: Large-Scale Architecture Search for Efficient Hybrid Language Models

As large language models scale to trillions of parameters, their computational and memory requirements present critical challenges for efficient training and deployment. While Mixture of Experts (MoE) architectures enable efficient scaling through sparse parameter activation, and state-space models like Mamba offer linear-time complexity, principled methods for combining these paradigms remain undeveloped. We introduce HARMONY (Hybrid Architecture Research for Mamba, Optimized with Neural efficiencY), a multi-objective evolutionary neural architecture search framework for discovering efficient hybrid language models that integrate Transformer attention mechanisms, Mixture-of-Experts routing, and Mamba state-space components. Through large-scale distributed search using 16,384 MI250X GPUs on the Frontier supercomputer, HARMONY explores a comprehensive design space encompassing six attention variants (MHA, MQA, GQA, MLA, SWA, and Mamba-2), variable MoE configurations with both routed and shared experts, and extensive Mamba hyperparameters. Our framework discovers heterogeneous architectures that balance training performance with computational efficiency through multi-objective optimization incorporating latency penalties and fitness-based selection. Analysis of discovered architectures reveals that optimal hybrid designs favor heterogeneous component mixing rather than homogeneous patterns, with Mamba-2 and Multi-Head Latent Attention (MLA) emerging as preferred mechanisms. Discovered architectures demonstrate superior training efficiency: our best configuration achieves a final perplexity of 1.0874 with 2.38B parameters while processing 4,320 tokens/second, outperforming significantly larger manually designed models. Full-scale evaluation shows HARMONY's top architectures achieve better loss trajectories than equivalently-sized models using state-of-the-art configurations including Mixtral, Jamba, and Samba. Additionally, we demonstrate 91% weak scaling efficiency when training discovered 36B-parameter models across 1,024 GPUs. HARMONY is released as an open framework with comprehensive tools for building and training hybrid models using expert-data-pipeline parallelism, democratizing access to automated architecture design for next-generation language models.

Herron, Emily [ORNL] (ORCID:0000000273008172)↗