SEARCH · Search NASA

Results for “Common data models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification" Willard et al. (2025).

This data release provides all data and code used in the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025)" to model stream temperature, evaluate, and assess results. The associated manuscript explores the effect of different ensemble construction techniques across different common machine learning (ML) architectures for predictions in unmonitored basins. Modeling was done using long short-term memory (LSTM), gated recurrent unit (GRU), temporal convolution network (TCN), and extreme gradient boosting (XGBoost) models, and stream site coverage spans 1362 locations across the conterminous United States. The ensemble construction techniques investigated include ensemble by random weight initialization, differing hyperparameters, different random subsets of training data, different subselections of input features, different architectures, and Monte Carlo Dropout. The data is organized into these items items:Code repository and data for the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025).Code: stream_temp_ml_regionalization.zip contains the code repositoryData to run the code:- data_dir.zip -- contains all files that should be moved to the "DATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- metadata_dir.zip -- contains all files that should be moved to the "METADATA_DIR" variable defined in the "set_env_vars.sh" script in the code repositoryData produced by the code and used in the paper:- outputs_dir.zip - contains model output and results (outputs_dir/results), model weights (outputs_dir/models), and all other outputs used for the paper including feature importances.To cite this code, please use the following BibTeX or MLA entries:bibtex:@misc{willard2025streamensembles,author = {Jared Willard and Charuleka Varadharajan},title = {Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification"},year = {2024},doi = {10.15485/2527393},publisher = {ESS-DIVE Repository},url = {https://data.ess-dive.lbl.gov/datasets/doi:10.15485/2527393}}MLA: Willard, Jared, et al. Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification". 2025. ESS-DIVE Repository, doi:10.15485/2448016.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Facial Named Entity Recognition by Attention-Based Graph Convolutional Neural Network

In the realm of facial recognition and analysis, the ability to accurately cluster large datasets of facial images stands as a cornerstone for various applications, ranging from security surveillance to user biometric identification. This project evolves a novel approach to facial data clustering by embedding facial images into a high-dimensional vector space using an advanced embedding model trained on separate data and assumes a graph-like structure on the high-dimensional vectors. We find our method works significantly better than common shallow methods.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Addressing genome scale design tradeoffs in Pseudomonas putida for bioconversion of an aromatic carbon source

Genome-scale metabolic models (GSMM) are commonly used to identify gene deletion sets that result in growth coupling and pairing product formation with substrate utilization and can improve strain performance beyond levels typically accessible using traditional strain engineering approaches. However, sustainable feedstocks pose a challenge due to incomplete high-resolution metabolic data for non-canonical carbon sources required to curate GSMM and identify implementable designs. Here we address a four-gene deletion design in the Pseudomonas putida KT2440 strain for the lignin-derived non-sugar carbon source, p-coumarate (p-CA), that proved challenging to implement. We examine the performance of the fully implemented design for p-coumarate to glutamine, a useful biomanufacturing intermediate. In this study glutamine is then converted to indigoidine, an alternative sustainable pigment and a model heterologous product that is commonly used to colorimetrically quantify glutamine concentration. Through proteomics, promoter-variation, and growth characterization of a fully implemented gene deletion design, we provide evidence that aromatic catabolism in the completed design is rate-limited by fumarase hydratase (FUM) enzyme activity in the citrate cycle and requires careful optimization of another fumarate hydratase protein (PP_0897) expression to achieve growth and production. A double sensitivity analysis also confirmed a strict requirement for fumarate hydratase activity in the strain where all genes in the growth coupling design have been implemented. Metabolic cross-feeding experiments were used to examine the impact of complete removal of the fumarase hydratase reaction and revealed an unanticipated nutrient requirement, suggesting additional functions for this enzyme. While a complete implementation of the design was achieved, this study highlights the challenge of completely inactivating metabolic reactions encoded by under-characterized proteins, especially in the context of multi-gene edits.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Physics-tailored machine learning reveals unexpected physics in dusty plasmas

Dusty plasma is a mixture of ions, electrons, and macroscopic charged particles that is commonly found in space and planetary environments. The particles interact through Coulomb forces mediated by the surrounding plasma, and as a result, the effective forces between particles can be nonconservative and nonreciprocal. Machine learning (ML) models are a promising route to learn these complex forces, yet their structure should match the underlying physical constraints to provide useful insight. Here, we demonstrate and experimentally validate an ML approach that incorporates physical intuition to infer force laws in a laboratory dusty plasma. Trained on 3D particle trajectories, the model accounts for inherent symmetries, nonidentical particles, and learns the effective nonreciprocal forces between particles with exquisite accuracy (R 2 > 0.99). We validate the model by inferring particle masses in two independent yet consistent ways. The model’s accuracy enables precise measurements of particle charge and screening length, identifying large deviations from common theoretical assumptions. Our ability to identify unknown physics from experimental data demonstrates how ML-powered approaches can guide new routes of scientific discovery in many-body systems. Furthermore, we anticipate our ML approach to be a starting point for inferring laws from dynamics in a wide range of many-body systems, from colloids to living organisms.

Science & Technology - Other Topics↗

DOE OSTI2025

Hyper Spectral Anomaly Detection

Anomaly detection is a common machine learning (ML) task with growing importance in the fields of imaging, quality assurance, and multiple security related disciplines. Anomaly detection is more difficult than traditional machine learning methods due to the inherent unlabeled nature of the datasets. Existing anomaly detection architectures commonly face challenges with explainability, retaining information related to the relational structure of the data, and false positive rates. Hyperspectral Imaging Anomaly Detection (HSI) is a statistical model that employs vertex and edge weighted graphs to preserve the data’s relationships on different topographical scales. The model is able to generalize from anomaly detection in 2D images to novel datasets related to cyber-security. Furthermore, the use of multi-spectral and other filtering methods results in fewer false positives and increases the explainability of model predictions. When applying HSI to cyber-security datasets, we are able to successfully detect malicious activity with a relatively high degree of accuracy.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Thermal performance and energy consumption validation of an occupied local government office building outfitted with ceiling tile phase change materials

Buildings present an opportunity for energy conservation and the modulation of peak energy demand through controlled Heating, Ventilation, and Air Conditioning (HVAC) energy use. The administrative and office building stock in the United States holds potential to achieve energy and demand savings through retrofits such as insulation, weatherization, and thermal energy storage. Specifically, there is a need to validate passive phase change material (PCM) applications in full scale in aging administrative buildings in the US to evaluate the energy benefits. Aim of this study was to conduct a whole building level thermal and energy validation of an operational building and explore an alternative method for evaluating energy efficiency. To accomplish this, the study employed PCMs in the drop ceiling and carry out an energy audit and on-site measurement of HVAC systems' energy demand and consumption. A full-scale EnergyPlus energy model, modeled by the authors, served as a baseline for evaluation. The results show that calibrated model's envelope temperature measures fall within the accepted errors. HVAC energy simulation results also fall within the accepted errors for monthly and hourly pre- and post- PCM retrofit electricity and natural gas data. The novelty of this study is that it employees energy scales per Heating Degree Hour and Cooling Degree Hour, in contrast to the commonly used Heating Degree Days and Cooling Degree Days as reported in the literature to analyze energy savings. These findings underscore the pivotal role of a calibrated model in assessing the efficacy of a singular energy measure, like a PCM-retrofitted ceiling, in an occupied office building.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2024

SAM Code Enhancements for Modeling of Liquid Metal-Cooled Fast Reactor Concepts

The SAM code is under development and supported by DOE-NE’s Nuclear Energy Advanced Modeling and Simulation program as a modern system-level modeling and simulation tool for advanced non-light water reactor safety analyses. These advanced reactor concepts incorporate novel and improved approaches to achieve safety and economic feasibility. This report summarizes two major efforts in addressing the modeling gaps in SAM for liquid-metal-cooled fast reactor (LMFR) applications, i.e. thermal mixing and stratification phenomena in large pools and corrosion-oxidation of components in flowing lead. A new one-dimensional model for thermal mixing and stratification effects in large pools and enclosures is developed and implemented. Thermal mixing and stratification occur when fluid enters a pool at a temperature different than the bulk fluid itself, a scenario often encountered during transients in pool-type systems. These phenomena are critical for the safety of reactors, impacting phenomena like natural circulation, which is essential for passive cooling. The improved model in SAM addresses limitations of state-of-the-art approaches by combining one-dimensional (1D) channels, representing the coolant jet flow, with lumped-parameter zero-dimensional (0D) pools, representing the rest of coolant in the tank. Energy exchange between the 1D jet and the 0D pools is based on heat transfer correlations calibrated against 3D simulations. It is verified that this model can handle various flow configurations, including hot jets in colder pools, cold jets in hotter pools, and the presence of features like ceilings, free surfaces, and obstacles. Additionally, validation against experimental data demonstrates the ability of the model to capture mixing and stratification effects in a wide range of conditions. The flexibility and improved accuracy of the new model make it a valuable tool for reactor safety analysis, allowing for the simulation of different geometries encountered in advanced reactors. A system-level corrosion modeling capability is developed and implemented in SAM to support Lead Fast Reactor (LFR) development. Although the initial focus of this capability will be on LFR application, this can later be expanded to include other liquid metals such as Lead-Bismuth Eutectic (LBE) and PbLi. This report summarizes the common corrosion mitigation strategies and outlines the progress on implementing and validating a corrosion-oxidation model in SAM. Verification and validation of the corrosion-oxidation portion of the model was performed using analytical solution and measured data from samples tested in the non-isothermal pumped lead loop at IPPE Obninsk. The iron transport and corrosion/precipitation portion of the model was assessed using an analytical model and measured corrosion depths from a natural convection lead loop experiment performed at CEA. It is demonstrated that the model implemented in SAM performed well in these assessments.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2025

Convective shells in the interior of Cepheid variable stars: Overshooting models based on hydrodynamic simulations

Context. Because Cepheid variable stars have long been used as a cosmic benchmark for scaling distances in our Galaxy and beyond, the accuracy of stellar evolution models for Cepheids have wide-reaching effects. However, our understanding of the dynamics in the interiors of these physically complex stars is limited. Aims. Our goal is to provide a detailed multi-dimensional picture of hydrodynamic convection and convective boundary mixing in the interior of Cepheids. Methods. Using the Modules for Experiments in Stellar Astrophysics (MESA), we studied the structure of intermediate-mass stars that cross the instability strip. Then, we performed two-dimensional hydrodynamic simulations of six stars with the fully compressible Multidimensional Stellar Implicit Code (MUSIC). Our simulations did not model the radial pulsations but focused on the interior structure of this family of stars. We developed and applied a new statistical analysis to examine convection and convective boundary mixing in the interior of these stellar simulations. Results. Based on a grid of MESA models, we demonstrated that a common structure for intermediate mass Cepheids includes an interior convective shell as well as a thin outer convective envelope. Using the extreme value theory approach to analyze our MUSIC simulation data, we found that overshooting above the convective shell fills the space between these convectively unstable layers. We developed a new statistical analysis that provides a clearer picture of how overshooting fills this layer; it also allowed us to formulate a detailed comparison between overshooting above and below the convective shell. Our analysis effectively decomposes the overshooting layer into two layers: a weak overshooting layer and a strong overshooting layer. Statistically, this is accomplished by decomposing the strongly non-Gaussian probability density function into a mixture of gamma distributions. Using our mixture model, we showed that the ratio of overshooting lengths above and below the convective shell depends directly on the radial extent of the convective shell as well as its depth in the star. We proposed a new form for the diffusion coefficient that addresses the need for overlapping overshooting layers between convective shells. We introduced the idea of a “super-mixing layer” where overshooting from both the convective shell and the convective envelope results in efficient mixing and could be viewed as merging the two adjacent convective zones.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Enhancing Short-Range Weather Forecasts through Temporal Variation Encoding: A Multiperiod Embedding Approach

Machine learning (ML) techniques have emerged as promising approaches to improve regional weather forecast accuracy and reliability through data-driven methods. We propose a novel ML-based weather forecasting model, the Multiperiod Embed Net (MPENet). A key distinguishing feature of MPENet is its explicit utilization of the inherent cyclic nature in weather dynamics, unlike the autoregressive strategies commonly used in other ML weather forecasting approaches. Critical cyclic structures are identified via Fourier analyses of dynamic time series. Cyclicity in the convolutional representation is achieved by transforming one-dimensional time series of meteorological variables into two-dimensional tensors based on identified periods. This approach enables the model to leverage intrinsic weather patterns, enhancing regional forecast performance. To demonstrate the effectiveness of MPENet, we conduct a comparative analysis with Nvidia’s FourCastNet. Both models are trained on High-Resolution Rapid Refresh (HRRR) data from 2015 to 2022, over a 192 km × 192 km region in Tennessee. The comparisons are performed locally at two specific locations known to have different weather dynamics due to orographic effects: Crossville, on the relatively flat Cumberland Plateau with fewer topographic airflow disruptions, and Oak Ridge, in the ridge-and-valley region, where airflow is heavily influenced by surrounding valleys and mountains. Our results indicate that FourCastNet achieves strong accuracy at very short lead times, while MPENet maintains competitive skill and shows advantages in capturing temporal evolution over longer periods. Cross-correlation analyses of MPENet and FourCastNet predictions with the HRRR data suggest that encoding critical cyclicity into the network architecture leads to improvements in the forecasting skill.

Artificial intelligence↗

DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗

DOE OSTI2026

Data‐Efficient Generation of Synthetic Microstructures of Polymer‐Bonded Energetic Material With Fine‐Tuned Stable Diffusion

Among current deep learning approaches for synthetic image generation, diffusion-based models stand out in terms of algorithmic stability and ability to retain high-fidelity image features with detailed resolution. Here, in this work, we employ Dreambooth, a method for fine-tuning Stable Diffusion, on X-ray CT images of microstructure of the polymer-bonded form (PBX) of a commonly used high explosive, Pentaerythritol tetranitrate (PETN), which yields generative models for creating synthetic PBX images. The models developed here represent five classes (or ‘lots’) of microstructures and demonstrate successful generation of images of each class with high fidelity, as verified by computed classification accuracy of ∼ 94% or higher. Data augmentation afforded by such image synthesis can be used to more reliably decipher underlying statistics, build processing-structure correlations, recognize off-normal structural anomalies, and identify age-related changes. Ideas related to converting image data into appropriate density mapping and performing mesoscale simulation or surrogate modeling of detonation are also discussed.

Dreambooth↗

DOE OSTI2024

phosaa14SB and phosaa19SB: Updated Amber Force Field Parameters for Phosphorylated Amino Acids

Phosphorylated amino acids are involved in many cell regulatory networks; proteins containing these post-translational modifications are widely studied both experimentally and computationally. Simulations are used to investigate a wide range of structural and dynamic properties of biomolecules, such as ligand binding, enzyme-reaction mechanisms, and protein folding. However, the development of force field parameters for the simulation of proteins containing phosphorylated amino acids using the Amber program has not kept pace with the development of parameters for standard amino acids, and it is challenging to model these modified amino acids with accuracy comparable to proteins containing only standard amino acids. In particular, the popular ff14SB and ff19SB models do not contain parameters for phosphorylated amino acids. Here, the dihedral parameters for the side chains of the most common phosphorylated amino acids are trained against reference data from QM calculations adopting the ff14SB approach, followed by validation against experimental data. Finally, library files and corresponding parameter files are provided, with versions that are compatible with both ff14SB and ff19SB.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

DOE OSTI2026

A Generalized Grain-Scale Model for the Non-Plasma and Plasma-Assisted Hydrogen Direct Reduction of Iron Ore

Direct Reduction of Iron ore using hydrogen (H-DRI) is a promising pathway towards efficient steelmaking and accurate predictive models are a necessity for scale-up and optimization of this technology. However, accurate models of this process remain limited because existing models oversimplify grain-scale phenomena, such as nonlinearity inside grain, self-sufficient porosity, surface reactions, and the role of plasma species. These phenomena are important for flash steelmaking and plasma-assisted H-DRI processes. To address this need, we present a phenomenological model for simulating H-DRI at the scale of a single micron-sized grain of the iron ore. We call this the Transient Reactive Grain Model (TRGM). TRGM incorporates key physical process: gas species transport, a chemical kinetics of material conversion, nanopore structural evolution and, adsorption-desorption surface kinetics at the reactive nanopore surface. The important contribution of this work is that the model provides a dependence on different reductant species, specifically hydrogen atoms versus molecules, so that role of hydrogen plasma reduction can be clarified compared to the use of pure hydrogen gas reduction. TRGM predictions agree well with experimental data for both molecular H2 reduction of Fe2O3 and plasma hydrogen reduction of Fe3O4. Results reveal species concentration gradients with a diffuse reaction zone, and enhanced hydrogen diffusion at the grain outer surface due to evolving porosity. These findings challenge common assumptions in existing models, including sharp reaction fronts, quasi-steady diffusion and kinetics, and the neglect of surface chemistry. As a generalized grain-scale model for H-DRI processes, TRGM has practical applications in flash steelmaking and in-flight reduction using both molecular and plasma hydrogen.

08 HYDROGEN↗

DOE OSTI2024

Continual learning in the presence of repetition

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.

Class-incremental learning↗

DOE OSTI2026

Multi-machine validation of plasma initiation modelling and prospects for future devices: Predicting plasma initiation using only hardware design and control room input data

This paper reports on the generic prediction capability of full electromagnetic plasma initiation modelling with DYON, which was carried out for the first time in fusion research by the joint modelling of the International Tokamak Physics Activity—Integrating Operation Scenario group. The following devices were included in the experiment database: VEST (spherical torus, copper coils, Stainless steel wall, R/a = 0.3 m/0.2 m, V v = 3.7 m 3 ), MAST-U (spherical torus, copper coils, C wall, R/a = 0.7 m/0.5 m, V v = 55 m 3 ), EAST (conventional tokamak, superconducting coils, metallic wall, R/a = 1.85 m/0.5 m, V v = 38 m 3 ), DIII-D (conventional tokamak, copper coils, C wall, R/a = 1.67 m/0.65 m, V v = 35 m 3 ), and KSTAR (conventional tokamak, superconducting coils, C wall, R/a = 1.8 m/0.5 m, V v = 55 m 3 ). Despite the different hardware features of the devices, the required operating spaces of the loop voltage induction and prefill gas pressure for inductive plasma initiation in each device were successfully reproduced by the predictive simulations with DYON using only the individual hardware design and the control room input data for each discharge. This successful validation across multiple machines demonstrates that the full electromagnetic DYON modelling can capture the essential physics of inductive plasma initiation. The simulation settings commonly employed for all modelling and the modifications necessary to account for the discrepancies between individual devices are reported. Predictions for ITER based on the multi-machine validation indicate that a wide range of prefill gas pressures exists for the Townsend breakdown and the plasma burn-through (0.01–1.5 mPa).

DYON↗

DOE OSTI2024

Are Atmospheric Models Too Cold in the Mountains? The State of Science and Insights from the SAIL Field Campaign

Mountains play an outsized role in water resource availability, and the amount and timing of water they provide depend strongly on temperature. To that end, we ask the question: How well are atmospheric models capturing mountain temperatures? We synthesize results showing that high-resolution, regionally relevant climate models produce 2-m air temperature (T2m) measurements colder than what is observed (a “cold bias”), particularly in snow-covered midlatitude mountain ranges during winter. We find common cold biases in 44 studies across global mountain ranges, including single-model and multimodel ensembles. We explore the factors driving these biases and examine the physical mechanisms, data limitations, and observational uncertainties behind T2m. Our analysis suggests that the biases are genuine and not due to observation sparsity or resolution mismatches. Cold biases occur primarily on mountain peaks and ridges, whereas valleys are often warm biased. Our literature review suggests that increasing model resolution does not clearly mitigate the bias. By analyzing data from the Surface Atmosphere Integrated Field Laboratory (SAIL) field campaign in the Colorado Rocky Mountains, we test various hypotheses related to cold biases and find that local wind circulations, longwave (LW) radiation, and surface-layer parameterizations contribute to the T2m biases in this particular location. We conclude by emphasizing the value of coordinated model evaluation and development efforts in heavily instrumented mountain locations for addressing the root cause(s) of T2m biases and improving predictive understanding of mountain climates.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Are Atmospheric Models Too Cold in the Mountains? The State of Science and Insights from the SAIL Field Campaign

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelism

Graph neural networks (GNNs), an emerging class of machine learning models for graphs, have gained popularity for their superior performance in various graph analytical tasks. Mini-batch training is commonly used to train GNNs on large graphs, and data parallelism is the standard approach to scale mini-batch training across multiple GPUs. Data parallel approaches contain redundant work as subgraphs sampled by different GPUs contain significant overlap. To address this issue, we introduce a hybrid parallel mini-batch training paradigm called Split parallelism. Split parallelism avoids redundant work by splitting the sampling, loading, and training of each mini-batch across multiple GPUs. Split parallelism, however, introduces communication overheads that can be more than the savings from removing redundant work. We further present a lightweight partitioning algorithm that probabilistically minimizes these overheads. We implement spllit parllelism in GSplit and show that it outperforms state-of-the-art mini-batch training systems like DGL, Quiver, and P3.

Lim, Seung-Hwan [ORNL] (ORCID:0000000194616866)↗