Search NASA⌕ Search

Engineering topics

Wang, Zheng

Publications and source records attributed to Wang, Zheng.

11 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

OPER: Optimality-Guided Embedding Table Parallelization for Large-scale Recommendation Model

With the sharp increasing volume of user data, Deep Learning Recommendation Model (DLRM) becomes an indispensable infrastructure in large technology companies. However, large-scale DLRM on the multi-GPU platform is still inefficient due to unbalanced workload partitioning and intensive inter-GPU communication. To this end, we propose OPER, an OPtimality guided Embedding table placement for large-scale Recommendation model training and inference. OPER explores the potential of mitigating remote memory access latency in DLRM through fine-grained embedding table placement. Specifically, OPER proposes a theoretical modeling that builds up the relationship between EMT placement and the embedding communication latency in both training and inference. OPER proves the NP hardness of finding the optimal embedding table placement and proposes a heuristic algorithm that yields near optimal placement. OPER implements a SHMEM-based embedding table training system and a unified embedding index mapping to support fine-grained embedding table sharding and placement. Comprehensive experiments reveal that OPER achieves on average 3.4× and 5.1× speedup on training and inference respectively over state-of-the-art DLRM frameworks.

Wang, Zheng↗
DOE OSTI2024

RAP: Resource-aware Automated GPU Sharing for Multi-GPU Recommendation Model Training and Input Preprocessing

Ensuring high-quality recommendations for newly onboarded users requires the continuous retraining of Deep Learning Recommendation Models (DLRMs) with freshly generated data. To serve the online DLRM retraining, existing solutions use hundreds of CPU computing nodes designated for input preprocessing, causing significant power consumption that surpasses even the power usage of GPU trainers. To this end, we propose RAP, an end-to-end DLRM training framework that supports Resource-aware Automated GPU sharing for DLRM input Preprocessing and Training. The core idea of RAP is to accurately capture the remaining GPU computing resources during DLRM training for input preprocessing, achieving superior training efficiency without requiring additional resources. Specifically, RAP utilizes a co-running cost model to efficiently assess the costs of various input preprocessing operations, and it implements a resource-aware horizontal fusion technique that adaptively merges smaller kernels according to GPU availability, circumventing any interference with DLRM training. In addition, RAP leverages a heuristic searching algorithm that jointly optimizes both the input preprocessing graph mapping and the co-running schedule to maximize the end-to-end DLRM training throughput. The comprehensive evaluation shows that RAP achieves 78.3× speedup on average over CPU-based DLRM input preprocessing frameworks. In addition, the end-to-end training throughput of RAP is only 2.04% lower than the ideal case, which has no input preprocessing overhead.

Wang, Zheng↗
DOE OSTI2022

Variation in carbon and nitrogen concentrations among peatland categories at the global scale

Peatlands account for 15 to 30% of the world’s soil carbon (C) stock and are important controls over global nitrogen (N) cycles. However, C and N concentrations are known to vary among peatlands contributing to the uncertainty of global C inventories, but there are few global studies that relate peatland classification to peat chemistry. We analyzed 436 peat cores sampled in 24 countries across six continents and measured C, N, and organic matter (OM) content at three depths down to 70 cm. Sites were distinguished between northern (387) and tropical (49) peatlands and assigned to one of six distinct broadly recognized peatland categories that vary primarily along a pH gradient. Peat C and N concentrations, OM content, and C:N ratios differed significantly among peatland categories, but few differences in chemistry with depth were found within each category. Across all peatlands C and N concentrations in the 10–20 cm layer, were 440 ± 85.1 g kg -1 and 13.9 ± 7.4 g kg -1 , with an average C:N ratio of 30.1 ± 20.8. Among peatland categories, median C concentrations were highest in bogs, poor fens and tropical swamps (446–532 g kg -1 ) and lowest in intermediate and extremely rich fens (375–414 g kg -1 ). The C:OM ratio in peat was similar across most peatland categories, except in deeper samples from ombrotrophic tropical peat swamps that were higher than other peatlands categories. Peat N concentrations and C:N ratios varied approximately two-fold among peatland categories and N concentrations tended to be higher (and C:N lower) in intermediate fens compared with other peatland types. This study reports on a unique data set and demonstrates that differences in peat C and OM concentrations among broadly classified peatland categories are predictable, which can aid future studies that use land cover assessments to refine global peatland C and N stocks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Plutonium reactive transport in fractured granite: Multi-species experiments and simulations

Plutonium (Pu) in the subsurface environment can transport in different oxidation states as an aqueous solute or as colloidal particles. The transport behavior of Pu is affected by the relative abundances of these species and can be difficult to predict when they simultaneously exist. This study investigates the concurrent transport of Pu intrinsic colloids, Pu(IV) (aq) and Pu(V-VI) (aq) through a combination of controlled experiments and semi-analytical dual-porosity transport modeling. Pu transport experiments were conducted in a fractured granite at high and low flow rates to elucidate sorption processes and their scaling behavior. In the experiments, Pu(IV) (aq) was the least mobile of the Pu species, Pu(V-VI) (aq) had intermediate mobility, and the colloidal Pu, which consisted mainly of precipitated and/or hydrolyzed Pu(IV), was the most mobile. The semi-analytical modeling revealed that the sorption of each Pu species was rate-limited, as the sorption could not be described by assuming local equilibrium in the experiments. The model was able to describe the sorption of the different Pu species that occurring either on fracture surfaces, in the pores of the rock matrix, or simultaneously in both locations. While equally good fits to the data could be achieved using any of these assumptions, a fracture-dominated process was considered to be the most plausible because it provided the most reasonable estimates of sorption rate constants. Importantly, a key result of this work is that the sorption rate constant of all Pu species tends to decrease with increasing time scales, which implies that Pu will tend to be more mobile at longer time scales than observations at shorter time scales suggest. This result has important implications for predicting the environmental impacts of Pu in the safety assessments of geologic repositories for radioactive waste disposal, and we explore potential mechanistic bases for upscaling the sorption rate constants to time and distance scales that cannot be practically evaluated in experiments.

58 GEOSCIENCES↗
DOE OSTI2022

Latitude, Elevation, and Mean Annual Temperature Predict Peat Organic Matter Chemistry at a Global Scale

Peatlands contain a significant fraction of global soil carbon, but how these reservoirs will respond to the changing climate is still relatively unknown. A global picture of the variations in peat organic matter chemistry will aid our ability to gauge peatland soil response to climate. Here, the goal of this research is to test the hypotheses that (a) peat carbohydrate content, an indicator of soil organic matter reactivity, will increase with latitude and decrease with mean annual temperatures, (b) while peat aromatic content, an indicator of recalcitrance, will vary inversely, and (c) elevation will have a similar effect to latitude. We used Fourier Transform Infrared Spectroscopy to examine variations in the organic matter functional groups of 1034 peat samples collected from 10 to 20, 30–40, and 60–70 cm depths at 165 individual sites across a latitudinal gradient of 79°N–65°S and from elevations of 0–4,773 m. Carbohydrate contents of high latitude peat were significantly greater than peat originating near the equator, while aromatic content showed the opposite trend. For peat from similar latitudes but different elevations, the carbohydrate content was greater and aromatic content was lower at higher elevations. Higher carbohydrate content at higher latitudes indicates a greater potential for mineralization, whereas the chemical composition of low latitude peat is consistent with their apparent relative stability in the face of warmer temperatures. The combination of low carbohydrates and high aromatics at warmer locations near the equator suggests the mineralization of high latitude peat until reaching recalcitrance under a new temperature regime.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Atomic-Scale Imaging of Polarization Switching in an (Anti-)Ferroelectric Memory Material: Zirconia (ZrO2)

Direct, atomic-scale visualization of polarization switching in a functional, polycrystalline, binary oxide via in-situ high-resolution transmission electron microscopy (HRTEM) biasing is reported for the first time. Antiferroelectric (AFE) ZrO2 was used as the model system, which is important for commercial DRAMs and as emerging NVMs (through work-function engineering). We observed (1) clear shifting and coalescing of domains within a single grain, and (2) dramatic changes of the atomic arrangements and crystalline phases—both at voltages above the critical voltage measured for AFE switching. Similar synergistic in-situ structural-electrical characterization can pave the way to understand and engineer microscopic mechanisms for retention, fatigue, variability, sub-coercive switching and analog states in ferroelectric and AFE-based memory devices.

Lombardo, Sarah↗
DOE OSTI2019

Catalysts in Coronas: A Surface Spatial Confinement Strategy for High-Performance Catalysts in Methane Dry Reforming

One of the grand challenges in industrial catalytic processes is the inevitable sintering and aggregation of conventional supported catalysts to large particles, leading to the decrease of activity and even deactivation with time. Herein, a surface spatial confinement strategy was employed to design high-performing catalysts for the dry reforming of methane (DRM). Specifically, active nickel (Ni) nanoparticles (NPs) were confined on the surface of a dendritic mesoporous silica (DMS) in the form of the “catalysts in coronas”. The Ni/DMS catalyst exhibited a high catalytic performance close to its equilibrium conversion (76% conversion for CH4 at 700 °C). More importantly, the prepared catalyst remained stable after 145 h time-on-stream at 700 °C without noticeable carbon deposition. This sintering and coking resistance was found to arise from the surface spatial confinement effect in which the three-dimensional dendritic layers in the corona posted a steric barrier against migration and aggregation of Ni NPs and size of Ni NPs was controlled below 5 nm, hence against sintering and coking. Meanwhile, the mesoporous feature of the layered wall facilitated mass transport of reactants to Ni species and further boosted catalysis. This strategy should be broadly applicable to a range of metal- and metal oxide-supported catalysts in high-temperature heterogeneous reactions, such as DRM, water gas shift reaction, and vehicle emission control related reactions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

Active and stable Pt-Ceria nanowires@silica shell catalyst: Design, formation mechanism and total oxidation of CO and toluene

Cerium oxide is one of the most important rare earth metal oxides in catalysis, however, the sintering problem of noble metals and CeO 2 at higher temperature (e.g., >700 °C) is still unresolved. Herein, Pt nanoparticles self-assembled on ultra-thin CeO 2 nanowires (NWs) and then confined inside a thermally robust porous silica shell (Pt-CeO2NW@SiO 2 ) were introduced. The thickness of CeO 2 NWs was just ~2.0 nm. Moreover, Pt-CeO 2 NW@SiO 2 showed significantly enhanced catalytic performances for total oxidation of CO and toluene. The increased catalytic properties are attributed to the strong metal-support interaction effect between Pt and CeO 2 NWs at sub-nanoscale. Most importantly, the special core-shell structure also affords excellent sintering resistance retention up to 700 °C for 100 h, due to the guarding effect of porous silica shell. Finally, the formation mechanism of Pt-CeO 2 NW@SiO 2 was investigated in detail. Current strategy should inspire many rational designs of rare-earth metal-based nanocatalysts for real-world catalysts.

36 MATERIALS SCIENCE↗