Search NASA⌕ Search

Engineering topics

Wang, Shusen

Publications and source records attributed to Wang, Shusen.

Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning

Although Transformer-based language representations achieve state-of-the-art accuracy on various natural language processing (NLP) tasks, the large model size has been challenging the resource constrained computing platforms. Weight pruning, as a popular and effective technique in reducing the number of weight parameters and accelerating the Transformer, has been investigated on GPUs. However, the Transformer acceleration using weight pruning on field-programmable gate array (FPGAs) remains unexplored. This paper investigates the column balanced block-wise pruning on Transformer and designs an FPGA acceleration engine to customize the balanced blockwise matrix multiplication. We implement the Transformer model with proper hardware scheduling, and the experiments show that the Transformer inference on FPGA achieves 10.35 ms latency with the batch size of 32, which is 10.96 × speed up comparing to CPU platform and 2.08 × speed up comparing to GPU platform.

Peng, Hongwu↗

Efficient Data-Driven Geologic Feature Characterization from Pre-stack Seismic Measurements using Randomized Machine-Learning Algorithm

Conventional seismic techniques for detecting the subsurface geologic features are challenged by limited data coverage, computational inefficiency and subjective human factors. We developed a novel data-driven geological feature characterization approach based on pre-stack seismic measurements. Our characterization method employs an efficient and accurate machine learning method to extract useful subsurface geologic features automatically. Specifically, we use kernel ridge regression to account for the nonlinear relationship between seismic data and geological features. We moreover employ kernel tricks to avoid the explicit nonlinear mapping and infinite dimension of feature space. Yet, the conventional kernel ridge regression can be computationally prohibitive because of the large volume of seismic measurements. We employ a data reduction technique in combination with the conventional kernel ridge regression method to improve the computational efficiency and reduce memory usage. In particular, we utilize a randomized numerical linear algebra technique, named Nyström method, to effectively reduce the dimensionality of the feature space without compromising the information content required for accurate characterization. We provide thorough computational cost analysis to show the efficiency of our new geological feature characterization methods. We validate the performance of our method in characterizing geologic fault zones because faults play an important role in various subsurface applications. Our numerical examples demonstrate that our new characterization method significantly improves the computational efficiency while maintaining comparable accuracy. Interestingly, we demonstrate that our approach yields a speedup ratio on the order of ~102 to ~103 in a multicore computational environment.

58 GEOSCIENCES↗

Comparing evapotranspiration from Eddy covariance measurements, water budgets, remote sensing, and land surface models over Canada a, b

Here, this study compares six evapotranspiration ET products for Canada's landmass, namely, eddy covariance EC measurements; surface water budget ET; remote sensing ET from MODIS; and land surface model (LSM) ET from the Community Land Model (CLM), the Ecological Assimilation of Land and Climate Observations (EALCO) model, and the Variable Infiltration Capacity model (VIC). The ET climatology over the Canadian landmass is characterized and the advantages and limitations of the datasets are discussed. The EC measurements have limited spatial coverage, making it difficult for model validations at the national scale. Water budget ET has the largest uncertainty because of data quality issues with precipitation in mountainous regions and in the north. MODIS ET shows relatively large uncertainty in cold seasons and sparsely vegetated regions. The LSM products cover the entire landmass and exhibit small differences in ET among them. Annual ET from the LSMs ranges from small negative values to over 600 mm across the landmass, with a countrywide average of 256 ± 15 mm. Seasonally, the countrywide average monthly ET varies from a low of about 3 mm in four winter months (November-February) to 67 ± 7 mm in July. The ET uncertainty is scale dependent. Larger regions tend to have smaller uncertainties because of the offset of positive and negative biases within the region. More observation networks and better quality controls are critical to improving ET estimates. Future techniques should also consider a hybrid approach that integrates strengths of the various ET products to help reduce uncertainties in ET estimation.

54 ENVIRONMENTAL SCIENCES↗

Using Ecosystem Experiments to Improve Vegetation Models

Ecosystem responses to rising CO2 concentrations are a major source of uncertainty in climate change projections. Data from ecosystem-scale Free-Air CO2 Enrichment (FACE) experiments provide a unique opportunity to reduce this uncertainty. The recent FACE Model–Data Synthesis project aimed to use the information gathered in two forest FACE experiments to assess and improve land ecosystem models. A new 'assumption-centred' model intercomparison approach was used, in which participating models were evaluated against experimental data based on the ways in which they represent key ecological processes. Identifying and evaluating the main assumptions caused differences among models, and the assumption-centered approach produced a clear roadmap for reducing model uncertainty. We explain this approach and summarize the resulting research agenda. We encourage the application of this approach in other model intercomparison projects to fundamentally improve predictive understanding of the Earth system.

54 ENVIRONMENTAL SCIENCES↗

Comprehensive ecosystem model-experiment synthesis using multiple datasets at two temperate forest free-air CO2 enrichment experiments: model performance and compensating biases

Free Air CO2 Enrichment (FACE) experiments provide a remarkable wealth of data to test the sensitivities of terrestrial ecosystem models (TEMs). In this study, a broad set of 11 TEMs were compared to 22 years of data from two contrasting FACE experiments in temperate forests of the south eastern US the evergreen Duke Forest and the deciduous Oak Ridge forest. We evaluated the models' ability to reproduce observed net primary productivity (NPP), transpiration and Leaf Area index (LAI) in ambient CO2 treatments. Encouragingly, many models simulated annual NPP and transpiration within observed uncertainty. Daily transpiration model errors were often related to errors in leaf area phenology and peak LAI. Our analysis demonstrates that the simulation of LAI often drives the simulation of transpiration and hence there is a need to adopt the most appropriate of hypothesis driven methods to simulate and predict LAI. Of the three competing hypotheses determining peak LAI (1) optimisation to maximise carbon export, (2) increasing SLA with canopy depth and (3) the pipe model the pipe model produced LAI closest to the observations. Modelled phenology was either prescribed or based on broader empirical calibrations to climate. In some cases, simulation accuracy was achieved through compensating biases in component variables. For example, NPP accuracy was sometimes achieved with counter-balancing biases in nitrogen use efficiency and nitrogen uptake. Combined analysis of parallel measurements aides the identification of offsetting biases; without which over-confidence in model abilities to predict ecosystem function may emerge, potentially leading to erroneous predictions of change under future climates.

Walker, Anthony P↗