Search NASA⌕ Search

Engineering topics

Evans, Katherine J.

Publications and source records attributed to Evans, Katherine J..

At least 19 records

An implicit barotropic mode solver for MPAS-ocean using a modern Fortran solver interface

Here, we demonstrate use of a modern Fortran solver interface to manage solver algorithms for an implicit barotropic mode solver in the Model for Predictions Across Scales-Ocean (MPAS-O). ForTrilinos, a Fortran interface to Trilinos that contains a large collection of solver capabilities written in C++, has been implemented in MPAS-O to provide access to a suite of linear solver options. By virtue of the simplified wrapper and interface generator (SWIG) automation tool that generates modern Fortran interfaces to C++ code, we were able to implement the Fortran solver interface in MPAS-O using a familiar Fortran coding style while minimizing performance degradation. The ForTrilinos solver interface is written within MPAS-O’s time stepping modules as a subroutine in conjunction with MPAS-O code. Applied to an idealized ocean and a high-resolution realistic ocean test case, parallel performance of ForTrilinos solvers is examined. It is found that parallel scalability of the ForTrilinos solvers is highly dependent on the number of global synchronization points per solver iteration in each iterative solver algorithm. ForTrilinos solvers perform best compared to the Fortran hand-crafted (FHC) solver when the amount of work per processor is large enough. However, parallel scalability is better with the FHC solver and so when the work per core is modest FHC outperforms ForTrilinos. The intercomparison between the ForTrilinos and FHC solvers reveals that this performance hit in the ForTrilinos solver mostly comes from the global synchronization process, while suggesting that the matrix-vector multiplication process in the FHC solver needs to be optimized for better performance.

97 MATHEMATICS AND COMPUTING↗

The influence of natural variability on extreme monsoons in Pakistan

The monsoons in Pakistan have been exceptionally harsh in recent decades, resulting in extraordinary drought conditions and record flooding events. The changing characteristics of extreme events are widely attributed to climate change. However, given this region’s long history of floods and droughts, the role of natural climate variability cannot be rejected without a careful diagnosis. Here, we examine how oceanic and atmospheric variability has contributed to unusual precipitation distributions in West South Asia. Variations in sea surface temperatures in the tropical Pacific and northern Arabian Sea, and internal atmospheric variability related to the circumglobal teleconnection pattern and the subtropical westerly jet stream, explain more than 70% of monthly summer precipitation variability in the 21st century. Several of these forcings have co-occurred with record strength during episodes of extreme monsoons, which have exacerbated the overall effect. Climate change may have contributed to increased variability and the in-phase co-occurrences of the identified mechanisms, but further research is required to confirm any such connection.

54 ENVIRONMENTAL SCIENCES↗

A Scalable Semi-Implicit Barotropic Mode Solver for the MPAS-Ocean

A scalable semi-implicit barotropic mode solver for the ocean component of the model for prediction across scales has been implemented as a competitor to an existing explicit-subcycling scheme to allow faster and more stable simulations while not sacrificing accuracy. The semi-implicit solver adopts the pipelined preconditioned bi-conjugate gradient stabilization algorithm as an iterative solver in conjunction with the restricted additive Schwarz preconditioner that accelerates the convergence rate of the iterative solver. The preconditioner is constructed from a linearized barotropic system that also reorders the system for optimal performance, while the semi-implicit solver deals with the fully nonlinear barotropic system that requires reassembly of the coefficient matrix for every time step. Several numerical experiments, from simple one-dimensional tests to three-dimensional real-world tests, demonstrate that the semi-implicit solver has almost the same accuracy and better parallel scalability compared with the existing scheme while allowing faster and more stable simulations. Furthermore, the semi-implicit solver accelerates the barotropic mode up to 2.9 times faster than the existing scheme on 16,320 processors, leading to an overall runtime speedup of 1.9.

97 MATHEMATICS AND COMPUTING↗

TRITON: A Multi-GPU open source 2D hydrodynamic flood model

A new open source multi-GPU 2D flood model called TRITON is presented in this work. The model solves the 2D shallow water equations with source terms using a time-explicit first order upwind scheme based on an Augmented Roe's solver that incorporates a careful estimation of bed strengths and a local implicit formulation of friction terms. Here, the scheme is demonstrated to be first order accurate, robust and able to solve for flows under various conditions. TRITON is implemented such that the model effectively utilizes heterogeneous architectures, from single to multiple CPUs and GPUs. Different test cases are shown to illustrate the capabilities and performance of the model, showing promising runtimes for large spatial and temporal scales when leveraging the computer power of GPUs. Under this hardware configuration, communication and input/output subroutines may impact the scalability. The code is developed under an open source license and can be freely downloaded in https://code.ornl.gov/hydro/triton.

2D flood model↗

Simulation of Hurricane Harvey flood event through coupled hydrologic-hydraulic models: Challenges and next steps

Using the 2017 Hurricane Harvey flood event as a test case, this study set up a series of sensitivity analyses to highlight three challenges associated with large-scale flood inundation modeling, including (a) model parameterization, (b) errors in digital elevation models, and (c) effects of reservoir retention. Driven by radar-based hourly rainfall data, a series of hydrologic-hydraulic models including the VIC hydrologic model, RAPID routing model, and Flood2D-GPU hydrodynamic model are set up over Harris County, Texas, to simulate flood inundation and hazards. The results demonstrate the importance of hydrologic parameters in improving flood modeling. For a large flood event such as Hurricane Harvey, the effect of the initial water depths is insignificant. The Manning's n values may increase the peak water depth by ~1%, the flood extents by 65km 2 , and the high danger zone by ~6%. On the contrary, the bathymetry correction factors may reduce the flood extent by ~1.4% and the high-danger zone by ~4%. Reducing the reservoir storage capacity to 1% may increase the flood extent by ~4% and the high-danger zone by ~17%. This study may provide supporting information to guide and prioritize the development of future high-performance computing hydrodynamic large-scale flood simulations.

54 ENVIRONMENTAL SCIENCES↗

Tropical Indian Ocean Mediates ENSO Influence Over Central Southwest Asia During the Wet Season

El Niño–Southern Oscillation (ENSO) modulates wet season (November–April) precipitation over Central Southwest Asia (CSWA), however, intraseasonal characteristics of its influence are largely unknown, which can be important for its subseasonal to seasonal hydroclimate predictability. Here we show that the ENSO-CSWA teleconnection varies intraseasonally and is a combination of direct and indirect positive influences. The direct influence is through a Rossby wave-like pattern in the tail months. The indirect influence is through an atmospheric dipole of diabatic heating anomalies in the tropical Indian Ocean (TIO) as a result of ENSO-forced response, which also generates a Rossby wave-like forcing and persists throughout the wet season. ENSO exerts its strongest influence when both direct and indirect modes are in phase, while the relationship breaks down when the two modes are out of phase. The atmospheric teleconnection through the atmospheric diabatic heating anomalies in the TIO is reproducible in numerical simulations.

54 ENVIRONMENTAL SCIENCES↗

Identification of major moisture sources across the Mediterranean Basin

We employ a Lagrangian based moisture back trajectory method on an ensemble of four reanalysis datasets to provide a comprehensive understanding of moisture sources over the Mediterranean land region (30° N–49.5° N and 9.75° W–61.5° E) at seasonal timescales for 1980–2013 period. Using a source region between 10° S–71.35° N along the latitude and 80° W–84.88° E along the longitude that is subdivided into ten complimentary sub-regions, our analyses is able to backtrack up to > 90% of seasonal precipitation at each grid point within the target region. Our results indicate a significant role of moisture advected from the North Atlantic and Mediterranean Sea, and locally recycled moisture over the target region in shaping the spatial organization of seasonal precipitation. However, a clear east–west contrast is witnessed in determining the relative importance of each of these major moisture sources where the North Atlantic dictates the moisture supply over the western Mediterranean while moisture from Mediterranean Sea and local recycling play a key role over the eastern Mediterranean. Our analyses also demonstrate a major footprint of the North Atlantic Oscillation (NAO) on precipitation variability over the Mediterranean land as dynamic and thermodynamic anomalies during the negative phase of NAO match with those during wet years and vice versa. The findings reported here are generally consistent across the four reanalysis datasets. Overall, this study establishes the relative roles of adjacent and far-off oceanic and terrestrial evaporative sources over the Mediterranean land and should help in understanding the drivers of precipitation variability and change at varying timescales.

54 ENVIRONMENTAL SCIENCES↗

Shift in seasonal climate patterns likely to impact residential energy consumption in the United States

Abstract We develop a highly-resolved ensemble of climate simulations and empirical relationships between weather and household energy consumption to provide one of the most detailed estimates to date for potential climate-driven changes in the United States residential energy demand under the highest greenhouse gas emissions pathway. Our results indicate that more intense and prolonged warm conditions will drive an increase in electricity demand while a shorter and milder cold season will reduce natural gas demand by the mid 21st century. The environmental conditions that favor more cooling degree days in summer and reduced heating degree days in winter are driven by changes in daily maximum temperatures and daily minimum temperatures in the respective seasons. Our results also indicate that climate-driven change can potentially reverse impacts of a projected decrease in rural population on residential energy demand. These projected changes in climate-driven energy demand have implications for future energy planning and management.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

LIVVkit 2.1: automated and extensible ice sheet model validation

A collection of scientific analyses, metrics, and visualizations for robustvalidation of ice sheet models is presented using the Land Ice Verificationand Validation toolkit (LIVVkit), version 2.1, and the LIVVkit Extensionsrepository (LEX), version 0.1. This software collection targets stand-aloneice sheet or coupled Earth system models, and handles datasets and analysesthat require high-performance computing and storage. LIVVkit aims to enableefficient and fully reproducible workflows for postprocessing, analysis, andvisualization of observational and model-derived datasets in a shareableformat, whereby all data, methodologies, and output are distributed to usersfor evaluation. Extending from the initial LIVVkit software framework, wedemonstrate Greenland ice sheet simulation validation metrics using thecoupled Community Earth System Model (CESM) as well as an idealizedstand-alone high-resolution Community Ice Sheet Model, version 2 (CISM2),coupled to the Albany/FELIX velocity solver (CISM-Albany or CISM-A). As oneexample of the capability, LIVVkit analyzes the degree to which modelscapture the surface mass balance (SMB) and identifies potential sources ofbias, using recently available in situ and remotely sensed data ascomparison. Related fields within atmosphere and land surface models, e.g.,surface temperature, radiation, and cloud cover, are also diagnosed. Appliedto the CESM1.0, LIVVkit identifies a positive SMB bias that is focusedlargely around Greenland's southwest region that is due to insufficientablation.

58 GEOSCIENCES↗

Ongoing solution reproducibility of earth system models as they progress toward exascale computing

We present a methodology for solution reproducibility for the Energy Exascale Earth System Model during its ongoing software infrastructure development to prepare for exascale computers. The nonlinear chaotic nature of climate system simulations precludes traditional model verification approaches since machine precision differences—resulting from code refactoring, changes in software environment, and so on—grow exponentially to a different weather state. Here, we leverage the nature of climate as a statistical description of the atmosphere in order to establish model reproducibility. We evaluate the degree to which two-sample equality of distribution tests can confidently detect the change in climate from minor tuning parameter changes on model output variables in order to establish the level of difference that indicates a new climate. To apply this (baselined test), we target a section of the model’s development cycle wherein no intentional science changes have been applied to its source code. We compare an ensemble of short simulations that were conducted using a verified model configuration against a new ensemble with the same configuration but with the latest software infrastructure (Common Infrastructure for Modeling the Earth, CIME5.0), compiler versions, and software libraries. We also compare these against ensemble simulations conducted using the original version of the software infrastructure (CIME4.0) of the earlier model configuration, but with the latest compilers and software libraries, to test the impact of new compilers and libraries in isolation from additional software infrastructure. The two-sample equality of distribution tests indicates that these ensembles indeed represent the same climate.

97 MATHEMATICS AND COMPUTING↗

Northern Hemisphere Blocking in ~25-km-Resolution E3SM v0.3 Atmosphere-Land Simulations

The fidelity of a prerelease Energy Exascale Earth System Model (E3SM) in reproducing atmospheric blocking is evaluated with ~100- and ~25-km horizontal resolution within ensembles of simulations with Atmospheric Modeling Intercomparison Protocol-type active atmosphere and land surface configurations. Here this evaluation is conducted via a hybrid bidimensional blocking index based on geopotential height. The lower-resolution model correctly reproduces the spatial distribution of blocking frequency maxima over the Northern Hemisphere. However, it overestimates the blocking frequency over western North America, the Pacific Northeast, and Eastern Europe regions and largely underestimates blocking frequency over the North Atlantic. The high-resolution model significantly reduces the bias over the North Pacific region, particularly over western North America, but the biases over the North Atlantic blocking sector mostly persist. A diagnosis of the mean flow reveals that the subtropical jet is displaced poleward in the high-resolution model with a jet core speed that is generally more realistic. A time scale analysis of eddies suggests that synoptic eddies associated with the blocking events are increased in the high-resolution configuration. The discrepancy between the models implies that both synoptic and low-frequency eddies play significant roles in determining North Pacific blocking, whereas North Atlantic blocking is mainly driven by low-frequency eddies. The characteristic time scale associated with blocking events is also investigated, and it shows that both model resolutions produce events that are overly persistent. However, the bias in the high-resolution model is reduced, which reflects more efficient energy dispersion with higher resolution.

54 ENVIRONMENTAL SCIENCES↗

Model Resolution Sensitivity of the Simulation of North Atlantic Oscillation Teleconnections to Precipitation Extremes

We evaluate a high-resolution (0.25°), four-member ensemble simulation of the global climate (1979–2005) with the U.S. Department of Energy's Energy Exascale Earth System Model v0.3—forced with observed ocean surface temperatures and sea ice extent—for its ability to represent the North Atlantic Oscillation (NAO) teleconnections to winter precipitation extremes over western Europe. As compared to the low-resolution model (1°), it simulates a stronger impact of NAO on daily precipitation extremes over the western slopes of mountain ranges over southwestern Norway, northwestern United Kingdom, and the Western Balkan states. Precipitation extremes and their linear relationship with NAO are quantified using the generalized extreme value distribution. NAO-dependent large-scale (stratiform) precipitation intensity strengthens in the high-resolution model on seasonal time scales and plays a dominant role during simulated daily precipitation extremes. Improvements in the high-resolution model over these varied-topography regions largely appear to be due to finer resolved scales of motion that amplify NAO-dependent seasonal vertical moisture fluxes and enhance stable condensation. However, the high-resolution model simulates a weaker than observed impact of NAO on extratropical cyclone activity and total precipitable water, generally underperforming the low-resolution model These effects possibly offset the impact of enhanced vertical moisture fluxes on NAO-dependent precipitation extremes in the high-resolution model in these regions. Over the southwestern Iberian peninsula, the high-resolution model underperforms the low-resolution model simulating weaker than observed impact of NAO on precipitation extremes. Finally, this appears to be due to the reduction in total precipitable water despite an increase in NAO-dependent extratropical activity there.

54 ENVIRONMENTAL SCIENCES↗

Characteristics of Bay of Bengal Monsoon Depressions in the 21st Century

The controversy surrounding the observed trends in the frequency of South Asian Monsoon Depressions (MDs)1-4, the association of MDs with the summer season extremes5,6, and a moisture driven future precipitation response in Global Climate Models (GCMs)7,8 have elevated the importance of understanding climate change impacts on the characteristics of MDs. GCMs exhibit a significant relationship between the long-term mean occurrences of MDs and the strength of the background upper (lower) tropospheric meridional (zonal) winds and atmospheric moisture in the core genesis region of MDs. Likewise, the strength of the meridional tropospheric temperature gradient (MTG) in GCMs determines the trajectories of MDs over land. While monsoon dynamics progressively weakens in the future period, atmospheric moisture exhibits a strong increase, limiting the impact of changes in background dynamics on the frequency of MDs. Moreover, the weakening of the future MTG is many times weaker than its inherent underestimation in the GCMs. Hence the 21st century increase in radiative forcing does not significantly reduce the frequency of MDs or impact their trajectories over land in the GCMs. Although more wet extremes are expected to occur during the MD lifecycle, non-depression day extremes dominate the overall increase in extreme precipitation, which may render the monsoon extremes less predictable in the future.

Rastogi, Deeksha↗

Performance analysis of fully explicit and fully implicit solvers within a spectral element shallow-water atmosphere model

In this study, explicit Runge–Kutta methods and implicit multistep methods utilizing a Newton–Krylov nonlinear solver are evaluated for a range of configurations of the shallow-water dynamical core of the spectral element community atmosphere model to evaluate their computational performance. These configurations are designed to explore the attributes of each method under different but relevant model usage scenarios including varied spectral order within an element, static regional refinement, and scaling to the largest problem sizes. This analysis is performed within the shallow-water dynamical core option of a full climate model code base to enable a wealth of simulations for study, with the aim of informing solver development within the more complete hydrostatic dynamical core used for climate research. The limitations and benefits to using explicit versus implicit methods, with different parameters and settings, are discussed in light of the trade-offs with Message Passing Interface (MPI) communication and memory and their inherent efficiency bottlenecks. Given the performance behavior across the configurations analyzed here, the recommendation for future work using the implicit solvers is conditional based on scale separation and the stiffness of the problem. For the regionally refined configurations, the implicit method has about the same efficiency as the explicit method, without considering efficiency gains from a preconditioner. The potential for improvement using a preconditioner is greatest for higher spectral order configurations, where more work is shifted to the linear solver. Finally, initial simulations with OpenACC directives to utilize a Graphics Processing Unit (GPU) when performing function evaluations show improvements locally, and that overall gains are possible with adjustments to data exchanges.

58 GEOSCIENCES↗

Performance Analysis of Large Scale HPC Workflows for Earth System Models

Climate assessment reports, such as the Intergovernmental Panel on Climate Change’s 5th assesment report [IPCC AR5; IPCC et al., 2014] or the United States National Climate Assessment [U.S. Global Change Research Program, 2014], increasingly rely on many-ensemble climate projections from fully-coupled Earth system models (ESM) to asses possible future climate states. ESM participate in a large set of Coupled Model Intercomparson Projects (CMIP) to better understand their models and the climate system, so that they may provide credible climate projections. However, fully participating in these intercomparison exercises represents an extremely large computational personnel burden for modeling centers [Eyring et al., 2016, Williams et al., 2016a]. However, there has been little attention within the climate modeling community paid to the in situ performance of ESMs up until now. Recently a computational performance model intercomparison project (CPMIP) for CMIP6 has begun [Balaji et al., 2017], but there has not yet been an effort to gather, process, and analyze the historical performance data that has been saved within archived log files from previous modeling efforts, nor the development of performance analysis system capable of providing detailed statistical analyses or visualizations of this performance data. Therefore, we’ve begun a joint effort to analyze the historical and current performance data on both GFDL and OLCF modeling systems. Here, we have developed a preliminary performance analysis system to captue, analyize, and visualize end-to-end in situ performance data and applied it to NOAA/GFDL’s post-processing system. Through the course of the project a number of performance improvements have been made, with the steps necessary to handle the expected volume of data (Big Data) likely to come out of the CMIP6 exercises (TB to PB) provided. We have also begun developing a statistical workload model that will allow modeling centers to determined their expected scientific throughput in the context of expected resource (computational and personnel) availability and accounting for typical workflow disruptions. This will allow them to develop a CMIP participation strategy and maximize their level of participation in the exercises. We expect this infrastructure, and the data it produces, to ultimately allow us to identify performance issues and workflow errors that exist across Earth system modeling centers, and possibly HPC modeling centers in general. Center-independent solutions to workflow or performance issues would help alleviate the computational burden for CMIP6+ modeling projects and should realize both labor and cost savings benefits throughout the CMIP community.

54 ENVIRONMENTAL SCIENCES↗

ForTrilinos Design Document

With the development of a Fortran Interface to Trilinos, ForTrilinos, modelers using modern Fortran will be able to provide their codes the capability to use solvers and other capabilities on exascale machines via a straightforward infrastructure that accesses Trilinos. This document outlines what ForTrilinos does and explains briefly how it works. We show it provides a general access to packages via an entry point and uses an xml file from fortran code. With the first release, ForTrilinos will enable Teuchos to take xml parameter lists from Fortran code and set up data structures. It will provide access to linear solvers and eigen solvers. Several examples are provided to illustrate the capabilities in practice. We explain what the user should have already with their code and what Trilinos provides and returns to the Fortran code. We provide information about the build process for ForTrilinos, with a practical example. In future releases, nonlinear solvers, time iteration, advanced preconditioning techniques, and inversion of control (IoC), to enable callbacks to Fortran routines, will be available.

97 MATHEMATICS AND COMPUTING↗

Exploring an Ensemble-Based Approach to Atmospheric Climate Modeling and Testing at Scale

A strict throughput requirement has placed a cap on the degree to which we can depend on the execution of single, long, fine spatial grid simulations to explore global atmospheric climate behavior. Alternatively, running an ensemble of short simulations is computationally more efficient. We test the null hypothesis that the climate statistics of a full-complexity atmospheric model derived from an ensemble of independent short simulation is equivalent to that from an equilibrated long simulation. The climate of short simulation ensembles is statistically distinguishable from that of a long simulation in terms of the distribution of global annual means, largely due to the presence of low-frequency atmospheric intrinsic variability in the long simulation. We also find that model climate statistics of the simulation ensemble are sensitive to the choice of compiler optimizations. While some answer-changing optimization choices do not effect the climate state in terms of mean, variability and extremes, aggressive optimizations can result in significantly different climate states.

54 ENVIRONMENTAL SCIENCES↗