Search NASA⌕ Search

Engineering topics

Schwartz, Peter

Publications and source records attributed to Schwartz, Peter.

At least 19 records

Enabling kilometer-scale E3SM land model simulation over North America: A new integrated framework solution

This study introduces a novel framework designed to enhance the performance, scalability, and portability of the kilometer-scale E3SM Land Model (km-ELM) within the E3SM modeling infrastructure. By seamlessly integrating cutting-edge data tools, we address existing challenges such as slow performance, limited scalability, and difficulties in software integration in current data-driven ELM simulation over large geographic areas. Our innovative approach leverages the KiloCraft data toolkit to generate unified inputs for simulations ranging from a single-cite case, to a 72,083-cell regional case to a continental configuration encompassing 21.6 million land grid cells at a 1 km × 1 km resolution. We conduct extensive strong- and weak-scaling experiments on three state-of-the-art supercomputers, utilizing up to 100,800 CPU cores across 2400 compute nodes to evaluate end-to-end metrics including wall-clock time, simulation-years-per-day (SYPD), initialization costs, and I/O throughput. Our results reveal the land (LND) component’s efficient scaling, demonstrating near-ideal weak scaling and strong-scaling parallel efficiencies reaching up to 87% at 50,400 cores. We confirm portability and reproducibility through bitwise-equivalent outputs across different machines using identical inputs over supported machines. Notably, at extreme scales, we identify I/O as a critical bottleneck and that leads to effective solution with the SCORPIO/ADIOS stack. Collectively, these findings validate the deployment of km-ELM at a continental scale with high parallel efficiency and provide essential guidance on configuration, decomposition, and I/O settings for optimized kilometer-scale land simulations in E3SM. This work emphasizes the innovative design and practical solutions that enhance the operational capabilities of km-ELM, focusing on software performance and scalability while leaving detailed scientific evaluations of simulated land processes for future investigations.

E3SM land model (ELM), km-ELM, scalability, perfor↗

An ultrahigh-resolution E3SM land model simulation framework and its first application to the Seward Peninsula in Alaska

The availability of supercomputers and state-of-science datasets has made it possible to conduct large-scale land simulations at an ultrahigh-resolution. This study reported a computational framework for land surface simulation using the E3SM land model (ELM) at an unprecedented resolution (1 km x 1 km gridcell). The ultrahigh-resolution ELM (uELM) simulation framework includes three parts: (1) high-resolution atmospheric forcing and surface properties dataset generation, (2) massive gridcell-based simulation, and (3) large-scale simulation results analysis. Additionally, we implemented the uELM simulation framework and completed the first 1 km x 1 km terrestrial ecosystem simulation (from 1850 to 2014) over the Seward Peninsula in Alaska (78,000 km 2 ). The experiment contained two phases: a spin-up simulation and a transient simulation, and required five weeks of calculations using 320 cores in a 44-node Linux HPC computer. It created approximately 1.3 TB of data from the transient simulation alone (1850 - present). We selected sample results (monthly and daily simulation outputs) to illustrate the temporal and spatial variations of several variables in high-latitude Arctic ecosystems’ water, energy, and carbon cycles. At last, we summarized the lessons learned and proposed new developments for full-scale uELM simulations over the entire North American continent (approximately 22,000,000 km 2 ).

54 ENVIRONMENTAL SCIENCES↗

Factors Influencing Building Demand Flexibility

The U.S. Department of Energy’s National Roadmap for Grid-interactive Efficient Buildings (GEB) acknowledged that building demand flexibility (DF) is both an important strategy to decarbonizing the buildings sector and an important resource for meeting the changing needs of the electrical grid such as improving grid reliability. However, understanding the complexity and uncertainties in real building field performance of DF strategies is a large gap hindering stakeholders on both grid and buildings side to make investments on deploying such strategies. The research work in this report intended to advance understanding of the variability and influential factors in building demand flexibility. Adding such knowledge based on lab testing results and measured performance data from real buildings is an important contribution. The report uses standardized metrics and methods to quantify DF performance from field-measured DF datasets of two significant building groups of big-box retail and medium office buildings to present the challenge of building DF variability in multiple dimensions. The report presents findings related to how several key factors influence building demand flexibility from implementing a common, cost-effective DF control strategy (i.e., adjusting zone temperatures). The findings are supported by full-scale lab testing, field data analysis and simulation research. The authors also provided application-oriented recommendations to stakeholders such as building aggregators, utility program design professionals, sophisticated building portfolio owners, and more.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Comparing simulated demand flexibility against actual performance in commercial office buildings

Commercial building energy benchmarking has been used as a mechanism to evaluate energy use of a single building over time, relative to other similar buildings, or to simulations of a reference building conforming to various energy standards. Lack of empirical demand flexibility data and consistent flexibility metrics has limited the ability to compare demand flexibility performance with estimated demand flexibility in buildings. In this study, we collected demand response performance data for a total of 831 demand response events from 192 sites as a first step to build such a building demand flexibility dataset, and propose a standard core data schema to consolidate field data from different sources. We also performed parametric simulations of a control strategy called “global temperature adjustment” using commercial office prototype building models. We then compared the simulated demand flexibility performance against the actual data for offices with global temperature adjustment strategy implemented. During demand response events with an average outside air temperature of 34 °C (range 23 °C–42 °C), the measured demand decrease intensity of the demand flexibility metrics were 6.1 watts per square meter (W/m 2 ), 10.0 W/m 2 , 11.1 W/m 2 , 7.1 W/m 2 , and 4.7 W/m 2 for small, small–medium, medium, medium–large, and large office buildings, respectively. Compared to the measured data in medium- and large-size buildings, the simulated demand decrease intensity was 0.7 W/m 2 (17%) lower on average. The discrepancy between simulated and measured peak demand intensities fell within one standard deviation of the mean measured data. Here, the comparison results validate the credibility of simulations in capturing real building data for assessing the technical potential of building demand flexibility.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Developing Ultrahigh-Resolution E3SM Land Model for GPU Systems

Designing and refactoring complex scientific code, such as the E3SM land model (ELM), for new computing architectures is challenging. This paper presents design strategies and technical approaches to develop a data-oriented, GPU-ready ELM model using compiler directives (OpenACC/OpenMP). We first analyze the datatypes and processes in the original ELM code. Then we present design considerations for ultrahigh-resolution ELM (uELM) development for massive GPU systems. These techniques include the global data-oriented simulation workflow, domain partition, code porting and data copy, memory reduction, parallel loop restructure and flattening, and race condition detection. We implemented the first version of uELM using OpenACC targeting the NVidia GPUs in the Summit supercomputer at Oak Ridge National Laboratory. During the implementation, we developed a software tool (named SPEL) to facilitate code generation, verification, and performance tuning using these techniques. The first uELM implementation for Nvidia GPUs on Summit delivered promising results: 1) over 98% of the ELM code was automatically generated and tuned by scripts. Most ELM modules had better computational performances than the original ELM code for CPUs. The GPU-ready uELM is more scalable than the CPU code on fully-loaded Summit nodes. Example profiling results from several modules are also presented to illustrate the performance improvements and race condition detection. The lessons learned and toolkit developed in the study are also suitable for further uELM deployment using OpenMP on the first US exascale computer, Frontier, equipped with AMD CPUs and GPUs.

Schwartz, Peter↗

Towards Ultra-high-resolution E3SM Land Modeling on Exascale Computers

Here we present an ultra-high-resolution E3SM land model (uELM) for high-fidelity land simulations targeting new Exascale computers. After considering modeling infrastructure compatibility and ELM software features, we designed a parallel model for the uELM development targeting hybrid architectures of new US Exascale computers. We also described a function unit test framework to expedite the piece-wise code porting (with compiler directives), verification, and global variable management. Furthermore, in this study, we report an early uELM model development using OpenACC within a function unit test framework on a pre-Exascale computer, demonstrate the performance of a uLEM submodel with a 3.0-time speedup, and summarize the code porting experience regarding global variable handling, deepcopy, memory reduction, and parallel loop reconstruction.

97 MATHEMATICS AND COMPUTING↗

SPEL: Software tool for Porting E3SM Land Model with OpenACC in a Function Unit Test Framework

Most high-end computers adopt hybrid architecture, porting a large-scale scientific code onto accelerators is necessary. The paper presents a generic method for porting large-scale scientific code onto accelerators using compiler directives within a modularized function unit test platform. We have implemented the method and designed a software tool (SPEL) to port the E3SM Land Model (ELM) onto the GPUs in the Summit computer. SPEL automatically generates GPU-ready test modules for all ELM functions, such as CanopyFlux, SoilTemperature, and EcosystemDynamics. SPEL breaks the ELM into a collection of standalone unit test programs for easy code verification and further performance improvement. We further optimize several ELM test modules with advanced techniques, including memory reduction, reconstructed parallel loops, and asynchronous GPU kernel launch. We hope our study will inspire new toolkit developments that expedite large-scale scientific code porting with compiler directives.

Schwartz, Peter↗

Defining and applying an electricity demand flexibility benchmarking metrics framework for grid-interactive efficient commercial buildings

Building demand flexibility (DF) research has recently gained attention. To unlock building DF as a predictable grid resource, we must establish a quantitative understanding of the resource size, performance variability, and predictability based on large empirical datasets. Researchers have proposed various sets of theoretical metrics to measure this performance. Some metrics have been applied to simulation results, but most fall short of exploring the complexities in real building applications. There are practical metrics used in individual demand response field studies but they alone cannot fulfil the job of DF benchmarking across a diverse group of buildings. The electrical grid's geographically diverse and changing nature presents challenges to comparing building DF performance measured under different conditions (i.e., benchmarking DF). To address this challenge, a novel DF benchmarking framework focused on load shedding and shifting is presented; the foundation is a set of simple, proven single-event metrics with attributes describing event conditions. These enable benchmarking and visualization in different dimensions for identifying trends that represent how these attributes influence DF. To test its feasibility and scalability, the DF framework was applied to two case studies of 11 office buildings and 121 big-box retail buildings with demand response participation data. Furthermore, these examples provided a pathway for using both building level benchmarking and aggregation to extract insights into building DF about magnitude, consistency, and influential factors. Potential applications of the framework and real-world values have been identified for grid and building stakeholders.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Developing an ELM Ecosystem Dynamics Model on GPU with OpenACC

Porting a complex scientific code, such as the E3SM land model (ELM), onto a new computing architecture is challenging. The paper presents design strategies and technical approaches to develop an ELM ecosystem dynamics model with compiler directives (OpenACC) on NVIDIA GPUs. The code has been refactored with advanced OpenACC features (such as deepcopy and routine directives) to reduce memory consumption and to increase the levels of parallelism through parallel loop reconstruction and new data structures. As a result, the optimized parallel implementation achieved more than a 140-time speedup (50 ms vs 7600 ms), compared to a naive implementation that uses OpenACC routine directive and parallelizes the code across existing loops on a single NVIDIA V100. On a fully loaded computing node with 44 CPUs and 6 GPUs, the code achieved over a 3.0-times speedup, compared to the original code on the CPU. Furthermore, the memory footprint of the optimized parallel implementation is 300 MB, which is around 15% of the 2.15 GB of memory consumed by a naive implementation. This study is the first effort to develop the ELM component on GPUs efficiently to support ultra-high-resolution land simulations at continental scales.

Schwartz, Peter↗

An evaluation of prognostic factors, oncologic outcomes, and management for primary and recurrent squamous cell carcinoma of the vulva

Objective: To evaluate prognostic factors, outcomes, and management patterns of patients treated for squamous cell carcinoma of the vulva. Methods: One hundred sixty-four women were retrospectively identified with primary squamous cell carcinoma of the vulva treated at our institution between 1/1996–12/2018. Descriptive statistics were performed on patient, tumor, and treatment characteristics. The χ 2 tests and t-tests were used to compare categorical variables and continuous variables, respectively. Recurrence free survival (RFS), overall survival (OS), and disease-specific survival (DSS) were analyzed with Kaplan-Meier estimates, the log-rank test, and Cox proportional hazards. Results: Median follow-up was 52.5 months. Five-year RFS was 67.9%, 60.0%, 42.1%, and 20.0% for stage I–IV, respectively. Five-year DSS was 86.2%, 81.6%, 65.0%, and 42.9% for stage I–IV, respectively. On multivariate analysis, positive margins predicted overall RFS (hazard ratio [HR]=3.55; 95% confidence interval [CI]=1.18–10.73; p=0.025), while presence of lichen sclerosus on pathology (HR=2.78; 95% CI=1.30–5.91; p=0.008) predicted local RFS. OS was predicted by nodal involvement (HR=2.51; 95% CI=1.02–6.13; p=0.043) and positive margins (HR=5.19; 95% CI=2.03–13.26; p=0.001). Adjuvant radiotherapy significantly improved RFS (p=0.016) and DSS (p=0.012) in node-positive patients. Median survival after treatment of local, groin, and pelvic/distant recurrence was 52, 8, and 5 months, respectively. Conclusion: For primary treatment, more conservative surgical approaches can be considered with escalation of treatment in patients with concurrent precursor lesions, positive margins, and/or nodal involvement. Further studies are warranted to improve risk stratification in order to optimize treatment paradigms for vulvar cancer patients.

60 APPLIED LIFE SCIENCES↗

GMLC Survey of Distributed Energy Resource Interconnection and Interoperability Standards

This document reports on the Grid Modernization Laboratory Consortium effort to identify gaps in standards for the interconnection and interoperability of distributed energy resources (DERs). The project extended a 3-year period from 2017 to 2019. Under the work plan, the team identified standards and test procedures related to interconnection and interoperability, grouped under the broad headings of their primary applicable technology domain: automotive, responsive loads, photovoltaic inverters, inverter-based energy storage, machine-based DERs, and microgrids. The team then conducted a gap analysis by comparing the current standards to the expected future requirements needed for specific grid services. Overall, gaps could be any activity needed to harmonize requirements among standards development organizations, minimize conflicting requirements among technology domains, or streamline conformance test procedures.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Developing Flexible, Networked Lighting Control Systems That Reliably Save Energy in California Buildings

An important strategy to meet California's ambitious energy efficiency goals is to use innovative wireless communications, embedded sensors, data analytics and controls to significantly reduce lighting energy use in commercial buildings. This project developed a suite of networked lighting solutions to further this goal. The technologies include a platform for low-cost sensing, distributed intelligence and communications, the “PermaMote,” which is a self-powered sensor and controller for lighting applications. The project team also developed a task ambient daylighting system that integrates sensors with data-driven daylighting control using an open communication interface, called the “Readings-At-Desk” (RAD) system. To address the problem of building occupants being confused about how to operate traditional lighting control systems, the research team created content that could be the basis for a user interface standard for lighting controls. Finally, to address the difficulty of ensuring that advanced lighting control systems actually deliver their promised energy savings, the project team developed a new method for evaluating and specifying lighting systems’ performance. The research team validated these technologies in the laboratory, showing significant lighting energy savings, up to 73% for the PermaMote sensor system from occupancy control and daylight dimming features, compared to the same light source (LED replacement lamps) operated via simple on/off scheduling. The project team also developed a proposed standard lighting data model and user interface elements, which were contributed to the ANSI Lighting Systems Committee (C137) for standardization. Existing data models are incomplete and inconsistent, whereas the lighting-specific data model developed here is clear and comprehensive, to serve as a starting point for creating common, universally agreed upon semantic definitions of key lighting parameters, to promote interoperability. For the task on verifiable performance of lighting systems, the project team developed a more effective metric for capturing the actual energy impact of a lighting system over time — the energy usage intensity (kWh/ft2/year). Three commercial lighting systems were tested in FLEXLAB® using this new metric, and the tests show a wide range in the accuracy of the self-reported energy-use metric, from 0.5% to 28% error compared to direct measurement of lighting energy using dedicated submeters. Overall, the project team estimates that these advanced technologies can reduce California office lighting energy use by 20% (above and beyond normal advanced lighting controls mandated by Title 24), resulting in about 1,600 GWh/year in savings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The Value Proposition for Cost-Effective, Demand Responsive-Enabling, Nonresidential Lighting System Retrofits in California Buildings

Commercial lighting represents a significant potential source of demand response (DR) for the electrical grid, via traditional load shedding and rapid-dispatch (fast-DR) ancillary services when DR is enabled by networked lighting controls (NLCs). However, despite the significant opportunity and a regulatory push, DR-enabled lighting is installed in relatively few buildings because most building owners do not recognize its strong value proposition. Although NLCs can reduce energy bills, optimize facilities, and increase revenue, these co-benefits are not well quantified. This project analyzed lighting DR resources and energy-related co-benefits for commercial buildings in California. Using more than 100,000 individual hourly load profiles, the team forecasted the potential DR resources likely available from commercial lighting in 2025 and estimated revenues available from participation of these DR resources in energy markets. Combining these results with field-study estimates for NLC installation costs and energy savings provided a detailed accounting of site-level cost and energy-related co-benefits by building type from NLC’s DR enablement. In many cases, energy savings alone can deliver significant net value, justifying NLC DR-enabled adoption. Additionally, the study considers the sometimes-larger non-energy benefits (NEBs).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Energy Cost Savings of Systems-Based Building Retrofits: A Study of Three Integrated Lighting Systems in Comparison with Component Based Retrofits

Most building retrofit projects are still component-based in that they typically address only one piece or type of equipment at a time. Systems-based retrofits that seek to address multiple components in an integrated manner have the potential to provide significantly greater energy savings. Lawrence Berkeley National Laboratory (LBNL) partnered with several utilities to develop and evaluate three different integrated retrofit packages involving lighting systems: automated shading with daylight dimming controls, workstation-specific lighting with daylight controls, and task/ambient lighting with plug load occupancy controls. Specifically, our analysis sought to quantify the marginal benefits of these systems (energy savings as well as lighting performance and visual comfort) relative to component based approaches. The analysis was based on a combination of measured performance data from LBNL’s FLEXLAB® test facility and energy simulations. All three systems were compared to a simple fluorescent-to-LED retrofit, which represents the component-based approach. While the simple LED upgrade provides significant lighting energy savings of 63%, the systems yielded energy savings of 81-93%, which equates to additional savings of 49-82% over the simple LED upgrade. All systems tested provided satisfactory visual comfort as well, indicating good potential for market acceptance.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Integrating Demand Response and Distributed Resources in Planning for Large-scale Renewable Energy Integration

The electricity grid is transitioning from a centralized and uncoordinated set of large generators and loads to a framework that also includes decentralized and coordinated "distributed energy resources" (DER). Advances in renewable generation, energy storage, efficiency, and controls technology present a significant opportunity for demand-side investment that is matched to the needs of the future grid infrastructure and operations, but the complex interplay of controls technology and grid operations makes estimating and realizing the potential of DER a significant challenge. Supply curves for conserved energy have long been used to synthesize energy efficiency opportunities for electricity system planners and show how demand side resources compete with building new power plants. We have developed a similar approach for supporting policymakers who now face a range of technology options for DER, with a focus on describing the potential for demand response (DR) to provide flexibility to the grid. We describe our modeling approach using supply curves for demand response across four key dimensions: reshaping with rates, shedding at critical time, shifting to capture renewables, and fast-response "shimmy" to balance the grid. In a California-focused study, we find a significant potential for DR to support the grid, and a need for integration between DR and energy efficiency. The combined efficiency benefits from a better-controlled and commissioned facility can lead to significant reductions in the cost of DR, increasing the quantity that is cost competitive by 5-200%. The benefit stream from DR can alternatively be framed to "buy down" the cost of EE investment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Mobilizing the anti-duck brigade: technology and market pathways for load-shifting demand response in California

With rapidly increasing renewable power generation, the California electricity system is experiencing the impacts of the so-called “duck curve,” with occasional over-generation in the middle of the day, followed by rapid ramping of net demand producing a sharp evening peak. Mitigating this pattern will require novel demand-side management approaches to shifting demand away from high net demand periods and toward high renewables generation periods. In this paper, drawing from more than 200,000 individual residential, commercial, and industrial customers’ hourly smart meter data, we disaggregate load shapes for certain key end-uses and then average the results across a highly granular set of customer segments by sector, building type, demand profile and geographic region, to yield a library of characteristic daily load profiles (“clusters”) for different end-uses and seasons. By categorizing the library into particularly desirable or undesirable load shapes in the context of mitigating the duck curve, we can gain insight into specific technologies, seasonal effects, or site-level attributes that may serve as helpful levers to use in developing future load-shifting programs. Based on these insights, we develop recommendations on technologies and new market structures and incentives that can create a future demand curve that more closely matches renewable generation patterns.

24 POWER TRANSMISSION AND DISTRIBUTION↗