Codesign for Extreme Heterogeneity: Integrating Custom Hardware With Commodity Computing Technology to Support Next-Generation HPC Converged Workloads
Not Available
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Not Available
Data provenance provides a way for scientists to observe how experimental data originates, conveys process history, and explains influential factors such as experimental rationale and associated environmental factors from system metrics measured at runtime. The US Department of Energy Office of Science Integrated end-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) project has developed a provenance harvester that is capable of collecting observations from file based evidence typically produced by distributed applications. To achieve this, file based evidence is extracted and transformed into an intermediate data format inspired in part by W3C CSV on the Web recommendations, called the Harvester Provenance Application Interface (HAPI) syntax. This syntax provides a general means to pre-stage provenance into messages that are both human readable and capable of being written to a provenance store, Provenance Environment (ProvEn). HAPI is being applied to harvest provenance from climate ensemble runs for Accelerated Climate Modeling for Energy (ACME) project funded under the U.S. Department of Energy’s Office of Biological and Environmental Research (BER) Earth System Modeling (ESM) program. ACME informally provides provenance in a native form through configuration files, directory structures, and log files that contain success/failure indicators, code traces, and performance measurements. Because of its generic format, HAPI is also being applied to harvest tabular job management provenance from Belle II DIRAC scheduler relational database tables as well as other scientific applications that log provenance related information.
Cybersecurity is one of the largest concerns in modern computing, impacting and dictating how governments, private corporations, and individuals interact with and live in an increasingly digital world. The NSA has recently released a memo [ 1] on “Software Memory Safety” where they highlight that both Microsoft and Google have stated around 70% of software vulnerabilities were due to memory safety issues. Although languages such as C and C++ provide freedom and flexibility with memory management, guaran- teeing safety falls mostly on the developer. The NSA recommends using “memory safe” languages whenever possible. In this paper we introduce Lamellar, an asynchronous tasking and PGAS HPC runtime written in Rust, one such "memory safe" language. We describe the entire Lamellar stack, from network interfaces to high- level abstractions such as distributed LamellarArrays and Active Messages. We conclude by showing comparable performance to legacy PGAS runtimes (e.g. OpenSHMEM) on a subset of the BALE kernel suite while maintaining strong memory safety principles.
With the appearance of multi-/many core machines, applications and runtime systems have evolved in order to exploit the new on-node concurrency brought by new software paradigms. POSIX threads (Pthreads) was widely-adopted for that purpose and it remains as the most used threading solution in current hardware. Lightweight thread (LWT) libraries emerged as an alternative offering lighter mechanisms to tackle the massive concurrency of current hardware. In this article, we analyze in detail the most representative threading libraries including Pthread- and LWT-based solutions. In addition, to examine the suitability of LWTs for different use cases, we develop a set of microbenchmarks consisting of OpenMP patterns commonly found in current parallel codes, and we compare the results using threading libraries and OpenMP implementations. Moreover, we study the semantics offered by threading libraries in order to expose the similarities among different LWT application programming interfaces and their advantages over Pthreads. This article exposes that LWT libraries outperform solutions based on operating system threads when tasks and nested parallelism are required.
—This paper proposes an optimization strategy to assist utility operators to recover power distribution systems after large outages. Specifically, a mixed-integer linear programming (MILP) model is developed for co-optimizing crews, resources, and network operations. The MILP model coordinates damage isolation, network reconfiguration, distributed generator re-dispatch, and crew/resource logistics. In addition, a framework for integrating different types of photovoltaic (PV) systems in the restoration process is developed. We consider two different types of crews, namely, line crews for damage repair and tree crews for obstacle removal. We also model the repair resource logistic constraints. Furthermore, a new algorithm is developed for solving the distribution system repair and restoration problem (DSRRP). The algorithm starts by solving DSRRP using an assignmentbased method, then a neighborhood search method is designed to iteratively improve the solution. The proposed method is validated on modified IEEE 123- and 8500-bus distribution test systems.
Improvements in the quality of synchrotron beamline x-ray optics required for next-generation light sources (e.g. the ALS Upgrade project) drive the need to improve the performance of the metrology instrumentation used to measure these components. The Long Trace Profiler (LTP) that is in use at many synchrotron metrology laboratories around the world has some known issues that affect the accuracy of its measurements. The main error source is optical path difference (OPD) phase error introduced into the probe beam by inhomogeneities in the glass components used in the optical head. We have developed a new optical head design, LTP-2020, that replaces the cube polarizing beamsplitter (PBS) with a thin wedge plate polarizing beamsplitter (WPBS) and replaces the cemented doublet lens with an aspheric singlet. Both of these components significantly reduce the glass volume traversed by the laser probe beam. Careful attention to ghost ray interference produced by back reflection from optical surfaces is necessary to minimize distortion in the primary image that translates into systematic error in the slope angle measurement. We make extensive use of a commercial raytracing program to model the back reflections and adjust component parameters as necessary to minimize distortion. Deliberate misalignment of components is necessary to make the system perform correctly. Stringent requirements are placed on the 45◦ incidence coatings on the WPBS and on the normal incidence coatings on the lens and camera window elements. We encourage our colleagues who wish to upgrade their current LTP systems to join us in the procurement of these custom optical components.
Purpose: The ultimate goal of radiotherapy treatment planning is to find a treatment that will yield a high tumor-control-probability(TCP) with an acceptable normal-tissue-complication probability(NTCP). Yet most treatment planning today is not based upon optimization of TCPs and NTCPs, but rather upon meeting physical dose and volume constraints defined by the planner. We design treatment plans that optimize TCP directly and contrast them with the clinical dose-based plans. PET image is incorporated to evaluate gain in TCP for dose escalation. Methods: We build a nonlinear mixed integer programming optimization model that maximizes TCP directly while satisfying the dose requirements on the targeted organ and healthy tissues. The solution strategy first fits the TCP function with a piecewise-linear approximation, then solves the problem that maximizes the piecewise linear approximation of TCP, and finally performs a local neighborhood search to improve the TCP value. To gauge the feasibility, characteristics, and potential benefit of PET-image guided dose escalation, initial validation consists of fifteen cervical cancer HDR patient cases. These patients have all received prior 45Gy of external radiation dose. For both escalated strategies, we consider 35Gy PTV-dose, and two variations (37Gy-boost to BTV vs 40Gy-boost) to PET-image-pockets. Results: TCP for standard clinical plans range from 59.4% - 63.6%. TCP for dose-based PET-guided escalated-dose-plan ranges from 63.8%–98.6% for all patients; whereas TCP-optimized plans achieves over 91% for all patients. There is marginal difference in TCP among those with 37Gy-boosted vs 40Gy-boosted. There is no increase in rectum and bladder dose among all plans. Conclusion: Optimizing TCP directly results in highly conformed treatment plans. The TCP-optimized plan is individualized based on the biological PET-image of the patients. The TCP-optimization framework is generalizable and has been applied successfully to other external-beam delivery modalities. A clinical trial is on-going to gauge the clinical significance. Partially supported by the National Science Foundation.
Large-scale parallel applications with complex global data dependencies beyond those of reductions pose significant scalability challenges in an asynchronous runtime system. Internodal challenges include identifying the all-to-all communication of data dependencies among the nodes. Intranodal challenges include gathering together these data dependencies into usable data objects while avoiding data duplication. This paper addresses these challenges within the context of a large-scale, industrial coal boiler simulation using the Uintah asynchronous many-task runtime system on GPU architectures. We show significant reduction in time spent analyzing data dependencies through refinements in our dependency search algorithm. Multiple task graphs are used to eliminate subsequent analysis when task graphs change in predictable and repeatable ways. Using a combined data store and task scheduler redesign reduces data dependency duplication ensuring that problems fit within host and GPU memory. Furthermore, these modifications did not require any changes to application code or sweeping changes to the Uintah runtime system. We report results running on the DOE Titan system on 119K CPU cores and 7.5K GPUs simultaneously. Our solutions can be generalized to other task dependency problems with global dependencies among thousands of nodes which must be processed efficiently at large scale.
Compute Express Link (CXL) is an important emerging standard for disaggregated memory. While this standard provisions coherency across numerous hosts and devices, implementing hardware support for type three devices is challenging. In this work, we look at the overhead of software synchronization and using software-based coherency. Moreover, we discuss the limits of software-based coherency in fully expressing modern synchronization techniques for a CXL-based disaggregate memory system. We demonstrate our approach using a CXL hardware prototype and running a version of the famous Peterson Lock (enhanced to run with more than two threads). We analyze its performance and share how more advanced synchronization techniques might interact with software-based coherence CXL hardware and program execution models.
GridPACK is a software framework consisting of a set of modules designed to simplify the development of programs that model the power grid and run on parallel, high performance computing platforms. It also contains several fully developed applications, including powerflow, dynamic simulation, state estimation, Kalman filter analysis (dynamic state estimation), contingency analysis and real time path rating. These applications can be used either standalone or as components in more complicated workflows that combine several different types of application together. The framework modules are available as a combination of libraries and software templates and consist of components for setting up and distributing power grid networks, support for modeling the behavior of individual buses and branches in the network, converting the network models to the corresponding algebraic equations, and parallel routines for manipulating and solving large algebraic systems. The framework also contains a module for distributing tasks evenly amongst computing resources, even if individual tasks vary widely in their execution times. Additional modules support input and output, basic statistical analysis of contingency based calculations, distributed data structures, as well as basic profiling and error management.
SAND2025-07450O Impala is a software tool that performs Bayesian model calibration for emulator-based models. The program uses R code to perform the calculations. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Multiphysics coupling presents a significant challenge in terms of both computational accuracy and performance. Achieving high performance on coupled simulations can be particularly challenging in a high-performance computing context. The US Department of Energy Exascale Computing Project has the mission to prepare mission-relevant applications for the delivery of the exascale computers starting in 2023. Many of these applications require multiphysics coupling, and the implementations must be performant on exascale hardware. In this special issue we feature six articles performing advanced multiphysics coupling that span the computational science domains in the Exascale Computing Project.
Higher-octane gasoline can enable increases in an internal combustion engine’s energy efficiency and a vehicle’s fuel economy by allowing an increase in the engine compression ratio and/or by enabling downspeeding and downsizing. Producing high-octane fuel (HOF) with the current level of ethanol blending (E10) could increase the energy and greenhouse gas (GHG) emissions intensity of the fuel product from refinery operations. Alternatively, increasing the ethanol blending level in final gasoline products could be a promising solution to HOF production because of the high octane rating and potentially low blended Reid vapor pressure (RVP) of ethanol at 25% and higher of the ethanol blending level by volume. In our previous HOF well-to-wheels (WTW) report (the so-called phase I report of the HOF WTW analysis), we conducted WTW analysis of HOF with different ethanol blending levels (i.e., E10, E25, and E40) and a range of vehicle efficiency gains with detailed petroleum refinery linear programming (LP) modeling by Jacobs Consultancy and showed that the overall WTW GHG emission changes associated with HOFVs were dominated by the positive impact associated with vehicle efficiency gains and ethanol blending levels, while the refining operations to produce gasoline blendstock for oxygenate blending (BOB) for various HOF blend levels had a much smaller impact on WTW GHG emissions (Han et al. 2015). The scope of the previous phase I study, however, was limited to evaluating PADDs 2 and 3 operation changes with various HOF market share scenarios and ethanol blending levels. Also, the study used three typical configuration models of refineries (cracking, light coking, and heavy coking) in each PADD, which may not be representative of the aggregate response of all refineries in each PADD to various ethanol blending levels and HOF market scenarios. Lastly, the phase I study assumed no new refinery expansion in the existing refineries, which limited E10 HOF production to the volume achievable by the cracking refinery configuration. To be able to satisfy large market demands of E10 HOF, that study arbitrarily relaxed the RVP requirements by replacing reformulated gasoline (RFG) RVP requirement of 7 psi in summer with conventional gasoline (CG) RVP requirement of 9 psi in summer. To examine the response by all refineries in major refinery regions, this phase II of the HOF WTW analysis employed regionally aggregated refinery models for the following six regions: PADDs 1, 2, 3, 4, and 5 excluding California (CA) and CA separately. Using aggregate refinery models, this phase II study examined the impacts of ethanol blending and HOF market shares on the refinery operations in these six regions. Also, this study included refinery expansion to produce a pre-determined HOF volume with 10% ethanol blending. In particular, this study examined several refinery expansion options using refinery configuration models to investigate a practical refinery response to the increase in E10 HOF market demand.
These are slides which are part of the ASC L2 Milestone Review. The following topics are covered: Legion Backend, Distributed-Memory Partitioning, Sparse Data Representations, and MPI-Legion Interoperability.
The primary objective of this project was to improve our ability to predict performance of an Enhanced Geothermal System (EGS) reservoir over time by relating, in a quantitative manner, microseismic imaging with fluid and temperature changes within the reservoir. Historically, microseismic data have been used qualitatively to place bounds on the growth of EGS reservoirs created by large hydraulic fracturing experiments. Previous investigators used an experimentally based fracture opening relationship (fracture aperture as a function of pressure), the spatial extent of microseismic events, and some assumptions about fracture frequency to determine the size of an EGS reservoir created during large pumping tests. We addressed a number of issues (1) locating microearthquakes that occur during hydraulic fracturing, (2) obtaining more information about a reservoir than the microearthquake locations from the microearthquake data, for example, information about the seismic velocity structure of the reservoir or the scattering of seismic waves within the reservoir, (3) developing an improved methodology for estimating properties of fractures that intersect wellbores in a reservoir, and (4) developing a conceptual model for explaining the downward growth of observed seismicity that accompanies some hydraulic injections into geothermal reservoirs. We used two primary microseismic datasets for our work. The work was motivated by a dataset from the Salak Geothermal Field in Indonesia where seismicity accompanying a hydraulic injection was observed to migrate downward. We also used data from the Soultz EGS site in France. We also used Vertical Seismic Profiling data from a well in the United States. The work conducted is of benefit for characterizing reservoirs that are created by hydraulic fracturing for both EGS and for petroleum recovery.
This report describes the basic aspects of control room modernization projects in the U.S. nuclear industry and the need for supplementary guidance on the integration of human factors considerations into the licensing and regulatory aspects of digital upgrades. The report pays specific attention to the integration of principles described in NUREG-0711 (Human Factors Engineering Program Review Model) and how supplementary guidance can help to raise general awareness in the industry regarding the complexities of control room modernization projects created by many interdependent regulations, standards and guidelines. The report also describes how human factors engineering principles and methods provided by various resources and international standards can help in navigating through the process of licensing digital upgrades. In particular, the integration of human factors engineering guidance and requirements into the process of licensing digital upgrades can help reduce uncertainty related to development of technical bases for digital upgrades that will avoid the introduction of new failure modes.
This report describes the basic aspects of control room modernization projects in the U.S. nuclear industry and the need for supplementary guidance on the integration of human factors considerations into the licensing and regulatory aspects of digital upgrades. The report pays specific attention to the integration of principles described in NUREG-0711 (Human Factors Engineering Program Review Model) and how supplementary guidance can help to raise general awareness in the industry regarding the complexities of control room modernization projects created by many interdependent regulations, standards and guidelines. The report also describes how human factors engineering principles and methods provided by various resources and international standards can help in navigating through the process of licensing digital upgrades. In particular, the integration of human factors engineering guidance and requirements into the process of licensing digital upgrades can help reduce uncertainty related to development of technical bases for digital upgrades that will avoid the introduction of new failure modes.
This progress report (Level 4 Milestone Number M4SF-19LL010301042) summarizes research conducted at Lawrence Livermore National Laboratory (LLNL) within the Argillite Activity Number SF-19LL010301041. LLNL is leading efforts in the development of thermodynamic databases in support of the Spent Fuel and Waste and Science Technology (SFWST) program. Thermodynamic models provide the basis for understanding the stability of solid phases and speciation of aqueous species and modeling the evolution of repository conditions. The LLNL effort is being performed in coordination with other US database development efforts. The effort includes a review of available thermochemical databases and a path forward for database integration. International coordination with the NEA-TDB is supported through crystalline international work package