Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54

Experiences with Testing the Largest Ground System NASA Has Ever Built

In the 1980s, the National Aeronautics and Space Administration (NASA) embarked upon a major Earth-focused program called Mission to Planet Earth. The Goddard Space Flight Center (GSFC) was selected to manage and develop a key component - the Earth Observing System (EOS). The EOS consisted of four major missions designed to monitor the Earth. The missions included 4 spacecraft. Terra (launched December 1999), Aqua (launched May 2002), ICESat (Ice, Cloud, and Land Elevation Satellite, launched January 2003), and Aura (scheduled for launch January 2004). The purpose of these missions was to provide support for NASA s long-term research effort for determining how human-induced and natural changes affect our global environment. The EOS Data and Information System (EOSDIS), a globally distributed, large-scale scientific system, was built to support EOS. Its primary function is to capture, collect, process, and distribute the most voluminous set of remotely sensed scientific data to date estimated to be 350 Gbytes per day. The EOSDIS is composed of a diverse set of elements with functional capabilities that require the implementation of a complex set of computers, high-speed networks, mission-unique equipment, and associated Information Technology (IT) software along with mission-specific software. All missions are constrained by schedule, budget, and staffing resources, and rigorous testing has been shown to be critical to the success of each mission. This paper addresses the challenges associated with the planning, test definition. resource scheduling, execution, and discrepancy reporting involved in the mission readiness testing of a ground system on the scale of EOSDIS. The size and complexity of the mission systems supporting the Aqua flight operations, for example, combined with the limited resources available, prompted the project to challenge the prevailing testing culture. The resulting success of the Aqua Mission Readiness Testing (MRT) program was due in no small measure to re-structuring the traditional programmatic and technical approach to a more efficient and robust program. Programmatically, it meant gaining the endorsement, commitment, and cooperation of the numerous subsystem element managers and other stakeholder organizations. Technically, it required an MRT program that was agile, could rapidly adapt to requirements changes, and was flexible in its overall approach. Furthermore, this paper addresses the following questions: 1. What are the key ingredients (e.g., test tools, organization) needed to conduct a successful MRT program? 2. What distinguishes EOS MRT from the traditional system testing approach? 3. Where should the focus of testing be since it is infeasible to test every element or subsystem? 4. How can MRT be applied effectively to other systems or missions? To provide answers to these questions, this paper relies heavily on real-life, hands-on experiences ("lessons learned") gained during mission readiness testing of the Terra ground system and, most recently, the Aqua and ICESat missions. Moreover, this paper explores how lessons learned were turned into lessons applied for the upcoming Aura mission. Although derived from the EOS missions, MRT techniques and strategies can be applied to enhance the testing of other missions.

Lehtonen, Ken↗

Pushing the Limits of Aquatic Remote Sensing: Synthetic Data and Deep Learning for Fast Inverse Emulation of A Coupled Ocean-Atmosphere Radiative Transfer Model

The inversion of electromagnetic information to physical and biological properties of the water column is a notoriously difficult problem, yet fundamental to our ability of understanding aquatic processes on large time and space scales. There is now a growing necessity to develop pragmatic approaches that allow timely and effective extrapolation of local processes, to spatially resolved global products, and to promote operational and sustainable resource policy management. This presentation will discuss research integrating advanced biological and radiative modeling, high-end computation, and machine learning to develop a portable global processor for simultaneous retrieval of atmosphere and water optics for diverse aquatic systems from the open and coastal ocean to optically extreme inland waters and harmful algal blooms. We will discuss some of the basic concepts behind the forward modeling approach including DEAP, the novel Distributed Equivalent Algal Populations model, for developing large spectral libraries of aquatic particle optics to aid in our ability to distinguish phytoplankton functional types (PFTs) and inorganic material, as well as other factors which enable comprehensive modeling from the benthos to top-of-atmosphere (TOA). This information is being used to understand how we can leverage next-generation deep learning methods for maximum information retrieval and rapid image processing, while also providing capabilities to identify minimum sensor spectral requirements necessary for certain aquatic applications. Further, I will touch on how we envision this research to enable the aquatic community for science discovery and how we are moving closer towards the capability for high-fidelity global analysis of aquatic ecosystems.

Jeremy Alan Kravitz↗

Cyber Resilient Design and Controls for Energy Systems

As the grid of today is evolving into a highly distributed and autonomous cyber-physical system with higher penetration of Distributed Energy Resources (DERs) and autonomous controls that are dependent on the cyber infrastructure, there are novel and increasing challenges to cyber-resilient operation of the grid. While the traditional mechanisms take a "bolt on" approach to cybersecurity and resiliency, this talk presents two novel technologies developed at NREL that "bake in" security and resilience into the design and control of the grid. An Adaptive Resilience Metrics framework is a novel mechanism for system owners and operators to understand the impact of cyber and physical system design and topology on the resilient operation, while also helping determine optimal policies in response to adverse cyber events. Along with that a zero-knowledge proof-based technique helps incorporate security into the controls layer by focusing on computational integrity rather than data integrity. By leveraging design and control properties unique to Operational Technology systems these technologies will not only help enhance cyber-resilience for the grid of today, but a highly distributed and autonomous grid of tomorrow.

cyber resilience↗

FAODEL

FAODEL is a collection of libraries for implementing data management services on high-performance computing (HPC) platforms. It is composed of several components: (1) an RDMA portability library called NNTI for transporting data, (2) an in-application, restful API service, (3) a memory management library for network memory named Lunasa, (4) an asynchronous communication engine, and (5) a distributed key/blob service names Kelpie. Application developers may use these libraries to move data between the distributed nodes of one job to another, as well as to distributed storage resources. FAODEL is different than other communication libraries in that it supports both intra- and inter-job communication, features event-driven mechanisms, and provides high-level primitives for developers to reason about a platform’s distributed resources.

ECP↗

Comparative analysis of energy deposition modes available in Serpent 2 within the framework of the supercritical water reactor - Fuel qualification test reactor physics benchmark

A joint European Canadian Chinese development of a supercritical water-cooled small modular reactor (SCW-SMR) technology is in progress since September 2020 in the framework of a Horizon 2020 project called ECC-SMART. As a main purpose of the project, proper estimates of energy deposition and its spatial distribution are prerequisites for the accurate analysis of safety related parameters of the SCW-SMR concept under development. A supercritical water reactor fuel computational benchmark model, provided by Canadian Nuclear Laboratories, was applied for detailed comparison of different energy deposition calculation options available in the Serpent 2 Monte Carlo code. The effect of energy deposition options on the normalization of the results as well as on the spatial distribution of the energy deposition are discussed. Consistent energy deposition calculation methods are presented between three Monte Carlo codes, viz., Serpent 2, MCNP6 and OpenMC. Although resource-intensive, the use of the coupled neutron-photon transport mode of Serpent 2 is recommended for accurate spatial and quantitative characterization of energy deposition in the SCW-SMR fuel assemblies, accounting for both neutron and photon heating of all the materials. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Challenges Facing Design and Analysis Tools

The design and analysis of future aerospace systems will strongly rely on advanced engineering analysis tools used in combination with risk mitigation procedures. The implications of such a trend place increased demands on these tools to assess off-nominal conditions, residual strength, damage propagation, and extreme loading conditions in order to understand and quantify these effects as they affect mission success. Advances in computer hardware such as CPU processing speed, memory, secondary storage, and visualization provide significant resources for the engineer to exploit in engineering design. The challenges facing design and analysis tools fall into three primary areas. The first area involves mechanics needs such as constitutive modeling, contact and penetration simulation, crack growth prediction, damage initiation and progression prediction, transient dynamics and deployment simulations, and solution algorithms. The second area involves computational needs such as fast, robust solvers, adaptivity for model and solution strategies, control processes for concurrent, distributed computing for uncertainty assessments, and immersive technology. Traditional finite element codes still require fast direct solvers which when coupled to current CPU power enables new insight as a result of high-fidelity modeling. The third area involves decision making by the analyst. This area involves the integration and interrogation of vast amounts of information - some global in character while local details are critical and often drive the design. The proposed presentation will describe and illustrate these areas using composite structures, energy-absorbing structures, and inflatable space structures. While certain engineering approximations within the finite element model may be adequate for global response prediction, they generally are inadequate in a design setting or when local response prediction is critical. Pitfalls to be avoided and trends for emerging analysis tools will be described.

Knight, Norman F., Jr.↗

Large-scale optimal control of interconnected natural gas and electrical transmission systems

We present a detailed optimal control model that captures spatiotemporal interactions between gas and electric transmission networks. We use the model to study flexibility and economic opportunities provided by coordination. A large-scale case study in the Illinois system reveals that coordination can enable the delivery of significantly larger amounts of natural gas to the power grid. In particular, under a coordinated setting, gas-fired generators act as distributed demand response resources that can be controlled by the gas pipeline operator. This enables more efficient control of pressures and flows in space and time and overcomes delivery bottlenecks. We demonstrate that the additional flexibility not only can benefit the gas operator but can also lead to more efficient power grid operations and results in increased revenue for gas-fired power plants. We also use the optimal control model to analyze computational issues arising in these complex models. We demonstrate that the interconnected Illinois system with full physical resolution gives rise to a highly nonlinear optimal control problem with 4400 differential and algebraic equations and 1040 controls that can be solved with a state-of-the-art sparse optimization solver. (C) 2016 Elsevier Ltd. All rights reserved.

Chiang, Nai-Yuan↗

The HPC Container Experience on the Summit Supercomputer

Containers are seeing widespread use in the world of High Performance Computing, with many HPC Centers either providing their own containerization solution or adopting existing ones like Singularity and Apptainer. The demand for containerization options come from users who want to take advantage of the portability and reproducibility containers can provide, as well as being able to build and use applications that are only distributed in container form or are otherwise unsuited to natively run in an HPC environment. The users served by the Oak Ridge Leadership Computing Facility are no exception. We go over the past and current containerization offerings at the Oak Ridge Leadership Computing Facility, mainly focusing on the Summit supercomputer. We arrive at using a combination of Podman and Singularity to allow users to build and run containers directly on Summit, without requiring external resources or hardware for any step of the process. We look at a couple of projects running on Summit that greatly benefited from being able to use containers on Summit. And we compare benchmarks running natively and in containers on Summit at different scales, observing minimal performance difference and consistent behavior across all tests.

Abraham, Subil↗

Evaluating the Performance of Single and Double Moment Microphysics Schemes During a Synoptic-Scale Snowfall Event

Increases in computing resources have allowed for the utilization of high-resolution weather forecast models capable of resolving cloud microphysical and precipitation processes among varying numbers of hydrometeor categories. Several microphysics schemes are currently available within the Weather Research and Forecasting (WRF) model, ranging from single-moment predictions of precipitation content to double-moment predictions that include a prediction of particle number concentrations. Each scheme incorporates several assumptions related to the size distribution, shape, and fall speed relationships of ice crystals in order to simulate cold-cloud processes and resulting precipitation. Field campaign data offer a means of evaluating the assumptions present within each scheme. The Canadian CloudSat/CALIPSO Validation Project (C3VP) represented collaboration among the CloudSat, CALIPSO, and NASA Global Precipitation Measurement mission communities, to observe cold season precipitation processes relevant to forecast model evaluation and the eventual development of satellite retrievals of cloud properties and precipitation rates. During the C3VP campaign, widespread snowfall occurred on 22 January 2007, sampled by aircraft and surface instrumentation that provided particle size distributions, ice water content, and fall speed estimations along with traditional surface measurements of temperature and precipitation. In this study, four single-moment and two double-moment microphysics schemes were utilized to generate hypothetical WRF forecasts of the event, with C3VP data used in evaluation of their varying assumptions. Schemes that incorporate flexibility in size distribution parameters and density assumptions are shown to be preferable to fixed constants, and that a double-moment representation of the snow category may be beneficial when representing the effects of aggregation. These results may guide forecast centers in optimal configurations of their forecast models for winter weather and identify best practices present within these various schemes.

Molthan, Andrew L.↗

An Innovative Approach to Modeling VIPER Rover Software Life Cycle Cost

NASA’s “Volatiles Investigating Polar Exploration Rover” (VIPER) will be the first robotic mission to prospect for water ice near the south pole of the Moon in late 2023 on a 100-Earth-day mission. The information that the VIPER rover provides will help improve understanding of the composition, distribution, and accessibility of Lunar polar volatiles and will help determine how the Moon’s resources can support future human space exploration. VIPER, however, represents a radical departure from the way that NASA has traditionally developed planetary robotic missions. A key consequence of these differences is that estimating the cost of VIPER’s rover software is challenging and complex.For example, VIPER is being developed using management procedures typically applied to NASA research and technology projects, rather than space flight programs. In addition, key portions of the rover’s software are being designed as ground software to run on mission control computers (rather than on-board the rover as flight software as with prior planetary missions) taking advantage of continuous, interactive data communications between the Moon and Earth and higher performance computing available on the ground. Moreover, the rover’s software is being engineered using Agile software development practices and incorporates a significant amount of open-source, rather than following traditional (spiral, waterfall, etc.) development methods and in-house code. In this paper, we present an innovative process to estimate the life cycle cost of VIPER’s rover software. We first describe how we modeled the architecture and code counts for three software elements: Rover Flight Software (RFSW), Rover Ground Software (RGSW), and Rover Simulation Software (RSIM). We then discuss key challenges and unique aspects of our approach, such as the lack of Lunar rover analogies, the need to integrate and test large open source software, and the strategies developed to account for use of non-space flight management practices and the impact of the COVID-19 pandemic. We conclude with a summary of our results, including cumulative distribution, nearest neighbors and cluster analysis, as well as heuristics used to confirm the reasonableness of the cost estimate.

Utz, Hans↗

Risk Management for Distributed Energy Resources

The National Institute of Standards and Technology will be hosting on Tuesday, February 2 and Wednesday, February 3, 2021, the second workshop in a new series focusing on the Open Security Controls Assessment Language. NREL extended the scope of the DERCF to include the NIST Risk Management Framework (RMF), addressing the challenges faced by federal energy managers when complying with the NIST RMF for DER systems. The NIST RMF is a cyclical process designed to incorporate principles of security and risk management into an organization’s system policies and procedures. The DER-RM will be downloadable application that runs locally and documents all the major requirements for achieving Authority to Operate the DER.

cybersecurity↗

Efficient and generalizable nested Fourier-DeepONet for three-dimensional geological carbon sequestration

Geological carbon sequestration (GCS) involves injecting CO2 into subsurface geological formationsfor permanent storage. Numerical simulations could guide decisions in GCS projects by predictingCO 2 migration pathways and the pressure distribution in storage formation. However, these simula-tions are often computationally expensive due to highly coupled physics and large spatial-temporalsimulation domains. Surrogate modelling with data-driven machine learning has become a promis-ing alternative to accelerate physics-based simulations. Among these, the Fourier neural operator(FNO) has been applied to three-dimensional synthetic subsurface models. Despite its good accuracyin simulating CO 2 plume migration, it requires large computational resources in training and alsolacks generalizability. Here, to further improve performance, we have developed a nested Fourier-DeepONet by combining the expressiveness of the FNO with the modularity of a deep operatornetwork (DeepONet). This new framework is twice as efficient as a nested FNO for training and has atleast 80% lower GPU memory requirement due to its flexibility to treat temporal coordinates sepa-rately. These performance improvements are achieved without compromising prediction accuracy.In addition, the generalization and extrapolation ability of nested Fourier-DeepONet beyond thetraining range has been thoroughly evaluated. Nested Fourier-DeepONet outperformed the nestedFNO for extrapolation in time with more than 50% reduced error. It also exhibited good extrapolationaccuracy beyond the training range in terms of reservoir properties, number of wells, and injectionrate.

Lee, Jonathan E. [Department of Chemical and Envir↗

WTFgenes: What's The Function of these genes? Static sites for model-based gene set analysis

A common technique for interpreting experimentally-identified lists of genes is to look for enrichment of genes associated with particular ontology terms. The most common test uses the hypergeometric distribution; more recently, a model-based test was proposed. These approaches must typically be run using downloaded software, or on a server. We develop a collapsed likelihood for model-based gene set analysis and present WTFgenes, an implementation of both hypergeometric and model-based approaches, that can be published as a static site with computation run in JavaScript on the user's web browser client. Apart from hosting files, zero server resources are required: the site can (for example) be served directly from Amazon S3 or GitHub Pages. A C++11 implementation yielding identical results runs roughly twice as fast as the JavaScript version. WTFgenes is available from https://github.com/evoldoers/wtfgenes under the BSD3 license. A demonstration for the Gene Ontology is usable at https://evoldoers.github.io/wtfgo.

Mungall, Christopher J.↗

Global Magellan-image map of Venus at full resolution

During its first 243-day mapping cycle, the Magellan spacecraft succeeded in imaging 84 percent of the surface of Venus at resolutions on the order of 100 meters; subsequent cycles have increased the total coverage to over 97 percent and provided redundant coverage of much of the planet with differing viewing geometries. Unfortunately, this full-resolution global dataset is in the form of thousands of individual orbit tracks (F-BIDR's) whose length-to-width ratio of nearly 1000:1 makes them minimally useful unless mosaicked. The Magellan project produced full-resolution mosaics (F-MIDR's) only for selected regions on the planet, whereas a global set of mosaics was made only at threefold degraded resolution (C1-MIDR's). Furthermore, although the F-MIDR's, which are approximately equidimensional, are much better suited for scientific interpretation than the F-BIDR's, they are still an unwieldy dataset: over 1500 quadrangles, each showing a region only about 600 km on a side, would be required to cover the entire planet. The USGS has therefore undertaken to produce and distribute a global, full resolution set of mosaics of the Magellan image data in a format that will be efficient for both hardcopy and digital use. The initial motivation was that it would provide an efficient means of verifying the integrity of the F-BIDR's to be archived on computer-compatible tape at the USGS Flagstaff facility. However, the resulting product, known as the FMAP, should also serve as an important resource for future scientific interpretation. It will offer several advantages beyond global coverage at full resolution. The first, alluded to above, is its division of the planet's surface to minimize the number of quadrangles and maximize their area, subject to the limits on the number of pixels imposed by state-of-the-art digital recording media and hardcopy output devices. The second, the use of improved 'cosmetic' processing techniques, will greatly reduce tonal discontinuities between component F-BIDR's in the FMAP compared to the standard Magellan mosaic products. Finally, wherever possible, the FMAP will incorporate data that were unavailable (e.g., because of processing delays) when the standard MIDR products were created, as well as data that were reprocessed to improve their radiometric or geometric quality.

Kirk, R. L.↗

Role of Nuclear Science User Facilities (NSUF) in Nuclear Energy Materials Research

The Nuclear Science User Facilities (NSUF) is one of a diverse number of U.S. Department of Energy (DOE) user facilities established to provide researchers with the most advanced tools of modern science. The NSUF is unique and represents a consortium of capabilities distributed across the U.S. at twenty-one institutions. The NSUF is centered at and managed from the Idaho National Laboratory (INL), where it was originally founded, but it coordinates activities at twenty “partner” institutions that include universities, the Center for Advanced Energy Studies (CAES), national laboratories, and a nuclear industry vendor. These institutions have capabilities that include neutron, ion, and gamma irradiation, hot cells, advanced materials characterization equipment, and high-performance computing resources. Many of these capabilities were beyond reach for most researchers before NSUF. The NSUF provides researchers to access these capabilities at no cost to nuclear energy researchers to produce the highest quality research results to increase understanding of advanced nuclear energy technologies important to DOE-NE and support national priorities by adapting to the needs of DOE-NE programs, industry, and new innovative concepts for sustainable nuclear future.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

HyperSpace: Distributed Bayesian Hyperparameter Optimization

As machine learning models continue to increase in complexity, so does the potential number of free model parameters commonly known as hyperparameters. While there has been considerable progress toward finding optimal configurations of these hyperparameters, many optimization procedures are treated as black boxes. We believe optimization methods should not only return a set of optimized hyperparameters, but also give insight into the effects of model hyperparameter settings. To this end, we present HyperSpace, a parallel implementation of Bayesian sequential model-based optimization. HyperSpace leverages high performance computing (HPC) resources to better understand unknown, potentially non-convex hyperparameter search spaces. We show that it is possible to learn the dependencies between model hyperparameters through the optimization process. By partitioning large search spaces and running many optimization procedures in parallel, we also show that it is possible to discover families of good hyperparameter settings over a variety of models including unsupervised clustering, regression, and classification tasks.

Computer Science↗

Langston University - High Energy Physics (LU-HEP)

This final report is presented by Langston University (LU) for the project entitled "Langston University High Energy Physics" (LUHEP) under the direction of principal investigator (PI) and project director Professor Joel Snow. The project encompassed high energy physics research performed at hadron colliders. The PI is a collaborator on the DZero experiment at Fermi National Accelerator Laboratory in Batavia, IL, USA and the ATLAS experiment at CERN in Geneva, Switzerland and was during the entire project period from April 1, 1999 until May 14, 2012. Both experiments seek to understand the fundamental constituents of the physical universe and the forces that govern their interactions. In 1999 as member of the Online Systems group for Run 2 the PI developed a cross-platform Python-based, Graphical User Interface (GUI) application for monitoring and control of EPICS based devices for control room use. This served as a model for other developers to enhance and build on for further monitoring and control tasks written in Python. Subsequently the PI created and developed a cross-platform C++ GUI utilizing a networked client-server paradigm and based on ROOT, the object oriented analysis framework from CERN. The GUI served as a user interface to the Examine tasks running in the D\O\ control room which monitored the status and integrity of data taking for Run 2. The PI developed the histogram server/control interface to the GUI client for the EXAMINE processes. The histogram server was built from the ROOT framework and was integrated into the D\O\ framework used for online monitoring programs and offline analysis. The PI developed the first implementation of displaying histograms dynamically generated by ROOT in a Web Browser. The PI's work resulted in several talks and papers at international conferences and workshops. The PI established computing software infrastructure at LU and U. Oklahoma (OU) to do analysis of DZero production data and produce simulation data for the experiment. Eventually this included the FNAL SAM data grid system, the SAMGrid (SG) infrastructure, and the Open Science Grid software stacks for computing and storage elements. At the end of 2003 Snow took on the role of global Monte Carlo production coordinator for the DØ experiment. A role which continues til this day. In January of 2004 Snow started working with the SAMGrid development team to help debug, deploy, and integrate SAMGrid with DØ Monte Carlo production. Snow installed and configured SG execution and client sites at LUHEP and OUHEP, and a SG scheduler site at LUHEP. The PI developed a python based GUI (DAJ) that acts as a front end for job submission to SAMGrid. The GUI interfaces to the DZero Mone Carlo (MC) request system that uses SAM to manage MC requests by the physics analysis groups. DAJ significantly simplified SG job submission and was deployed in DZero in an effort to increase the user base of SG. The following year was the advent of SAMGrid job submission to the Open Science Grid (OSG) and LHC Computing Grid (LCG) through a forwarding mechanism. The PI oversaw the integration of these grids into the existing production infrastructure. The PI developed an automatic MC (Automc) request processing system capable of operating without user intervention (other than getting grid credentials), and able to submit to any number of sites on various grids. The system manages production at all but 2 sites. The system was deployed at Fermilab and remains operating there today. The PI's work in distributed computing resulted in several talks at international conferences. UTA, OU, and LU were chosen as the collaborating institutions that form the Southwest Tier 2 Center (SWT2) for ATLAS. During the project period the PI contributed to the online and offline software infrastructure through his work with the Run 2 online group, and played a major role in Monte Carlo production for DZero. During the part of the project period in which the PI served as MC production coordinator MC production increased very significantly. In the first year of the PI's tenure as production coordinator production was 159M events and 6.7~TB of data. During the last year of the project period production was 2,342~M events and 262~TB of data. That is a factor of 15 increase in events and 39 in data volume. The increase occurred with improvements in computer hardware and networks, through the use of grid technology on diverse resources, and through increased automation and efficiency of the production process. LU HEP developed and deployed the automatic MC request processing system in use at FNAL. The complementary strategies of automation and grid production served DZero well. Fermilab has recognized LU HEP's contribution to DZero by allowing the PI to devote full time to research activities by appointing him a guest scientist for the last six years of the project period.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

ORCHA: A performance portability system for extreme heterogeneity

Heterogeneity is the prevalent trend in the rapidly evolving high-performance computing (HPC) landscape in both hardware and application software. The diversity in hardware platforms, currently comprising various accelerators and a future possibility of specializable chiplets, poses a significant challenge for scientific software developers aiming to harness optimal performance across different computing platforms while maintaining the quality of solutions when their applications are simultaneously growing more complex. Code synthesis and code generation can provide mechanisms to mitigate this challenge. We have developed a divide and conquer approach where different aspects of performance are handled by different stand-alone tools that are interfaced with the application through generated code. This portability system, ORCHA, enables users to configure and orchestrate their computations among available resources on a platform by specifying a high-level recipe, thereby permitting a many-to-many paradigm where each recipe results in a different variant of the application. The core design goal is to let users decide the application’s hardware mapping and orchestration by editing only the high-level recipe—without modifying the maintained source code or binding the application to a particular runtime system. Tools in ORCHA distribution are: CG-Kit for translating the recipe into an execution graph; Milhoja to execute the graph by orchestrating data and task movement among hardware resources; and Macroprocessor that enables users to define their own code-shorthand for higher composability and easier management of code variants. Additionally, the design of ORCHA permits tools to work in a plug-and-play mode where the application can build and run without CG-Kit and Milhoja, and either tool can be swapped out for other tools with similar capabilities by modifying the code generation portion of ORCHA. In this paper, we describe the design of ORCHA and the role that code-generation plays in isolating applications from tools. We demonstrate the breadth of configurations ORCHA enables with a case study in which an application configuration is realized on three distinct hardware mappings—a GPU-centric, a CPU/GPU balanced, and a CPU/GPU concurrent layouts by using different recipes.

Lee, Youngjun↗