Re-evaluating Network Onload vs. Offload for the Many-Core Era.
Abstract not provided.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
The OpenMP language features have been evolving to meet the rapid development in hardware platforms. DOE applications tend to push the bleeding edge of features ratified in the OpenMP specification and tend to expose the rough edges of the features' implementations. The software harness on DOE supercomputers such as Titan and (upcoming) Summit include Cray, Clang, Flang, XL and GCC compilers. It is critical, especially for Summit, that the compilers support OpenMP offloading features. This paper focuses on evaluating support for OpenMP 4.5 target offload directives across compiler implementations on Titan and Summitdev, an early access system, which is one generation removed from Summit's architecture enabling application teams to test the systems' architecture. Our tests not only evaluate the OpenMP implementations but also expose ambiguities in the OpenMP 4.5 specification. We also evaluate compiler implementations using kernels extracted from production DOE applications. This helps in assessing the interaction of different OpenMP directives independent of other application artifacts. We are aware that the implementations are constantly evolving and are advertised as having only partial OpenMP 4.x support. We see this as a synergistic effort to help identify and correct features that are required by DOE applications and prevent deployment delays later on. Going forward, we also plan to interact with standard benchmarking bodies like SPEC/HPG to donate our tests and mini-apps/kernels for potential inclusion in the next release versions of SPEC OMP and SPEC ACCEL benchmark suites.
The Active Response Gravity Offload System (ARGOS) at NASA Johnson Space Center (JSC) is an analog environment that can offload pressurized suited subjects for various reduced gravity simulations. The suit is suspended from a robotic overhead crane by a cable connected to the suit via a gimbal with an adjustable pivot point (i.e. offload attachment). There has been increased interest in providing planetary pressurized suited training at ARGOS in preparation for lunar missions. Determination of the appropriate gimbal pivot point location for a given subject is vital for a high-fidelity functional lunar simulation. Interactions between the pivot point location and human-spacesuit center of gravity (CG) can result in righting moments that may lead to artificially stable or unrealistically challenging configurations. Changing the pivot point location is time consuming and repeated adjustment can result in loss of valuable pressurized suited time. This paper aims to share knowledge obtained from the offloading characterization efforts during pressurized suited testing at ARGOS and document the ongoing process to define an appropriate pivot point location through iterative quantitative and qualitative assessments. Human-spacesuit CG locations for the ARGOS lunar simulation were estimated using a 3D body scan and density model combined with spacesuit hardware CAD and specifications. Early pilot testing of the gimbal revealed that setting the pivot point coincident with the modeled CG location was not always possible due to the current gimbal design, and small pivot point shifts had noticeable effects on subject stability. Fourteen subjects performed a series of CG-related tasks in the Exploration Extravehicular Mobility Unit (xEMU) to assess simulation characteristics. Through iterative testing, this task list evolved to streamline the process needed to efficiently identify a suitable pivot point for a given subject. The developed methodology will be critical for pivot point selection during astronaut training in the ARGOS environment.
Explore the source record for details and available documents.
BACKGROUND: The Active Response Gravity Offload System (ARGOS) provides an analog environment for extravehicular activity (EVA) testing and training. Discomfort has been observed during longer suited test sessions. While the subject’s core is offloaded during surface EVA evaluations, his/her arms experience full Earth gravity and can become overly fatigued, especially during suited tests which involve reaching and prolonged arm extensions. A device (ARGOS Negation of Gravitational Effects on the Limbs: ANGEL) to offload the weight of the arms and suit sleeves is being developed by JSC’s Flight Systems Branch of the Software, Robotics, and Simulation Division. Previously we have shared preliminary modeling of that device and kinematics based on motion capture data. Here we present an alternative approach to determine device kinematics by calculating ANGEL component angles with an OpenSim plugin. We compare calculated angles to inverse kinematics (IK) derived ones with the goal of validating the model. This new method can be further informative for device design and analytically testing different configurations to achieve desired reduced gravity conditions (e.g., lunar gravity (Lg) or Martian gravity (Mg)). We have compared calculated angles with IK-derived angles in tests with a shirt-sleeve subject positioned in a test stand with a Mark-III Hard Upper Torso (HUT) and Portable Life Support System (PLSS) mockup and arm weights to emulate the weight of the suit sleeve as well as a suited subject in ARGOS with a Mark-III suit. A variety of upper body tasks were completed in the former and full-body tasks in the latter. METHODS AND RESULTS: To model the offload device, we augment the OpenSim human model topology with the offload mechanism components and joints, using CAD models to represent the mechanism graphically. The joint angles of the device are calculated in the OpenSim plugin by modeling how the components configure themselves under the offloading spring tension given a particular IK-derived arm position. There are four ANGEL components with a total of 5 degrees of freedom (DOFs), each component has a single DOF except for the cuff which is modeled as 2 DOFs. The sickle/yaw bracket and cuff rotation angles are determined statically based on the assumptions that the sickle will track the attachment point of the cuff and that the cuff will rotate such that the attachment point is at its highest point. The cuff tilt, linker and V-bracket angles are then determined by optimizing their positions to approach a mechanical equilibrium. The calculated linker angle is compared to three different methods of determining the linker line-of-force kinematically (from V-bracket to center-cuff, cuff highest point or marker-derived position). Given the joint angles of the device, the spring force and resulting force on the arm is computed by the plugin and applied as an external load in inverse dynamics (ID) to enable study of overall shoulder joint torques as well as offload achieved. We verify the calculated joint angles by using the inverse kinematic data. The average difference in angles is the smallest for the V-bracket and linker, around 1 to 5 degrees for most trials. The resulting offload and shoulder torque are comparable between calculated and IK-derived angles. In summary, we have developed a method to calculate the joint angles of an exoskeleton-like upper limb offloading device currently in development. We have also developed a custom plugin which will be a valuable tool to optimize device configurations for a desired gravitational environment, probe the offload achieved for motions recorded outside of our test suite, and inform future design improvements.
BACKGROUND: The Active Response Gravity Offload System (ARGOS) provides an analog environment for extravehicular activity (EVA) testing and training. Discomfort has been observed during longer suited test sessions. While the subject’s core is offloaded during surface EVA evaluations, his/her arms experience full Earth gravity and can become overly fatigued, especially during suited tests which involve reaching and prolonged arm extensions. A device (ARGOS Negation of Gravitational Effects on the Limbs: ANGEL) to offload the weight of the arms and suit sleeves is being developed by JSC’s Flight Systems Branch of the Software, Robotics, and Simulation Division. Previously we have shared preliminary modeling of that device and kinematics based on motion capture data. Here we present an alternative approach to determine device kinematics by calculating ANGEL component angles with an OpenSim plugin. We compare calculated angles to inverse kinematics (IK) derived ones with the goal of validating the model. This new method can be further informative for device design and analytically testing different configurations to achieve desired reduced gravity conditions (e.g., lunar gravity (Lg) or Martian gravity (Mg)). We have compared calculated angles with IK-derived angles in tests with a shirt-sleeve subject positioned in a test stand with a Mark-III Hard Upper Torso (HUT) and Portable Life Support System (PLSS) mockup and arm weights to emulate the weight of the suit sleeve as well as a suited subject in ARGOS with a Mark-III suit. A variety of upper body tasks were completed in the former and full-body tasks in the latter. METHODS AND RESULTS: To model the offload device, we augment the OpenSim human model topology with the offload mechanism components and joints, using CAD models to represent the mechanism graphically. The joint angles of the device are calculated in the OpenSim plugin by modeling how the components configure themselves under the offloading spring tension given a particular IK-derived arm position. There are four ANGEL components with a total of 5 degrees of freedom (DOFs), each component has a single DOF except for the cuff which is modeled as 2 DOFs. The sickle/yaw bracket and cuff rotation angles are determined statically based on the assumptions that the sickle will track the attachment point of the cuff and that the cuff will rotate such that the attachment point is at its highest point. The cuff tilt, linker and V-bracket angles are then determined by optimizing their positions to approach a mechanical equilibrium. The calculated linker angle is compared to three different methods of determining the linker line-of-force kinematically (from V-bracket to center-cuff, cuff highest point or marker-derived position). Given the joint angles of the device, the spring force and resulting force on the arm is computed by the plugin and applied as an external load in inverse dynamics (ID) to enable study of overall shoulder joint torques as well as offload achieved. We verify the calculated joint angles by using the inverse kinematic data. The average difference in angles is the smallest for the V-bracket and linker, around 1 to 5 degrees for most trials. The resulting offload and shoulder torque are comparable between calculated and IK-derived angles. In summary, we have developed a method to calculate the joint angles of an exoskeleton-like upper limb offloading device currently in development. We have also developed a custom plugin which will be a valuable tool to optimize device configurations for a desired gravitational environment, probe the offload achieved for motions recorded outside of our test suite, and inform future design improvements.
The Common Habitat is a large, long-duration habitat being explored as part of a conceptual study (not an active NASA program) that uses an SLS core stage liquid oxygen (LOX) tank as its primary structure. Measuring 8.4 meters in diameter and 15.6 meters in length, it is manufactured as a habitat and launched as such into space. It is intended for use on the Moon as part of a permanently occupied outpost, on Mars as part of an outpost that will be occupied for hundreds of days at a time, and in deep space as part of the Deep Space Exploration Vehicle where it will support crewed missions up to 1200 days in duration. A study of internal orientation and crew size resulted in a Common Habitat configuration sized for a crew of eight with a three-deck horizontal orientation. There are obvious challenges associated with the delivery of such a large habitat, which may mass as much as 90-tons when initially deployed. The Mars destination in particular imposes extreme challenges due to Martian gravity. This paper identifies initial options for the offloading of a 90-ton Common Habitat from a lander spacecraft on the surface of Mars. On Mars, the Common Habitat is part of a surface outpost where a Habitation Zone includes the Common Habitat docked to a two-chamber airlock node, up to two logistics modules, and up to two pressurized rovers. It is connected by underground conduit to a radiator farm and communications tower assembly. These elements and other surface infrastructure, including robotic systems for surface preparation, are landed prior to the Common Habitat. In the baseline Common Habitat Architecture, the Common Habitat is delivered on the third heavy cargo flight. The Habitation Zone configuration dictates that the Common Habitat needs to be offloaded from the lander. All of the docked elements require direct access to the surface and the Common Habitat must actually be placed in a trench to lower its docking ports to be level with those of the mated elements. Additionally, the habitat must be emplaced in a horizontal configuration, while for any conceivable Earth launch system it must be launched in a vertical configuration. It is true that the Common Habitat must be offloaded from its lander on both the Moon and Mars and a common offloading system must therefore work in both destinations. Mars, however, is considered the driving case for offloading in most, but not all, aspects. A four-day internal study in 2021 recommended that a modified Starship be used to land the Common Habitat on Mars and considered multiple approaches to offload the Common Habitat from the payload section and lower it to the surface. The topic was presented at a public hackathon organized by the Johnson Space Center’s Emerge Employee Resource Group. One team took on the challenge and proposed a concept in some ways similar to the previously considered jib crane. Despite the excellent innovation in the team’s work, a number of study refinements are necessary to truly establish feasibility. These and other future work needed to mature the concept are discussed in this work.
In the last few years, many scientific applications have been developed for powerful graphics processing units (GPUs) and have achieved remarkable speedups. This success can be partially attributed to high performance host callable GPU library routines that are offloaded to GPUs at runtime. These library routines are based on C/C++-like programming toolkits such as CUDA from NVIDIA and have the same calling signatures as their CPU counterparts. Recently, with the sufficient support of C++ templates from CUDA, the emergence of template libraries have enabled further advancement in code reusability and rapid software development for GPUs. However, Expression Templates (ET), which have been very popular for implementing data parallel scientific software for host CPUs because of their intuitive and mathematics-like syntax, have been underutilized by GPU development libraries. The lack of ET usage is caused by the difficulty of offloading expression templates from hosts to GPUs due to the inability to pass instantiated expressions to GPU kernels as well as the absence of the exact form of the expressions for the templates at the time of coding. This paper presents a general approach that enables automatic offloading of C++ expression templates to CUDA enabled GPUs by using the C++ metaprogramming technique and Just-In-Time (JIT) compilation methodology to generate and compile CUDA kernels for corresponding expression templates followed by executing the kernels with appropriate arguments. This approach allows developers to port applications to run on GPUs with virtually no code modifications. More specifically, this paper uses a large ET based data parallel physics library called QDP++ as an example to illustrate many aspects of the approach to offload expression templates automatically and to demonstrate very good speedups for typical QDP++ applications running on GPUs against running on CPUs using this method of offloading. In addition, this approach of automatic offlo- ding expression templates could be applied to other many-core accelerators that provide C++ programming toolkits with the support of C++ template.
This study investigates the feasibility of using a heavy-lift variant of the Lunar Surface Manipulator System (LSMS-H) to lift and handle a 12 metric ton payload. Design challenges and requirements particular to handling heavy cargo were examined. Differences between the previously developed first-generation LSMS and the heavy-lift version are highlighted. An in-depth evaluation of the tip-over risk during LSMS-H operations has been conducted using the Synergistic Engineering Environment and potential methods to mitigate that risk are identified. The study investigated three specific offloading scenarios pertinent to current Lunar Campaign studies. The first involved offloading a large element, such as a habitat or logistics module, onto a mobility chassis with a lander-mounted LSMS-H and offloading that payload from the chassis onto the lunar surface with a surface-mounted LSMS-H. The second scenario involved offloading small pressurized rovers with a lander-mounted LSMS-H. The third scenario involved offloading cargo from a third-party lander, such as the proposed ESA cargo lander, with a chassis-mounted LSMS-H. In all cases, the analyses show that the LSMS-H can perform the required operations safely. However, Chariot-mounted operations require the addition of stabilizing outriggers, and when operating from the Lunar surface, LSMS-H functionality is enhanced by adding a simple ground anchoring system.
The HPC industry is inexorably moving towards an era of extremely heterogeneous architectures, with more devices configured on any given HPC platform and potentially more kinds of devices, some of them highly specialized. Writing a separate code suitable for each target system for a given HPC application is not practical. The better solution is to use directive-based parallel programming models such as OpenMP. OpenMP provides a number of options for offloading a piece of code to devices like GPUs. To select the best option from such options during compilation, most modern compilers use analytical models to estimate the cost of executing the original code and the different offloading code variants. Building such an analytical model for compilers is a difficult task that necessitates a lot of effort on the part of a compiler engineer. Recently, machine learning techniques have been successfully applied to build cost models for a variety of compiler optimization problems. In this paper, we present COMPOFF, a cost model which uses the multi-layer perceptrons to statically estimates the Cost of OpenMP OFFloading. We used six different transformations on a parallel code of Wilson Dslash Operator to support GPU offloading, and we predicted their cost of execution on different GPUs using COMPOFF during compile time. Our results show that this model can predict offloading costs with a root mean squared error in prediction of less than 0.5 seconds. Our preliminary findings indicate that this work will make it much easier and faster for scientists and compiler developers to port legacy HPC applications that use OpenMP to new heterogeneous computing environment.
The International Space Station (ISS) Space-to-Ground Antenna (SGANT) is used for ISS communication with earth through the Tracking and Data Relay Satellite (TDRSS). Due to the different speeds of travel between earth, ISS and TDRSS, a steerable SGANT was required on the ISS. The mechanical design of SGANT is an unbalanced mechanism with insufficient strength and driving torque to support and drive itself in a 1G environment. For ground testing, a specially designed offloading mechanism is required. Basically, the test mechanism must offload the SGANT in a two-axis operation, allowing the SGANT to move within a specific range, speed and acceleration; therefore the SGANT can move from elevation 0 to 90 deg and be tested at both the 0 and 90 deg positions. The load introduced by the test equipment should be less than 10.17 N-m (7.5 ft-lbf). The on-ground group level tracking test is quite challenging due to the unbalanced antenna mechanical design and tough specification requirements. This paper describes the detailed design, fabrication, and calibration of the test mechanism, and how the above requirements are met. The overall antenna is simplified to a mass model in order to facilitate the offloading mechanism design and analysis. An actual SGANT mass dummy was made to calibrate the system. This paper brings together the theoretical analysis and the industrial experience that were relied upon to meet the above-mentioned requirements for the ground test. The lessons learned during the calibration phase are extremely important for future double or multiple offloading system designs. The ISS SGANT QM and FM units passed their ground test and the SGANT/Boom fit check successfully, and the Flight Model (FM) was delivered to SSPF in April 1998. It is now installed on ISS and functioning well.
An extremely fast offload engine system has been developed that operates at 60 Gigabits per second (Gbps), and has scalability to 100 Gbps full-duplex (f-d). This system is based on unique coding and architecture derived from splintered UDP (User Datagram Protocol) offload technology, resulting in unique FPGA (field programmable gate array) intellectual property core and firmware. This innovation improves the networking speed of supercomputer clusters by providing an ultra-fast network protocol processing offload from a CPU (central processing unit) by inserting an offload engine into a host backplane and network connections. This runs on protocol firmware.
This work addresses distributed data management and access dynamically configurable high-speed access to data distributed and shared over wide-area high-speed network environments. An offload engine NIC (network interface card) is proposed that scales at nX10-Gbps increments through 100-Gbps full duplex. The Globus de facto standard was used in projects requiring secure, robust, high-speed bulk data transport. Novel extension mechanisms were derived that will combine these technologies for use by GridFTP, bandwidth management resources, and host CPU (central processing unit) acceleration. The result will be wire-rate encrypted Globus grid data transactions through offload for splintering, encryption, and compression. As the need for greater network bandwidth increases, there is an inherent need for faster CPUs. The best way to accelerate CPUs is through a network acceleration engine. Grid computing data transfers for the Globus tool set did not have wire-rate encryption or compression. Existing technology cannot keep pace with the greater bandwidths of backplane and network connections. Present offload engines with ports to Ethernet are 32 to 40 Gbps f-d at best. The best of ultra-high-speed offload engines use expensive ASICs (application specific integrated circuits) or NPUs (network processing units). The present state of the art also includes bonding and the use of multiple NICs that are also in the planning stages for future portability to ASICs and software to accommodate data rates at 100 Gbps. The remaining industry solutions are for carrier-grade equipment manufacturers, with costly line cards having multiples of 10-Gbps ports, or 100-Gbps ports such as CFP modules that interface to costly ASICs and related circuitry. All of the existing solutions vary in configuration based on requirements of the host, motherboard, or carriergrade equipment. The purpose of the innovation is to eliminate data bottlenecks within cluster, grid, and cloud computing systems, and to add several more capabilities while reducing space consumption and cost. Provisions were designed for interoperability with systems used in the NASA HEC (High-End Computing) program. The new acceleration engine consists of state-ofthe- art FPGA (field-programmable gate array) core IP, C, and Verilog code; novel communication protocol; and extensions to the Globus structure. The engine provides the functions of network acceleration, encryption, compression, packet-ordering, and security added to Globus grid or for cloud data transfer. This system is scalable in nX10-Gbps increments through 100-Gbps f-d. It can be interfaced to industry-standard system-side or network-side devices or core IP in increments of 10 GigE, scaling to provide IEEE 40/100 GigE compliance.
Efficient collective operations are a major component of application scalability. Offload of collective operations onto the network interface reduces many of the latencies that are inherent in network communications and, consequently, reduces the time to perform the collective operation. To support offload, it is desirable to expose semantic building blocks that are simple to offload and yet powerful enough to implement a variety of collective algorithms. This paper presents the implementation of barrier and broadcast leveraging triggered operations - a semantic building block for collective offload. Triggered operations are shown to be both semantically powerful and capable of improving performance.
Abstract We present Jay , a software framework for offloading applications in hybrid edge clouds. Jay provides an API, services, and tools that enable mobile application developers to implement, instrument, and evaluate offloading applications using configurable cloud topologies, offloading strategies, and job types. We start by presenting Jay 's job model and the concrete architecture of the framework. We then present the programming API with several examples of customization. Then, we turn to the description of the internal implementation of Jay instances and their components. Finally, we describe the Jay Workbench, a tool that allows the setup, execution, and reproduction of experiments with networks of hosts with different resource capabilities organized with specific topologies. The complete source code for the framework and workbench is provided in a GitHub repository.
Recent reliability issues with one of the fastest supercomputers in the world, Titan at Oak Ridge National Laboratory (ORNL), demonstrated the need for resilience in large-scale heterogeneous computing. OpenMP currently does not address error and failure behavior. This paper takes a first step toward resilience for heterogeneous systems by providing the concepts for resilient OpenMP offload to devices. Using real-world error and failure observations, the paper describes the concepts and terminology for resilient OpenMP target offload, including error and failure classes and resilience strategies. It details the experienced general-purpose computing graphics processing unit (GPGPU) errors and failures in Titan. It further proposes improvements in OpenMP, including a preliminary prototype design, to support resilient offload to devices for efficient handling of errors and failures in heterogeneous high-performance computing (HPC) systems.
This work evaluated the use of OpenMP tasking with target GPU offloading as a potential solution for programming productivity and performance on heterogeneous systems. Also, it is proposed a new OpenMP specification to make the implementation of heterogeneous codes simpler by using OpenMP target task, which integrates both OpenMP tasking and target GPU offloading in a single OpenMP pragma. As a test case, the authors used one of the most popular and widely used Basic Linear Algebra Subprogram Level-3 routines: triangular solver (TRSM). To benefit from the heterogeneity of the current high-performance computing systems, the authors propose a different parallelization of the algorithm by using a nonuniform decomposition of the problem. This work used target GPU offloading inside OpenMP tasks to address the heterogeneity found in the hardware. This new approach can outperform the state-of-the-art algorithms, which use a uniform decomposition of the data, on both the CPU-only and hybrid CPU-GPU systems, reaching speedups of up to one order of magnitude. The performance that this approach achieves is faster than the IBM ESSL math library on CPU and competitive relative to a highly optimized heterogeneous CUDA version. One node of Oak Ridge National Laboratory’s supercomputer, Summit, was used for performance analysis.
Modern High-Performance Computing (HPC) centers are facing a data deluge from emerging scientific applications. Supporting large data entails a significant commitment of the highthroughput center storage system, scratch space. However, the scratch space is typically managed using simple purge policies, without sophisticated end-user data services to balance resource consumption and user serviceability. End-user data services such as offloading are performed using point-to-point transfers that are unable to reconcile center s purge and users delivery deadlines, unable to adapt to changing dynamics in the end-toend data path and are not fault-tolerant. Such inefficiencies can be prohibitive to sustaining high performance. In this paper, we address the above issues by designing a framework for the timely, decentralized offload of application result data. Our framework uses an overlay of user-specified intermediate and landmark sites to orchestrate a decentralized fault-tolerant delivery. We have implemented our techniques within a production job scheduler (PBS) and data transfer tool (BitTorrent). Our evaluation using both a real implementation and supercomputer job log-driven simulations show that: the offloading times can be significantly reduced (90.4% for a 5 GB data transfer); the exposure window can be minimized while also meeting center-user Service Level Agreements.