SEARCH · Search NASA

Results for “scheduling software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Influence of Fuel Properties on Gasoline Direct Injection Particulate Matter Emissions Over First 200 s of World-Harmonized Light-Duty Test Procedure Using an Engine Dynamometer and Novel “Virtual Drivetrain” Software

Abstract The influence of fuel properties on particulate matter (PM) emissions from a catalytic gasoline particulate filter (GPF) equipped gasoline direct injection (GDI) engine was investigated using novel “virtual drivetrain” software and an engine mated to an engine dynamometer. The virtual drivetrain software was developed in labview to operate the engine on an engine dynamometer as if it were in a vehicle undergoing a driving cycle. The software uses a physics-based approach to determine vehicle acceleration and speed based on engine load and a programed “shift” schedule to control engine speed. The software uses a control algorithm to modulate engine load and braking to match a calculated vehicle speed with the prescribed speed trace of the driving cycle of choice. The first 200 s of the World-harmonized Light-duty Test Procedure (WLTP) driving cycle was tested using six different fuel formulations of varying volatility, aromaticity, and ethanol concentration. The first 200 s of the WLTP was chosen as the test condition because it is the most problematic section of the driving cycle for controlling PM emissions due to the cold start and cold drive off. It was found that there was a strong correlation between aromaticity of the fuel and the engine-out PM emissions, with the highest emitting fuel producing more than double the mass emissions of the low PM production fuel. However, the post-GPF PM emissions depended greatly on the soot loading state of the GPF. The fuel with the highest engine-out PM emissions produced comparable post-GPF emissions to the lowest PM producing fuel over the driving cycle when the GPF was loaded over three cycles with the respective fuels. These results demonstrate the importance of GPF loading state when aftertreatment systems are used for PM reduction. It also shows that GPF control may be more important than fuel properties, and that regulatory compliance for PM can be achieved with proper GPF control calibration irrespective of fuel type.

Energy & Fuels↗

DOE OSTI2026

Alchemy: A Model-Based Approach for 2D to 3D Autonomous Nuclear System Design

Engineering design of nuclear power plant (NPP) piping and equipment systems frequently bypasses crucial 2D system planning, instead moving straight to 3D modeling. This often leads to designs that exceed building envelope constraints, forcing expensive and time-consuming redesigns. When 2D modeling is employed, it typically involves labor-intensive manual workflows that convert 2D drawings into 3D models, resulting in inefficiencies and errors across design iterations. These workflows further suffer from poor software interoperability and dependence on proprietary software ecosystems, thereby contributing to schedule delays and cost overruns. This paper presents Alchemy, an autonomous framework that transforms 2D system definitions into Industry Foundation Classes (IFC)-compliant 3D building information models (BIMs) for expediting nuclear facility design at the conceptual preliminary phase. Using a model-based approach, the framework treats the 2D system diagram as the central reference model employed to automatically generate all subsequent outputs, ensuring consistency between the system definition and the resulting physical design. A web-based interface enables engineers to define hierarchical system topologies including associated equipment, geometric properties, and connectivity requirements. A two-phase equipment layout optimization algorithm automatically computes collision-free spatial configurations within predefined building envelopes. An artificial intelligence (AI)-assisted pipe routing module then generates orthogonal, collision-free routing paths, allowing the user to select either an A* search-based method or an Ant Colony Optimization (ACO)-based method. All outputs are authored natively in IFC format, relying on open-source technologies and standardized formats in order to ensure extensibility and eliminate proprietary software dependencies. The proposed framework is validated on two representative pressurized-water reactor (PWR)-based case studies, for which it autonomously generates IFC-compliant 3D models in minutes, drastically reducing workflows that typically require hours of manual effort. The generated model demonstrates topologically correct equipment placement, physically plausible spatial relationships, and collision-free pipe routing consistent with known PWR loop configurations. This work represents a foundational step toward digital engineering for nuclear facility preliminary design, with future ongoing development targeting design code compliance and expanded system complexity.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2017

LArSoft: Toolkit for Simulation, Reconstruction and Analysis of Liquid Argon TPC Neutrino Detectors

LArSoft is a set of detector-independent software tools for the simulation, reconstruction and analysis of data from liquid argon (LAr) neutrino experiments The common features of LAr time projection chambers (TPCs) enable sharing of algorithm code across detectors of very dierent size and conguration. LArSoft is currently used in production simulation and reconstruction by the ArgoNeuT, DUNE, LArIAT, MicroBooNE, and SBND experiments. The software suite oers a wide selection of algorithms and utilities, including those for associated photo-detectors and the handling of auxiliary detectors outside the TPCs. Available algorithms cover the full range of simulation and reconstruction, from raw waveforms to high- level reconstructed objects, event topologies and classication. The common code within LArSoft is contributed by adopting experiments, which also provide detector-specic geometry descriptions, and code for the treatment of electronic signals. LArSoft is also a collaboration of experiments, Fermilab and associated software projects which cooperate in setting requirements, priorities, and schedules. In this talk, we outline the general architecture of the software and the interaction with external libraries and detector-specic code. We also describe the dynamics of LArSoft software development between the contributing experiments, the projects supporting the software infrastructure LArSoft relies on, and the core LArSoft support project.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

Virtual Engineering Software Framework for Integrated Biomass Conversion Modeling

This presentation covers the design and implementation of a software tool to systematically connect computational models of unit operations to simulate an integrated process of low-temperature conversion of biomass to fuel. This virtual engineering (VE) software was designed with the overarching goal of connecting unit models written in various programming languages and requiring different computational resources within a single, flexible framework. The models and features currently considered for the VE library include mechanistic models for pretreatment, enzymatic hydrolysis, and aerobic bioreaction; high-fidelity computational fluid dynamics (CFD) simulations for enzymatic hydrolysis and aerobic bioreaction; and the capability to perform techno-economic analyses (TEA) using Aspen Plus, a commercial software package. The CFD models require access to high-performance computing (HPC) resources, so in addition to handling multiple programming languages and interfaces, the VE software must also be capable of interacting with an HPC scheduler to submit, run, and post-process jobs. Using the Python programming language, a new VE software package has been developed that contains functionality to manage the input-output communication between various unit models, schedule simulations to run on NREL's HPC and analyze those results, and interface with existing TEA software workflows. A Jupyter-notebook GUI was also created to solicit user input and provide documentation. In cases where multiple models for a particular unit-operation exist, selection between models is accomplished through a simple checkbox, with the appropriate inputs and outputs being parsed and converted seamlessly in the background. Each operation makes use of a different programming language, but the flow of information from pretreatment to enzymatic hydrolysis to bioreaction is managed with an intuitive, centralized file-communication strategy. In this talk, the programming approach and implementation details of the notebook are presented for multiple possibilities of the conversion process, including a demonstration of the ability to manage HPC resources. Additionally, an example of a sensitivity study of treatment parameters governing the overall conversion outcome is shown which highlights the ease of defining new problems using the VE Notebook workflow and leads into a discussion of ongoing work to enable outer-loop optimization studies.

biofuel↗

DOE OSTI2012

Integrated Cost and Schedule using Monte Carlo Simulation of a CPM Model - 12419

This discussion of the recommended practice (RP) 57R-09 of AACE International defines the integrated analysis of schedule and cost risk to estimate the appropriate level of cost and schedule contingency reserve on projects. The main contribution of this RP is to include the impact of schedule risk on cost risk and hence on the need for cost contingency reserves. Additional benefits include the prioritizing of the risks to cost, some of which are risks to schedule, so that risk mitigation may be conducted in a cost-effective way, scatter diagrams of time-cost pairs for developing joint targets of time and cost, and probabilistic cash flow which shows cash flow at different levels of certainty. Integrating cost and schedule risk into one analysis based on the project schedule loaded with costed resources from the cost estimate provides both: (1) more accurate cost estimates than if the schedule risk were ignored or incorporated only partially, and (2) illustrates the importance of schedule risk to cost risk when the durations of activities using labor-type (time-dependent) resources are risky. Many activities such as detailed engineering, construction or software development are mainly conducted by people who need to be paid even if their work takes longer than scheduled. Level-of-effort resources, such as the project management team, are extreme examples of time-dependent resources, since if the project duration exceeds its planned duration the cost of these resources will increase over their budgeted amount. The integrated cost-schedule risk analysis is based on: - A high quality CPM schedule with logic tight enough so that it will provide the correct dates and critical paths during simulation automatically without manual intervention. - A contingency-free estimate of project costs that is loaded on the activities of the schedule. - Resolves inconsistencies between cost estimate and schedule that often creep into those documents as project execution proceeds. - Good-quality risk data that are usually collected in risk interviews of the project team, management and others knowledgeable in the risk of the project. The risks from the risk register are used as the basis of the risk data in the risk driver method. The risk driver method is based in the fundamental principle that identifiable risks drive overall cost and schedule risk. - A Monte Carlo simulation software program that can simulate schedule risk, burn WM2012 rate risk and time-independent resource risk. The results include the standard histograms and cumulative distributions of possible cost and time results for the project. However, by simulating both cost and time simultaneously we can collect the cost-time pairs of results and hence show the scatter diagram ('football chart') that indicates the joint probability of finishing on time and on budget. Also, we can derive the probabilistic cash flow for comparison with the time-phased project budget. Finally the risks to schedule completion and to cost can be prioritized, say at the P-80 level of confidence, to help focus the risk mitigation efforts. If the cost and schedule estimates including contingency reserves are not acceptable to the project stakeholders the project team should conduct risk mitigation workshops and studies, deciding which risk mitigation actions to take, and re-run the Monte Carlo simulation to determine the possible improvement to the project's objectives. Finally, it is recommended that the contingency reserves of cost and of time, calculated at a level that represents an acceptable degree of certainty and uncertainty for the project stakeholders, be added as a resource-loaded activity to the project schedule for strategic planning purposes. The risk analysis described in this paper is correct only for the current plan, represented by the schedule. The project contingency reserve of time and cost that are the main results of this analysis apply if that plan is to be followed. Of course project managers have the option of re-planning and re-scheduling in the face of new facts, in part by mitigating risk. This analysis identifies the high-priority risks to cost and to schedule, which assist the project manager in planning further risk mitigation. Some project managers reject the results and argue that they cannot possibly be so late or so overrun. Those project managers may be wasting an opportunity to mitigate risk and get a more favorable outcome. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2022

Thermal-Hydraulics Modeling and Simulations of Hot Pool Using the SAS-CFD Coupled Code

The main goal of this activity is to test the dynamic coupling of the SAS4A/SASSYS-1 (SAS) and CFD models, using a recently patched version of the SAS code intended to address an undocumented limitation that hindered the Versatile Test Reactor (VTR) simulation efforts in FY21. As described in previous VTR calculation reports, the undocumented limitation in SAS v5.4 does not allow the user to activate the CFD coupling option during restart calculations. Since the analysts were unaware of this limitation, prior SAS-CFD simulation results for the protected station blackout (PSBO) transient were erroneous. Root-cause analysis was performed to determine the cause of this undocumented limitation in SAS v5.4, the SAS software was updated in a new patch, and the SAS-CFD simulations were repeated with this patched software. The results of the SAS-CFD simulations documented in this report show that the software patch does address the cited issue, and that the patched software indeed supports the activation of the CFD coupling model in restart calculations. The SAS development team will determine the schedule for implementing the patch in an official software release. This report documents updated SAS-CFD simulations of the PSBO transient response in the VTR. The hot pool is modeled with the CFD code STAR-CCM+, which is coupled at the flow boundaries to the SAS model of the primary heat transport system. SAS computes the mass flow rate and temperature at each core subassembly outlet, the thermal insulation cavity bypass, and the IHX inlet windows. CFD in turn computes the absolute pressure and temperature at each of these boundaries. The SAS code will ignore the temperature data at flow boundaries where flow is directed into the hot pool, i.e., at the core subassembly outlets unless flow reversal occurs. Similarly, CFD will ignore temperature data at boundaries where the flow is directed out of the hot pool, i.e., at the IHX inlets except under flow reversal. The focus of this work is to ensure that the SAS software patch addresses the undocumented limitation described in prior VTR calculation reports, rather than the accurate assessment of thermal stratification in the VTR during protected transients. This motivates the development of a new, simplified CFD model with a coarser mesh to accelerate the testing process. The updated model, and simplifying assumptions, are documented in this report. In future work, the thermal stratification assessment should be performed in more detail. The simplified CFD model can be improved by performing grid convergence studies sensitivity studies of turbulence parameters (e.g., Prandtl number, turbulence production and dissipation parameters) on temperature distributions and thermal stratification.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2022

Ambassador Project (Final Report)

SEL, in collaboration with its partners, performed research, development and demonstration of a trust, data, and resource management software layer enabling multiple software applications from different suppliers to operate in a Software Defined Infrastructure. The Energy sector has successfully deployed Software Defined Networking (SDN) technology and is rapidly scaling deployments. One major benefit of SDN is the abstracted centralized nature of the orchestration and management of the network. This architecture enables end users to deploy new technology and capabilities as quickly as they perform firmware upgrades to field devices. When an end user needs to roll trucks to install new equipment, they simply deploy new software to run on top of this SD-Infrastructure. This has attracted many software companies to develop value added features and introduce new capabilities, which is all good for the industry but has also introduced new challenges, including how to manage trust between these software applications, how to manage the data integrity and access, and how to manage access to the network resources each of these software applications are demanding. Completion of this project developed and demonstrated a safe and reliable resource scheduler and cybersecurity trust management communication platform allowing multiple software applications to operate in a SD-Infrastructure resulting in the commercialization of a software eco-system and architecture that maintains interoperability, cybersecurity, and reliability where the end user can safely scale out their software to leverage the powerful benefits of SDN.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

TU-C-BRE-11: 3D EPID-Based in Vivo Dosimetry: A Major Step Forward Towards Optimal Quality and Safety in Radiation Oncology Practice

Purpose: To develop a 3D in vivo dosimetry method that is able to substitute pre-treatment verification in an efficient way, and to terminate treatment delivery if the online measured 3D dose distribution deviates too much from the predicted dose distribution. Methods: A back-projection algorithm has been further developed and implemented to enable automatic 3D in vivo dose verification of IMRT/VMAT treatments using a-Si EPIDs. New software tools were clinically introduced to allow automated image acquisition, to periodically inspect the record-and-verify database, and to automatically run the EPID dosimetry software. The comparison of the EPID-reconstructed and planned dose distribution is done offline to raise automatically alerts and to schedule actions when deviations are detected. Furthermore, a software package for online dose reconstruction was also developed. The RMS of the difference between the cumulative planned and reconstructed 3D dose distributions was used for triggering a halt of a linac. Results: The implementation of fully automated 3D EPID-based in vivo dosimetry was able to replace pre-treatment verification for more than 90% of the patient treatments. The process has been fully automated and integrated in our clinical workflow where over 3,500 IMRT/VMAT treatments are verified each year. By optimizing the dose reconstruction algorithm and the I/O performance, the delivered 3D dose distribution is verified in less than 200 ms per portal image, which includes the comparison between the reconstructed and planned dose distribution. In this way it was possible to generate a trigger that can stop the irradiation at less than 20 cGy after introducing large delivery errors. Conclusion: The automatic offline solution facilitated the large scale clinical implementation of 3D EPID-based in vivo dose verification of IMRT/VMAT treatments; the online approach has been successfully tested for various severe delivery errors.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2013

Physics-based seismic hazard analysis on petascale heterogeneous supercomputers, In: SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

We have developed a highly scalable and efficient GPU-based finite-difference code (AWP) for earthquake simulation that implements high throughput, memory locality, communication reduction and communication/computation overlap and achieves linear scalability on Cray XK7 Titan at ORNL and NCSA's Blue Waters system. We simulate realistic 0-10 Hz earthquake ground motions relevant to building engineering design using high-performance AWP. Moreover, we show that AWP provides a speedup by a factor of 110 in key strain tensor calculations critical to probabilistic seismic hazard analysis (PSHA). These performance improvements to critical scientific application software, coupled with improved co-scheduling capabilities of our workflow-managed systems, make a statewide hazard model a goal reachable with existing supercomputers. The performance improvements of GPU-based AWP are expected to save millions of core-hours over the next few years as physics-based seismic hazard analysis is developed using heterogeneous petascale supercomputers.

Computer Science↗

DOE OSTI2024

SmartFuse: Reconfigurable Smart Switches to Accelerate Fused Collectives in HPC Applications

Communication switches have sometimes been augmented to process collectives (e.g., the IBM BlueGene project and the Mellanox SHArP switch). In this work, we find that there is a great acceleration opportunity through the further augmentation of switches to accelerate more complex functions that combine communication with computation. We consider three types of such functions. The first is fully-fused collectives built by fusing multiple existing collectives like Allreduce with Alltoall. The second is semi-fused collectives built by combining a collective with another computation. The third we refer to as higher-order collectives built by combining multiple computations and communications, such as to perform matrix-matrix multiply (PGEMM). In this work, we propose a framework called SmartFuse to accelerate fused collective functions. The core of SmartFuse is a reconfigurable smart switch to support these operations. The semi/fully fused collectives are implemented with a CGRAlike architecture, while higher-order collectives are implemented with a more specialized computational unit that can also schedule communication. Supporting our framework is software to evaluate and translate relevant parts of the input program, compile them into a control data flow graph, and then map this graph to the switch hardware. The proposed framework, once deployed, has the strong potential to accelerate existing HPC applications transparently by encapsulation within an MPI implementation. Experimental results show that this approach improves the performance of the PGEMM kernel, MINIFE, and AMG by, on average, 94%, 15%, and 13%, respectively.

Haghi, Pouya↗

DOE OSTI2015

Brahman

The Adversary & Interdiction Methods (AIM) program provides training and capability assessment services to government agencies around the country. Interdisciplinary teams equipped with gear and radioactive sources are repeatedly fielded to offsite events to collaborate with law enforcement agencies at all levels of government. AIM has grown rapidly over the past three years. A knowledge management system as evolved along with the program but it has failed to keep pace. A new system is needed. The new system must comply with cybersecurity and information technology solutions already in place at an institutional level. The offsite nature of AIM activities must also be accommodated. Cost and schedule preclude the commissioning of new software and the procurement of expensive hardware. The new system must exploit in-house capabilities and be established quickly. A novel system is proposed. This solution centers on a recently introduced institutional file sharing capability called Syncplicity. AIM-authored software will be combined with a dedicated institutional account to vastly extend the capability of this resource. The new knowledge management system will reduce error and increase efficiency through automation and be accessible offsite via mobile devices.

42 ENGINEERING↗

DOE OSTI2023

PSI/J: A Portable Interface for Submitting, Monitoring, and Managing Jobs

It is generally desirable for high-performance computing (HPC) applications to be portable between HPC systems, for example to make use of more performant hardware, make effective use of allocations, and to co-locate compute jobs with large datasets. Unfortunately, moving scientific applications between HPC systems is challenging for various reasons, most notably that HPC systems have different HPC schedulers. We introduce PSI/J, a job management abstraction API intended to simplify the construction of software components and applications that are portable over various HPC scheduler implementations. We argue that such a system is both necessary and that no viable alternative currently exists. We analyze similar notable APIs and attempt to determine the factors that influenced their evolution and adoption by the HPC community. We base the design of PSI/J on that analysis. We describe how PSI/J has been integrated in three workflow systems and one application, and also show via experiments that PSI/J imposes minimal overhead.

Hategan, Mihael↗

DOE OSTI2011

Software Technology Readiness for the Smart Grid

Abstract Budget and schedule overruns in product development due to the use of immature technologies constitute an important matter for program managers. Moreover, unexpected lack of technology maturity is also a problem for buyers. Both sides of the situation would benefit from an unbiased measure of technology maturity. This paper presents the use of a software maturity metric called Technology Readiness Level (TRL), in the milieu of the smart grid. For most of the time they have been in existence, power utilities have been protected monopolies, guaranteed a return on investment on anything they could justify adding to the rate base. Such a situation did not encourage innovation, and instead led to widespread risk-avoidance behavior in many utilities. The situation changed at the end of the last century, with a series of regulatory measures, beginning with the Public Utility Regulatory Policy Act of 1978. However, some bad experiences have actually served to strengthen the resistance to innovation by some utilities. Some aspects of the smart grid, such as the addition of computer-based control to the power system, face an uphill battle. It is our position that the addition of TRLs to the decision-making process for smart grid power-system projects, will lead to an environment of more confident adoption.

technology maturity, software technology readiness↗

DOE OSTI2020

Automated Data Acquisition Software for National Instruments Hardware with Timed Acquisition, Real-Time Display and Spectrogram, and Data Redundancy. Current version is Version 7 of DataAcq.exe.RElease1.2.4.26.

This software automates the collection of waveform data on a timed schedule so that an operator does not need to be at the computer to initiate the collection of data sets.It provides real time display of up to 16 channels of data as a time display, frequency display, or frequency vs time (spectrogram) display. The software provides the ability to collect data sets of a user selected time length at a selected interval. This operation can be set to occur during a selected time interval during the day. This software provides the capability to write two copies of the data files simultaneously so that the data is backed up as collected. This software automates the collection of waveform data on a timed schedule so that an operator does not need to be at the computer to initiate the collection of data sets. It provides real time display of up to 16 channels of data as a time display, frequency display, or frequency vs time (spectrogram) display. The software provides the ability to collect data sets of a user selected time length at a selected interval. This operation can be set to occur during a selected time interval during the day. This software provides the capability to write two copies of the data files simultaneously so that the data is backed up as collected.

Wysor, Robert W↗

DOE OSTI2015

Database Performance Monitoring for the Photovoltaic Systems

The Database Performance Monitoring (DPM) software (copyright in processes) is being developed at Sandia National Laboratories to perform quality control analysis on time series data. The software loads time indexed databases (currently csv format), performs a series of quality control tests defined by the user, and creates reports which include summary statistics, tables, and graphics. DPM can be setup to run on an automated schedule defined by the user. For example, the software can be run once per day to analyze data collected on the previous day. HTML formatted reports can be sent via email or hosted on a website. To compare performance of several databases, summary statistics and graphics can be gathered in a dashboard view which links to detailed reporting information for each database. The software can be customized for specific applications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

ExaWorks software development kit: a robust and scalable collection of interoperable workflows technologies

Scientific discovery increasingly requires executing heterogeneous scientific workflows on high-performance computing (HPC) platforms. Heterogeneous workflows contain different types of tasks (e.g., simulation, analysis, and learning) that need to be mapped, scheduled, and launched on different computing. That requires a software stack that enables users to code their workflows and automate resource management and workflow execution. Currently, there are many workflow technologies with diverse levels of robustness and capabilities, and users face difficult choices of software that can effectively and efficiently support their use cases on HPC machines, especially when considering the latest exascale platforms. We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK). The SDK is a curated collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms. We present our experience with (1) curating those technologies, (2) integrating them to provide users with new capabilities, (3) developing a continuous integration platform to test the SDK on DOE HPC platforms, (4) designing a dashboard to publish the results of those tests, and (5) devising an innovative documentation platform to help users to use those technologies. Our experience details the requirements and the best practices needed to curate workflow technologies, and it also serves as a blueprint for the capabilities and services that DOE will have to offer to support a variety of scientific heterogeneous workflows on the newly available exascale HPC platforms.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Kubernetes Bindings for the Flux Workload Manager

Flux is a framework for writing resource managers as well as a resource manager for HPC systems. It allows for hierarchical nesting and providing a portable interface for job scheduling across systems. The flux-k8s software provides an interface between Flux and Kubernetes so the two frameworks can use each other's functionality. The code is currently licensed under the LGPL-3.0 License, but we request to re-license the code to promote outside collaboration and community adoption.

Ahn, DongH↗

DOE OSTI2015

Analyzing the Interplay of Failures and Workload on a Leadership-Class Supercomputer

The unprecedented computational power of cur- rent supercomputers now makes possible the exploration of complex problems in many scientific fields, from genomic analysis to computational fluid dynamics. Modern machines are powerful because they are massive: they assemble millions of cores and a huge quantity of disks, cards, routers, and other components. But it is precisely the size of these machines that glooms the future of supercomputing. A system that comprises many components has a high chance to fail, and fail often. In order to make the next generation of supercomputers usable, it is imperative to use some type of fault tolerance platform to run applications on large machines. Most fault tolerance strategies can be optimized for the peculiarities of each system and boost efficacy by keeping the system productive. In this paper, we aim to understand how failure characterization can improve resilience in several layers of the software stack: applications, runtime systems, and job schedulers. We examine the Titan supercomputer, one of the fastest systems in the world. We analyze a full year of Titan in production and distill the failure patterns of the machine. By looking into Titan s log files and using the criteria of experts, we provide a detailed description of the types of failures. In addition, we inspect the job submission files and describe how the system is used. Using those two sources, we cross correlate failures in the machine to executing jobs and provide a picture of how failures affect the user experience. We believe such characterization is fundamental in developing appropriate fault tolerance solutions for Cray systems similar to Titan.

Cray↗