SEARCH · Search NASA

Results for “computer system performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Geomancy: Automated Performance Enhancement through Data Layout Optimization

Large distributed storage systems such as high- performance computing (HPC) systems used by national or international laboratories require sufficient performance and scale for demanding scientific workloads and must handle shifting workloads with ease. Ideally, data is placed in locations to optimize performance, but the size and complexity of large storage systems inhibit rapid effective restructuring of data layouts to maintain performance as workloads shift. To address these issues, we have developed Geomancy, a tool that models the placement of data within a distributed storage system and reacts to drops in performance. Using a combination of machine learning techniques suitable for temporal modeling, Geomancy determines when and where a bottleneck may happen due to changing workloads and suggests changes in the layout that mitigate or prevent them. Our approach to optimizing throughput offers benefits for storage systems such as avoiding potential bottlenecks and increasing overall I/O throughput from 11% to 30%.

Bel, Oceane M.↗

DOE OSTI2014

Non-volatile memory for checkpoint storage

A system, method and computer program product for supporting system initiated checkpoints in high performance parallel computing systems and storing of checkpoint data to a non-volatile memory storage device. The system and method generates selective control signals to perform checkpointing of system related data in presence of messaging activity associated with a user application running at the node. The checkpointing is initiated by the system such that checkpoint data of a plurality of network nodes may be obtained even in the presence of user applications running on highly parallel computers that include ongoing user messaging activity. In one embodiment, the non-volatile memory is a pluggable flash memory card.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

FY 18 Status Report on the Computing Systems for the Yucca Mountain Project TSPA-LA Models and Inventory of Software used for Process Models

Sandia National Laboratories (SNL) continued evaluation of total system performance assessment (TSPA) computing systems for the previously considered Yucca Mountain Project (YMP). This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment (TSPA) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. This work is a continuation of the ongoing readiness evaluation reported in Lee and Hadgu (2014), Hadgu et al. (2015) and Hadgu and Appel (2016), and Hadgu et al. (2017). The TSPA computing hardware (2014 server cluster -CL2014) and storage system described in Hadgu et al. (2015) were used for the current analysis. One floating license of Gold Sim with Versions 9.60.300, 10.5, 11.1 and 12.0 was installed on the cluster head node, and its distributed processing capability was mapped on the cluster processors. Other supporting software were tested and installed to support the TSPA-type analysis on the server cluster. The FY18 task included developing an inventory of software used for the Yucca Mountain Project process models and preliminary assessment of status of the software; enhancing security of the cluster and setting a backup system. The 2014 server cluster and supporting software systems are fully operational to support TSPA-LA type analysis.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2020

FY19 Status Report on the Computing Systems for the Yucca Mountain Project TSPA-LA Models and Preliminary Testing of a Selected Process Model

Sandia National Laboratories continued evaluation of total system performance assessment (TSPA) computing systems for the previously considered Yucca Mountain Project. This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. The FY19 task included continued operation of the cluster; maintenance of the TSPA-LA models (with Gold Sim 9.60.300); preliminary assessment of the status of the Infiltration Model (a process model that feeds the TSPA-LA). In addition, precautionary actions were needed to extend the life of the cluster hardware. To do that, three new nodes were added to the cluster. In the event any of the original nodes fail they will be replaced with the new nodes, thereby maintaining the core capability. The 2014 cluster and supporting software systems are currently fully operational to support TSPA-LA type analysis.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Parallel Programming Models and Systems for High Performance Computing

Explore the source record for details and available documents.

Gorentla Venkata, Manjunath↗

DOE OSTI2017

Preemptive resource management for dynamically arriving tasks in an oversubscribed heterogeneous computing system

We design resource management heuristics that assign serial tasks to the nodes of a heterogeneous high performance computing (HPC) system. The value of completing these tasks is modeled using monotonically decreasing utility functions that represent the time-varying importance of the task. The value of completing a task is equal to its utility function at the time of its completion. The overall performance of this system is measured using the total utility earned by all tasks during some interval of time. To maximize the performance of such a system where the preemption of tasks is possible, we have designed, analyzed, and compared a set of resource allocation heuristic techniques. We combine two utility-aware heuristics with three different preemption techniques to create six preemption-capable heuristics. We also consider the two utility-aware heuristics without preemption. We use simulation studies to evaluate this set of eight heuristics and compare them with an FCFS heuristic, which is often used in real systems, and random assignments. In general, our set of eight heuristics is able to significantly outperform the comparison heuristics, and the preemption-capable heuristics are able to significantly increase the utility earned compared to the heuristics that do not use preemption. We analyze the performance tradeoffs among the different preemption-capable heuristics under a variety of oversubscribed workload environments.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

The Institute for Sustained Performance, Energy, and Resilience Final Project Report

Over the course of the five years of the Scientific Discovery through Advanced Computation (SciDAC-3) program, computational scientists working on behalf of the Department of Energy's Office of Science (DOE SC) exploited a new generation of petascale computing resources to make previously inaccessible discoveries in a broad range of disciplines including physics, chemistry, and materials science. The computing systems underpinning this work increased in performance potential from tens to hundreds of PFlop/s, but in the process evolving significantly from architectures in 2011, when SciDAC-3 began. Although Moore's law continues unabated, the end of Dennard scaling has necessitated a fundamental shift in computer architecture focused on power efficiency. To that end, processors are increasingly varied as they strive to satisfy performance, productivity, reliability, and energy efficiency in the face of divergent computational requirements. The diversity among these machines presents a number of challenges to merely porting today's scientific applications, much less achieving good performance. Therefore, at the beginning of SciDAC-3, USC and its collaborators anticipated that both vastly increased scale (e.g., more chips, more cores per chip, wider SIMD) and heterogeneity would exacerbate performance optimization challenges while simultaneously promoting the issues of energy and heterogeneity will exacerbate performance optimization challenges while simultaneously promoting the issues of energy consumption and resilience to the forefront. To ensure that DOE's computational scientists could successfully exploit emerging high performance computing (HPC) systems, DOE SC’s Advanced Scientific Computing Research Program (ASCR) funded the SciDAC-3 Institute for Sustained Performance, Energy, and Resilience (SUPER). Led by USC, this was a broadly based project with expertise in compilers and system tools, performance engineering, energy management, and resilience. SUPER followed the successful model developed in the previous SciDAC-2 Performance Engineering Research Institute (PERI) of leveraging the research investments DOE and others have made and integrating the results to create new capabilities beyond the reach of any one group. To accomplish this, SUPER was organized into multi-institution teams conducting the research activities in performance portability, energy efficiency, exascale resilience, and overall optimization. SUPER researchers also collaborated with other SciDAC-3 institutes, 13 of the 19 Scientific Computation Application Partnerships (SAP), as well as the broader DOE computational sciences community. These collaborations both focused our research on the real challenges facing petascale scientific computing as well as ensured the broad and immediate impact of the results of our research.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Reproduction of the Yucca Mountain Project TSPA-LA Uncertainty and Sensitivity Analyses and Preliminary Upgrade of Models

Sandia National Laboratories (SNL) continued evaluation of total system performance assessment (TSPA) computing systems for the previously considered Yucca Mountain Project (YMP). This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment (TSPA) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. This work is a continuation of the ongoing readiness evaluation reported in Lee and Hadgu (2014) and Hadgu et al. (2015). The TSPA computing hardware (CL2014) and storage system described in Hadgu et al. (2015) were used for the current analysis. One floating license of GoldSim with Versions 9.60.300, 10.5 and 11.1.6 was installed on the cluster head node, and its distributed processing capability was mapped on the cluster processors. Other supporting software were tested and installed to support the TSPA-type analysis on the server cluster. The current tasks included verification of the TSPA-LA uncertainty and sensitivity analyses, and preliminary upgrade of the TSPA-LA from Version 9.60.300 to the latest version 11.1. All the TSPA-LA uncertainty and sensitivity analyses modeling cases were successfully tested and verified for the model reproducibility on the upgraded 2014 server cluster (CL2014). The uncertainty and sensitivity analyses used TSPA-LA modeling cases output generated in FY15 based on GoldSim Version 9.60.300 documented in Hadgu et al. (2015). The model upgrade task successfully converted the Nominal Modeling case to GoldSim Version 11.1. Upgrade of the remaining of the modeling cases and distributed processing tasks will continue. The 2014 server cluster and supporting software systems are fully operational to support TSPA-LA type analysis.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2016

Rolex: Resilience-oriented language extensions for extreme-scale systems

Future exascale high-performance computing (HPC) systems will be constructed from VLSI devices that will be less reliable than those used today, and faults will become the norm, not the exception. This will pose significant problems for system designers and programmers, who for half-a-century have enjoyed an execution model that assumed correct behavior by the underlying computing system. The mean time to failure (MTTF) of the system scales inversely to the number of components in the system and therefore faults and resultant system level failures will increase, as systems scale in terms of the number of processor cores and memory modules used. However every error detected need not cause catastrophic failure. Many HPC applications are inherently fault resilient. Yet it is the application programmers who have this knowledge but lack mechanisms to convey it to the system. In this paper, we present new Resilience Oriented Language Extensions (Rolex) which facilitate the incorporation of fault resilience as an intrinsic property of the application code. We describe the syntax and semantics of the language extensions as well as the implementation of the supporting compiler infrastructure and runtime system. Furthermore, our experiments show that an approach that leverages the programmer's insight to reason about the context and significance of faults to the application outcome significantly improves the probability that an application runs to a successful conclusion.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

The Applicability of Unit Systems to High-Performance Computing Applications

Dimensional analysis is a key technique used to verify the soundness of scientific models. Most experts agree that engineering and scientific software would be made more reliable by integrating dimensional analysis in their type system. We explored how High Performance Computing (HPC) applications could integrate compile-time dimensional analysis. We started by investigating various implementation of unit systems for C++. Eventually, selecting the latest (and most advanced) one to apply to our test codes. We worked with code of increasing complexity, from a projectile trajectory calculation to the proxy-application Lulesh. This included our code, Springs-3D, which focuses on demonstrating language features while performing simple physic computations. Finally, our main contribution is a source-code analysis which extracts constraints on the dimension of all variables, functions, and constants in an application. This resulting system of equations is solved using the dimensions of a few of these objects. This analysis has the potential to greatly reduce the time spent performing dimensional analysis when refactoring application to use a representation of units.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Workload-Adaptive Scheduling for Efficient Use of Parallel File Systems in High-Performance Computing Clusters

Results from work on feedback control of a queuing system for HPC jobs with shared storage resources.

Allan, Benjamin Andrew↗

DOE OSTI2023

Enabling Scalable VQE Simulation on Leading HPC Systems

Large-scale simulations of quantum circuits pose significant challenges, especially in the context of quantum chemistry, due to the number of qubits, circuit depth, and the number of circuits needed per problem. High-performance computing (HPC) systems offer massive computational capabilities that could help overcome these obstacles. We developed a high-performance quantum circuit simulator, called NWQ-Sim, and demonstrate its capability to simulate large quantum chemistry problems on NERSC's Perlmutter supercomputer. Integrating NWQ-Sim with XACC, we have executed QPE and VQE algorithms for downfolded quantum chemistry systems at unprecedented scales. Our work demonstrates the potential of leveraging HPC resources to advance quantum chemistry and other applications of near-term quantum devices.

Wang, Meng↗

DOE OSTI2014

A Report on Simulation-Driven Reliability and Failure Analysis of Large-Scale Storage Systems

High-performance computing (HPC) storage systems provide data availability and reliability using various hardware and software fault tolerance techniques. Usually, reliability and availability are calculated at the subsystem or component level using limited metrics such as, mean time to failure (MTTF) or mean time to data loss (MTTDL). This often means settling on simple and disconnected failure models (such as exponential failure rate) to achieve tractable and close-formed solutions. However, such models have been shown to be insufficient in assessing end-to-end storage system reliability and availability. We propose a generic simulation framework aimed at analyzing the reliability and availability of storage systems at scale, and investigating what-if scenarios. The framework is designed for an end-to-end storage system, accommodating the various components and subsystems, their interconnections, failure patterns and propagation, and performs dependency analysis to capture a wide-range of failure cases. We evaluate the framework against a large-scale storage system that is in production and analyze its failure projections toward and beyond the end of lifecycle. We also examine the potential operational impact by studying how different types of components affect the overall system reliability and availability, and present the preliminary results

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Reproduction of the Yucca Mountain Project TSPA-LA Model Runs using TSPA Computing Systems

Sandia National Laboratories (SNL) conducted an evaluation of total system performance assessment (TSPA) related computing systems for the previously considered Yucca Mountain Project (YMP). This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment (TSPA) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. This work is a continuation of the ongoing readiness evaluation reported in Lee and Hadgu (2014). The current work examined main components of the computing system identified in the previous work (Lee and Hadgu, 2014) to ensure the operational readiness of the TSPA-LA model capability on the server cluster. The TSPA computing hardware and storage system were replaced in late 2014 to maintain core capability and improve computation efficiency. One floating license of GoldSim Version 9.60.300 was installed on the upgraded cluster head node, and its distributed processing capability was mapped on the cluster processors. Other supporting software was tested and installed to support the TSPA-type analysis on the server cluster. All the TSPA-LA modeling cases were tested and verified for the model reproducibility on the upgraded 2014 server cluster (CL2014). All test runs were executed on multiple processors on the server cluster utilizing the GoldSim distributed processing capability, and all runs completed successfully. The model reproducibility verification was evaluated by two approaches: numerical value comparison and graphical comparison. The analysis demonstrated an excellent reproducibility of the TSPA-LA model runs on the upgraded server cluster. The 2014 server cluster and supporting software systems are fully operational to support TSPA- LA type analysis.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Reproduction of the Yucca Mountain Project TSPA-LA Model Runs using TSPA Computing Systems

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Ad Hoc File Systems for High-Performance Computing

Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include nodelocal SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 “Challenges and Opportunities of User-Level File Systems for HPC” and discusses application scenarios as well as design strategies for ad hoc file systems using node-local storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗

DOE OSTI2021

Sentiment Analysis based Error Detection for Large-Scale Systems

Today's large-scale systems such as High Performance Computing (HPC) Systems are designed/utilized towards exascale computing, inevitably decreasing its reliability due to the increasing design complexity. HPC systems conduct extensive logging of their execution behaviour. In this paper, we leverage the inherent meaning behind the log messages and propose a novel sentiment analysis-based approach for the error detection in large-scale systems, by automatically mining the sentiments in the log messages. Our contributions are four-fold. (1) We develop a machine learning (ML) based approach to automatically build a sentiment lexicon, based on the system log message templates. (2) Using the sentiment lexicon, we develop an algorithm to detect system errors. (3) We develop an algorithm to identify the nodes and components with erroneous behaviors, based on sentiment polarity scores. (4) We evaluate our solution vs. other state-of-the-art machine/deep learning algorithms based on three representative supercomputers' system logs. Experiments show that our error detection algorithm can identify error messages with an average MCC score and f -score of 91% and 96% respectively, while state of the art ML/deep learning model (LSTM) obtains only 67% and 84%. To the best of our knowledge, this is the first work leveraging the sentiments embedded in log entries of large-scale systems for system health analysis.

error detection↗