Search NASA⌕ Search

SEARCH · Search NASA

Results for “Workload”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,441 records · Page 80

procmon

procmon performs large-scale monitoring, data collection, and analysis of user processes running on Linux systems. It is intended to be used to aid in performing an automated workload analysis of very large scale systems, but may also be useful for live monitoring of the running processes on large scale systems.

Jacobsen, Douglas↗

Sirocco Storage Server v. pre-alpha 0.1

Sirocco is a parallel storage system under development, designed for write-intensive workloads on large-scale HPC platforms. It implements a keyvalue object store on top of a set of loosely federated storage servers that cooperate to ensure data integrity and performance. It includes support for a range of different types of storage transactions. This software release constitutes a conformant storage server, along with the client-side libraries to access the storage over a network.

Curry, MatthewL.↗

Processor Emulator with Benchmark Applications

A processor emulator and a suite of benchmark applications have been developed to assist in characterizing the performance of data-centric workloads on current and future computer architectures. Some of the applications have been collected from other open source projects. For more details on the emulator and an example of its usage, see reference [1].

Lloyd, G. Scott↗

ExaSP2

ExaSP2 is a reference implementation of typical linear algebra algorithms and workloads for a quantum molecular dynamics (QMD) electronic structure code. The algorithm is based on a recursive second-order Fermi-Operator expansion method (SP2) and is tailored for density functional based tight-binding calculations of material systems.

Negre, Christian↗

MCR Container Tools

MathWorks' MATLAB is widely used in academia and industry for prototyping, data analysis, data processing, etc. Many users compile their programs using the MATLAB Compiler to run on workstations/computing clusters via the free MATLAB Compiler Runtime (MCR). The MCR facilitates the execution of code calling Application Programming Interfaces (API) functions from both base MATLAB and MATLAB toolboxes. In a Linux environment, a sizable number of third-party runtime dependencies (i.e. shared libraries) are necessary. Unfortunately, to the MTLAB community's knowledge, these dependencies are not documented, leaving system administrators and/or end-users to find/install the necessary libraries either as runtime errors resulting from them missing or by inspecting the header information of Executable and Linkable Format (ELF) libraries of the MCR to determine which ones are missing from the system. To address various shortcomings, Docker Images based on Community Enterprise Operating System (CentOS) 7, a derivative of Redhat Enterprise Linux (RHEL) 7, containing recent (2015-2017) MCR releases and their dependencies were created. These images, along with a provided sample Docker Compose YAML Script, can be used to create a simulated computing cluster where MATLAB Compiler created binaries can be executed using a sample Slurm Workload Manager script.

Haas, NicholasQ↗

Advanced Tri-lab Software Environment (ATSE)

The Advanced Tri-lab Software Environment (ATSE) is an effort to build an open, modular, extensible, community-engaged, and vendor-adaptable software ecosystem that enables the prototyping of new technologies for improving the ASC computing environment. The initial target for ATSE is to accelerate the maturity of the Arm ecosystem for supporting ASC computing and the high-performance computing community more broadly. ATSE provides an integrated and optimized software stack that includes: 1) Application development environment and libraries including compilers, math libraries, tools, MPI, and OpenMP, 2) Low-level system software including optimized Linux, network stack, file systems, containers, and virtual machines, 3) Job scheduling and management including workload manager, application launcher, and user tools, and 4) System administration and management tools supporting booting, monitoring, and operating system image management. SAND2020-12377 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Younge, Andrew J.↗

TaskWorks v1.0

TaskWorks is a portable, high-level, task engine designed for HPC workloads. Applications can create tasks and define dependencies between them with the task engine. Once the task is defined and submitted to TaskWorks, the TaskWorks engine will execute it according to the specified dependencies, without additional input from the application. TaskWorks has an integrated event manager monitors files, sockets, timers, and MPI operations, and these events can be used as task dependencies. TaskWorks is compatible with MPI and is designed to work efficiently with HPC applications that perform.

Koziol, Quincey↗

Customs: Inspect and Report Python Imports (customs) v1.0.0

We have developed a simple, minimally invasive monitoring framework that enables us to capture key metrics from Python user processes. This is part of a larger effort to collect data across our entire data-intensive science workload at NERSC. The approach for Python is inspired by a similar framework developed for Blue Waters at NCSA. Both frameworks leverage standard Python features, sitecustomize and atexit, to capture Python imports of interest and other job data. The customs package decomposes the information capture into separate inspection and reporting interfaces.

Thomas, Rollin↗

NERSC Slurm Software Configurations and Plugins

This software can be used to tune the HPC operations of a Cray system running Slurm workload manager software. This includes many of the customizations that NERSC has made over the past three years. In particular these customizations allow for complex site-local policies to be implemented and managed in a scalable way, or add unique capabilities to slurm via our plugins, including: * performance software integration and compute node staging * temporary directory/dynamic user content management automation * software defined networking integration

Jacobsen, Douglas↗

vvtools v. 1.0

Vvtools is a suite of testing tools, with a focus on reproducible verification and validation. They are written in pure Python, and contain a test harness and an automated process management tool. Users of vvtools can develop suites of verification and validation tests and run them on small to large high performance computing resources in an automated and reproducible way. The test harness enables complex processes to be performed in each test and even supports a one-level parent/child dependency between tests. It includes a built in capability to manage workloads requiring multiple processors and platforms that use batch queueing systems.

Drake, RichardR.↗

DLIO: A DATA-CENTRIC BENCHMARK FOR DEEP LEARNING APPLICATIONS

SF-22-136 Deep learning has been shown as a successful method for various tasks, and its popularity results in numerous open-source deep learning software tools. Deep learning has been applied to a broad spectrum of scientific domains such as cosmology, particle physics, computer vision, fusion, and astrophysics. Scientists have performed a great deal of work to optimize the computational performance of deep learning frameworks. However, the same cannot be said for I/O performance. As deep learning algorithms rely on big-data volume and variety to effectively train neural networks accurately, I/O is a significant bottleneck on large-scale distributed deep learning training. DLIO, is a novel representative benchmark suite built based on the I/O profiling of the selected workloads. DLIO can be utilized to accurately emulate the I/O behavior of modern deep learning applications. Using DLIO, application developers and system software solution architects can identify potential I/O bottlenecks in their applications and guide optimizations to boost the I/O performance leading to lower training times. The storage vendor can also use DLIO as a guide for designing and optimize the storage and filesystem targeting at deep learning application.

ZHENG, HUIHUO↗

LCIO

Performance of file systems shift during their life cycles. Evaluating this performance change over time is not trivial. Complexity arises in the interplay between external (i.e. application I/O workloads) and internal (i.e. the filesystem state) factors. Many benchmarks can test how a filesystem performs at the current snapshot state, but to observe the change over time necessitates that the filesystem state mutate (age) between benchmark runs. For a large-scale HPC parallel filesystem, the sheer scale and amount of interacting components during I/O operations magnify these challenges. LCIO addresses the question - how will the filesystem perform at different stages of its life cycle? LCIO is a synthetic benchmark, which provides the file system aging process to increase the amount of information that existing benchmarks like IOR and MDTest yield, as well as provide additional points of data that will be useful to system architects and engineers.

Bachstein, Matthew↗

An Integrated Tool for Functional Requirements Analysis, Function Allocation and Task Analysis

The software application is called "HFE-Trace". This is an integrated method and tool for the management of Human Factors Engineering analyses and related data. Its primary purpose is to support the coherent and consistent application of the nuclear industry's best practices for human factors engineering work. The software is a custom Microsoft® Access® application. The application is used (in conjunction with other tools such as spreadsheets, checklists and normal documents where necessary) to collect data on the design of a new nuclear power plant from subject matter experts and other sources. This information is then used to identify potential system and functional breakdowns of the intended power plant design. This information is expanded by developing extensive descriptions of all functions, as well as system performance parameters, operating limits and constraints, and operational conditions. Once these have been verified, the human factors elements are added to each function, including intended operator role, function allocation considerations, prohibited actions, primary task categories, and primary work station. In addition, the application includes a computational method to assess a number of factors such as system and process complexity, workload, environmental conditions, procedures, regulations, etc.) that may shape operator performance. This is a unique methodology based upon principles described in NUREG/CR-3331 ("A methodology for allocating nuclear power plant control functions to human or automatic control") and it results in a semi-quantified allocation of functions to three or more levels of automation for a conceptual automation system. The aggregate of all this information is then linked to the Task Analysis section of the application where the existing information on all operator functions is transformed into task information and ultimately into design requirements for Human-System Interfaces and Control Rooms. This final step includes assessment of methods to prevent potential operator errors.

Hugo, Jacques↗

pnnl/emp-gnn

Efficient Graph Neural Network for Predicting Molecular Properties software can compare the prediction quality with ab initio DFT results reported by the high-performance state-of-theart NWChem quantum chemistry package [1] through Mean Absolute Error (MAE) obtained by the fitting between DFT and model predictions i.e, MPNN [2]. We also demonstrated the performance benefits by grouping large molecules by atom sizes, and executing GNN models on different types of resources. Since the training times depend on the number of atoms, we demonstrate the impact of distributing the workloads on two GPUs with varying capabilities (e.g., NVIDIA A100 vs. GeForce RTX 2080 Ti) to optimize the efficiency. We are at the precipice of broad adoption of GNNs for molecular property prediction tasks; hence, our work is timely by comparing model prediction against classical approaches with the intent of providing initial screening for specific classes of molecules.

Lee, Hyungro↗

High Velocity AI Cache (HVAC)

HVAC is a high-performance caching system designed for large-scale AI training workloads.

Khan, Awais [Oak Ridge National Laboratory (ORNL),↗

Adaptive Computing (AC) [SWR-24-106]

The Adaptive Computing (AC) software stack supports goal-based computing, for which a simulation workload is created on the fly adapting to the results of calculations. Application-specific code defines an objective, which may be to solve an optimization problem or to train a surrogate model with minimal uncertainty. Then, the AC driver decides where in the design parameter space to run simulations to best achieve that objective. This process is iterative and online; as new data is returned from simulations, the AC driver chooses new simulations to run. The AC driver can strategically run simulations on distributed hardware resources (including high performance computing machines, cloud resources, and edge devices) to maximize throughput and obey resource constraints.

Griffin, Kevin [National Renewable Energy Laborato↗

ESnet-JLab FPGA Accelerated Transport (control plane) [EJFAT (udplbd2)] v2.0

The ESnet-JLab FPGA Accelerated Transport system is a solution for streaming high-speed scientific measurement data from Data Acquisition Systems (DAQs) to high-performance computing facilties. It is generally compatible with many science workflows, and makes no assumptions about the specifics of any particular experiment. This program (udplbd version 2) implements the control plane for the system. It is responsible for programming network forwarding rules into the data plane (implemented by the hardware designed named udplb, described separately). It also implements the control loop necessary to match up offered workload with available capacity on high-performance compute nodes.

Howard, Derek [Lawrence Berkeley National Laborato↗

BULKI-Store v0.3.2

BULKI-Store is a distributed object storage system optimized for high-performance computing environments. Built with a Rust core and Python bindings, it efficiently manages scientific and machine learning datasets across HPC clusters. The system employs a client-server architecture with MPI integration, enabling seamless scaling on supercomputers like Perlmutter. BULKI-Store's object-oriented approach provides intuitive data organization with rich metadata support, contrasting with traditional file-based solutions. Key optimizations include selective checkpoint loading, unified checkpoint files, and object chunking for large data transfers. For machine learning workloads, BULKI-Store offers advantages through fine-grained access patterns, dynamic data sharing between training instances, and reduced memory pressure. Memory management features include strategic Python GC calls, minimized data copies, and batch processing capabilities. The system leverages Rayon's thread pool for asynchronous data prefetching and supports multiple CPU architectures (ARM64, x86, AMD, RISC-V). By combining performance optimizations with developer-friendly APIs, BULKI-Store addresses the complex data management challenges of modern HPC applications while maintaining compatibility across heterogeneous computing environments.

Zhang, Wei [Lawrence Berkeley National Laboratory ↗