Search NASA⌕ Search

SEARCH · Search NASA

Results for “Resource scheduling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Reaching new peaks for the future of the CMS HTCondor Global Pool

The CMS experiment at CERN employs a distributed computing infrastructure to satisfy its data processing and simulation needs. The CMS Submission Infrastructure team manages a dynamic HTCondor pool, aggregating mainly Grid clusters worldwide, but also HPC, Cloud and opportunistic resources. This CMS Global Pool, which currently involves over 70 computing sites worldwide and peaks at 350k CPU cores, is employed to successfully manage the simultaneous execution of up to 150k tasks. While the present infrastructure is sufficient to harness the current computing power scales, CMS latest estimates predict a noticeable expansion in the amount of CPU that will be required in order to cope with the massive data increase of the High-Luminosity LHC (HL-LHC) era, planned to start in 2027. This contribution presents the latest results of the CMS Submission Infrastructure team in exploring and expanding the scalability reach of our Global Pool, in order to preventively detect and overcome any barriers in relation to the HL-LHC goals, while maintaining high effciency in our workload scheduling and resource utilization.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Characterizing, Modeling, and Accurately Simulating Power and Energy Consumption of I/O-intensive Scientific Workflows

While distributed computing infrastructures can provide infrastructure-level techniques for managing energy consumption, application-level energy consumption models have also been developed to support energy-efficient scheduling and resource provisioning algorithms. In this work, we analyze the accuracy of a widely-used application-level model that has been developed and used in the context of scientific workflow executions. To this end, we profile two production scientific workflows on a distributed platform instrumented with power meters. We then conduct an analysis of power and energy consumption measurements. This analysis shows that power consumption is not linearly related to CPU utilization and that I/O operations significantly impact power, and thus energy, consumption. We then propose a power consumption model that accounts for I/O operations, including the impact of waiting for these operations to complete, and for concurrent task executions on multi-socket, multi-core compute nodes. We implement our proposed model as part of a simulator that allows us to draw direct comparisons between real-world and modeled power and energy consumption. Here, we find that our model has high accuracy when compared to real-world executions. Furthermore, our model improves accuracy by about two orders of magnitude when compared to the traditional models used in the energy-efficient workflow scheduling literature.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

On Evaluating Cycling and Movement of Conventional Generators for Balancing Services with Large Solar Penetration

Abstract— The increasing uncertainty caused by a higher penetration level of renewable energy points to the need for sufficient balancing services to ensure secure operation. It is thus of necessity to accurately quantify these services and evaluate the cycling and movement of conventional generators providing load following and regulation services for optimally planning and scheduling multiple resources. This paper first introduces a systematic approach to calculating load following and regulation requirements with a large amount of solar integration. Then, two methods to quantitatively determine the cycling of conventional generators are proposed, including (1) mileage and number of direction changes in balancing service and (2) ramp statistics (or half-cycle) analysis. The proposed methods are tested on the NV Energy power system model. Simulation results demonstrate a significant impact of increased solar capacity on balancing service and cycling of conventional generators. Busy hours are identified for different study scenarios with various solar penetration levels. This study provides a basis for evaluating the wear and tear of the conventional generators in the solar integration process in the Nevada power system. Index Terms—balancing requirement, conventional generator, cycling and movement, load following, regulation, renewable energy, solar integration

ok↗
DOE OSTI2025

Maskman

SAND2025-04369O Maskman is a user-friendly tool designed to create hex masks, which are essential for optimizing application performance in high-performance computing environments. By converting a list of integers into binary and then hex masks, Maskman simplifies the process of setting application affinity. This ensures that software runs efficiently on specific nodes within a computing cluster. Ideal for researchers and developers, Maskman streamlines the preparation of inputs for HPC schedulers, enhancing resource management and improving overall system performance. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Pase, Douglas [Sandia National Lab. (SNL-CA), Live↗
DOE OSTI2023

Design and implementation of I/O performance prediction scheme on HPC systems through large-scale log analysis

Abstract Large-scale high performance computing (HPC) systems typically consist of many thousands of CPUs and storage units used by hundreds to thousands of users simultaneously. Applications from large numbers of users have diverse characteristics, such as varying computation, communication, memory, and I/O intensity. A good understanding of the performance characteristics of each user application is important for job scheduling and resource provisioning. Among these performance characteristics, I/O performance is becoming increasingly important as data sizes rapidly increase and large-scale applications, such as simulation and model training, are widely adopted. However, predicting I/O performance is difficult because I/O systems are shared among all users and involve many layers of software and hardware stack, including the application, network interconnect, operating system, file system, and storage devices. Furthermore, updates to these layers and changes in system management policy can significantly alter the I/O behavior of applications and the entire system. To improve the prediction of the I/O performance on HPC systems, we propose integrating information from several different system logs and developing a regression-based approach to predict the I/O performance. Our proposed scheme can dynamically select the most relevant features from the log entries using various feature selection algorithms and scoring functions, and can automatically select the regression algorithm with the best accuracy for the prediction task. The evaluation results show that our proposed scheme can predict the write performance with up to 90% prediction accuracy and the read performance with up to 99% prediction accuracy using the real logs from the Cori supercomputer system at NERSC.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

COOLR: A New System for Dynamic Thermal-Aware Computing

Extreme-scale systems need to walk a fine line between the amount of cooling they receive and the thermal-induced performance and reliability degradation they can sustain. System managers are extremely motivated to battle cooling-energy cost: the largest line item in their total operating cost. On the other hand, pushing the nodes to peak performance in tightened cooling regimes places the burden on the dynamic thermal management (DTM) to protect the hardware. DTM schemes throttle performance to relieve heat accumulation within nodes when cooling cannot mitigate the problem. These interventions prevent fatal failures but introduce inevitable performance degradations and variations. Such variations can have drastic consequences on the performance of extreme-scale systems. Furthermore, distribution of heat across different system nodes depends not only on the amount of workload. Even if a strictly equal share of computational load is assigned to all nodes, there are physical attributes and topological features of each system that inherently cause uneven accumulation of heat. These can cause one subcomponent to trigger DTM prematurely and penalize the overall system. Our goal in this project is to create a holistic thermal-aware view of the system, capturing both inherent physical attributes and dynamic system state. We developed COOLR, a dynamic system with the ability to evaluate the interplay between management of computation, data, power dissipation, and thermal state. Thereby, COOLR will achieve higher overall performance at the same energy and cooling cost. First, we performed a systematic power and thermal modeling and of high performance computing architectures. Analytical and empirical techniques were blended to generate a light-weight thermal model of target systems. Second, we developed a thermal-aware OS and runtime system. Policies for managing resources and schedules for processes has been designed to co-optimize the thermal state and performance of the system. The main outcome of our proposed thermal-aware dynamic computing system are scheduling and allocation of processes and memory accesses leading to an appropriately “skewed” distribution of activity across the system. The ultimate result of this new arrangement resulted in reduced occurrences of hotspots and reduced episodes of DTM intervention, leading to improvements in performance. The overall thermal characterization and modeling methodology (thermal instrumentation techniques, model building, and systematic model reduction) resulting from this project can be applied to systems beyond our immediate scope and will be applicable to future scaling. The thermal-aware dynamic computing paradigm will impact the efficiency of extreme-scale systems. The gained “thermal slack” can be given back to further tighten the cooling budget, benefiting the management cost of next generation extreme-scale systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Risk-based System Upgrade Planning for AOT-IC Presentation [Slides]

The number of deployed systems and different HW/SW form factors makes it difficult to maintain a complete understanding of their health which leads to a reduced level of certainty when assessing risk and trying to prioritize system upgrade efforts. AOT-IC must constantly plan and execute system upgrade and replacement projects to ensure the longevity of the control system. There are many more projects needed than the group can realistically accomplish given schedule and resource constraints. The group must prioritize their upgrade projects to choose only the highest priority projects to execute each year. Currently plans upgrade and replacement projects based on alignment with mission needs, group vision, resource availability, and qualitative risk assessments based on SME estimates. The group maintains a risk register. This is useful as a tool for shaping strategic planning but currently lacks the quantitative data and process needed to generate accurate risk assessments.

43 PARTICLE ACCELERATORS↗
NASA NTRS1975

Experiences in using modal synthesis within project requirements

Modal synthesis methods were developed for use by engineers for cost-effective solutions of large structural dynamics problems. Different methods were proposed by various investigators based upon comparative solutions of relatively simple structures using different forms of displacement functions. The application of modal synthesis methods to a large complex structure in a project environment is described. The considerations include analysis, hardware interfaces, schedules, tests, resources, and other project requirements. Good technical results were obtained through the use of an integrated analysis/test modal synthesis on both substructures and systems.

Garba, J. A.↗
NASA NTRS1976

MCC/shuttle test plan. Volume 1: Philosophy and guidelines

The Mission Control Center/Shuttle Test Plan is defined from development through operations to a level of detail which will support the National Aeronautics and Space Administration and contractor management in the following areas: test management, test tool development, and resource and schedule planning.

Source record↗
NASA NTRS1980

Mission planning and analysis division development plan for STS-2 through STS-4

The baseline products, schedules, and resource requirements for the Mission Planning and Analysis Division's support of Space Transportation System flights 2, 3, and 4 are presented. Major functions addressed are: orbiter software, Mission Control Center software, flight design, flight operations support, simulation tools, and postflight analysis.

Source record↗
NASA NTRS1979

Experiments for dedicated life science missions

Over 80 investigations were selected recently as candidate experiments for Life Science Spacelab Missions in the 1983-1985 time frame. A payload consisting of perhaps 20 of these experiments will be selected in 1981, approximately two years before the first dedicated Life Sciences Spacelab Mission. Implementation of a carefully designed management process will be a prerequisite for accomplishing the development, flight, and post-flight analysis of a maximum number of these investigations, working within currently approved resources and schedules. This paper provides an overview of the Life Sciences Flight Experiments Program, and describes the approach being taken at NASA Headquarters in its planning and implementation.

Farrell, R. M.↗
NASA NTRS1985

Software Cost-Estimation Model

Software Cost Estimation Model SOFTCOST provides automated resource and schedule model for software development. Combines several cost models found in open literature into one comprehensive set of algorithms. Compensates for nearly fifty implementation factors relative to size of task, inherited baseline, organizational and system environment and difficulty of task.

Tausworthe, R. C.↗
NASA NTRS1984

The USL NASA PC R and D development environment standards

The development environment standards which have been established in order to control usage of the IBM PC/XT development systems and to prevent interference between projects being currently developed on the PC's are discussed. The standards address the following areas: scheduling PC resources; login/logout procedures; training; file naming conventions; hard disk organization; diskette care; backup procedures; and copying policies.

Dominick, Wayne D.↗
NASA NTRS1989

AdaNET Dynamic Software Inventory (DSI) prototype component acquisition plan

A component acquisition plan contains the information needed to evaluate, select, and acquire software and hardware components necessary for successful completion of the AdaNET Dynamic Software Inventory (DSI) Management System Prototype. This plan will evolve and be applicable to all phases of the DSI prototype development. Resources, budgets, schedules, and organizations related to component acquisition activities are provided. A purpose and description of a software or hardware component which is to be acquired are presented. Since this is a plan for acquisition of all components, this section is not applicable. The procurement activities and events conducted by the acquirer are described and who is responsible is identified, where the activity will be performed, and when the activities will occur for each planned procurement. Acquisition requirements describe the specific requirements and standards to be followed during component acquisition. The activities which will take place during component acquisition are described. A list of abbreviations and acronyms, and a glossary are contained.

Hanley, Lionel↗