SEARCH · Search NASA

Results for “Heuristic Scheduling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2018

The research on meta-job scheduling heuristics in heterogeneous environments

As the core of rational utilization of computing resources, the scheduling algorithm has gained the interest of many researchers. Aimed at meta-job scheduling in heterogeneous environments, this paper puts forward three algorithms: the meta-job scheduling algorithm based on deviation (MaxD-min heuristic), the meta-job scheduling algorithm based on relative deviation (MaxRD-min heuristic), and the Cordwood algorithm (CA). The former two Algorithm are improvements of the classic Min-min algorithm. The latter CA algorithm is a new method that tries to place each building block on the tray with minimal amount of Sufferage. In conclusion, the result of experimenting shows that the three algorithms, in comparison to Min-Min and Max-Min algorithm, can effectively reduce the total span of scheduling (makespan).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

HPC Node Performance and Energy Modeling with the Co-Location of Applications

Multicore processors have become an integral part of modern large-scale and high-performance parallel and distributed computing systems. This advance became necessary as the demand for increased system performance has exceeded the limits that single core processors can provide. Unfortunately, applications co-located on multicore processors can suffer from decreased performance and increased dynamic energy use as a result of access to shared resources, such as memory. Consequently, it is increasingly important to characterize the performance of applications that execute on these architectures. This work investigates some of the disadvantages of co-location, and presents a methodology for building models capable of utilizing varying amounts of information about a target application and its co-located applications to make predictions about the target application’s execution time and the system’s energy use under arbitrary co-locations of a wide range of application types. The proposed methodology is validated on three different server class Intel Xeon multicore processors using eleven applications from two scientific benchmark suites. The model’s utility for scheduling is also demonstrated in a simulated large-scale high-performance computing environment through the creation of a co-location aware scheduling heuristic. This heuristic demonstrates that scheduling using information generated with the proposed modeling methodology is capable of making significant improvements over a scheduling heuristic that is naive to co-location interference.

performance modeling, energy modeling, resource ma↗

DOE OSTI2022

DRAS: Deep Reinforcement Learning for Cluster Scheduling in High Performance Computing

Cluster schedulers are crucial in high-performance computing (HPC). They determine when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with specific HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling. An efficient training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a specific scheduling objective given by the system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. We implement DRAS into a HPC scheduling platform called CQGym. CQGym provides a common platform allowing users to flexibly evaluate DRAS and other scheduling methods such as heuristic and optimization methods. Furthermore, the experiments using CQGym with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 50%.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Mixed-integer programming models for optimal constellation scheduling given cloud cover uncertainty

We introduce the problem of scheduling observations on a constellation of remote sensors, to maximize the aggregate quality of the collections obtained. While automated tools exist to schedule remote sensors, they are often based on heuristic scheduling techniques, which typically fail to provide bounds on the quality of the resultant schedules. To address this issue, we first introduce a novel deterministic mixed-integer programming (MIP) model for scheduling a constellation of one to n satellites, which relies on extensive pre-computations associated with orbital propagators and sensor collection simulators to mitigate model size and complexity. Our MIP model captures realistic and complex constellation-target geometries, with solutions providing optimality guarantees. We then extend our base deterministic MIP model to obtain two-stage and three-stage stochastic MIP models that proactively schedule to maximize expected collection quality across a set of scenarios representing cloud cover uncertainty. Our experimental conclusions on instances of one and two satellites demonstrate that our stochastic MIP models yield significantly improved collection quality relative to our base deterministic MIP model. We further demonstrate that commercial off-the-shelf MIP solvers can produce provably optimal or near-optimal schedules from these models in time frames suitable for sensor operations.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Efficient Parallelization of Irregular Applications on GPU Architectures

With the enlarging computation capacity of general Graphics Processing Units (GPUs), leveraging GPUs to accelerate parallel applications has become a critical topic in academia and industry. However, a wide range of irregular applications with the computation-/memory-intensive nature cannot easily achieve high GPU utilization. The challenges mainly involve the following aspects: first, data dependence leads to coarse-grained kernel and inefficient parallelism; second, heavy GPU memory usage may cause frequent memory evictions and extra overhead of I/O; third, specific computation patterns produce memory redundancies; last, workload balance and data reusability conjunctly benefit the overall performance, but there may exist a dynamic trade-off between them. Targeting these challenges, this dissertation proposes multiple optimizations to accelerate two real-world applications: many-body correlation functions to simulate nuclear physics in a large-scale scientific system; the other is the eALS-based matrix factorization recommendation system. To accelerate the calculations of many-body correlation functions, this dissertation presents three frameworks in GPU memory management and multi-GPU scheduling. Firstly, an optimized systematic GPU memory management framework, MemHC, utilizes a series of new memory reduction designs in GPU memory allocation, CPU/GPU communications, and GPU memory oversubscription. Secondly, an enhanced multi-GPU scheduling framework, MICCO, particularly by taking both data dimension (e.g., data reuse and data eviction) and computation dimension into account. MICCO designs a heuristic scheduling algorithm and a machine learning-based regression model to generate the optimal settings of a proposed new concept to manage the trade-off. Thirdly, a locality-aware multi-GPU scheduling framework. This scheduler leverages pipeline batch generation with a looking-ahead strategy by building local dependency graphs for memory transfer reduction and better data reuse, achieving up to 79.92% memory cost reduction and 1.67x speedup. To parallelize the eALS-based recommendation system, this dissertation proposes an efficient CPU/GPU heterogeneous recommendation system, HEALS. HEALS employs newly designed architecture-adaptive data formats to achieve load balance and good data locality on CPU and GPU. To mitigate the data dependence, HEALS presents a CPU/GPU collaboration model for both task parallelism and data parallelism with multiple kernel computation optimizations. In summary, this dissertation efficiently accelerates two typical irregular applications on GPUs by building four frameworks, including CPU/GPU collaboration, GPU memory management, and multi-GPU scheduling.

Wang, Qihan↗

DOE OSTI2022

MICCO: An Enhanced Multi-GPU Scheduling Framework for Many-Body Correlation Functions

Calculation of many-body correlation functions is one of the critical kernels utilized in many scientific computing areas, especially in Lattice Quantum Chromodynamics (Lattice QCD). It is formalized as a sum of a large number of contraction terms each of which can be represented by a graph consisting of vertices describing quarks inside a hadron node and edges designating quark propagations at specific time intervals. Due to its computation- and memory-intensive nature, real-world physics systems (e.g., multi-meson or multi-baryon systems) explored by Lattice QCD prefer to leverage multi-GPUs. Different from general graph processing, many-body correlation function calculations show two specific features: a large number of computation-/data-intensive kernels and frequently repeated appearances of original and intermediate data. The former results in expensive memory operations such as tensor movements and evictions. The latter offers data reuse opportunities to mitigate the data-intensive nature of many-body correlation function calculations. However, existing graph-based multi-GPU schedulers cannot capture these data-centric features, thus resulting in a sub-optimal performance for many-body correlation function calculations. To address this issue, this paper presents a multi-GPU scheduling framework, MICCO, to accelerate contractions for correlation functions particularly by taking the data dimension (e.g., data reuse and data eviction) into account. This work first performs a comprehensive study on the interplay of data reuse and load balance, and designs two new concepts: local reuse pattern and reuse bound to study the opportunity of achieving the optimal trade-off between them. Based on this study, MICCO proposes a heuristic scheduling algorithm and a machine-learning-based regression model to generate the optimal setting of reuse bounds. Specifically, MICCO is integrated into a real-world Lattice QCD system, Redstar, for the first time running on multiple GPUs. The evaluation demonstrates MICCO outperforms other state-of-art works, achieving up to 2.25× speedup in synthesized datasets, and 1.49× speedup in real-world correlation functions.

Wang, Qihan↗

DOE OSTI2014

PLAStiCC: Predictive Look-Ahead Scheduling for Continuous dataflows on Clouds

Scalable stream processing and continuous dataflow systems are gaining traction with the rise of big data due to the need for processing high velocity data in near real time. Unlike batch processing systems such as MapReduce and workflows, static scheduling strategies fall short for continuous dataflows due to the variations in the input data rates and the need for sustained throughput. The elastic resource provisioning of cloud infrastructure is valuable to meet the changing resource needs of such continuous applications. However, multi-tenant cloud resources introduce yet another dimension of performance variability that impacts the application’s throughput. In this paper we propose PLAStiCC, an adaptive scheduling algorithm that balances resource cost and application throughput using a prediction-based look-ahead approach. It not only addresses variations in the input data rates but also the underlying cloud infrastructure. In addition, we also propose several simpler static scheduling heuristics that operate in the absence of accurate performance prediction model. These static and adaptive heuristics are evaluated through extensive simulations using performance traces obtained from public and private IaaS clouds. Our results show an improvement of up to 20% in the overall profit as compared to the reactive adaptation algorithm.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗

DOE OSTI2018

Heuristic approach to Satellite Range Scheduling with Bounds using Lagrangian Relaxation

Here, this paper focuses on scheduling antennas to track satellites using a novel heuristic method. The objectives pursued in developing a schedule are two-fold: (1) minimize the priority weighted number of time periods that satellites are not tracked; and (2) equalize the percent of time each satellite is uncovered. The heuristic method is a population-based local search tailored to the unique characteristics of this problem. In order to validate the performance of the heuristic, bounds are developed using Lagrangian relaxation. The heuristic method and the bounds are applied to several test problems. In all cases, the heuristic identifies a solution that is better than the upper bound and is generally quite close (but obviously larger) than the lower bound with about an order of magnitude reduction in computation time. Lastly, a comparison with CPLEX 12.7 is provided.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

ALTSched: Improved Scheduling for Time-domain Science with LSST

Abstract Telescope scheduling is the task of determining the best sequence of observations (pointings and filter choices) for a survey system. Because it is computationally intractable to optimize over all possible multiyear sequences of observations, schedulers use heuristics to pick the best observation at a given time. A greedy scheduler selects the next observation by choosing whichever one maximizes a scalar merit function, which serves as a proxy for the scientific goals of the telescope. This sort of bottom-up approach for scheduling is not guaranteed to produce a schedule for which the sum of merit over all observations is maximized. As an alternative to greedy schedulers, we introduce ALTS ched, which takes a top-down approach to scheduling. Instead of considering only the next observation, ALTS ched makes global decisions about which area of sky and which filter to observe in, and then refines these decisions into a sequence of observations taken along the meridian to minimize airmass. We implement ALTS ched for the Large Synoptic Survey Telescope (LSST) and show that it equals or outperforms the baseline greedy scheduler in essentially all quantitative performance metrics. Due to its simplicity, our implementation is considerably faster than O p S im , the simulated greedy scheduler currently used by the LSST Project: a full 10-yr survey can be simulated in 4 minutes, as opposed to tens of hours for O p S im . LSST’s hardware is fixed, so improving the scheduling algorithm is one of the only remaining ways to optimize LSST’s performance. We see ALTS ched as a prototype scheduler that gives a lower bound on the performance achievable by LSST.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2016

Dynamic Multi-Sensor Multi-Mission Optimal Planning Tool

Remote sensing systems have firmly established a role in providing immense value to commercial industry, scientific exploration, and national security. Continued maturation of sensing technology has reduced the cost of deploying highly-capable sensors while at the same time increased reliance on the information these sensors can provide. The demand for time on these sensors is unlikely to diminish. Coordination of next-generation sensor systems, larger constellations of satellites, unmanned aerial vehicles, ground telescopes, etc. is prohibitively complex for existing heuristics-based scheduling techniques. The project was a two-year collaboration spanning multiple Sandia centers and included a partnership with Texas A&M University. We have developed algorithms and software for collection scheduling, remote sensor field-of-view pointing models, and bandwidth-constrained prioritization of sensor data. Our approach followed best practices from the operations research and computational geometry communities. These models provide several advantages over state of the art techniques. In particular, our approach is more flexible compared to heuristics that tightly couple models and solution techniques. First, our mixed-integer linear models afford a rigorous analysis so that sensor planners can quantitatively describe a schedule relative to the best possible. Optimal or near-optimal schedules can be produced with commercial solvers in operational run-times. These models can be modified and extended to incorporate different scheduling and resource constraints and objective function definitions. Further, we have extended these models to proactively schedule sensors under weather and ad hoc collection uncertainty. This approach stands in contrast to existing deterministic schedulers which assume a single future weather or ad hoc collection scenario. The field-of-view pointing algorithm produces a mosaic with the fewest number of images required to fully cover a region of interest. The bandwidth-constrained algorithms find the highest priority information that can be transmitted. All of these are based on mixed-integer linear programs so that, in the future, collection scheduling, field-of-view, and bandwidth prioritization can be combined into a single problem. Experiments conducted using the developed models, commercial solvers, and benchmark datasets have demonstrated that proactively scheduling against uncertainty regularly and significantly outperforms deterministic schedulers.

47 OTHER INSTRUMENTATION↗

DOE OSTI2020

RLScheduler: An Automated HPC Batch Job Scheduler Using Reinforcement Learning

Today’s high-performance computing (HPC) platforms are still dominated by batch jobs. Accordingly, effective batch job scheduling is crucial to obtain high system efficiency. Existing HPC batch job schedulers typically leverage heuristic priority functions to prioritize and schedule jobs. But, once configured and deployed by the experts, such priority functions can hardly adapt to the changes of job loads, optimization goals, or system settings, potentially leading to degraded system efficiency when changes occur. To address this fundamental issue, we present RLScheduler, an automated HPC batch job scheduler built on reinforcement learning. RLScheduler relies on minimal manual interventions or expert knowledge, but can learn high-quality scheduling policies via its own continuous ‘trial and error’. We introduce a new kernel-based neural network structure and trajectory filtering mechanism in RLScheduler to improve and stabilize the learning process. Through extensive evaluations, we confirm that RLScheduler can learn high-quality scheduling policies towards various workloads and various optimization goals with relatively low computation cost. Moreover, we show that the learned models perform stably even when applied to unseen workloads, making them practical for production use.

Zhang, Di↗

DOE OSTI2018

Flexible Bandwidth Scheduling for Streaming Data Movement Over Dedicated Networks

A wide range of scientific disciplines are generating large amounts of data at a high speed, which must be transferred to remote sites for real-time processing. Reserving bandwidths over dedicated channels in high-performance networks (HPNs) within a specified time interval has proved to be an effective solution to such high-demanding data transfer. Given a bandwidth reservation request, if the desired bandwidth within the specified time interval cannot be satisfied, most of the existing scheduling algorithms simply reject the request, which would immediately terminate the application. One reasonable approach to mitigate this issue is to provide an alternative bandwidth reservation option to schedule the desired bandwidth within the time interval closest to the specified one. We propose a flexible bandwidth reservation algorithm that considers both the best and alternative bandwidth reservation options for a given request. Extensive simulations are conducted to show the superior performance of the proposed scheduling algorithm compared with a heuristic approach adapted from existing scheduling algorithms.

Zuo, Liudong↗

DOE OSTI2026

CRADA Number NFE-24-10110 with Qubit Engineering Inc. (CRADA Final Report)

Over the past year, the Qubit Engineering team has pushed the frontiers of power‑grid optimization, working in close collaboration with Oak Ridge National Laboratory (ORNL) and the Tennessee Valley Authority (TVA). Their progress is reflected in three newly submitted conference papers, “Unified Relational GNN Architecture for AC Optimal Power Flow Calculations in Electric Grids,” “Graph‑Based Attention Mechanisms for Solving the AC Optimal Power Flow Problem in Electrical‑Power Networks,” and “Enhanced Power‑Grid Maintenance Planning and Quantum‑Inspired Combinatorial Prospects.” These publications showcase state‑of‑the‑art graph‑neural‑network methods for AC‑OPF and novel quantum‑inspired heuristics for maintenance scheduling. Beyond the academic results, the Qubit team has converted the research into two production‑grade tools built on TVA data: Neuro‑Grid, an AI‑driven power‑flow simulator that provides instant, interactive full‑grid load‑flow visualizations, and Quanta‑Grid, a quantum‑inspired maintenance‑scheduling engine to support logistics optimization for power utilities. Together, these advances demonstrate how Qubit’s partnership with ORNL and TVA is delivering practical, physics‑grounded analytics for next‑generation grid management.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Heuristic solutions to the single depot electric vehicle scheduling problem with next day operability constraints

This study focuses on the single depot electric vehicle scheduling problem (SDEVSP) within the broader context of the vehicle scheduling problem (VSP). By developing an effective scheduling model using mixed-integer linear programming, we generate bus blocks that accommodate electric vehicles (EVs), ensuring successful completion of each block while considering recharging requirements between blocks and during off-hours. Next day operability constraints are also incorporated, allowing for seamless repetition of blocks on subsequent days. The SDEVSP is known to be computationally complex, deriving optimal solutions unattainable for large-scale problems within reasonable timeframes. To address this, we propose a two-step solution approach: first solving the single depot VSP, and then addressing the block chaining problem (BCP) using the blocks generated in the first step. The BCP focuses on optimizing block combinations to facilitate recharging between consecutive blocks, considering operational constraints. Further, a case study conducted reveals that nearly 100% electrification for Chicago, IL and Austin, TX transit buses is viable yet requires 1.6 EVs at 150-mile range per diesel vehicle.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2018

A Heterogeneity-Aware Task Scheduler for Spark

Big data processing systems such as Spark are employed in an increasing number of diverse applications-such as machine learning, graph computation, and scientific computing-each with dynamic and different resource needs. These applications increasingly run on heterogeneous hardware, e.g., with out-of-core accelerators. However, big data platforms do not factor in the multi-dimensional heterogeneity of applications and hardware. This leads to a fundamental mismatch between the application and hardware characteristics, and the resource scheduling adopted in big data platforms. For example, Hadoop and Spark consider only data locality when assigning tasks to nodes, and typically disregard the hardware capabilities and suitability to specific application requirements. In this paper, we present RUPAM, a heterogeneity-aware task scheduling system for big data platforms, which considers both task-level resource characteristics and underlying hardware characteristics, as well as preserves data locality. RUPAM adopts a simple yet effective heuristic to decide the dominant scheduling factor (e.g., CPU, memory, or I/O), given a task in a particular stage. Our experiments show that RUPAM is able to improve the performance of representative applications by up to 62.3% compared to the standard Spark scheduler.

Computer Science↗

DOE OSTI2018

Exact and heuristic algorithms for minimizing Tardy/Lost penalties on a single-machine scheduling problem

This paper addresses minimizing Tardy/Lost penalties with common due dates on a single machine. According to this penalty criterion, if tardiness of a job exceeds a predefined value, the job will be lost and penalized by a fixed value. The problem is formulated as an integer programming model, and a heuristic algorithm is constructed. Then, using the proposed dominance rules and lower bounds, we develop two dynamic programming algorithms as well as a branch and bound. Experimental results show that the heuristic algorithm has an average optimality gap less than 2 % in all problem sizes. Instances up to 250 jobs with low variety of process times are optimally solved and for high process time varieties, the algorithms solved all instances up to 75 jobs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration

Here, this paper proposes a safe soft actor-critic reinforcement learning (RL) algorithm–based controller for networked microgrid restoration. It formulates the post black-start start as a finite-horizon constrained Markov decision process. The RL agent co-optimizes real and reactive power set-points for both grid-forming and grid-following inverters under explicit voltage and frequency constraints, while enforcing proper power sharing via the Mean Active Power Sharing Index (MPSI) and Mean Reactive Power Sharing Index (MQSI). Numerical results obtained on the IEEE 123-bus distribution system show that the proposed method achieves a mean voltage build-up time of 0.01 s without breaching the 5% sharing-violation budget under various load scenarios, considering MPSI and MQSI indices. These findings demonstrate that the proposed method yields fast and safe black-start schedules without resorting to heuristic penalties.

Selim, Alaa [Dartmouth College, Hanover, NH (Unite↗