SEARCH · Search NASA

Results for “network scheduling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Resource Management for Processing Wide Area Data Streams on Supercomputers

Modern scientific instruments generate enormous amount of data. Typically, the data collected from the instruments are stored in one or more files that are then moved to a distant supercomputer for processing. The final results are sent back to the user. In order to make effective use of the time on expensive instruments, experimenters want to process the data as they are generated. They want to stream the data from instruments' memory directly to a supercomputer's memory for analysis. Since the compute nodes in a supercomputer are not connected directly to the wide area network, the data streams need to be passed through intermediate gateway nodes. As opposed to the best effort file transfers, data streaming applications require resources at a specific time for a specific period. In this paper, we present a system model for enabling data streaming through gateway nodes and an algorithm to efficiently allocate gateway node resources along with compute nodes. We evaluate the algorithm using real-world traces on the Chameleon Cloud. The results show that our system can schedule compute and gateway resources efficiently for streaming analysis.

data streaming↗

DOE OSTI2025

Robustness of Deep Learning Classification to Adversarial Input on GPUs: Asynchronous Parallel Accumulation Is a Source of Vulnerability

The ability of machine learning (ML) classification models to resist small, targeted input perturbations—known as adversarial attacks—is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets used in robustness assessments. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.

Shanmugavelu, Sanjif [Maxeler Technologies, a Groq↗

DOE OSTI2021

Enhancing Active Distribution Systems Resilience by Fully Distributed Self-Healing Strategy

Distributed restoration can exploit smart grid technologies to enhance the resilience of active distribution networks toward a self-healing smart grid. However, the large number of decision variables, especially the binary ones for reconfiguration, bring challenges to developing scalable distributed distribution service restoration (DDSR) strategies. This paper proposes a fully distributed solution procedure based on the alternating direction method of multipliers (ADMM) for mixed-integer programming problems and applies to develop the DDSR framework. The method consists of relax-drive-polish phases, 1) relaxing binary variables, and applying the convex ADMM as a warm start; 2) driving the solutions toward Boolean values through a proximal operator; 3) fixing the obtained binding binary variables and solving the rest of the problem to polish results and achieve a high-quality suboptimal solution. Then, an autonomous clustering strategy and consensus ADMM are integrated with the proposed method to realize the fully distributed cluster-based framework of DDSR. This framework can first determine DER scheduling and switch status for reconfiguration to energize the out-of-service areas from local faults, and then provide the load restoration solution in a distributed manner for total blackouts in large-scale distribution networks. Furthermore, the effectiveness and scalability of the proposed DDSR framework are demonstrated through testing on the IEEE 123-node, IEEE 8500-node, and synthetic 100k-node test feeders.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

An MILP-Based Distributed Energy Management for Coordination of Networked Microgrids

An MILP-based distributed energy management for the coordination of networked microgrids is proposed in this paper. Multiple microgrids and the utility grid are coordinated through iteratively adjusted price signals. Based on the price signals received, the microgrid controllers (MCs) and distribution management system (DMS) update their schedules separately. Then, the price signals are updated according to the generation–load mismatch and distributed to MCs and DMS for the next iteration. The iteration continues until the generation–load mismatch is small enough, i.e., the generation and load are balanced under agreed price signals. Through the proposed distributed energy management, various microgrids and the utility grid with different economic, resilient, emission and socio-economic objectives are coordinated with generation–load balance guaranteed and the microgrid customers’ privacy preserved. In particular, a piecewise linearization technique is employed to approximate the augmented Lagrange term in the alternating direction method of multipliers (ADMM) algorithm. Thus, the subproblems are transformed into mixed integer linear programming (MILP) problems and efficiently solved by open-source MILP solvers, which would accelerate the adoption and deployment of microgrids and promote clean energy. The proposed MILP-based distributed energy management is demonstrated through various case studies on a networked microgrids test system with three microgrids.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

How Many Trip Requests Could We Support? An Activity-Travel Based Vehicle Scheduling Approach

In a world of ever-changing travel behavior and ever-increasing modal options, is vital to have integrated models that could capture the interactions between supply and demand layers of travel. Addressing this need, we propose three different versions of network representation and mathematical models for the activity-based vehicle routing problem to connect activity-travel graphs of passengers (demand layer) to spatio-temporal networks of vehicles (supply layer). Versions I and II are arc-based, while version III is path-based. In version I, we introduce the concept of activity-travel graphs for passengers. For vehicles, we construct space–time networks and add a new dimension, called “under-service state”, to track the execution status of trip requests at any location and time. In version II, we reduce the complexity of the network structure by eliminating the state dimension and some other modifications in the structure of the passengers’ and vehicles’ network. Although both versions can capture various behavioral constraints of the activity-based vehicle routing problem (e.g., mandatory and optimal activities, duration of activities, chain of activities, preferred starting and ending times of activities), due to the high level of complexity of the network structure, both versions can only solve small-sized problems. To tackle the computational complexity, we propose a path-based network representation in version III, and to make a balance between the disutility of passengers and vehicles, we present a tolled user equilibrium problem. Mathematical models are coded in C and GAMS and implemented on real-world Phoenix regional transportation network with more than 39 million trip requests, which demonstrate the effectiveness of the proposed solution for the original and restricted master problems.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2020

Repair and Resource Scheduling in Unbalanced Distribution Systems Using Neighborhood Search

—This paper proposes an optimization strategy to assist utility operators to recover power distribution systems after large outages. Specifically, a mixed-integer linear programming (MILP) model is developed for co-optimizing crews, resources, and network operations. The MILP model coordinates damage isolation, network reconfiguration, distributed generator re-dispatch, and crew/resource logistics. In addition, a framework for integrating different types of photovoltaic (PV) systems in the restoration process is developed. We consider two different types of crews, namely, line crews for damage repair and tree crews for obstacle removal. We also model the repair resource logistic constraints. Furthermore, a new algorithm is developed for solving the distribution system repair and restoration problem (DSRRP). The algorithm starts by solving DSRRP using an assignmentbased method, then a neighborhood search method is designed to iteratively improve the solution. The proposed method is validated on modified IEEE 123- and 8500-bus distribution test systems.

Outage management↗

DOE OSTI2021

Communication system and method for applying security for a time sensitive network

A method includes identifying power connections between plural components of a time sensitive network (TSN) that are interconnected via a predetermined connection plan. The method also includes determining a topology of the components of the TSN based on the power connections. Also, the method includes scheduling flows for the TSN based on the topology determined based on the power connections.

Bush, Stephen Francis↗

DOE OSTI2021

Comparison of Deterministic and Statistical Models for Water Quality Compliance Forecasting in the San Joaquin River Basin, California

Model selection for water quality forecasting depends on many factors including analyst expertise and cost, stakeholder involvement and expected performance. Water quality forecasting in arid river basins is especially challenging given the importance of protecting beneficial uses in these environments and the livelihood of agricultural communities. In the agriculture-dominated San Joaquin River Basin of California, real-time salinity management (RTSM) is a state-sanctioned program that helps to maximize allowable salt export while protecting existing basin beneficial uses of water supply. The RTSM strategy supplants the federal total maximum daily load (TMDL) approach that could impose fines associated with exceedances of monthly and annual salt load allocations of up to $1 million per year based on average year hydrology and salt load export limits. The essential components of the current program include the establishment of telemetered sensor networks, a web-based information system for sharing data, a basin-scale salt load assimilative capacity forecasting model and institutional entities tasked with performing weekly forecasts of river salt assimilative capacity and scheduling west-side drainage export of salt loads. Web-based information portals have been developed to share model input data and salt assimilative capacity forecasts together with increasing stakeholder awareness and involvement in water quality resource management activities in the river basin. Two modeling approaches have been developed simultaneously. The first relies on a statistical analysis of the relationship between flow and salt concentration at three compliance monitoring sites and the use of these regression relationships for forecasting. The second salt load forecasting approach is a customized application of the Watershed Analysis Risk Management Framework (WARMF), a watershed water quality simulation model that has been configured to estimate daily river salt assimilative capacity and to provide decision support for real-time salinity management at the watershed level. Analysis of the results from both model-based forecasting approaches over a period of five years shows that the regression-based forecasting model, run daily Monday to Friday each week, provided marginally better performance. However, the regression-based forecasting model assumes the same general relationship between flow and salinity which breaks down during extreme weather events such as droughts when water allocation cutbacks among stakeholders are not evenly distributed across the basin. A recent test case shows the utility of both models in dealing with an exceedance event at one compliance monitoring site recently introduced in 2020.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Joint optimization of electric bus charging infrastructure, vehicle scheduling, and charging management

High upfront costs of vehicles and charging infrastructure as well as the lack of knowledge related to infrastructure planning and electric bus system operation are major obstacles to the implementation of battery electric buses (BEBs). To tackle the obstacles and promote BEB adoption, a comprehensive optimization framework was developed to address the combined charging infrastructure planning, vehicle scheduling, and charging management problem for BEB systems, with the goal to minimize the total cost of ownership. The problem was formulated as a mixed-integer non-linear problem. A genetic algorithm-based approach was then proposed to solve the problem. Last, three alternative scenarios based on a sub-transit network in Salt Lake City, Utah, were analyzed and compared with the optimal scenario results in the numerical experiments. Our comparison results demonstrate the effectiveness of the proposed model and solution algorithm in determining a cost-efficient planning strategy for BEB systems.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2020

Innovations in Sensors and Controls for Building Energy Management: Research and Development Opportunities Report for Emerging Technologies

Sensors, actuators, and controllers, which collectively serve as the backbone of cyberphysical systems for building energy management, are one of the core technical areas of investment for achieving the U.S. Department of Energy (DOE) Building Technologies Office's (BTO's) goals for energy affordability in the national building stock - both commercial and residential. In fact, an aggregated annual energy savings of 29% is estimated in the commercial sector alone through the implementation of efficiency measures using current state-of-the-art sensors and controls to retune buildings by optimizing programmable settings based on occupant schedules and comfort requirements, as well as detecting and diagnosing equipment operation and installation problems (Fernandez et al. 2017). Monitoring and control of building conditions and operations has advanced significantly, from the invention of the modern thermostat just before the start of the 20th century to the midcentury incorporation of direct digital control into devices, the introduction of open protocols and network communications at the end of the century, and finally the invention of cloud-based computing and additional advancements that have enabled remote operation and a proliferation of connected and intelligent devices in building automation. Despite this potential, however, two main challenges hinder widespread adoption of sensors and controls in building operations that can ensure savings for high-efficiency components and equipment (e.g., heat pumps, windows, and lighting devices), as well as additional savings from more sophisticated control architectures and algorithms. energy savings of 29% is estimated in the commercial sector alone through the implementation of efficiency measures using current state-of-the-art sensors and controls to retune buildings by optimizing programmable settings based on occupant schedules and comfort requirements, as well as detecting and diagnosing equipment operation and installation problems (Fernandez et al. 2017). Monitoring and control of building conditions and operations has advanced significantly, from the invention of the modern thermostat just before the start of the 20th century to the midcentury incorporation of direct digital control into devices, the introduction of open protocols and network communications at the end of the century, and finally the invention of cloud-based computing and additional advancements that have enabled remote operation and a proliferation of connected and intelligent devices in building automation. Despite this potential, however, two main challenges hinder widespread adoption of sensors and controls in building operations that can ensure savings for high-efficiency components and equipment (e.g., heat pumps, windows, and lighting devices), as well as additional savings from more sophisticated control architectures and algorithms.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2020

Ad Hoc File Systems for High-Performance Computing

Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include nodelocal SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 “Challenges and Opportunities of User-Level File Systems for HPC” and discusses application scenarios as well as design strategies for ad hoc file systems using node-local storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Network-Aware and Welfare-Maximizing Dynamic Pricing for Energy Sharing

The proliferation of behind-the-meter (BTM) distributed energy resources (DER) within the electrical distribution network presents significant supply and demand flexibilities, but also introduces operational challenges such as voltage spikes and reverse power flows. In response, this paper proposes a network-aware dynamic pricing framework tailored for energy-sharing coalitions that aggregate small, but ubiquitous, BTM DER downstream of a distribution system operator's (DSO) revenue meter that adopts a generic net energy metering (NEM) tariff. By formulating a Stackelberg game between the energy-sharing market leader and its prosumers, we show that the dynamic pricing policy induces the prosumers toward a network-safe operation and decentrally maximizes the energysharing social welfare. The dynamic pricing mechanism involves a combination of a locational ex-ante dynamic price and an ex-post allocation, both of which are functions of the energy sharing's BTM DER. The ex-post allocation is proportionate to the price differential between the DSO NEM price and the energy-sharing locational price. Simulation results using real DER data and the IEEE 13-bus test systems illustrate the dynamic nature of network-aware pricing at each bus, and its impact on voltage.

aggregates↗

DOE OSTI2024

The Galaxy platform for accessible, reproducible, and collaborative data analyses: 2024 update

Galaxy (https://galaxyproject.org) is deployed globally, predominantly through free-to-use services, supporting user-driven research that broadens in scope each year. Users are attracted to public Galaxy services by platform stability, tool and reference dataset diversity, training, support and integration, which enables complex, reproducible, shareable data analysis. Applying the principles of user experience design (UXD), has driven improvements in accessibility, tool discoverability through Galaxy Labs/subdomains, and a redesigned Galaxy ToolShed. Galaxy tool capabilities are progressing in two strategic directions: integrating general purpose graphical processing units (GPGPU) access for cutting-edge methods, and licensed tool support. Engagement with global research consortia is being increased by developing more workflows in Galaxy and by resourcing the public Galaxy services to run them. The Galaxy Training Network (GTN) portfolio has grown in both size, and accessibility, through learning paths and direct integration with Galaxy tools that feature in training courses. Code development continues in line with the Galaxy Project roadmap, with improvements to job scheduling and the user interface. Environmental impact assessment is also helping engage users and developers, reminding them of their role in sustainability, by displaying estimated CO 2 emissions generated by each Galaxy job.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Applications which fuse machine learning and simulation are rarely best served by a single computing resource. Highly parallel simulation codes are best deployed on super- computers, while AI tasks used to decide which simulations to perform may be best suited to specialized accelerators. Here we present a Function-as-a-Service (FaaS) system for executing complex, distributed computational campaigns that achieves performance parity with conventional workflow systems without the complexities of secure network connections between compute providers. One innovation enabling high performance is a subsystem that directly moves task data between sites, separate from the cloud-hosted FaaS system used to distribute task instructions. We also introduce a flexible scheduling system that allows us access factor of 2 trade offs between the amount of resources required to solve a problem at each compute site. We anticipate that this system will upgrade multi-site applications from demonstration projects to routine practice in computational science.

Ward, Logan↗

DOE OSTI2021

Systems, devices and methods for communicating data with unmanned aerial vehicles using underlay broadcast channel

An unmanned aerial vehicle (UAV) and base station are disclosed that communicate within a first cell via schedule requests to set up transmission of up-link data. The UAV additionally communicates up-link data via a grant-free underlay broadcast channel to one or more neighboring base stations of the terrestrial cellular network. Transmitters, receivers, related methods are also disclosed for modulation and demodulation of the transmission packets.

Moradi, Hussein↗

DOE OSTI2026

Frontier Job-Centric Telemetry Dataset

Comprehensive analysis of high-performance computing (HPC) systems requires linking workload execution to system behavior. This kind of analysis is vital for diagnosing performance issues, managing capacity, detecting anomalous workloads, and understanding how applications interact with system hardware. This job-centric telemetry dataset unifies scheduler job records with node-level measurements, enabling direct association between workloads and their corresponding power, thermal, and performance characteristics. It contains sanitized, scheduler related metadata for 152,400 individual jobs that ran on the Frontier supercomputer and ended on selected days throughout 2024 and 2025, a subpopulation of ~6.8% of the total number of allocated jobs with non-zero run time on the system over that same period. Each is linked with files that contain telemetry time series records of the power utilization and temperature behavior of its allocated nodes and their processors during the run time of the job. Where available, a portion of the job files also contain network performance time series. Jobs are sampled from select days that reflect normal levels of user activity and possess job size distributions with large numbers of leadership class jobs (>20% of Frontier nodes). Jobs in this dataset attempt to best represent successful user workflows.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2024

Electric vehicle supply equipment location and capacity allocation for fixed-route networks

Electric vehicle (EV) supply equipment location and allocation (EVSELCA) problems for freight vehicles are becoming more important because of the trending electrification shift. Some previous works address EV charger location and vehicle routing problems simultaneously by generating vehicle routes from scratch. Although such routes can be efficient, introducing new routes may violate practical constraints, such as drive schedules, and satisfying electrification requirements can require dramatically altering existing routes. To address the challenges in the prevailing adoption scheme, we approach the problem from a fixed -route perspective. We develop a mixed -integer linear program, a clustering approach, and a metaheuristic solution method using a genetic algorithm (GA) to solve the EVSELCA problem. The clustering approach simplifies the problem by grouping customers into clusters, while the GA generates solutions that are shown to be nearly optimal for small problem cases. A case study examines how charger costs, energy costs, the value of time (VOT), and battery capacity impact the cost of the EVSELCA. Charger equipment costs were found to be the most significant component in the objective function, leading to a substantial reduction in cost when decreased. VOT costs exhibited a significant decrease with rising energy costs. Further, an increase in VOT resulted in a notable rise in the number of fast chargers. Longer EV ranges decrease total costs up to a certain point, beyond which the decrease in total costs is negligible.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2022

A Comparison of Machine Learning Methods for Frequency Nadir Estimation in Power Systems

An increasing penetration level of inverter-based renewable energy resources changes the inertia of power systems, posing challenges for maintaining the desired system frequency stability. An accurate frequency nadir estimation is crucial for power system operators to prepare preventive actions against large frequency excursions. In this paper, five machine learning methods - linear regression, gradient boosting, support vector regression, an artificial neural network, and XGBoost - are applied to two different datasets, i.e., 1) the unit generation dataset and 2) the system total inertia and headroom dataset, for the prediction of the frequency nadir. The training and testing datasets are generated through extensive generation scheduling simulations using Multi-timescale Integrated Dynamic and Scheduling (MI-DAS) toolbox on the Western Electricity Coordinating Council 240-bus system with high renewable penetration levels. Numerical results show that all five machine learning methods perform well in predicting the nadir frequency of the system. Among them, the gradient boosting and the XGBoost are clear winners yielding the best prediction accuracy in terms of four evaluation metrics.

data driven↗