SEARCH · Search NASA

Results for “Queues”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Method and apparatus for front end gather/scatter memory coalescing

A system for processing gather and scatter instructions can implement a front-end subsystem, a back-end subsystem, or both. The front-end subsystem includes a prediction unit configured to determine a predicted quantity of coalesced memory access operations required by an instruction. A decode unit converts the instruction into a plurality of access operations based on the predicted quantity, and transmits the plurality of access operations and an indication of the predicted quantity to an issue queue. The back-end subsystem includes a load-store unit that receives a plurality of access operations corresponding to an instruction, determines a subset of the plurality of access operations that can be coalesced, and forms a coalesced memory access operation from the subset. A queue stores multiple memory addresses for a given load-store entry to provide for execution of coalesced memory accesses.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Method and apparatus for back end gather/scatter memory coalescing

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Modeling information flow in a computer processor with a multi-stage queuing model

In this paper, we introduce a nonlinear stochastic model to describe the propagation of information inside a computer processor. In this model, a computational task is divided into stages, and information can flow from one stage to another. The model is formulated as a spatially-extended, continuous-time Markov chain where space represents different stages. This model is equivalent to a spatially-extended version of the M/M/s queue. The main modeling feature is the throttling function which describes the processor slowdown when the amount of information falls below a certain threshold. We derive the stationary distribution for this stochastic model and develop a closure for a deterministic ODE system that approximates the evolution of the mean and variance of the stochastic model. In conclusion, we demonstrate the validity of the closure with numerical simulations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Approach for energy efficient building design during early phase of design process

Energy consumption in the building sector is about 40% of total energy consumed globally and is trending upwards, along with its contribution to greenhouse gas (GHG) emissions. Given the adverse impacts of GHG emissions, it is crucial to integrate energy efficiency into building designs. The most significant opportunities for enhancing energy performance are present during the initial phases of building design, when there is less impact of other design constraints. Various tools exist for simulating different design options and providing feedback in terms of energy consumption and comfort parameters. These simulation outputs must then be analyzed to derive design solutions. This paper presents an innovative approach that utilizes user input parameters, processes them through cloud computing, and outputs easily understandable strategies for energy-efficient building design. The methodology employs Asynchronous Distributed Task Queues (DTQ) - a more scalable and reliable alternative to conventional speedup techniques-for conducting parametric energy simulations in the cloud. The goal of this approach is to assist design teams in identifying, visualizing, and prioritizing energy-saving design strategies from a range of possible solutions for each project. Furthermore, a tool ‘eDOT’ has been developed utilizing the discussed methodology. Unlike existing tools, eDOT leverages artificial intelligence to dynamically generate and provide design strategies during the early phases of design process. By simplifying the simulation process, eDOT enables design teams to make informed, data-driven decisions without needing to interpret complex simulation outputs. A case study simulated for two locations is provided in this paper to demonstrate the effectiveness of eDOT, further underscoring its practical impact on energy-efficient building design.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2022

Online Learning and Pricing for Network Revenue Management with Reusable Resources

We consider a price-based network revenue management problem with multiple products and multiple reusable resources. Each randomly arriving customer requests a product (service) that needs to occupy a sequence of reusable resources (servers). We adopt an incomplete information setting where the firm does not know the price-demand function for each product and the goal is to dynamically set prices of all products to maximize the total expected revenue of serving customers. We propose novel batched bandit learning algorithms for finding near-optimal pricing policies, and show that they admit a near-optimal cumulative regret bound of $O(J\sqrt{XT})$, where J, X, and T, are the numbers of products, candidate prices, and service periods, respectively. As part of our regret analysis, we develop the first finite-time mixing time analysis of an open network queueing system (i.e., the celebrated Jackson Network), which could be of independent interest. Our numerical studies show that the proposed approaches perform consistently well.

42 ENGINEERING↗

DOE OSTI2021

Utility-Scale Solar, 2021 Edition

Berkeley Labs "Utility-Scale Solar", 2021 Edition presents analysis of empirical plant-level data from the U.S. fleet of ground-mounted photovoltaic (PV), PV+battery, and concentrating solar-thermal power (CSP) plants with capacities exceeding 5 MWAC. While focused on key developments in 2020, this report explores trends in deployment, technology, capital and operating costs, capacity factors, the levelized cost of solar energy (LCOE), power purchase agreement (PPA) prices, wholesale market value, and interconnection queue data.

2021↗

DOE OSTI2023

Utility-Scale Solar, 2023 Edition: Analysis of Empirical Plant-level Data from U.S. Ground-mounted PV, PV+battery, and CSP Plants (exceeding 5 MWAC)

Berkeley Labs "Utility-Scale Solar", 2023 Edition presents analysis of empirical plant-level data from the U.S. fleet of ground-mounted photovoltaic (PV), PV+battery, and concentrating solar-thermal power (CSP) plants with capacities exceeding 5 MWAC. While focused on key developments in 2022, this report explores trends in deployment, technology, capital and operating costs, capacity factors, the levelized cost of solar energy (LCOE), power purchase agreement (PPA) prices, wholesale market value, and interconnection queue data.

analysis↗

DOE OSTI2022

Utility-Scale Solar, 2022 Edition: Analysis of Empirical Plant-level Data from U.S. Ground-mounted PV, PV+battery, and CSP Plants (exceeding 5 MWAC)

Berkeley Labs "Utility-Scale Solar", 2022 Edition presents analysis of empirical plant-level data from the U.S. fleet of ground-mounted photovoltaic (PV), PV+battery, and concentrating solar-thermal power (CSP) plants with capacities exceeding 5 MWAC. While focused on key developments in 2021, this report explores trends in deployment, technology, capital and operating costs, capacity factors, the levelized cost of solar energy (LCOE), power purchase agreement (PPA) prices, wholesale market value, and interconnection queue data.

2022↗

DOE OSTI2024

Utility-Scale Solar, 2024 Edition: Analysis of Empirical Plant-level Data from U.S. Ground-mounted PV, PV+battery, and CSP Plants (exceeding 5 MWAC)

Berkeley Labs "Utility-Scale Solar", 2024 Edition presents analysis of empirical plant-level data from the U.S. fleet of ground-mounted photovoltaic (PV), PV+battery, and concentrating solar-thermal power (CSP) plants with capacities exceeding 5 MWAC. While focused on key developments in 2023, this report explores trends in deployment, technology, capital and operating costs, capacity factors, the levelized cost of solar energy (LCOE), power purchase agreement (PPA) prices, wholesale market value, net value, and interconnection queue data.

analysis↗

DOE OSTI2021

Exploring filamentous fungi depolymerization of corn stover in the context bioenergy queuing operations

Recalcitrance of lignocellulosic feedstocks to depolymerization is a significant barrier for bioenergy production approaches that require conversion of monomeric carbohydrates to renewable energy sources. This study assesses how low-cost modifications in the feedstock supply chain can be transformed into targeted pretreatments in the context of the entire bioenergy supply chain. The aim of this research is to overcome the physiochemical barriers in corn stover that necessitate increased severity in conversion in terms of chemical loading, temperature, and residence time. Corn stover samples were inoculated with a selective (Ceriporiopsis subvermispora) and non-selective (Phaenarochaete chrysosporium) lignin degrading filamentous fungal strains, then stored aerobically to determine the working envelope for fungal pretreatment to achieve lignin degradation. Dry matter loss and gross chemical makeup of corn stover varied by the length of treatment (2 and 4 weeks) and by the moisture content of the treated corn stover samples (40 and 60%, wet basis). Dry matter loss in P. chrysosporium inoculated biomass was elevated compared to the C. subvermispora inoculated biomass; however, treatment also induced additional chemical composition changes suggestive of depolymerization. Scanning electron microscope images reveal hyphae attached within cell lumen and suggest structural changes within P. chrysosporium treated corn stover after 60% moisture storage. These results highlight that fungal treatment approaches must balance loss of convertible material with the potential for reduction in recalcitrance. Techno-economic assessment (TEA) of fungal pretreatment in a short-term queuing system indicated the viability of this approach compared to conventional queuing operations. The total queuing system cost was estimated at $\$$1.65/tonne of biomass stored. After applying the credit of $\$$1.48/tonne from energy savings in the conversion phase using fungal pretreated biomass, the total system cost was $0.80 lower than traditional biomass queueing approach. While the TEA results suggested that treating biomass with C. subvermispora is the most economically viable storage method in the designed fungal-assisted queuing system, future research should focus on additional fungal depolymerization such as those observed in the P. chrysosporium inoculated biomass.

09 BIOMASS FUELS↗

DOE OSTI2022

Unveiling User Behavior on Summit Login Nodes as a User

We observe and analyze usage of the login nodes of the leadership class Summit supercomputer from the perspective of an ordinary user—not a system administrator—by periodically sampling user activities (job queues, running processes, etc.) for two full years (2020–2021). Our findings unveil key usage patterns that evidence misuse of the system, including gaming the policies, impairing I/O performance, and using login nodes as a sole computing resource. Our analysis highlights observed patterns for the execution of complex computations (workflows), which are key for processing large-scale applications.

Wilkinson, Sean↗

DOE OSTI2026

Cross-domain digital twin architecture for predictive maintenance via machine learning and Large Language Models

This research introduces a comprehensive framework for creating and deploying a digital twin platform for continuous monitoring and predictive maintenance within industrial settings. Through utilizing advanced technologies, including Unreal Engine 5, Unity 3D, the Message Queue Telemetry Transport protocol, Random Forest machine learning algorithms, and Large Language Models (LLMs), we establish a platform that digitally reproduces physical equipment and translates digital controls into real-world actions. This facilitates preventive maintenance approaches and improves operational effectiveness. The digital twin platform gathers sensor data from operational equipment, analyzes it using machine learning, and delivers practical insights to prevent potential malfunctions and enhance equipment performance. Furthermore, the incorporation of a web portal enables efficient monitoring and access to historical data, educational materials, and equipment status information. Preliminary findings indicate that digital twins can transform industrial equipment management and maintenance methodologies.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Predicting runtime and resource utilization of jobs on integrated cloud and HPC systems

Recent advances in virtualization technologies used in cloud computing offer performance that closely approaches bare-metal levels. Combined with specialized instance types and high-speed networking services for cluster computing, cloud platforms have become a compelling option for high-performance computing (HPC). However, most current batch job schedulers in HPC systems are designed for homogeneous clusters and make decisions based on limited information about jobs and system status. Scientists typically submit computational jobs to these schedulers with a requested runtime that is often over- or under-estimated. More accurate runtime predictions can help schedulers make better decisions and reduce job turnaround times. Here, they can also support decisions about migrating jobs to the cloud to avoid long queue wait times in HPC systems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Feasibility of critical infrastructure protection using network functions for programmable and decoupled ICS policy enforcement over WAN

Industrial control systems (ICS) represent a major component of our critical infrastructure. With the increasing need for more control and monitoring of such systems, ICS have seen an increase in connectivity to wide area networks (WAN) exposing aging equipment to rapidly evolving cybersecurity threats. Furthermore, the ICS data requires a reliability measure from the networks for critical functions for infrastructure monitoring and control. Especially when remote plant sites are involved such as pipelines, energy distribution networks, and transportation, WAN transport impairments most often provide a best effort delivery with no strict reliability guarantees. Network functions can provide a vendor agnostic, programmable critical infrastructure protection with a single maintenance, policy determination, and reliability assurance surface. A network function (NF) can be utilized for policy enforcement over the communication between remote entities and the main control office. This paper presents the research on transparent integration with existing ICS without disrupting communications, resulting in minimal downtime while decoupling the fast paced evolution of defensive security measures from the upgrade cycle of expensive long term hardware. We report our measurements on the resource requirements and overhead in the network for successful NF insertion under a wide variety of network impairments (network packet delay, reordering, and loss). Our paired NF implementation provides a policy enforcement platform extensible to cover myriad cybersecurity-related communication goals, including packet signing for verification, encryption for data privacy, packet filtering and data diode operation (i.e. protecting against eavesdropping, packet injection, and denial-of-service). Furthermore, bundling communication specifications into packet flows allows for tunability in applying policies as coarse- or fine-grained as the needs of the operator. We report on network function resource requirements in the form of required queue depth and network utilization overhead to inform the decision making against hardware cost constraints.

42 ENGINEERING↗

DOE OSTI2023

Economic evaluation of variable renewable energy participation in U.S. ancillary services markets

Variable renewable energy (VRE) is not yet meaningfully participating in U.S. ancillary services (AS) markets. VRE participation in AS markets could provide a new source of revenue for VRE resource owners to offset declining energy and capacity values and a new tool for power system operators to address emerging system constraints. This paper uses a price-taker dispatch model and historical prices to estimate the economic value of standalone and hybrid (battery-paired) VRE participation in AS markets, from the resource owner and electricity system perspectives, in each of the seven U.S. independent system operator and regional transmission organization (ISO/RTO) markets where ancillary service prices are set. Across ISO/RTO markets, average (2015–2019) simulated incremental revenues from power regulation market participation were 0.0–2.9 USD/MWh (+0–15% of revenue without participation) for standalone VRE owners and 1–33 USD/MWh (+1–69%) for hybrid VRE owners. However, ISO/RTO reserve markets are relatively thin and have the potential to become saturated by energy storage projects that are currently in ISO/RTO interconnection queues. In most markets, standalone and hybrid VRE could provide power regulation reserves during periods with high power regulation prices, suggesting that VRE participation in AS markets could have high system value. The analysis highlights the relevance of separate upward and downward power regulation products and indicates that ISOs/RTOs might consider initially focusing on enabling hybrid VRE provision of AS.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2023

Integration of discrete-event dynamics and machining dynamics for machine tool: Modeling, analysis and algorithms

Machining dynamics research lays a solid foundation for machining operations by providing stable combinations of spindle speed and depth of cut. Furthermore, machine learning has been applied to predict tool life as a function of cutting speed. However, the existing research does not consider the discrete-event dynamics in machine shop, i.e., the machine tool needs to process a series of parts in queue under various practical production requirements. This paper addresses the integration of discrete-event dynamics and machining dynamics to achieve cost savings in machining. A learning-based cost function is first proposed for the studied integrated optimization problem of machine tool. The proposed cost function utilizes the predicted tool life under different stable cutting speeds for further optimizing speed selection of machine tool to deal with the discrete-event dynamics in machine shop. Then, according to the practical production requirements, effective mathematical optimization models are developed for the related integrated optimization problems with the consideration of cost, makespan and due date, respectively. Numerical results show the effectiveness of our proposed methods and also the potential to be used in practice.

Ma, Mason↗

DOE OSTI2026

Cascading economic losses from port disruptions under capacity constrained multimodal freight networks

This study quantifies how throughput disruptions at major seaports cascade through capacity-constrained multimodal freight networks and interregional production systems. We couple an agent-based model (ABM) multimodal freight simulation that resolves rerouting, terminal queueing, and inventory drawdown under binding modal and facility capacities with a multiregional output loss input-output (MRIIM) model that propagates realized delivery shortfalls across regions and sectors. The framework is demonstrated for the Port of Los Angeles using Freight Analysis Framework flows and Bureau of Economic Analysis input-output accounts and is evaluated over a 52-week horizon under deterministic sector targeted shocks and stochastic disruption realizations with uncertain severity and duration. Results indicate nonlinear amplification: realized national losses concentrate in manufacturing and transportation/warehousing even when exogenous port shocks are dispersed, suggesting that congestion spillback and limited short-run substitution can dominate the initial shock allocation. We further evaluate a tabular reinforcement-learning (Q-learning) intervention layer that selects among a small set of implementable system level levers (truck-to-rail and truck-to-barge shift settings) without overriding shipper routing, finding that such interventions reduce total losses for moderate disruptions but yield diminishing returns once substitute modes approach capacity. By linking operational freight behavior to system wide impacts under uncertainty, the proposed ABM-MRIIM pipeline provides a reusable workflow for port disruption stress testing, identification of structurally critical sectors/corridors, and evaluation of resilience interventions under realistic capacity limits.

42 ENGINEERING↗