Search NASA⌕ Search

SEARCH · Search NASA

Results for “Resource allocation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

FLEET: Flexible Efficient Ensemble Training for Heterogeneous Deep Neural Networks

Parallel training of an ensemble of Deep Neural Networks (DNN) on a cluster of nodes is an effective approach to shorten the process of neural network architecture search and hyper-parameter tuning for a given learning task. Prior efforts have shown that data sharing, where the common preprocessing operation is shared across the DNN training pipelines, saves computational resources and improves pipeline efficiency. Data sharing strategy, however, performs poorly for a heterogeneous set of DNNs where each DNN has varying computational needs and thus different training rate and convergence speed. This paper proposes FLEET, a flexible ensemble DNN training framework for efficiently training a heterogeneous set of DNNs. We build FLEET via several technical innovations. We theoretically prove that an optimal resource allocation is NP-hard and propose a greedy algorithm to efficiently allocate resources for training each DNN with data sharing. We integrate data-parallel DNN training into ensemble training to mitigate the differences in training rates and introduce checkpointing into this context to address the issue of different convergence speeds. Experiments show that FLEET significantly improves the training efficiency of DNN ensembles without compromising the quality of the result.

Guan, Hui↗

Storm-DEPART (Damage Estimate Prediction and Recovery Tool)

Storm-DEPART (Damage Estimate Prediction and Restoration Tool): Each year hurricanes and tropical storms in the United States damage critical infrastructure assets, disrupt the services they provide, and cause millions to billions of dollars in economic impacts due to extended recovery times. The Storm-DEPART tool and analytical output enable more impactful data-driven decision-making capabilities and strengthen national-level disaster preparedness, response, and recovery. Storm-DEPART, built through multi-month collaboration between Entergy and INL, combines Entergy’s critical infrastructure inventory data with weather forecasts to predict damages to Electric utility’s assets due to natural disasters and the estimated recovery support needed, including time, materials, and resource allocation. In the event of an approaching hurricane, this innovative solution can assess potential damage to power generation capacity, transmission grids, distribution networks, and communications assets from wind bands, storm surge, and flooding. With more effective predictions, Entergy can more efficiently allocate resources to mitigate impacts and optimize recovery for customers. Storm-DEPART also allows Electric utilities the ability to apply a planning scenario and model expected damage to better inform infrastructure restoration needs leading to enhance system resiliency. The technology is fully transferrable to other electric utilities with the same damage estimating challenges. The INL team is working on the evolution of Storm-DEPART to include ice event damage prediction framework.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Storm-DEPART (Damage Estimate Prediction and Recovery Tool)

Storm-DEPART (Damage Estimate Prediction and Restoration Tool): Each year hurricanes and tropical storms in the United States damage critical infrastructure assets, disrupt the services they provide, and cause millions to billions of dollars in economic impacts due to extended recovery times. The Storm-DEPART tool and analytical output enable more impactful data-driven decision-making capabilities and strengthen national-level disaster preparedness, response, and recovery. Storm-DEPART, built through multi-month collaboration between Entergy and INL, combines Entergy’s critical infrastructure inventory data with weather forecasts to predict damages to Electric utility’s assets due to natural disasters and the estimated recovery support needed, including time, materials, and resource allocation. In the event of an approaching hurricane, this innovative solution can assess potential damage to power generation capacity, transmission grids, distribution networks, and communications assets from wind bands, storm surge, and flooding. With more effective predictions, Entergy can more efficiently allocate resources to mitigate impacts and optimize recovery for customers. Storm-DEPART also allows Electric utilities the ability to apply a planning scenario and model expected damage to better inform infrastructure restoration needs leading to enhance system resiliency. The technology is fully transferrable to other electric utilities with the same damage estimating challenges. The INL team is working on the evolution of Storm-DEPART to include ice event damage prediction framework.

24 POWER TRANSMISSION AND DISTRIBUTION↗

iFair: Achieving Fairness in the Allocation of Scarce Resources for Senior Health Care

Efficient resource allocation is crucial in many domains, particularly in senior care, where assigning resources to older adults must consider uncertainties associated with vulnerable populations. In collaboration with Senior Health Facilities (SHFs) and domain experts, this paper presents iFair, a novel framework designed to assist decision-makers in equitably allocating scarce resources to older adults. iFair was prototyped in the context of ongoing work on a data exchange platform, CAREDEX, used for enhancing older adults' resilience during disasters. A key novelty of iFair focuses on aligning resident preferences with resources in urgent situations, expediting care, and enhancing task efficiency. We integrate static and dynamic environmental data, including facility layouts and sensor data, with detailed resident profiles to cater to the individual needs and preferences of residents. While our framework primarily focuses on allocation within facilities, it also extends to a regional scale to support the planning and transfer of seniors to mutual aid facilities. Our experiments adapt data from a real SHF to emulate resource allocation in an emergency fire evacuation setting and highlight the delicate balance that decision-makers can achieve between efficiency and fairness.

Kenne, Modeste Mefenya↗

The Minos Computing Library: Efficient Parallel Programming for Extremely Heterogeneous Systems

Hardware specialization has become the silver bullet to achieve efficient high performance, from Systems-on-Chip systems, where hardware specialization can be ``extreme'', to large-scale HPC systems. As the complexity of the systems increases, so does the complexity of programming such architectures in a portable way. This work introduces the Minos Computing Library (MCL), as system software, programming model, and programming model runtime that facilitate programming extremely heterogeneous systems. MCL supports the execution of several multi-threaded applications within the same compute node, performs asynchronous execution of application tasks, efficiently balances computation across hardware resources, and provides performance portability. We show that code developed on a personal desktop automatically scales up to fully utilize powerful workstations with 8 GPUs and down to power-efficient embedded systems. MCL provides up to 17.5x speedup over OpenCL on NVIDIA DGX-1 systems and up to 1.88x speedup on single-GPU systems. In multi-application workloads, MCL dynamically resource allocation provides up to 2.43x performance improvement over manual, static allocation of computing resources.

Gioiosa, Roberto↗

Calibers: A Bandwidth Calendaring Paradigm for Science Workflows

Many scientific workflows require large data transfers between distributed instrument facilities, storage and computing resources. To ensure that these resources are maximally utilized, R&E networks connecting these resources must ensure that an inherently unpredictable network behaves predictably. In practice, this amounts to the per-application over-provisioning of network resources in an attempt to guarantee that adequate throughput is provided to users. This often results in resource under-utilization over time. One promising solution is the use of deadlines and bandwidth calendaring. In this approach, "fair" resource allocation is replaced with deadline-based resource allocation. However, these approaches often suffer from issues in efficiently regulating resource allocation and failure modes. Therefore, our solution, Calibers, approaches bandwidth calendaring and deadline-awareness in a different way. Calibers uses shaping, metering, and pacing at the edge of the network and end-system to provide participating clients the ability to schedule bandwidth reservations without having to worry about network noise from non-participating clients. Calibers can also fail back to the fair resource allocation of underlying transport protocols if necessary. For example, if a non-participating flow somehow enters the core of the network, or a sudden network change causes the available bandwidth to be exceeded, the underlying transport protocol congestion avoidance implementation will be able to handle the congestion as it normally would. Furthermore, Calibers provides a novel simulation method and resource allocation algorithm.

97 MATHEMATICS AND COMPUTING↗

MROrchestrator: A Fine-Grained Resource Orchestration Framework for MapReduce Clusters

Efficient resource management in data centers and clouds running large distributed data processing frameworks like MapReduce is crucial for enhancing the performance of hosted applications and boosting resource utilization. However, existing resource scheduling schemes in Hadoop MapReduce allocate resources at the granularity of fixed-size, static portions of nodes, called slots. In this work, we show that MapReduce jobs have widely varying demands for multiple resources, making the static and fixed-size slot-level resource allocation a poor choice both from the performance and resource utilization standpoints. Furthermore, lack of co-ordination in the management of mul- tiple resources across nodes prevents dynamic slot reconfigura- tion, and leads to resource contention. Motivated by this, we propose MROrchestrator, a MapReduce resource Orchestrator framework, which can dynamically identify resource bottlenecks, and resolve them through fine-grained, co-ordinated, and on- demand resource allocations. We have implemented MROrches- trator on two 24-node native and virtualized Hadoop clusters. Experimental results with a suite of representative MapReduce benchmarks demonstrate up to 38% reduction in job completion times, and up to 25% increase in resource utilization. We further show how popular resource managers like NGM and Mesos when augmented with MROrchestrator can hike up their performance.

99 GENERAL AND MISCELLANEOUS↗

Value of Information in Fuel Cycle Decision-making

Transitions between nuclear fuel cycles with uncertain key parameters (e.g., SNF and HLW disposal costs, fast reactor capital costs) have been examined using a decision-making under uncertainty methodology. The methodology explicitly incorporates uncertainties in key parameters into the decision-making process by constructing a stochastic model and embedding uncertainties as bifurcations in the decision-tree. The framework of this methodology allows for interpretation using a real options valuation or game theoretic approach, reviewed in. The work presented here extends the decision-making under uncertainty framework to calculate the value of information, which may be used to inform a game against nature. These games against nature have been applied to resource allocation problems, where scarce resources are allocated to maximize expected payoffs. More broadly, the value of developing alternative technologies may be considered under the same framework, where an investment portfolio for technology deployment may be developed. Here, the value of information is taken as the value of reducing uncertainty in key parameters, and these values may be substituted for the traditional payoff matrix when considering allocating scarce resources. Games against nature may provide a unique perspective on R and D funding decisions when considering transitions between nuclear fuel cycles. By determining the value of reducing uncertainties (of gaining information on probability distributions) scarce resources may be allocated in such a way to maximize their utility. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

combinatorial optimization with demands (LDRD Final Report)

This report summarizes the research conducted and results obtained under the Laboratory Directed Research and Development (LDRD) project, “Combinatorial Optimization with Demands ,” from October 2010 to September 2012. Complex resource allocation problems are ubiquitous in mission-driven applications. We investigated resource allocation problems endowed with additional parameters called demands, which allow for richer modeling. Demand endowed resource allocation problems are considerably harder to solve than their traditional counterparts, and we developed a conceptually simple but powerful new framework called iterative packing which allowed us to address the added complexity of demands. By leveraging iterative packing , we designed approximation algorithms to solve a large class of hard resource allocation problems with demands. Approximation algorithms are efficient heuristics which also offer a performance guarantee on the worst case deviation from the cost of an optimal solution. Our algorithms offer better performance guarantees over previously known algorithms; moreover, we are able to show that our algorithms provide performance guarantees that are likely close to best possible.

Parekh, Ojas↗

Performance assessment of ensembles of in situ workflows under resource constraints

Summary Scientific breakthroughs in biomolecular methods and improvements in hardware technology have shifted from a long‐running simulation to a large set of shorter simulations running simultaneously, called an ensemble. In an ensemble, simulations are usually coupled with analyses of data produced by the simulations. In situ methods can be used to analyze large volumes of data generated by scientific simulations at runtime (i.e., simulations and analyses are performed concurrently). In this work, we study the execution of ensemble‐based simulations paired with in situ analyses using in‐memory staging methods. Using an ensemble of molecular dynamics in situ workflows with multiple simulations and analyses, we first show that collecting traditional metrics such as makespan, instructions per cycle, memory usage, or cache miss ratio is not sufficient to characterize complex behaviors of ensembles. We propose a method to evaluate the performance of ensembles of workflows that captures multiple resource usage aspects: resource efficiency, resource allocation, and resource provisioning. Experimental results demonstrate that the proposed method can effectively distinguish the performance of different component placements in an ensemble with up to 32 ensemble members. By evaluating different co‐location scenarios, our proposed performance indicators demonstrate benefits of co‐locating simulation and coupled analyses within a compute node.

Do, Tu Mai Anh↗

Microbiome Adaptation Could Amplify Modeled Projections of Global Soil Carbon Loss With Climate Warming

Warming alters soil microbial traits through ecological and evolutionary processes, directly influencing the decomposition of organic matter, which significantly affects global soil carbon emissions. Yet, soil carbon models largely ignore these processes and their implications for global responses to warming. Here, we incorporate eco-evolutionary theory into a mechanistic model describing microbial soil carbon decomposition to address the question of whether such processes could have consequential effects on climate carbon feedbacks globally. We assume that a key trait of microbes, their resource allocation to production of exoenzymes (which facilitate decomposition of organic matter)—is optimized to environmental temperatures by natural selection. We find that eco-evolutionary optimization results in microbes allocating more resources to enzyme production under warming. When applied at the global scale, eco-evolutionary optimization enhances the biological realism of soil carbon models and significantly amplifies global soil carbon loss by 2100. Our results highlight the significant potential of microbial eco-evolutionary responses to influence carbon cycle feedbacks to climate change, and motivate an urgent need for more comprehensive data to accurately quantify the adaptive potential of microbiomes in response to climate change.

Biodiversity & Conservation↗

A Systems Biology and Pond Culture-based Understanding and Improvement of Metabolic Processes Related to Productivity in Diverse Microalgal Classes for Viable Biofuel Production

The overarching goal of this project was to understand and manipulate fundamental molecular mechanisms involved in maximizing growth rate and lipid accumulation in diverse classes of microalgae under authentic diurnal conditions to enhance production capabilities for biofuels. The research was aimed at filling two critical knowledge gaps by addressing fundamental cellular issues that govern growth and fuel molecule accumulation in conjunction with realistic climate-simulated cultivation: 1) The insufficient understanding of the effect of metabolic topology on cellular carbon partitioning and its regulation with regards to productivity. 2) The understudied effect of the diurnal and cell cycles on cellular metabolism, resource allocation, and resource mobilization under simulated production conditions. To unravel broader as well as lineage-specific regulatory mechanisms that determine productivity, we compared green algae with diatoms. Complementing the systems approach with a pond-based cultivation method under authentic diurnal conditions, we anticipated an understanding of how the effect of diurnal and cell cycles affects growth and lipid productivity. Elucidation of the topology and regulation of relevant metabolic networks in proposed new model organisms was expected to enable improvements in functional capabilities and biosynthetic pathways. The major goals of the project were achieved by the following accomplishments. Genome assemblies and functional annotations for the green algae S. obliquus strain DOE0152Z and Coleastrella sp. strain DOE0202. Development of a metabolic map for the core carbon metabolism for S. obliquus. Analyzed the cellular topology for the core carbon network and proposed differences in core carbon metabolism for the Chlorophyceae versus the Trebouxiophyceae. Discovered the genes for enzymes involved in dicarboxylic acid metabolism and proposed the existence of C4-type metabolism for S. obliquus. Calculated the energy and reducing equivalencies for TAG and isoprenoid biosynthesis based on the core carbon network created in this project. Identified one newly created strain SOM-P004 (UV-Mutagenesis), which performed better at higher culture densities than wild type in LEAPS reactors under simulated environmental conditions. Results obtained within the course of this project were used in 2017 to apply for a Community Science Program (CSP) at the DOE Joint Genome Institute (JGI). The project (JGI Proposal Id: 503423) with the Title “Comparative genomics and germ plasm diversity in Acutodesmus, a green microalgal bioenergy feedstock candidate with potential for breeding and hybridization.” was selected for CSP 2018 for sequencing of up to 30 genomes of the alga A. obliquus (=S. obliquus). Our genome of the strain DOE0152Z serves as a reference for this new CSP project. Characterization of synchronized cultures of Cyclotella cryptica and the implications on productivity. Determination that some cellular pigments respond to light, but others to the cell cycle. Characterization of productivity of C. cryptica in PNNL ponds. Genome sequence and metabolic mapping of Marinichlorella kaistiae KAS603. Demonstration that some green algae store and utilize starch in the cytoplasm. Photorespiratory pathway clarified in T. pseudonana leading to a better understanding of carbon flux pathways in diatoms. Genome sequence determination, annotation, and comparison completed for C. cryptica, identifying potential metabolic differences that lead to high biomass and lipid accumulation ability for this species. Transcriptomic comparison of silicon (Si) and nitrogen (N) limited triacylglycerol accumulation in Cyclotella cryptica.

59 BASIC BIOLOGICAL SCIENCES↗

A sequential Attacker-Defender game for distribution systems resilience enhancement against extreme weather events

Improving distribution system resilience against frequent extreme weather events is important for reliable power system operations. Especially when dealing with events such as hurricanes that have short-term predictions, proactive pre-event preparedness plays a vital role in system resilience performance. In this paper, we propose a novel approach to construct pre-event resource allocation plans for system operators to cope with upcoming threats through a sequential attacker-defender game framework. The sequential attacker-defender game is designed to model the interaction between the extreme weather and the system operator. In each round of the game, the attacker and the defender sequentially update their current strategies by accounting for the opponent’s action set. The attacker model is formulated as a bi-level problem to identify the severe outage scenarios, and the defender model is formulated as a two-stage optimization problem to determine the allocation of restoration resources including mobile responsive resources and repair crews. Two scale-reduction strategies are proposed to ensure the scalability of the game scheme. Finally, case studies on the IEEE 33-bus and a 7149-node practical utility system validate the effectiveness of the proposed sequential game and the efficiency of the scale-reduction strategies.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Water Resources Utilization and Protection in the Coal Mining Area of Northern China

This study investigates multiple types of water resources in the western mining area in China, their supply-demand balance by using the same dimension gray recurrence dynamic model GM (1, 1), and water content coefficients of mines to ease water supply-demand contradiction. A multi-objective programming model is proposed for optimal water resources allocation management. Optimal technical schemes for water resources allocation among different users are obtained. The optimization model improves upon the previous studies by using water demand and water supply forecast. Coordinated development for mining safety, mine water utilization, and remediation and control of water environment is achieved.

54 ENVIRONMENTAL SCIENCES↗

Stochastic pre-event preparation for enhancing resilience of distribution systems

Extreme weather events are the common causes for power supply interruptions and power outages in electrical distribution systems. Improving the distribution system and enhancing its resilience is becoming crucial due to the increased frequency of extreme weather events. Preparation and allocation of multiple flexible resources, such as mobile resources, fuel resources, and labor resources before extreme weather events can mitigate the effects of extreme weather events and enhance the resilience of power distribution systems. Here, in this paper, a two-stage stochastic mixed-integer linear programming (SMILP) is proposed to optimize the preparation and resource allocation process for upcoming extreme weather events, which leads to faster and more efficient post-event restoration. The objective of the proposed two-stage SMILP is to maximize the served load and minimize the operating cost of flexible resources. The first stage in the optimization problem selects the amounts and locations of different resources. The second stage considers the operational constraints of the distribution system and repair crew scheduling constraints. The proposed stochastic pre-event preparation model is solved by a scenario decomposition method, Progressive Hedging (PH), to ease the computational complexity introduced by a large number of scenarios. Furthermore, to show the impact of solar photovoltaic (PV) generation on system resilience, three types of PV systems are considered during a power outage and the resilience improvements with different PV penetration levels are compared. Numerical results from simulations on a large-scale (more than 10,000 nodes) distribution feeder have been used to validate the effectiveness and scalability of the proposed method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimization of Composite Cloud Service Processing with Virtual Machines

By leveraging virtual machine (VM) technology, we optimize cloud system performance based on refined resource allocation, in processing user requests with composite services. Our contribution is three-fold. (1) We devise a VM resource allocation scheme with a minimized processing overhead for task execution. (2) We comprehensively investigate the best-suited task scheduling policy with different design parameters. (3) We also explore the best-suited resource sharing scheme with adjusted divisible resource fractions on running tasks in terms of Proportional-share model (PSM), which can be split into absolute mode (called AAPSM) and relative mode (RAPSM). We implement a prototype system over a cluster environment deployed with 56 real VM instances, and summarized valuable experience from our evaluation. As the system runs in short supply, lightest workload first (LWF) is mostly recommended because it can minimize the overall response extension ratio (RER) for both sequential-mode tasks and parallel-mode tasks. In a competitive situation with over-commitment of resources, the best one is combining LWF with both AAPSM and RAPSM. It outperforms other solutions in the competitive situation, by 16 + % w.r.t. the worst-case response time and by 7.4 + % w.r.t. the fairness.

cloud resource allocation↗

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗

Controlling circuitry underlies the growth optimization of Saccharomyces cerevisiae

Microbial growth emerges from coordinated synthesis of various cellular components from limited resources. In Saccharomyces cerevisiae, cyclic AMP (cAMP)-mediated signaling is shown to orchestrate cellular metabolism; however, it remains unclear quantitatively how the controlling circuit drives resource partition and subsequently shapes biomass growth. Here we combined experiment with mathematical modeling to dissect the signaling-mediated growth optimization of S. cerevisiae. We showed that, through cAMP-mediated control, the organism achieves maximal or nearly maximal steady-state growth during the utilization of multiple tested substrates as well as under perturbations impairing glucose uptake. However, the optimal cAMP concentration varies across cases, suggesting that different modes of resource allocation are adopted for varied conditions. Under settings with nutrient alterations, S. cerevisiae tunes its cAMP level to dynamically reprogram itself to realize rapid adaptation. Moreover, to achieve growth maximization, cells employ additional regulatory systems such as the GCN2-mediated amino acid control. This study establishes a systematic understanding of global resource allocation in S. cerevisiae, providing insights into quantitative yeast physiology as well as metabolic strain engineering for biotechnological applications.

59 BASIC BIOLOGICAL SCIENCES↗