Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud resource allocation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

A survey and taxonomy on energy efficient resource allocation techniques for cloud computing systems

In a cloud computing paradigm, energy efficient allocation of different virtualized ICT resources (servers, storage disks, and networks, and the like) is a complex problem due to the presence of heterogeneous application (e.g., content delivery networks, MapReduce, web applications, and the like) workloads having contentious allocation requirements in terms of ICT resource capacities (e.g., network bandwidth, processing speed, response time, etc.). Several recent papers have tried to address the issue of improving energy efficiency in allocating cloud resources to applications with varying degree of success. However, to the best of our knowledge there is no published literature on this subject that clearly articulates the research problem and provides research taxonomy for succinct classification of existing techniques. Hence, the main aim of this paper is to identify open challenges associated with energy efficient resource allocation. In this regard, the study, first, outlines the problem and existing hardware and software-based techniques available for this purpose. Furthermore, available techniques already presented in the literature are summarized based on the energy-efficient research dimension taxonomy. The advantages and disadvantages of the existing techniques are comprehensively analyzed against the proposed research dimension taxonomy namely: resource adaption policy, objective function, allocation method, allocation operation, and interoperability.

Cloud computing↗

Scaling Optimal Allocation of Cloud Resources Using Lagrange Relaxation

The rapid growth of Cloud Computing (CC) has increased the variety of computing resources, storage, and communication services that pose significant new challenges for the efficient use of cloud resources. The cost-efficient allocation of cloud resources has become a decisive premise for the adoption of CC services. The cost-efficient selection and scheduling of these resources to meet the demands of a scientific workflow is a challenging problem that is exacerbated by the inclusion of multiple CC providers. In this paper, we present a novel strategy for the cost-efficient selection of CC resources using Lagrange relaxation. Our approach is based on preselection of resources and demand decomposition to create a series of smaller sub-problems, which allow the estimation of the best cost-structures and selection of CC service providers for a subset of the time period of the planning horizon. Decomposition of the demand is achieved through the boundary analysis of a continuous relaxation of the problem. Using the metrics defined in terms of the cost and time of completion, we demonstrate excellent performance in relation to optimal solutions. Our method reduced the computational time from hours to seconds for a representative 36-month problem and provided high-quality solutions (< 0.05% relative error). Given the importance of selecting resources and scheduling complex scientific workflows, we believe that this novel strategy will be beneficial for many researchers and users of cloud computing resources.

De La Torre, Luis↗

Co-Operative Resource Allocation: Building an Open Cloud Market Using Shared Infrastructure

In this paper we present DRIVE, a distributed service-based system designed to facilitate an open economic market for federating Cloud providers. Additionally, to address the challenges associated with market ownership and operation we propose the use of a cooperative (co-op) infrastructure in which the services that make up DRIVE are hosted across participants' resources. To prevent malicious behavior we use cryptographic, secure and privacy preserving allocation protocols as a means of establishing trust in the allocation infrastructure. Finally, we investigate through simulation the effect of different strategies, pricing functions, and penalty models on allocation performance and revenue, and show that the overhead of running DRIVE's services on commodity infrastructure is modest.

97 MATHEMATICS AND COMPUTING↗

GRACC: New generation of the OSG accounting

Throughout the last decade the Open Science Grid (OSG) has been fielding requests from user communities, resource owners, and funding agencies to provide information about utilization of OSG resources. Requested data include traditional accounting - core-hours utilized - as well as users certicate Distinguished Name, their affiliations, and eld of science. The OSG accounting service, Gratia, developed in 2006, is able to provide this information and much more. However, with the rapid expansion and transformation of the OSG resources and access to them, we are faced with several challenges in adapting and maintaining the current accounting service. The newest changes include, but are not limited to, acceptance of users from numerous university campuses, whose jobs are flocking to OSG resources, expansion into new types of resources (public and private clouds, allocation-based HPC resources, and GPU farms), migration to pilot-based systems, and migration to multicore environments. In order to have a scalable, sustainable and expandable accounting service for the next few years, we are embarking on the development of the next-generation OSG accounting service, GRACC, that will be based on open-source technology and will be compatible with the existing system. It will consist of swappable, independent components, such as Logstash, Elasticsearch, Grafana, and RabbitMQ, that communicate through a data exchange. GRACC will continue to interface EGI and XSEDE accounting services and provide information in accordance with existing agreements. Lastly, we will present the current architecture and working prototype.

97 MATHEMATICS AND COMPUTING↗

Policy Analysis of Water Availability and Use Issues for Domestic Oil Shale and Oil Sands Development

Oil shale and oil sands resources located within the intermountain west represent a vast, and as of yet, commercially untapped source of energy. Development will require water, and demand for scarce water resources stands at the front of a long list of barriers to commercialization. Water requirements and the consequences of commercial development will depend on the number, size, and location of facilities, as well as the technologies employed to develop these unconventional fuels. While the details remain unclear, the implication is not – unconventional fuel development will increase demand for water in an arid region where demand for water often exceeds supply. Water demands in excess of supplies have long been the norm in the west, and for more than a century water has been apportioned on a first-come, first-served basis. Unconventional fuel developers who have not already secured water rights stand at the back of a long line and will need to obtain water from willing water purveyors. However, uncertainty regarding the nature and extent of some senior water claims combine with indeterminate interstate river management to cast a cloud over water resource allocation and management. Quantitative and qualitative water requirements associated with Endangered Species protection also stand as barriers to significant water development, and complex water quality regulations will apply to unconventional fuel development. Legal and political decisions can give shape to an indeterminate landscape. Settlement of Northern Ute reserved rights claims would help clarify the worth of existing water rights and viability of alternative sources of supply. Interstate apportionment of the White River would go a long way towards resolving water availability in downstream Utah. And energy policy clarification will help determine the role oil shale and oil sands will play in our nation’s future.

04 OIL SHALES AND TAR SANDS↗

Optimization of Composite Cloud Service Processing with Virtual Machines

By leveraging virtual machine (VM) technology, we optimize cloud system performance based on refined resource allocation, in processing user requests with composite services. Our contribution is three-fold. (1) We devise a VM resource allocation scheme with a minimized processing overhead for task execution. (2) We comprehensively investigate the best-suited task scheduling policy with different design parameters. (3) We also explore the best-suited resource sharing scheme with adjusted divisible resource fractions on running tasks in terms of Proportional-share model (PSM), which can be split into absolute mode (called AAPSM) and relative mode (RAPSM). We implement a prototype system over a cluster environment deployed with 56 real VM instances, and summarized valuable experience from our evaluation. As the system runs in short supply, lightest workload first (LWF) is mostly recommended because it can minimize the overall response extension ratio (RER) for both sequential-mode tasks and parallel-mode tasks. In a competitive situation with over-commitment of resources, the best one is combining LWF with both AAPSM and RAPSM. It outperforms other solutions in the competitive situation, by 16 + % w.r.t. the worst-case response time and by 7.4 + % w.r.t. the fairness.

cloud resource allocation↗

A Reference Model for Virtual Machine Launching Overhead

Cloud bursting is one of the key research topics in the cloud computing communities. A well designed cloud bursting module enables private clouds to automatically launch virtual machines (VMs) to public clouds when more resources are needed. One of the main challenges in developing a cloud bursting module is to decide when and where to launch a VM so that all resources are most effectively and efficiently utilized and the system performance is optimized. However, based on system operational data obtained from FermiCloud, a private cloud developed by the Fermi National Accelerator Laboratory for scientific workflows, the VM launching overhead is not a constant. It varies with physical resource utilization, such as CPU and I/O device utilizations, at the time when a VM is launched. Hence, to make judicious decisions as to when and where a VM should be launched, a VM launching overhead reference model is needed. In this paper, we first develop a VM launching overhead reference model based on operational data we have obtained on FermiCloud. Second, we apply the developed reference model on FermiCloud and compare calculated VM launching overhead values based on the model with measured overhead values on FermiCloud. Our empirical results on FermiCloud indicate that the developed reference model is accurate. We believe, with the guidance of the developed reference model, efficient resource allocation algorithms can be developed for cloud bursting process to minimize the operational cost and resource waste.

Wu, Hao↗

Overhead-Aware-Best-Fit (OABF) Resource Allocation Algorithm for Minimizing VM Launching Overhead

FermiCloud is a private cloud developed in Fermi National Accelerator Laboratory to provide elastic and on-demand resources for different scientific research experiments. The design goal of the FermiCloud is to automatically allocate resources for different scientific applications so that the QoS required by these applications is met and the operational cost of the FermiCloud is minimized. Our earlier research shows that VM launching overhead has large variations. If such variations are not taken into consideration when making resource allocation decisions, it may lead to poor performance and resource waste. In this paper, we show how we may use an VM launching overhead reference model to minimize VM launching overhead. In particular, we first present a training algorithm that automatically tunes a given refer- ence model to accurately reflect FermiCloud environment. Based on the tuned reference model for virtual machine launching overhead, we develop an overhead-aware-best-fit resource allocation algorithm that decides where and when to allocate resources so that the average virtual machine launching overhead is minimized. The experimental results indicate that the developed overhead-aware-best-fit resource allocation algorithm can significantly improved the VM launching time when large number of VMs are simultaneously launched.

Wu, Hao↗

A Multi-Objective Approach for Optimizing Edge-Based Resource Allocation Using TOPSIS

Existing approaches for allocating resources on edge environments are inefficient and lack the support of heterogeneous edge devices, which in turn fail to optimize the dependency on cloud infrastructures or datacenters. To this extent, we propose in this paper OpERA, a multi-layered edge-based resource allocation optimization framework that supports heterogeneous and seamless execution of offloadable tasks across edge, fog, and cloud computing layers and architectures. By capturing offloadable task requirements, OpERA is capable of identifying suitable resources within nearby edge or fog layers, thus optimizing the execution process. Throughout the paper, we present results which show the effectiveness of our proposed optimization strategy in terms of reducing costs, minimizing energy consumption, and promoting other residual gains in terms of processing computations, network bandwidth, and task execution time. We also demonstrate that by optimizing resource allocation in computation offloading, it is then possible to increase the likelihood of successful task offloading, particularly for computationally intensive tasks that are becoming integral as part of many IoT applications such robotic surgery, autonomous driving, smart city monitoring device grids, and deep learning tasks. The evaluation of our OpERA optimization algorithm reveals that the TOPSIS MCDM technique effectively identifies optimal compute resources for processing offloadable tasks, with a 96% success rate. Moreover, the results from our experiments with a diverse range of use cases show that our OpERA optimization strategy can effectively reduce energy consumption by up to 88%, and operational costs by 76%, by identifying relevant compute resources.

97 MATHEMATICS AND COMPUTING↗

Intelligently-Automated Facilities Expansion with the HEPCloud Decision Engine

The next generation of High Energy Physics experiments are expected to generate exabytes of data---two orders of magnitude greater than the current generation. In order to reliably meet peak demands, facilities must either plan to provision enough resources to cover the forecasted need, or find ways to elastically expand their computational capabilities. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and to choose an appropriate scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources---spanning multiple cloud providers, multiple HPC centers, and grid computing federations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Spatial–temporal changes in potential evaporation patterns based on the Cloud model and their possible causes

It is of importance to comprehensively investigate the spatial-temporal changes in potential evaporation patterns, which helps guide the long-term water resource allocation and irrigation managements. In this study, the Cloud model was adopted to quantify the average, uniformity, and stability of annual potential evaporation in the Wei River Basin (WRB), a typical arid and semi-arid region in China.. The cross wavelet analysis was then applied to explore the correlations between potential evaporation and Arctic Oscillation (AO)/El Niño Southern Oscillation (ENSO) with an aim to determine the possible causes of potential evaporation variations. Results indicated that: (1) the average of annual potential evaporation in the WRB first declined and then increased, which was similar with its stability, whilst its dispersion degree exhibited a decreasing trend, implying that potential evaporation has a small inter-annual variation; (2) the average of potential evaporation in the western basin was obviously smaller than that in the other areas, while its uniformity and stability in the Guanzhong plain and the Loess Plateau areas are larger than those in other areas, particularly in the western basin where the uniformity and stability are the smallest; (3) both AO and ENSO exhibited strong correlations with potential evaporation variations, indicating that both AO and ENSO have played an important role in the annual potential evaporation variations in the WRB.

54 ENVIRONMENTAL SCIENCES↗

HEPCloud, an Elastic Hybrid HEP Facility using an Intelligent Decision Support System

HEPCloud is rapidly becoming the primary system for provisioning compute resources for all Fermilab-affiliated experiments. In order to reliably meet the peak demands of the next generation of High Energy Physics experiments, Fermilab must plan to elastically expand its computational capabilities to cover the forecasted need. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and at which scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources spanning multiple cloud providers, multiple HPC centers, and grid computing federations. In this paper, we discuss the goals and architecture of the HEPCloud Facility, the architecture of the IDSS, and our early experience in using the IDSS for automated facility expansion both at Fermi and Brookhaven National Laboratory.

Mhashilkar, Parag↗

Energy-efficient cooperative resource allocation and task scheduling for Internet of Things environments

Offloading Internet of Things (IoT) tasks to the cloud for further processing might not always lead to an optimal execution time, particularly in situations such as resource contention, under-provisioning, over-provisioning, and fragmentation. In addition, dynamically optimizing the number of Virtual Machines (VMs) for resource scheduling in order to meet application requirements remains a major research challenge. Further, existing resource scheduling algorithms focus primarily on minimizing operational costs while maximizing resource sharing and utilization. Considering energy utilization as part of the resource allocation and scheduling process as an optimization objective for maintaining load balancing has often been neglected. To address these challenges and more, we propose a cooperative energy-aware resource allocation and scheduling strategy based on a Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) multi-criteria decision-making method. Here we used the Grid Workloads Archive dataset to evaluate our proposed approach named TOPREAL. Experimental results with respect to the allocation of VM resources when considering processing a large segment of tasks indicate that TOPREAL outperforms existing algorithms in terms of energy savings, with an average improvement of 40.25%, while maintaining an average improvement of 16.21% when it comes to execution time. Results also demonstrate that our method can save an average of 78.06 processing hours and 63,215kJ of energy when compared to existing scheduling algorithms. These results demonstrate the effectiveness of our proposed model and the viability of using multi-criteria decision-making techniques such as TOPSIS to solve the resource allocation and scheduling problem in edge environments.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Grid accounting service: state and future development

During the last decade, large-scale federated distributed infrastructures have been continually developed and expanded. One of the crucial components of a cyber-infrastructure is an accounting service that collects data related to resource utilization and identity of users using resources. The accounting service is important for verifying pledged resource allocation per particular groups and users, providing reports for funding agencies and resource providers, and understanding hardware provisioning requirements. It can also be used for end-to-end troubleshooting as well as billing purposes. In this work we describe Gratia, a federated accounting service jointly developed at Fermilab and Holland Computing Center at University of Nebraska-Lincoln. The Open Science Grid, Fermilab, HCC, and several other institutions have used Gratia in production for several years. The current development activities include expanding Virtual Machines provisioning information, XSEDE allocation usage accounting, and Campus Grids resource utilization. We also identify the direction of future work: improvement and expansion of Cloud accounting, persistent and elastic storage space allocation, and the incorporation of WAN and LAN network metrics.

Levshina, T.↗

A Digital Twin of Scalable Quantum Clouds

Quantum computing has emerged as a transformative technology capable of solving complex problems beyond the limit of classical systems. The rapid development of quantum processors has led to the proliferation of cloud-based quantum computing services offered by platforms such as IBM, Google, and Amazon. These platforms introduce unique challenges in resource allocation, job scheduling, and multi-device orchestration as quantum workloads become increasingly complex. In this work, we present a digital twin of quantum cloud infrastructures: a framework designed to model and simulate the behavior of real quantum cloud systems. Developed in Python using the SimPy discrete-event simulation library, the framework replicates key aspects of quantum cloud environments, including detailed quantum device modeling, job lifecycle management, and job fidelity. It incorporates noise-aware fidelity estimation, making it the first of its kind to simulate superconducting gate-based quantum cloud systems at an administrative level with job fidelity. We present use cases as proof of concept, demonstrating that our quantum cloud simulation framework can act as a digital twin of a quantum cloud and support the modeling and implementation of practical systems.

Luo, Waylon [Kent State University]↗

Automatic Cloud Bursting under FermiCloud

Cloud computing is changing the infrastructure upon which scientific computing depends from supercomputers and distributed computing clusters to a more elastic cloud-based structure. The service-oriented focus and elasticity of clouds can not only facilitate technology needs of emerging business but also shorten response time and reduce operational costs of traditional scientific applications. Fermi National Accelerator Laboratory (Fermilab) is currently in the process of building its own private cloud, FermiCloud, which allows the existing grid infrastructure to use dynamically provisioned resources on FermiCloud to accommodate increased but dynamic computation demand from scientists in the domains of High Energy Physics (HEP) and other research areas. Cloud infrastructure also allows to increase a private cloud’s resource capacity through “bursting” by borrowing or renting resources from other community or commercial clouds when needed. This paper introduces a joint project on building a cloud federation to support HEP applications between Fermi National Accelerator Laboratory and Korea Institution of Science and Technology Information, with technical contributions from the Illinois Institute of Technology. In particular, this paper presents two recent accomplishments of the joint project: (a) cloud bursting automation and (b) load balancer. Automatic cloud bursting allows computer resources to be dynamically reconfigured to meet users’ demands. The load balance algorithm which the cloud bursting depends on decides when and where new resources need to be allocated. Our preliminary prototyping and experiments have shown promising success, yet, they also have opened new challenges to be studied

Wu, Hao↗

Best Paper Award: Gathering Threat Intelligence Through Computer Network Deception

Presented at the 2016 IEEE International Symposium on Technologies for Homeland Security (HST), this LDRD-funded research received a Best Paper award, the highest award presented at the conference. The research focused on developing, implementing, and testing a novel computer network operations architecture that enables proactive defense by managing and monitoring enterprise resource allocations and network flows. The architecture leverages three emerging concepts: software-defined networks, cloud computing, and deception, to enable the detection and identification of anomalous access and intrusions to adjust to the dynamic nature of the adversary and to provide a mechanism to discover and react to the adversary’s attacks in a methodical and proactive manner. Additionally, the work allows network defenders to gather information on an adversary’s tools, tactics and procedures, providing insight into the “what, why, and how” of the adversary’s operations. A summary of the work presented at IEEE can be found at http://tiny.sandia.gov/8d8lu.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Disaster-survivable cloud-network mapping

Cloud-computing services are provided to consumers through a network of servers and network equipment. Cloud-network (CN) providers virtualize resources [e.g., virtual machine (VM) and virtual network (VN)] for efficient and secure resource allocation. Disasters are one of the worst threats for CNs as they can causemassive disruptions and CN disconnection. A disaster may also induce post-disaster correlated, cascading failures which can disconnect more CNs. Survivable virtual-network embedding (SVNE) approaches have been studied to protect VNs against single physical-link/- node and dual physical-link failures in communication infrastructure, but massive disruptions due to a disaster and their consequences can make SVNE approaches insufficient to guarantee cloud-computing survivability. In this work, we study the problem of survivable CN mapping from disaster. We consider risk assessment, VM backup location, and post-disaster survivability to reduce the risk of failure and probability of CN disconnection and the penalty paid by operators due to loss of capacity.We formulate the proposed approach as an integer linear program and study two scenarios: a natural disaster, e.g., earthquake and a human-made disaster, e.g., weapons-of-mass-destruction attack. Finally, our illustrative examples show that our approach reduces the risk of CN disconnection and penalty up to 90% compared with a baseline CN mapping approach and increases the CN survivability up to 100% in both scenarios.

97 MATHEMATICS AND COMPUTING↗