Grid and Cloud Scheduling and Resource Management
|
|
- Toby Lambert
- 7 years ago
- Views:
Transcription
1 Grid and Cloud Scheduling and Resource Management Dick Epema ASCI a24 1 Parallel and Distributed Group/
2 Outline Introduction Resource Management in Grids versus Clouds Condor Flocking in Grids Processor Co-allocation Cycle Scavenging Performance Variability in Clouds ASCI a24 2
3 Grids and Clouds Grids: large, multi-organizational collections of heterogeneous machines (desktops, clusters, supercomputers) best-effort, no cost model Clouds: large, uni-organizational, homogeneous data centers weak SLAs, cost model ASCI a24 3
4 Grids versus Clouds heterogeneous homogeneous many types of systems datacenters multi-organizational clearly defined organization scientific applications web/ applications authentication issues authentication by credit card one user - many resources elasticity no industry involvement much industry involvement free (?) payment model energy awareness Grids Clouds ASCI a24 4
5 Job types in Grids and Clouds Sequential jobs/bags-of-tasks E.g., parameter sweep applications (PSAs) These account for the vast majority of jobs of grids Leads to high-throughput computing Parallel jobs Extension of high-performance computing In most grids, only a small fraction Workflows Multi-stage data filtering, etc Represented by directed (acyclic) graphs MapReduce applications (in clouds) Miscellaneous Interactive simulations Data-intensive applications (also in clouds) Web applications (in clouds) ASCI a24 5
6 A General Resource Management Framework (1) ASCI a24 6
7 A General Resource Management Framework (2) Local resource management system (Resource Level) Basic resource management unit Provides low level resource allocation and scheduling e.g., PBS, SGE, LSF Global resource management system (Grid Level) Coordinates the local resource management systems within multiple or distributed sites Provides high-level functionalities to efficiently use resources: Job submission Resource discovery and selection Authentication, authorization, accounting Scheduling Job monitoring Fault tolerance ASCI a24 7
8 How to select resources in the grid? Grid scheduling is the process of assigning jobs to grid resources Grid Resource Broker/ Scheduler Job Job Job Local Resource Manager Local Resource Manager Local Resource Manager Clusters (Space Shared Allocation) Single CPU (Time Shared Allocation) Clusters (Space Shared Allocation) ASCI a24 8
9 Resource Characteristics in Grids (1) Autonomous each resource has its own management policy or scheduling mechanism no central control/multi-organizational sets of resources Heterogeneous hardware (processor architectures, disks, network) basic software (OS, libraries) grid software (middleware) systems management (security set-up, runtime limits) -/+ ASCI a24 9
10 Resource Characteristics in Grids (2) Size large numbers of nodes, providers, consumers large amounts of data Varying Availability resources can join or leave to the grid at any time due to maintenance, policy reasons, and failures Insecure and unreliable environment prone to various types of attacks + -/+? ASCI a24 10
11 Problems in Grid Scheduling (1) 1. Grid schedulers do not own resources themselves they have to negotiate with autonomous local schedulers authentication/multi-organizational issues 2. Grid schedulers have to interface to different local schedulers some may have support for reservations, others are queuing-based some may support checkpointing, migration, etc 3. Structure of applications many different structures (parallel, PSAs, workflows, database, etc.) need for application adaptation modules =? ever more support ASCI a24 11
12 Problems in Grid scheduling (2) 4. Lack of a reservation mechanism but with such a mechanism we need good runtime estimates 5. Heterogeneity - see above 6. Failures monitor the progress of applications/sanity of systems only thing we know to do upon failures: restart (possibly from a checkpoint)? 7. Performance metric turn-around time cost 8. Reproducibility of performance experiments +/- ASCI a24 12
13 Example Grids: The Globus Toolkit Widely used grid middleware Negotiate and control access to resources GRAM component for control of local execution Globus toolkit includes software services and libraries for: resource monitoring resource specification language resource discovery and management security file management (gridftp) Was the dominant grid middleware in the period Main designers: Ian Foster (Chicago) and Carl Kesselman (LA) Achievement award talk Ian Foster at HPDC 12 in Delft online at ASCI a24 13
14 Condor (1/8): batch queuing system + grids Condor is a high-throughput scheduling system started around 1986 as one of many batch queuing systems for clusters (of desktop machines), and has survived! supports cycle scavenging: use idle time on clusters of machines introduced the notions of matchmaking and classads provides remote system calls, a queuing mechanism, scheduling policies, priority scheme, resource monitoring initiated and still being developed by Miron Livny, Madison, Wisc. D.H.J. Epema, M. Livny, R. van Dantzig, X. Evers, and J. Pruyne, "A Worldwide Flock of Condors: Load Sharing among Workstation Clusters, Future Generation Computer Systems, Vol. 12, pp , ASCI a24 14
15 Condor (2/8): matchmaking Basic operation of Condor: 1c 1a jobs send classads to the matchmaker 1b machines send classads to the matchmaker 1c the matchmaker matches jobs and machines 1d and notifies the submission machine 2a which starts a shadow process is that represents the remote job on the execution machine 2b/c and contacts the execution machine 3b/c on the execution machine, the actual remote user job is started ASCI a24 15
16 Condor (3/8): combining pools (design 1) Federation with a Flock Central Manager: FCM Disadvantages: who maintains the FCM? FCM is single point of failure the matchmakers have to be modified modified ASCI a24 16
17 Condor (4/8): combining pools (design 2) Protocol between Central Managers: Disadvantage: the matchmaker has to be modified ASCI a24 17
18 Condor (5/8): combining pools (design 3) Connecting pools network-style with GateWays: Condor flocking GW GW GW Advantages: transparent to the matchmaker no component to be maintained by a third party in the GWs, any access policies and resource sharing policies can be implemented ASCI a24 18
19 Condor (6/8): flocking as before submission pool execution pool as before MM MM Conclusions: Design considerations for Condor Flocking are still very valid when joining systems (cloud federations?) Nice, clear, transparent research solution that was too complex in practice submission machine GateWay presents itself as machine of other pool GateWay presents remote job as if it is its own execution machine ASCI a24 19
20 Condor (7/8): user flocking Conclusion: Condor and Condor flocking have survived 20 years of grid computing!! ASCI a24 20
21 Experimentation: DAS-4 VU (148 CPUs) SURFnet6 10 Gb/s lambdas UvA/MultimediaN (72) UvA (32) System purely for CS research Operational since oct Specs: 1,600 cores (quad cores) 2.4 GHz CPUs accelerators (GPUs) 180 TB storage 10 Gb/s Infiniband Gb Ethernet TU Delft (64) Leiden (32) Astron (46) ASCI a24 21
22 KOALA: a co-allocating grid scheduler Original goals: 1. processor co-allocation: parallel applications 2. data co-allocation: job affinity based on data locations 3. load sharing: in the absence of co-allocation while being transparant for local schedulers Additional goals: research vehicle for grid and cloud research support for (other) popular application types KOALA is written in Java is middleware independent (initially Globus-based) has been deployed on the DAS2 - DAS4 since sept 2005 ASCI a24 22
23 KOALA: the runners The KOALA runners are adaptation modules for different application types: set up communication / name server / environment launch applications + perform application-level scheduling scheduling policies Current runners: CSRunner: for cycle-scavenging applications (PSAs) IRunner: for applications using the Ibis Java library Mrunner: for malleable parallel applications OMRunner: for co-allocated parallel OpenMPI applications Wrunner: for co-allocated workflows MR-runner: for MapReduce applications (under construction) ASCI a24 23
24 Co-Allocation (1) In grids, jobs may use resources in multiple sites: co-allocation or multi-site operation Reasons: to benefit from available resources (e.g., processors) to access and/or process geographically spread data application characteristics (e.g., simulation in one location, visualization in another) Resource possession in different sites can be: simultaneous (e.g., parallel applications) coordinated (e.g., workflows) With co-allocation: more difficult resource-discovery process single global job need to coordinate allocations by autonomous resource managers ASCI a24 24
25 Co-allocation (2): job types fixed job job components non-fixed job job component placement fixed scheduler decides on component placement flexible job total job same total job size scheduler decides on split up and placement single cluster of same total size ASCI a24 25
26 Co-allocation (3): slowdown Co-allocated applications are less efficient due to the relatively slow wide-area communications Slowdown of a job: execution time on multicluster execution time on single cluster (>1 usually) Processor co-allocation is a trade-off between faster access to more capacity, and higher utilization shorter execution times ASCI a24 26
27 Co-allocation (4): scheduling policies Placement policies dictate where the components of a job go Placement policies for non-fixed jobs: 1. Load-aware: Worst Fit (WF) (balance load in clusters) 2. Input-file-location-aware: Close-to-Files (CF) (reduce file-transfer times) 3. Communication-aware: Cluster Minimization (CM) (reduce number of wide-area messages) Placement policies for flexible jobs: 1. Communication-aware: Flexible Cluster (CM for flexible) Minimization (FCM) 2. Network-aware: Communication-Aware (take latency into account) (CA) ASCI a24 27
28 Co-allocation (5): simulations/analysis Model has a host of parameters Main conclusions: Co-allocation is beneficial when the slowdown 1.20 Unlimited co-allocation is no good: limit the number of job components limit the maximum job-component size Give local jobs some but not absolute priority over global jobs Mathematical analysis for maximal utilization See, e.g.: 1. A.I.D. Bucur and D.H.J. Epema, Trace-Based Simulations of Processor Co-Allocation Policies in Multiclusters, IEEE/ACM High Performance Distributed Computing (HPDC) A.I.D. Bucur and D.H.J. Epema, Scheduling Policies for Processor Co-Allocation in Multicluster Systems," IEEE Trans. on Parallel and Distributed Systems, Vol. 18, pp , ASCI a24 28
29 Co-Allocation (6): Experiments on the DAS3 average execution time (s) average job response time (s) Delft poorly connected number of clusters combined FCM CA FCM CA [w/o Delft] [with Delft] O.O. Sonmez, H.H. Mohamed, and D.H.J. Epema, On the Benefit of Processor Co-Allocation in Multicluster Grid Systems, IEEE Trans. on Parallel and Distributed Systems, Vol.21, pp , ASCI a24 29
30 Cycle Scavenging (1): System Requirements 1. Unobtrusiveness Minimal delay for (higher priority) local and grid jobs 2. Fairness Multiple cycle scavenging applications running concurrently should be assigned comparable CPU-Time 3. Dynamic Resource Allocation Cycle scavenging applications have to Grow/Shrink at runtime 4. Efficiency As much use of dynamic resources as possible 5. Robustness and Fault Tolerance Long-running, complex system: problems will occur, and must be dealt with ASCI a24 30
31 Cycle Scavenging (2): Policies 1. Equipartition-All (grid-wide basis) CS User-1 Clusters CS User-2 CS User-3 Non-CS jobs 2. Equipartition-PerSite (per-cluster basis) CS User-1 Clusters CS User-2 CS User-3 Non-CS jobs ASCI a24 31
32 Cycle Scavenging (3): Experimental Results Number of Completed Tasks Equi-All Equi-All Equi-PerSite Equi-PerSite WBlock WBurst WBlock WBurst Equi-PerSite is fair and superior to Equi-All O.O. Sonmez, B. Grundeken, H.H. Mohamed, A. Iosup, and D.H.J. Epema, Scheduling Strategies for Cycle Scavenging in Multicluster Grid Systems," 9th IEEE/ACM Int'l Symposium on Cluster Computing and the Grid (CCGRID09), May ASCI a24 32
33 Research Questions A. Iosup, M.N. Yigitbasi, and D.H.J. Epema, On the Performance Variability of Production Cloud Services, 11th IEEE/ACM Int'l Symp. on Cluster, Cloud, and Grid Computing (CCGrid), pp , ASCI a24 33
34 Outline 1. Introduction and Motivation 2. Method 3. Q1: Analysis of performance variability 1. Analysis of the AWS Dataset 2. Analysis of the GAE Dataset 4. Q2: Impact of variability on large-scale applications 1. Grid & PPE Job Execution 2. Selling Virtual Goods in Social Networks 3. Game Status Maintenance for Social Games ASCI a24 34
35 Production Cloud Services Amazon Web Services (AWS) IaaS EC2 (Elastic Compute Cloud) S3 (Simple Storage Service) SQS (Simple Queueing Service) SDB (Simple Database) FPS (Flexible Payment Service) Google App Engine (GAE) PaaS Run (Python/Java runtime) Datastore (Database) Memcache (Caching) URL Fetch (Web crawling) ASCI a24 35
36 Our Method (1): Performance Traces CloudStatus ( now defunct) real-time values and weekly averages for most of the AWS and GAE services values obtained with periodic performance probes (sampling rate is under 2 minutes) data obtained from the cloudstatus website using web crawls ASCI a24 36
37 Our Method (2): Analysis 1. Data selection Select several months worth of data to see whether a performance metric is highly variable 2. Find the characteristics of variability Basic statistics: the five quartiles (Q0-Q4) including the median (Q2), the mean, the standard deviation Derivative statistic: the IQR (Q3-Q1) E.g., coefficient of variation > 1.1 indicates high variability 3. Analyze the performance variability time patterns Investigate for each performance metric the presence of daily/ monthly/weekly/yearly time patterns E.g., for monthly patterns divide the dataset into twelve subsets and for each subset compute the statistics and plot for visual inspection ASCI a24 37
38 Our Method (3): Is Variability Present? Use statistics or visual inspection EC2 min Q1 median Q3 Max IQR mean Std Dev ASCI a24 38
39 Outline 1. Introduction and Motivation 2. Method 3. Q1: Analysis of performance variability 1. Analysis of the AWS Dataset 2. Analysis of the GAE Dataset 4. Q2: Impact of variability on large-scale applications 1. Grid & PPE Job Execution 2. Selling Virtual Goods in Social Networks 3. Game Status Maintenance for Social Games ASCI a24 39
40 AWS Dataset (1): EC2 Deployment Latency [s]: Time it takes to start a small instance, from the startup to the time the instance is available Higher IQR and range in weeks due to increasing EC2 user base? has impact on applications using EC2 auto-scaling ASCI a24 40
41 AWS Dataset (2): S3 Get Throughput [bytes/s]: Estimated rate at which an object in a bucket is read The last five months of the year exhibit much lower IQR and range more stable performance for the last five months may be due to software/infrastructure upgrades ASCI a24 41
42 AWS Dataset (3): SQS Variable Performance Stable Performance Average Lag Time [s]: Time it takes for a posted message to become available to read (average taken over multiple queues) Long periods of stability (low IQR and range) Periods of high performance variability also exist ASCI a24 42
43 GAE Dataset (1): Run Service Fibonacci [ms]: Time it takes to calculate the 27 th Fibonacci number Highly variable performance until September Last three months have stable performance (low IQR and range) ASCI a24 43
44 GAE Dataset (2): Datastore Read Latency [s]: Time it takes to read a User Group Yearly pattern from January to August? The last four months of the year exhibit much lower IQR and range ASCI a24 44
45 GAE Dataset (3): Memcache PUT [ms]: Time it takes to put 1 MB of data in memcache Median performance per month has an increasing trend over the first 10 months The last three months of the year exhibit stable performance ASCI a24 45
46 Outline 1. Introduction and Motivation 2. Background 3. Q1: Analysis of performance variability 1. Analysis of the AWS Dataset 2. Analysis of the GAE Dataset 4. Q2: Impact of variability on large-scale applications 1. Grid & PPE Job Execution 2. Selling Virtual Goods in Social Networks 3. Game Status Maintenance for Social Games ASCI a24 46
47 Experimental Setup (1): Simulations Trace-based simulations for three applications Input Grid Workload Archive traces (job execution) Number of daily unique users (other two apps) Monthly performance availibility/variability Application Service Job Execution Selling Virtual Goods Game Status Maintenance GAE Run AWS FPS AWS SDB + GAE Datastore ASCI a24 47
48 Experimental Setup (2): Metrics For Job Execution: average Response Time and Average Bounded Slowdown cost in millions of consumed CPU hours For the other two applications: Aggregate Performance Penalty -- APR(t) P(t): Pref : U(t): max U(t): performance at time t (in some metric) reference performance: Average of the twelve monthly medians number of users at time t max number of users over the whole trace the lower the better (how many users are hit?) ASCI a24 48
49 Game Status Maintenance (1): Scenario Maintenance of game status for a large-scale social game such as Farm Town, which has millions of unique users daily For such games, status maintenance is done with simple database operations Trace: number of users of Farm Town AWS SDB and GAE Datastore We assume that the number of database operations depends linearly on the number of daily unique users ASCI a24 49
50 Game Status Maintenance (2): Results AWS SDB GAE Datastore Big discrepancy between SDB and Datastore services Sep 09-Jan 10: APR of Datastore is well below than that of SDB due to increasing performance of Datastore APR of Datastore ~1 => no performance penalty APR of SDB ~1.4 => 40% larger performance penalty than SDB ASCI a24 50
51 Other issues in cloud performance How to efficiently use elasticity? How to use virtual machine migration? How to insulate VMs of different users from each other? How to meet SLAs (e.g., use predictions) How to benefit from data locality (same unit/rack/data center)? ASCI a24 51
52 The Archives: Grid Workloads Motivation: little is known about real grid use No grid workloads available (except my grid ) No standard way to share them The Grid Workloads Archive facilitates sharing grid workload traces and associated research: understand how real grids are used address challenges facing grid resource management develop and test solutions (simulations, experiments) A. Iosup, H. Li, M. Jan, S. Anoep, C. Dumitrescu, L. Wolters, D.H.J. Epema, The Grid Workloads Archive, Future Generation Computer Systems, Vol. 24, , ASCI a24 52
Scheduling and Resource Management in Grids and Clouds
Scheduling and Resource Management in Grids and Clouds Dick Epema Parallel and Distributed Systems Group Delft University of Technology Delft, the Netherlands may 10, 2011 1 Outline Resource Management
More informationTowards an Optimized Big Data Processing System
Towards an Optimized Big Data Processing System The Doctoral Symposium of the IEEE/ACM CCGrid 2013 Delft, The Netherlands Bogdan Ghiţ, Alexandru Iosup, and Dick Epema Parallel and Distributed Systems Group
More informationKeywords: Cloudsim, MIPS, Gridlet, Virtual machine, Data center, Simulation, SaaS, PaaS, IaaS, VM. Introduction
Vol. 3 Issue 1, January-2014, pp: (1-5), Impact Factor: 1.252, Available online at: www.erpublications.com Performance evaluation of cloud application with constant data center configuration and variable
More informationHeterogeneous Workload Consolidation for Efficient Management of Data Centers in Cloud Computing
Heterogeneous Workload Consolidation for Efficient Management of Data Centers in Cloud Computing Deep Mann ME (Software Engineering) Computer Science and Engineering Department Thapar University Patiala-147004
More informationThe Lattice Project: A Multi-Model Grid Computing System. Center for Bioinformatics and Computational Biology University of Maryland
The Lattice Project: A Multi-Model Grid Computing System Center for Bioinformatics and Computational Biology University of Maryland Parallel Computing PARALLEL COMPUTING a form of computation in which
More informationCluster, Grid, Cloud Concepts
Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of
More informationDelft University of Technology Parallel and Distributed Systems Report Series. On the Performance Variability of Production Cloud Services
Delft University of Technology Parallel and Distributed Systems Report Series On the Performance Variability of Production Cloud Services Alexandru Iosup, Nezih Yigitbasi, and Dick Epema A.Iosup@tudelft.nl,
More informationService Oriented Distributed Manager for Grid System
Service Oriented Distributed Manager for Grid System Entisar S. Alkayal Faculty of Computing and Information Technology King Abdul Aziz University Jeddah, Saudi Arabia entisar_alkayal@hotmail.com Abstract
More informationGrid Scheduling Dictionary of Terms and Keywords
Grid Scheduling Dictionary Working Group M. Roehrig, Sandia National Laboratories W. Ziegler, Fraunhofer-Institute for Algorithms and Scientific Computing Document: Category: Informational June 2002 Status
More informationNetwork Infrastructure Services CS848 Project
Quality of Service Guarantees for Cloud Services CS848 Project presentation by Alexey Karyakin David R. Cheriton School of Computer Science University of Waterloo March 2010 Outline 1. Performance of cloud
More informationCondor for the Grid. 3) http://www.cs.wisc.edu/condor/
Condor for the Grid 1) Condor and the Grid. Douglas Thain, Todd Tannenbaum, and Miron Livny. In Grid Computing: Making The Global Infrastructure a Reality, Fran Berman, Anthony J.G. Hey, Geoffrey Fox,
More informationPERFORMANCE ANALYSIS OF PaaS CLOUD COMPUTING SYSTEM
PERFORMANCE ANALYSIS OF PaaS CLOUD COMPUTING SYSTEM Akmal Basha 1 Krishna Sagar 2 1 PG Student,Department of Computer Science and Engineering, Madanapalle Institute of Technology & Science, India. 2 Associate
More informationC-Meter: A Framework for Performance Analysis of Computing Clouds
C-Meter: A Framework for Performance Analysis of Computing Clouds Nezih Yigitbasi, Alexandru Iosup, and Dick Epema {M.N.Yigitbasi, D.H.J.Epema, A.Iosup}@tudelft.nl Delft University of Technology Simon
More informationManjrasoft Market Oriented Cloud Computing Platform
Manjrasoft Market Oriented Cloud Computing Platform Innovative Solutions for 3D Rendering Aneka is a market oriented Cloud development and management platform with rapid application development and workload
More informationIntroduction to grid technologies, parallel and cloud computing. Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber
Introduction to grid technologies, parallel and cloud computing Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber OUTLINES Grid Computing Parallel programming technologies (MPI- Open MP-Cuda )
More informationC-Meter: A Framework for Performance Analysis of Computing Clouds
9th IEEE/ACM International Symposium on Cluster Computing and the Grid C-Meter: A Framework for Performance Analysis of Computing Clouds Nezih Yigitbasi, Alexandru Iosup, and Dick Epema Delft University
More informationThis is an author-deposited version published in : http://oatao.univ-toulouse.fr/ Eprints ID : 12902
Open Archive TOULOUSE Archive Ouverte (OATAO) OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible. This is an author-deposited
More informationIntroduction to Cloud Computing
Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services
More informationAn Evaluation of Economy-based Resource Trading and Scheduling on Computational Power Grids for Parameter Sweep Applications
An Evaluation of Economy-based Resource Trading and Scheduling on Computational Power Grids for Parameter Sweep Applications Rajkumar Buyya, Jonathan Giddy, and David Abramson School of Computer Science
More informationCONDOR And The GRID. By Karthik Ram Venkataramani Department of Computer Science University at Buffalo kv8@cse.buffalo.edu
CONDOR And The GRID By Karthik Ram Venkataramani Department of Computer Science University at Buffalo kv8@cse.buffalo.edu Abstract Origination of the Condor Project Condor as Grid Middleware Condor working
More informationOn the Performance Variability of Production Cloud Services
On the Performance Variability of Production Cloud Services Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology, Mekelweg 4, 2628 CD Delft A.Iosup@tudelft.nl Nezih Yigitbasi
More informationCloud computing. Intelligent Services for Energy-Efficient Design and Life Cycle Simulation. as used by the ISES project
Intelligent Services for Energy-Efficient Design and Life Cycle Simulation Project number: 288819 Call identifier: FP7-ICT-2011-7 Project coordinator: Technische Universität Dresden, Germany Website: ises.eu-project.info
More informationCloudAnalyst: A CloudSim-based Visual Modeller for Analysing Cloud Computing Environments and Applications
CloudAnalyst: A CloudSim-based Visual Modeller for Analysing Cloud Computing Environments and Applications Bhathiya Wickremasinghe 1, Rodrigo N. Calheiros 2, and Rajkumar Buyya 1 1 The Cloud Computing
More informationComputing in clouds: Where we come from, Where we are, What we can, Where we go
Computing in clouds: Where we come from, Where we are, What we can, Where we go Luc Bougé ENS Cachan/Rennes, IRISA, INRIA Biogenouest With help from many colleagues: Gabriel Antoniu, Guillaume Pierre,
More informationMultilevel Communication Aware Approach for Load Balancing
Multilevel Communication Aware Approach for Load Balancing 1 Dipti Patel, 2 Ashil Patel Department of Information Technology, L.D. College of Engineering, Gujarat Technological University, Ahmedabad 1
More informationGaruda: a Cloud-based Job Scheduler
Garuda: a Cloud-based Job Scheduler Ashish Patro, MinJae Hwang, Thanumalayan S., Thawan Kooburat We present the design and implementation details of Garuda, a cloud based job scheduler using Google App
More informationDistribution transparency. Degree of transparency. Openness of distributed systems
Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science steen@cs.vu.nl Chapter 01: Version: August 27, 2012 1 / 28 Distributed System: Definition A distributed
More informationBig Data and Cloud Computing for GHRSST
Big Data and Cloud Computing for GHRSST Jean-Francois Piollé (jfpiolle@ifremer.fr) Frédéric Paul, Olivier Archer CERSAT / Institut Français de Recherche pour l Exploitation de la Mer Facing data deluge
More informationKeywords Cloud computing, virtual machines, migration approach, deployment modeling
Volume 3, Issue 8, August 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Effective Scheduling
More informationInternational Journal of Advance Research in Computer Science and Management Studies
Volume 3, Issue 6, June 2015 ISSN: 2321 7782 (Online) International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online
More informationManjrasoft Market Oriented Cloud Computing Platform
Manjrasoft Market Oriented Cloud Computing Platform Aneka Aneka is a market oriented Cloud development and management platform with rapid application development and workload distribution capabilities.
More information- Behind The Cloud -
- Behind The Cloud - Infrastructure and Technologies used for Cloud Computing Alexander Huemer, 0025380 Johann Taferl, 0320039 Florian Landolt, 0420673 Seminar aus Informatik, University of Salzburg Overview
More informationDISTRIBUTED SYSTEMS AND CLOUD COMPUTING. A Comparative Study
DISTRIBUTED SYSTEMS AND CLOUD COMPUTING A Comparative Study Geographically distributed resources, such as storage devices, data sources, and computing power, are interconnected as a single, unified resource
More informationAn approach to grid scheduling by using Condor-G Matchmaking mechanism
An approach to grid scheduling by using Condor-G Matchmaking mechanism E. Imamagic, B. Radic, D. Dobrenic University Computing Centre, University of Zagreb, Croatia {emir.imamagic, branimir.radic, dobrisa.dobrenic}@srce.hr
More informationVirtual Machine Based Resource Allocation For Cloud Computing Environment
Virtual Machine Based Resource Allocation For Cloud Computing Environment D.Udaya Sree M.Tech (CSE) Department Of CSE SVCET,Chittoor. Andra Pradesh, India Dr.J.Janet Head of Department Department of CSE
More informationECE6130 Grid and Cloud Computing
ECE6130 Grid and Cloud Computing Howie Huang Department of Electrical and Computer Engineering School of Engineering and Applied Science Cloud Computing Hardware Software Outline Research Challenges 2
More informationEucalyptus: An Open-source Infrastructure for Cloud Computing. Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com
Eucalyptus: An Open-source Infrastructure for Cloud Computing Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com Exciting Weather Forecasts Commercial Cloud Formation Eucalyptus - Confidential What
More informationLSKA 2010 Survey Report Job Scheduler
LSKA 2010 Survey Report Job Scheduler Graduate Institute of Communication Engineering {r98942067, r98942112}@ntu.edu.tw March 31, 2010 1. Motivation Recently, the computing becomes much more complex. However,
More informationCondor and the Grid Authors: D. Thain, T. Tannenbaum, and M. Livny. Condor Provide. Why Condor? Condor Kernel. The Philosophy of Flexibility
Condor and the Grid Authors: D. Thain, T. Tannenbaum, and M. Livny Presenter: Ibrahim H Suslu What is Condor? Specialized job and resource management system (RMS) for compute intensive jobs 1. User submit
More informationIBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud
IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud February 25, 2014 1 Agenda v Mapping clients needs to cloud technologies v Addressing your pain
More informationIaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures
IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Introduction
More informationGrid Computing Vs. Cloud Computing
International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 6 (2013), pp. 577-582 International Research Publications House http://www. irphouse.com /ijict.htm Grid
More informationCloud Application Development (SE808, School of Software, Sun Yat-Sen University) Yabo (Arber) Xu
Lecture 4 Introduction to Hadoop & GAE Cloud Application Development (SE808, School of Software, Sun Yat-Sen University) Yabo (Arber) Xu Outline Introduction to Hadoop The Hadoop ecosystem Related projects
More informationOpen Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)
Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University
More informationJSSSP, IPDPS WS, Boston, MA, USA May 24, 2013 TUD-PDS
A Periodic Portfolio Scheduler for Scientific Computing in the Data Center Kefeng Deng, Ruben Verboon, Kaijun Ren, and Alexandru Iosup Parallel and Distributed Systems Group JSSSP, IPDPS WS, Boston, MA,
More informationGrid Computing Approach for Dynamic Load Balancing
International Journal of Computer Sciences and Engineering Open Access Review Paper Volume-4, Issue-1 E-ISSN: 2347-2693 Grid Computing Approach for Dynamic Load Balancing Kapil B. Morey 1*, Sachin B. Jadhav
More informationVolunteer Computing and Cloud Computing: Opportunities for Synergy
Volunteer Computing and Cloud Computing: Opportunities for Synergy Derrick Kondo INRIA, France Performance vs. Reliability vs. Costs high Cost Reliability high low low low Performance high Performance
More informationHigh Throughput Computing on P2P Networks. Carlos Pérez Miguel carlos.perezm@ehu.es
High Throughput Computing on P2P Networks Carlos Pérez Miguel carlos.perezm@ehu.es Overview High Throughput Computing Motivation All things distributed: Peer-to-peer Non structured overlays Structured
More informationSystem Models for Distributed and Cloud Computing
System Models for Distributed and Cloud Computing Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Classification of Distributed Computing Systems
More informationReal Time Network Server Monitoring using Smartphone with Dynamic Load Balancing
www.ijcsi.org 227 Real Time Network Server Monitoring using Smartphone with Dynamic Load Balancing Dhuha Basheer Abdullah 1, Zeena Abdulgafar Thanoon 2, 1 Computer Science Department, Mosul University,
More informationCloud Computing Simulation Using CloudSim
Cloud Computing Simulation Using CloudSim Ranjan Kumar #1, G.Sahoo *2 # Assistant Professor, Computer Science & Engineering, Ranchi University, India Professor & Head, Information Technology, Birla Institute
More informationMonitoring Elastic Cloud Services
Monitoring Elastic Cloud Services trihinas@cs.ucy.ac.cy Advanced School on Service Oriented Computing (SummerSoc 2014) 30 June 5 July, Hersonissos, Crete, Greece Presentation Outline Elasticity in Cloud
More informationTask Scheduling in Hadoop
Task Scheduling in Hadoop Sagar Mamdapure Munira Ginwala Neha Papat SAE,Kondhwa SAE,Kondhwa SAE,Kondhwa Abstract Hadoop is widely used for storing large datasets and processing them efficiently under distributed
More informationAmazon EC2 Product Details Page 1 of 5
Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Functionality Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of
More informationCloud Computing. Chapter 1 Introducing Cloud Computing
Cloud Computing Chapter 1 Introducing Cloud Computing Learning Objectives Understand the abstract nature of cloud computing. Describe evolutionary factors of computing that led to the cloud. Describe virtualization
More informationVirtual machine interface. Operating system. Physical machine interface
Software Concepts User applications Operating system Hardware Virtual machine interface Physical machine interface Operating system: Interface between users and hardware Implements a virtual machine that
More informationInfrastructure as a Service (IaaS)
Infrastructure as a Service (IaaS) (ENCS 691K Chapter 4) Roch Glitho, PhD Associate Professor and Canada Research Chair My URL - http://users.encs.concordia.ca/~glitho/ References 1. R. Moreno et al.,
More informationWhite Paper. Cloud Native Advantage: Multi-Tenant, Shared Container PaaS. http://wso2.com Version 1.1 (June 19, 2012)
Cloud Native Advantage: Multi-Tenant, Shared Container PaaS Version 1.1 (June 19, 2012) Table of Contents PaaS Container Partitioning Strategies... 03 Container Tenancy... 04 Multi-tenant Shared Container...
More informationFigure 1. The cloud scales: Amazon EC2 growth [2].
- Chung-Cheng Li and Kuochen Wang Department of Computer Science National Chiao Tung University Hsinchu, Taiwan 300 shinji10343@hotmail.com, kwang@cs.nctu.edu.tw Abstract One of the most important issues
More informationCondor-G: A Computation Management Agent for Multi-Institutional Grids
: A Computation Management Agent for Multi-Institutional Grids James Frey, Todd Tannenbaum, Miron Livny Ian Foster, Steven Tuecke Department of Computer Science Mathematics and Computer Science Division
More informationSynthetic Grid Workloads with Ibis, KOALA, and GrenchMark
Synthetic Grid Workloads with Ibis, KOALA, and GrenchMark Alexandru Iosup 1, Jason Maassen 2, Rob van Nieuwpoort 2, and Dick H.J. Epema 1 1 Faculty of Electrical Engineering, Mathematics, and Computer
More informationDiPerF: automated DIstributed PERformance testing Framework
DiPerF: automated DIstributed PERformance testing Framework Ioan Raicu, Catalin Dumitrescu, Matei Ripeanu Distributed Systems Laboratory Computer Science Department University of Chicago Ian Foster Mathematics
More informationRound Robin with Server Affinity: A VM Load Balancing Algorithm for Cloud Based Infrastructure
J Inf Process Syst, Vol.9, No.3, September 2013 pissn 1976-913X eissn 2092-805X http://dx.doi.org/10.3745/jips.2013.9.3.379 Round Robin with Server Affinity: A VM Load Balancing Algorithm for Cloud Based
More informationDatacenters and Cloud Computing. Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html
Datacenters and Cloud Computing Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html What is Cloud Computing? A model for enabling ubiquitous, convenient, ondemand network
More informationPayment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load
Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load Pooja.B. Jewargi Prof. Jyoti.Patil Department of computer science and engineering,
More informationCloud Design and Implementation. Cheng Li MPI-SWS Nov 9 th, 2010
Cloud Design and Implementation Cheng Li MPI-SWS Nov 9 th, 2010 1 Modern Computing CPU, Mem, Disk Academic computation Chemistry, Biology Large Data Set Analysis Online service Shopping Website Collaborative
More informationwu.cloud: Insights Gained from Operating a Private Cloud System
wu.cloud: Insights Gained from Operating a Private Cloud System Stefan Theußl, Institute for Statistics and Mathematics WU Wirtschaftsuniversität Wien March 23, 2011 1 / 14 Introduction In statistics we
More informationCUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com
` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and
More informationEFFICIENT SCHEDULING STRATEGY USING COMMUNICATION AWARE SCHEDULING FOR PARALLEL JOBS IN CLUSTERS
EFFICIENT SCHEDULING STRATEGY USING COMMUNICATION AWARE SCHEDULING FOR PARALLEL JOBS IN CLUSTERS A.Neela madheswari 1 and R.S.D.Wahida Banu 2 1 Department of Information Technology, KMEA Engineering College,
More informationA Service for Data-Intensive Computations on Virtual Clusters
A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent
More informationDeploying Business Virtual Appliances on Open Source Cloud Computing
International Journal of Computer Science and Telecommunications [Volume 3, Issue 4, April 2012] 26 ISSN 2047-3338 Deploying Business Virtual Appliances on Open Source Cloud Computing Tran Van Lang 1 and
More informationInternational Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 36 ISSN 2229-5518
International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 36 An Efficient Approach for Load Balancing in Cloud Environment Balasundaram Ananthakrishnan Abstract Cloud computing
More informationLoad Balancing in the Cloud Computing Using Virtual Machine Migration: A Review
Load Balancing in the Cloud Computing Using Virtual Machine Migration: A Review 1 Rukman Palta, 2 Rubal Jeet 1,2 Indo Global College Of Engineering, Abhipur, Punjab Technical University, jalandhar,india
More informationPart V Applications. What is cloud computing? SaaS has been around for awhile. Cloud Computing: General concepts
Part V Applications Cloud Computing: General concepts Copyright K.Goseva 2010 CS 736 Software Performance Engineering Slide 1 What is cloud computing? SaaS: Software as a Service Cloud: Datacenters hardware
More informationAxceleon s CloudFuzion Turbocharges 3D Rendering On Amazon s EC2
Axceleon s CloudFuzion Turbocharges 3D Rendering On Amazon s EC2 In the movie making, visual effects and 3D animation industrues meeting project and timing deadlines is critical to success. Poor quality
More informationNew resource provision paradigms for Grid Infrastructures: Virtualization and Cloud
CISCO NerdLunch Series November 7, 2008 San Jose, CA New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed Systems Architecture Research
More informationWORKFLOW ENGINE FOR CLOUDS
WORKFLOW ENGINE FOR CLOUDS By SURAJ PANDEY, DILEBAN KARUNAMOORTHY, and RAJKUMAR BUYYA Prepared by: Dr. Faramarz Safi Islamic Azad University, Najafabad Branch, Esfahan, Iran. Workflow Engine for clouds
More informationVolunteer Computing, Grid Computing and Cloud Computing: Opportunities for Synergy. Derrick Kondo INRIA, France
Volunteer Computing, Grid Computing and Cloud Computing: Opportunities for Synergy Derrick Kondo INRIA, France Outline Cloud Grid Volunteer Computing Cloud Background Vision Hide complexity of hardware
More informationABSTRACT. KEYWORDS: Cloud Computing, Load Balancing, Scheduling Algorithms, FCFS, Group-Based Scheduling Algorithm
A REVIEW OF THE LOAD BALANCING TECHNIQUES AT CLOUD SERVER Kiran Bala, Sahil Vashist, Rajwinder Singh, Gagandeep Singh Department of Computer Science & Engineering, Chandigarh Engineering College, Landran(Pb),
More informationCloudFTP: A free Storage Cloud
CloudFTP: A free Storage Cloud ABSTRACT: The cloud computing is growing rapidly for it offers on-demand computing power and capacity. The power of cloud enables dynamic scalability of applications facing
More informationWhat Is It? Business Architecture Research Challenges Bibliography. Cloud Computing. Research Challenges Overview. Carlos Eduardo Moreira dos Santos
Research Challenges Overview May 3, 2010 Table of Contents I 1 What Is It? Related Technologies Grid Computing Virtualization Utility Computing Autonomic Computing Is It New? Definition 2 Business Business
More informationCost-Benefit Analysis of Cloud Computing versus Desktop Grids
Cost-Benefit Analysis of Cloud Computing versus Desktop Grids Derrick Kondo, Bahman Javadi, Paul Malécot, Franck Cappello INRIA, France David P. Anderson UC Berkeley, USA Cloud Background Vision Hide complexity
More informationDeveloping Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control
Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control EP/K006487/1 UK PI: Prof Gareth Taylor (BU) China PI: Prof Yong-Hua Song (THU) Consortium UK Members: Brunel University
More informationPerformance Analysis of Cloud-Based Applications
Performance Analysis of Cloud-Based Applications Peter Budai and Balazs Goldschmidt Budapest University of Technology and Economics, Department of Control Engineering and Informatics, Budapest, Hungary
More informationENERGY EFFICIENT VIRTUAL MACHINE ASSIGNMENT BASED ON ENERGY CONSUMPTION AND RESOURCE UTILIZATION IN CLOUD NETWORK
International Journal of Computer Engineering & Technology (IJCET) Volume 7, Issue 1, Jan-Feb 2016, pp. 45-53, Article ID: IJCET_07_01_006 Available online at http://www.iaeme.com/ijcet/issues.asp?jtype=ijcet&vtype=7&itype=1
More informationEffective Virtual Machine Scheduling in Cloud Computing
Effective Virtual Machine Scheduling in Cloud Computing Subhash. B. Malewar 1 and Prof-Deepak Kapgate 2 1,2 Department of C.S.E., GHRAET, Nagpur University, Nagpur, India Subhash.info24@gmail.com and deepakkapgate32@gmail.com
More informationSURFsara HPC Cloud Workshop
SURFsara HPC Cloud Workshop doc.hpccloud.surfsara.nl UvA workshop 2016-01-25 UvA HPC Course Jan 2016 Anatoli Danezi, Markus van Dijk cloud-support@surfsara.nl Agenda Introduction and Overview (current
More informationDeployment Strategies for Distributed Applications on Cloud Computing Infrastructures
Deployment Strategies for Distributed Applications on Cloud Computing Infrastructures Jan Sipke van der Veen 1,2, Elena Lazovik 1, Marc X. Makkes 1,2, Robert J. Meijer 1,2 1 TNO, Groningen, The Netherlands
More informationTowards Autonomic Grid Data Management with Virtualized Distributed File Systems
Towards Autonomic Grid Data Management with Virtualized Distributed File Systems Ming Zhao, Jing Xu, Renato Figueiredo Advanced Computing and Information Systems Electrical and Computer Engineering University
More informationA REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM
A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information
More informationComputing. M< MORGAN KAUfMANM. Distributed and Cloud. Processing to the. From Parallel. Internet of Things. Geoffrey C. Fox. Dongarra.
Distributed and Cloud From Parallel Computing Processing to the Internet of Things Kai Hwang Geoffrey C. Fox Jack J. Dongarra AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO
More informationLOGO Resource Management for Cloud Computing
LOGO Resource Management for Cloud Computing Supervisor : Dr. Pham Tran Vu Presenters : Nguyen Viet Hung - 11070451 Tran Le Vinh - 11070487 Date : April 16, 2012 Contents Introduction to Cloud Computing
More informationAdvanced Load Balancing Mechanism on Mixed Batch and Transactional Workloads
Advanced Load Balancing Mechanism on Mixed Batch and Transactional Workloads G. Suganthi (Member, IEEE), K. N. Vimal Shankar, Department of Computer Science and Engineering, V.S.B. Engineering College,
More informationGrid Scheduling Architectures with Globus GridWay and Sun Grid Engine
Grid Scheduling Architectures with and Sun Grid Engine Sun Grid Engine Workshop 2007 Regensburg, Germany September 11, 2007 Ignacio Martin Llorente Javier Fontán Muiños Distributed Systems Architecture
More informationEnterprise Desktop Grids
Enterprise Desktop Grids Evgeny Ivashko Institute of Applied Mathematical Research, Karelian Research Centre of Russian Academy of Sciences, Petrozavodsk, Russia, ivashko@krc.karelia.ru WWW home page:
More informationCloud Computing. Lecture 5 Grid Case Studies 2014-2015
Cloud Computing Lecture 5 Grid Case Studies 2014-2015 Up until now Introduction. Definition of Cloud Computing. Grid Computing: Schedulers Globus Toolkit Summary Grid Case Studies: Monitoring: TeraGRID
More informationSLA Driven Load Balancing For Web Applications in Cloud Computing Environment
SLA Driven Load Balancing For Web Applications in Cloud Computing Environment More Amar amarmore2006@gmail.com Kulkarni Anurag anurag.kulkarni@yahoo.com Kolhe Rakesh rakeshkolhe139@gmail.com Kothari Rupesh
More informationArticle on Grid Computing Architecture and Benefits
Article on Grid Computing Architecture and Benefits Ms. K. Devika Rani Dhivya 1, Mrs. C. Sunitha 2 1 Assistant Professor, Dept. of BCA & MSc.SS, Sri Krishna Arts and Science College, CBE, Tamil Nadu, India.
More informationLecture 02a Cloud Computing I
Mobile Cloud Computing Lecture 02a Cloud Computing I 吳 秀 陽 Shiow-yang Wu What is Cloud Computing? Computing with cloud? Mobile Cloud Computing Cloud Computing I 2 Note 1 What is Cloud Computing? Walking
More informationGroup Based Load Balancing Algorithm in Cloud Computing Virtualization
Group Based Load Balancing Algorithm in Cloud Computing Virtualization Rishi Bhardwaj, 2 Sangeeta Mittal, Student, 2 Assistant Professor, Department of Computer Science, Jaypee Institute of Information
More information