The Virtual Grid Application Development Software (VGrADS) Project

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "The Virtual Grid Application Development Software (VGrADS) Project"

Transcription

1 The Virtual Grid Application Development Software (VGrADS) Project VGrADS: Enabling e-science Workflows on Grids and Clouds with Fault Tolerance

2 VGrADS Goal: Distributed Problem Solving Where We Want To Be o Transparent computing In an increasingly distributed space Applications to HPC Applications to cloud computing Where We Were (circa 2003) o Low-level hand programming o Programmer had to manage: Heterogeneous resources Scheduling of computation and data movement Fault tolerance and performance adaptation What Progress Have We Made? o Separate application development from resource management o VGrADS provides a uniform virtual grid abstraction atop widely differing resources Provide tools to bridge the gap Scheduling, resource management, distributed launch, simple programming models, fault tolerance, grid economies

3 Overview of SCʼxx Activities for VGrADS Built on previous SC demonstrations o Gradually built up system to handle LEAD workflow o Previous years focused improved performance estimates, scheduling methods, fault tolerance o Use LEAD as an application driver Current status o VGrADS integrates HPC and cloud resources Using TeraGrid (HPC), Amazon EC2 (cloud), Eucalyptus (cloud) resources Using reservations, batch queues, and on-demand clouds o Scheduling for balancing deadlines, reliability, and cost vges supports search for best set of resources Application-specific trade-offs of reliability, time, cost o Abstractions really do work!

4 VGrADS Components o Virtual Grid Execution System (vges) Uses Amazon EC2 tools to interact with cloud resources Uses QBETS and Globus to provision batch resources Uses Personal PBS to control execution on batch resources Provides a resource gantt chart view of resources to aid higher level workflow orchestration tool o Eucalyptus Developed for VGrADS (now Eucalyptus Inc.) Implements cloud computing on Xen-enabled clusters Open-source software infrastructure that is compatible with Amazon EC2 vges thinks a Eucalyptus cloud is EC2 o Fault Tolerance (FTR) Schedules a task to increase the probability of successful execution of a task up to a desired level, constrained by resource availability and application deadlines

5 Executing LEAD Workflow Sets o Demonstrate planning and execution of LEAD workflow sets execution atop virtualized cloud and Grid resources. o LEAD Workflow Orchestration schedules a set of independent workflows with characteristics a deadline D (e.g. 2 hours) fraction F such that at least F of the workflows finish by the deadline (e.g. 3/8) o Virtual Grid Execution System (vges) provides an abstraction over batch and cloud systems including Amazon EC2 and Eucalyptus cloud sites.

6 Workflow Orchestration o Workflow Orchestration o Phase 1: Minimal Scheduling Schedule minimum fraction of workflows using simple probabilistic DAG scheduler o Phase 2: Fault Tolerance Tradeoff Compare scheduling additional workflows with increasing fault-tolerance of one or more tasks of the scheduled workflows o Phase 3: Additional Scheduling Use available slots for other scheduling o Phase 4: EC2 Scheduling - For tasks below certain threshold, schedule on EC2 o Execution Manager A prototype for ordered execution of tasks on the slot based on the schedule determined by the orchestration.

7 Comparison of the LEAD-VGrADS collaboration system with cyberinfrastructure production deployments Without VGrADS User With VGrADS User Portal Portal Workflow Engine Application Service Execution Manager resource planning Workflow Planner Virtual Grid Execution System query execution plan Workflow Engine Application Service Execution Manager Globus HPC Resources specific protocol based execution EC2 interfaces Cloud resource binding Globus HPC Resources EC2 interfaces Cloud standard execution

8 Example Scheduling of Workflows (a) Example Workflow Set need F=2/3 B A D C F G E H I J K L (b) Uncoordinated Schedule Without VGrADS Batch Queue Cloud J K L A B C D E F G H I Start Deadline D (c) Coordinated Schedule With VGrADS Batch Queue Cloud J K L A B C D E F G H I E F G H I Start Deadline D

9 Interaction of system components for resource procurement and planning Workflow Planner resource find & bind schedule DAG Scheduler Fault tolerance EC2 Phase 1,2,3 Phase 2,3 Phase 4 Virtual Grid Execu3on System query batch queue times BQP virtual res. VARQ submit job setup virtual machines GT4, Personal PBS Globus EC2 interfaces HPC Resources Cloud

10 LEAD / VGrADS Architecture : Putting It All Together Portal LEAD System LEAD BPEL Workflow Engine Event Broker Workflow Planner schedule Application Service App. Factory resource find & bind DAG Scheduler Fault tolerance failures Execution Manager failures Virtual Grid Execution System BQP / VAR VGrADS System Batch Systems Globus HPC and Cloud Resources HPC Cloud GT4, Personal PBS

11 Visualization Key Resource Name Workflow Color Slot Width Done Failed Run Time Task Status

12 Snapshot of Execution of 6 Workflows

13 6 Workflows on 7 Clusters

14 Infrastructure Timing Metrics Binding time Planning time line Workflow execution time line

15 Fault Tolerance Exploration Low Reliability High Reliability Medium Reliability

16 Conclusions VGrADS virtual grid abstraction simplifies o Programming grid and cloud systems for e-science workflows o Managing QoS (performance and reliability) The VGrADS system unifies workflow execution over o batch queue systems (with and without advanced reservations), and o cloud computing sites (including Amazon EC2 and Eucalyptus) The system provides an enabling technology for executing deadline-driven, fault-tolerant workflows The integrated cyber-infrastructure from the LEAD and VGrADS system components provides a strong foundation for next-generation dynamic and adaptive environments for scientific workflows

17 Thank You VGrADS: Enabling e-science Workflows on Grids and Clouds with Fault Tolerance, L. Ramakrishnan, D. Nurmi, A. Mandal, C. Koelbel, D. Gannon, T. M. Huang, Y. S. Kee, G. Obertelli, K. Thyagaraja, R. Wolski, A. Yarkhan and D. Zagorodnov Paper to be presented at 3:30 p.m., Thursday, Nov. 19 in Room E Thank you..

Towards a New Model for the Infrastructure Grid

Towards a New Model for the Infrastructure Grid INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS Cetraro (Italy), June 30 - July 4, 2008 Panel: From Grids to Cloud Services Towards a New Model for the Infrastructure Grid

More information

The Eucalyptus Open-source Cloud Computing System

The Eucalyptus Open-source Cloud Computing System The Eucalyptus Open-source Cloud Computing System Chris Grzegorczyk, Dan Nurmi, Graziano Obertelli, Rich Wolski, Sunil Soman, Lamia Youseff, Dmitrii Zagorodnov University of California, Santa Barbara Cloud

More information

Infrastructure for Cloud Computing

Infrastructure for Cloud Computing Eucalyptus: an Open-Source Infrastructure for Cloud Computing Rich Wolski Chris Grzegorczyk, Dan Nurmi, Graziano Obertelli, Woody Rollins, Sunil Soman, Lamia Youseff, Dmitrii Zagorodnov The Eucalyptus

More information

Data Services @neurist and beyond

Data Services @neurist and beyond s @neurist and beyond Siegfried Benkner Department of Scientific Computing Faculty of Computer Science University of Vienna http://www.par.univie.ac.at Department of Scientific Computing Parallel Computing

More information

An Enhanced Workflow Scheduling Strategy for Deadline Guarantee on Hybrid Grid/Cloud Infrastructure

An Enhanced Workflow Scheduling Strategy for Deadline Guarantee on Hybrid Grid/Cloud Infrastructure Journal of Applied Science and Engineering, Vol. 18, No. 1, pp. 67 78 (2015) DOI: 10.6180/jase.2015.18.1.09 An Enhanced Workflow Scheduling Strategy for Deadline Guarantee on Hybrid Grid/Cloud Infrastructure

More information

INDIGO DataCloud. Technical Overview RIA-653549. Giacinto.Donvito@ba.infn.it. INFN-Bari

INDIGO DataCloud. Technical Overview RIA-653549. Giacinto.Donvito@ba.infn.it. INFN-Bari INDIGO DataCloud Technical Overview RIA-653549 Giacinto.Donvito@ba.infn.it INFN-Bari Agenda Gap analysis Goals Architecture WPs activities Conclusions 2 Gap Analysis Support federated identities and provide

More information

3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India

3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India 3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India Call for Papers Cloud computing has emerged as a de facto computing

More information

Enabling Execution of Service Workflows in Grid/Cloud Hybrid Systems

Enabling Execution of Service Workflows in Grid/Cloud Hybrid Systems Enabling Execution of Service Workflows in Grid/Cloud Hybrid Systems Luiz F. Bittencourt, Carlos R. Senna, and Edmundo R. M. Madeira Institute of Computing University of Campinas - UNICAMP P.O. Box 6196,

More information

Research IT Plan. UCD IT Services. Seirbhísí TF UCD

Research IT Plan. UCD IT Services. Seirbhísí TF UCD Research IT Plan UCD IT Services Research IT Plan The main goal of this plan is to provide a sustainable and evolving campus Cyberinfrastructure for the UCD research community. We will continue the development

More information

FREE AND OPEN SOURCE SOFTWARE FOR CLOUD COMPUTING SERENA SPINOSO (serena.spinoso@polito.it) FULVIO VALENZA (fulvio.valenza@polito.

FREE AND OPEN SOURCE SOFTWARE FOR CLOUD COMPUTING SERENA SPINOSO (serena.spinoso@polito.it) FULVIO VALENZA (fulvio.valenza@polito. + FREE AND OPEN SOURCE SOFTWARE FOR CLOUD COMPUTING SERENA SPINOSO (serena.spinoso@polito.it) FULVIO VALENZA (fulvio.valenza@polito.it) + OUTLINE INTRODUCTION OF CLOUD DEFINITION OF CLOUD BASIC CLOUD COMPONENTS

More information

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud CISCO NerdLunch Series November 7, 2008 San Jose, CA New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed Systems Architecture Research

More information

1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India

1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India 1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India Call for Papers Colossal Data Analysis and Networking has emerged as a de facto

More information

Building Platform as a Service for Scientific Applications

Building Platform as a Service for Scientific Applications Building Platform as a Service for Scientific Applications Moustafa AbdelBaky moustafa@cac.rutgers.edu Rutgers Discovery Informa=cs Ins=tute (RDI 2 ) The NSF Cloud and Autonomic Compu=ng Center Department

More information

Cisco Cloud Portal Delivers Self-Service Provisioning for Data Center Services

Cisco Cloud Portal Delivers Self-Service Provisioning for Data Center Services Product Bulletin Cisco Cloud Portal Delivers Self-Service Provisioning for Data Center Services The Cisco Cloud Portal provides a self-service portal, service catalog, and lifecycle management solution

More information

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing New Paradigms: Clouds, Virtualization and Co. EGEE08, Istanbul, September 25, 2008 An Introduction to Virtualization and Cloud Technologies to Support Grid Computing Distributed Systems Architecture Research

More information

A Batch Computing Service for the Spot Market. Supreeth Subramanya, Tian Guo, Prateek Sharma, David Irwin, Prashant Shenoy

A Batch Computing Service for the Spot Market. Supreeth Subramanya, Tian Guo, Prateek Sharma, David Irwin, Prashant Shenoy SpotOn: A Batch Computing Service for the Spot Market Supreeth Subramanya, Tian Guo, Prateek Sharma, David Irwin, Prashant Shenoy University of Massachusetts Amherst infrastructure cloud Cost vs. Availability

More information

Solution for private cloud computing

Solution for private cloud computing The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details System requirements and installation How to get it? 2 What is CC1? The CC1 system is a complete solution

More information

Brian Amedro CTO. Worldwide Customers

Brian Amedro CTO. Worldwide Customers Denis Caromel CEO Brian Amedro CTO Cloud Enterprise Applications (B2B) Reduce Costs (IT) + Reduce Pains (Time) Worldwide Customers 1 1 Software company born of INRIA in 2007 Software Editor, Open Source

More information

The GridWay Meta-Scheduler

The GridWay Meta-Scheduler The GridWay Meta-Scheduler Committers Ignacio M. Llorente Ruben S. Montero Eduardo Huedo Contributors Tino Vazquez Jose Luis Vazquez Javier Fontan Jose Herrera 1 Goals of the Project Goals of the Project

More information

CLEVER: a CLoud-Enabled Virtual EnviRonment

CLEVER: a CLoud-Enabled Virtual EnviRonment CLEVER: a CLoud-Enabled Virtual EnviRonment Francesco Tusa Maurizio Paone Massimo Villari Antonio Puliafito {ftusa,mpaone,mvillari,apuliafito}@unime.it Università degli Studi di Messina, Dipartimento di

More information

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING Carlos de Alfonso Andrés García Vicente Hernández 2 INDEX Introduction Our approach Platform design Storage Security

More information

Cloud-pilot.doc 12-12-2010 SA1 Marcus Hardt, Marcin Plociennik, Ahmad Hammad, Bartek Palak E U F O R I A

Cloud-pilot.doc 12-12-2010 SA1 Marcus Hardt, Marcin Plociennik, Ahmad Hammad, Bartek Palak E U F O R I A Identifier: Date: Activity: Authors: Status: Link: Cloud-pilot.doc 12-12-2010 SA1 Marcus Hardt, Marcin Plociennik, Ahmad Hammad, Bartek Palak E U F O R I A J O I N T A C T I O N ( S A 1, J R A 3 ) F I

More information

Adapting Market-Oriented Scheduling Policies for Cloud Computing

Adapting Market-Oriented Scheduling Policies for Cloud Computing Adapting Market-Oriented Scheduling Policies for Cloud Computing Mohsen Amini Salehi Rajkumar Buyya Cloud Computing and Distributed Systems (CLOUDS) Laboratory, Department of Computer Science and Software

More information

Welcome. Grid on Demand. Willem Toorop and Alain van Hoof. {wtoorop,ahoof}@os3.nl. June 30, 2010

Welcome. Grid on Demand. Willem Toorop and Alain van Hoof. {wtoorop,ahoof}@os3.nl. June 30, 2010 Welcome Grid on Demand Willem Toorop and Alain van Hoof {wtoorop,ahoof}@os3.nl June 30, 2010 Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, 2010 1 / 39 Research Question Introduction Research

More information

A Model-Based Proxy for Unified IaaS Management

A Model-Based Proxy for Unified IaaS Management A Model-Based Proxy for Unified IaaS Management Shixing Yan HP Labs Singapore 1 Fusionopolis Way Singapore 138632 Email: shixing.yan@hp.com Bu Sung Lee HP Labs Singapore, Singapore 138632 & School of Computer

More information

Magellan A Test Bed to Explore Cloud Computing for Science Shane Canon and Lavanya Ramakrishnan Cray XE6 Training February 8, 2011

Magellan A Test Bed to Explore Cloud Computing for Science Shane Canon and Lavanya Ramakrishnan Cray XE6 Training February 8, 2011 Magellan A Test Bed to Explore Cloud Computing for Science Shane Canon and Lavanya Ramakrishnan Cray XE6 Training February 8, 2011 Magellan Exploring Cloud Computing Co-located at two DOE-SC Facilities

More information

SUSE Cloud 2.0. Pete Chadwick. Douglas Jarvis. Senior Product Manager pchadwick@suse.com. Product Marketing Manager djarvis@suse.

SUSE Cloud 2.0. Pete Chadwick. Douglas Jarvis. Senior Product Manager pchadwick@suse.com. Product Marketing Manager djarvis@suse. SUSE Cloud 2.0 Pete Chadwick Douglas Jarvis Senior Product Manager pchadwick@suse.com Product Marketing Manager djarvis@suse.com SUSE Cloud SUSE Cloud is an open source software solution based on OpenStack

More information

Comparison of Several Cloud Computing Platforms

Comparison of Several Cloud Computing Platforms Second International Symposium on Information Science and Engineering Comparison of Several Cloud Computing Platforms Junjie Peng School of computer science & High performance computing center Shanghai

More information

Comparing Open Source Private Cloud (IaaS) Platforms

Comparing Open Source Private Cloud (IaaS) Platforms Comparing Open Source Private Cloud (IaaS) Platforms Lance Albertson OSU Open Source Lab Associate Director of Operations lance@osuosl.org / @ramereth About me OSU Open Source Lab Server hosting for Open

More information

Cloud and Virtualization to Support Grid Infrastructures

Cloud and Virtualization to Support Grid Infrastructures ESAC GRID Workshop '08 ESAC, Villafranca del Castillo, Spain 11-12 December 2008 Cloud and Virtualization to Support Grid Infrastructures Distributed Systems Architecture Research Group Universidad Complutense

More information

Administrative Issues

Administrative Issues Administrative Issues Make use of office hours We will have to make sure that you have tried yourself before you ask Monitor AWS expenses regularly Always do the cost calculation before launching services

More information

Workprogramme 2014-15

Workprogramme 2014-15 Workprogramme 2014-15 e-infrastructures DCH-RP final conference 22 September 2014 Wim Jansen einfrastructure DG CONNECT European Commission DEVELOPMENT AND DEPLOYMENT OF E-INFRASTRUCTURES AND SERVICES

More information

salsadpi: a dynamic provisioning interface for IaaS cloud

salsadpi: a dynamic provisioning interface for IaaS cloud salsadpi: a dynamic provisioning interface for IaaS cloud Tak-Lon (Stephen) Wu Computer Science, School of Informatics and Computing Indiana University, Bloomington, IN taklwu@indiana.edu Abstract On-demand

More information

Data Management in the Cloud. Zhen Shi

Data Management in the Cloud. Zhen Shi Data Management in the Cloud Zhen Shi Overview Introduction 3 characteristics of cloud computing 2 types of cloud data management application 2 types of cloud data management architecture Conclusion Introduction

More information

INRIA Job Scheduler GE webinar WP4 Cloud Hosting Chapter

INRIA Job Scheduler GE webinar WP4 Cloud Hosting Chapter INRIA Job Scheduler GE webinar WP4 Cloud Hosting Chapter Lorenzo CANTELMI, INRIA lorenzo.cantelmi@inria.fr Main requirements across Computing Systems eras Batch Systems Personal Computers Grids & Clusters

More information

Grid Scheduling Architectures with Globus GridWay and Sun Grid Engine

Grid Scheduling Architectures with Globus GridWay and Sun Grid Engine Grid Scheduling Architectures with and Sun Grid Engine Sun Grid Engine Workshop 2007 Regensburg, Germany September 11, 2007 Ignacio Martin Llorente Javier Fontán Muiños Distributed Systems Architecture

More information

Solution for private cloud computing

Solution for private cloud computing The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2 What

More information

An Experimental Study of Load Balancing of OpenNebula Open-Source Cloud Computing Platform

An Experimental Study of Load Balancing of OpenNebula Open-Source Cloud Computing Platform An Experimental Study of Load Balancing of OpenNebula Open-Source Cloud Computing Platform A B M Moniruzzaman 1, Kawser Wazed Nafi 2, Prof. Syed Akhter Hossain 1 and Prof. M. M. A. Hashem 1 Department

More information

Dynamic Resource Distribution Across Clouds

Dynamic Resource Distribution Across Clouds University of Victoria Faculty of Engineering Winter 2010 Work Term Report Dynamic Resource Distribution Across Clouds Department of Physics University of Victoria Victoria, BC Michael Paterson V00214440

More information

Technology Enablement

Technology Enablement SOLUTION OVERVIEW 1 ABOUT TECHMILEAGE Founded in 2008 / Tempe, Arizona Over 100 engagements Full range of business & technology services Software Development, Big Data, Cloud/AWS, BI, Advanced Analytics

More information

A Service for Data-Intensive Computations on Virtual Clusters

A Service for Data-Intensive Computations on Virtual Clusters A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent

More information

Daniel J. Adabi. Workshop presentation by Lukas Probst

Daniel J. Adabi. Workshop presentation by Lukas Probst Daniel J. Adabi Workshop presentation by Lukas Probst 3 characteristics of a cloud computing environment: 1. Compute power is elastic, but only if workload is parallelizable 2. Data is stored at an untrusted

More information

Elastic Management of Cluster based Services in the Cloud

Elastic Management of Cluster based Services in the Cloud First Workshop on Automated Control for Datacenters and Clouds (ACDC09) June 19th, Barcelona, Spain Elastic Management of Cluster based Services in the Cloud Rafael Moreno Vozmediano, Ruben S. Montero,

More information

VMM-Level Distributed Transparency Provisioning Using Cloud Infrastructure Technology. Mahsa Najafzadeh, Hadi Salimi, Mohsen Sharifi

VMM-Level Distributed Transparency Provisioning Using Cloud Infrastructure Technology. Mahsa Najafzadeh, Hadi Salimi, Mohsen Sharifi VMM-Level Distributed Transparency Provisioning Using Cloud Infrastructure Technology Mahsa Najafzadeh, Hadi Salimi, Mohsen Sharifi and Ali Hamidi Agenda Introduction to Virtualization Technology (VT)

More information

Cluster, Grid, Cloud Concepts

Cluster, Grid, Cloud Concepts Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of

More information

Logical Data Models for Cloud Computing Architectures

Logical Data Models for Cloud Computing Architectures Logical Data Models for Cloud Computing Architectures Augustine (Gus) Samba, Kent State University Describing generic logical data models for two existing cloud computing architectures, the author helps

More information

BPM Architecture Design Based on Cloud Computing

BPM Architecture Design Based on Cloud Computing Intelligent Information Management, 2010, 2, 329-333 doi:10.4236/iim.2010.25039 Published Online May 2010 (http://www.scirp.org/journal/iim) BPM Architecture Design Based on Cloud Computing Abstract Zhenyu

More information

HP CLOUD STRATEGY AND SOLUTIONS THE PATH TO HYBRID DELIVERY. Copyright 2011 Hewlett-Packard Development Company, L.P.

HP CLOUD STRATEGY AND SOLUTIONS THE PATH TO HYBRID DELIVERY. Copyright 2011 Hewlett-Packard Development Company, L.P. HP CLOUD STRATEGY AND SOLUTIONS THE PATH TO HYBRID DELIVERY 1 INSTANT-ON ENTERPRISE Everything and everyone s connected Everyone expects immediate gratification and instant results Business & IT one and

More information

Comparing Ganeti to other Private Cloud Platforms. Lance Albertson Director lance@osuosl.org @ramereth

Comparing Ganeti to other Private Cloud Platforms. Lance Albertson Director lance@osuosl.org @ramereth Comparing Ganeti to other Private Cloud Platforms Lance Albertson Director lance@osuosl.org @ramereth About me OSU Open Source Lab Server hosting for Open Source Projects Open Source development projects

More information

C-Meter: A Framework for Performance Analysis of Computing Clouds

C-Meter: A Framework for Performance Analysis of Computing Clouds C-Meter: A Framework for Performance Analysis of Computing Clouds Nezih Yigitbasi, Alexandru Iosup, and Dick Epema {M.N.Yigitbasi, D.H.J.Epema, A.Iosup}@tudelft.nl Delft University of Technology Simon

More information

Extending Hadoop beyond MapReduce

Extending Hadoop beyond MapReduce Extending Hadoop beyond MapReduce Mahadev Konar Co-Founder @mahadevkonar (@hortonworks) Page 1 Bio Apache Hadoop since 2006 - committer and PMC member Developed and supported Map Reduce @Yahoo! - Core

More information

Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases

Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases NASA Ames NASA Advanced Supercomputing (NAS) Division California, May 24th, 2012 Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases Ignacio M. Llorente Project Director OpenNebula Project.

More information

Data as a Service Virtualization with Enzo Unified

Data as a Service Virtualization with Enzo Unified Data as a Service Virtualization with Enzo Unified White Paper by Blue Syntax Abstract: This white paper explains how companies can benefit from a Data as a Service virtualization layer and build a data

More information

Virtual InfiniBand Clusters for HPC Clouds

Virtual InfiniBand Clusters for HPC Clouds Virtual InfiniBand Clusters for HPC Clouds April 10, 2012 Marius Hillenbrand, Viktor Mauch, Jan Stoess, Konrad Miller, Frank Bellosa SYSTEM ARCHITECTURE GROUP, 1 10.04.2012 Marius Hillenbrand - Virtual

More information

Integrating Clouds and Cyberinfrastructure: Research Challenges

Integrating Clouds and Cyberinfrastructure: Research Challenges Integrating Clouds and Cyberinfrastructure: Research Challenges Manish Parashar, Rutgers University, parashar@rutgers.edu Geoffrey Fox, Indiana University, gcf@indiana.edu Kate Keahey, Argonne National

More information

Resource Scalability for Efficient Parallel Processing in Cloud

Resource Scalability for Efficient Parallel Processing in Cloud Resource Scalability for Efficient Parallel Processing in Cloud ABSTRACT Govinda.K #1, Abirami.M #2, Divya Mercy Silva.J #3 #1 SCSE, VIT University #2 SITE, VIT University #3 SITE, VIT University In the

More information

CiteSeer x in the Cloud

CiteSeer x in the Cloud Published in the 2nd USENIX Workshop on Hot Topics in Cloud Computing 2010 CiteSeer x in the Cloud Pradeep B. Teregowda Pennsylvania State University C. Lee Giles Pennsylvania State University Bhuvan Urgaonkar

More information

WORKFLOW ENGINE FOR CLOUDS

WORKFLOW ENGINE FOR CLOUDS WORKFLOW ENGINE FOR CLOUDS By SURAJ PANDEY, DILEBAN KARUNAMOORTHY, and RAJKUMAR BUYYA Prepared by: Dr. Faramarz Safi Islamic Azad University, Najafabad Branch, Esfahan, Iran. Workflow Engine for clouds

More information

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms Volume 1, Issue 1 ISSN: 2320-5288 International Journal of Engineering Technology & Management Research Journal homepage: www.ijetmr.org Analysis and Research of Cloud Computing System to Comparison of

More information

Cloud Computing. Walfredo Cirne

Cloud Computing. Walfredo Cirne Cloud Computing Walfredo Cirne Agenda What is the cloud? o Motivation o Cloud provisioning o SLO variation o IaaS, PaaS, SaaS Using the cloud o Dependency and locking o Pets x Cattle Trust & security What

More information

Cloud Computing. Cloud computing:

Cloud Computing. Cloud computing: Cloud computing: Cloud Computing A model of data processing in which high scalability IT solutions are delivered to multiple users: as a service, on a mass scale, on the Internet. Network services offering:

More information

Cloud Computing for HPC

Cloud Computing for HPC Cloud Computing for HPC Extending Clusters to Clouds Solution Briefing TORONTO 10/25/2011 Company Background 2 Platform Computing, Inc. Platform Clusters, Grids, Clouds Computing The leader in cluster,

More information

Scalable Services for Digital Preservation

Scalable Services for Digital Preservation Scalable Services for Digital Preservation A Perspective on Cloud Computing Rainer Schmidt, Christian Sadilek, and Ross King Digital Preservation (DP) Providing long-term access to growing collections

More information

Cloud Sobriety. Technical challenges in mapping Informatics to the cloud. Chris Dagdigian 2010 NHGRI Cloud Workshop

Cloud Sobriety. Technical challenges in mapping Informatics to the cloud. Chris Dagdigian 2010 NHGRI Cloud Workshop Cloud Sobriety Technical challenges in mapping Informatics to the cloud Chris Dagdigian 2010 NHGRI Cloud Workshop Welcome To Day 2! Excellent talk lineup for today! Focus on implementation, architecture

More information

Availability Digest. www.availabilitydigest.com. @availabilitydig. HPE Helion Private Cloud and Cloud Broker Services February 2016

Availability Digest. www.availabilitydigest.com. @availabilitydig. HPE Helion Private Cloud and Cloud Broker Services February 2016 the Availability Digest @availabilitydig HPE Helion Private Cloud and Cloud Broker Services February 2016 HPE Helion is a complete portfolio of cloud products and services that offers enterprise security,

More information

A Very Brief Introduction To Cloud Computing. Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman

A Very Brief Introduction To Cloud Computing. Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman A Very Brief Introduction To Cloud Computing Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman What is The Cloud Cloud computing refers to logical computational resources accessible via a computer

More information

Windows Azure platform What is in it for you? Dominick Baier (dbaier@develop.com) Christian Weyer (cweyer@develop.com

Windows Azure platform What is in it for you? Dominick Baier (dbaier@develop.com) Christian Weyer (cweyer@develop.com Windows Azure platform What is in it for you? Dominick Baier (dbaier@develop.com) Christian Weyer (cweyer@develop.com Objectives Motivation Status quo Cloud Computing Windows Azure platform Windows Azure

More information

An Efficient Checkpointing Scheme Using Price History of Spot Instances in Cloud Computing Environment

An Efficient Checkpointing Scheme Using Price History of Spot Instances in Cloud Computing Environment An Efficient Checkpointing Scheme Using Price History of Spot Instances in Cloud Computing Environment Daeyong Jung 1, SungHo Chin 1, KwangSik Chung 2, HeonChang Yu 1, JoonMin Gil 3 * 1 Dept. of Computer

More information

AWS Account Setup and Services Overview

AWS Account Setup and Services Overview AWS Account Setup and Services Overview 1. Purpose of the Lab Understand definitions of various Amazon Web Services (AWS) and their use in cloud computing based web applications that are accessible over

More information

Improving MapReduce Performance in Heterogeneous Environments

Improving MapReduce Performance in Heterogeneous Environments UC Berkeley Improving MapReduce Performance in Heterogeneous Environments Matei Zaharia, Andy Konwinski, Anthony Joseph, Randy Katz, Ion Stoica University of California at Berkeley Motivation 1. MapReduce

More information

High Throughput Computing, Grid Computing, Cloud Computing, Etc. Definitions & Thoughts. Some Definitions

High Throughput Computing, Grid Computing, Cloud Computing, Etc. Definitions & Thoughts. Some Definitions High Throughput Computing, Grid Computing, Cloud Computing, Etc. Definitions & Thoughts Jay Boisseau Texas Advanced Computing Center July 16, 2008 But before the definitions, some disclaimers: These my

More information

LSKA 2010 Survey Report I Device Drivers & Cloud Computing

LSKA 2010 Survey Report I Device Drivers & Cloud Computing LSKA 2010 Survey Report I Device Drivers & Cloud Computing Yu Huang and Hao-Chung Yang {r98922015, r98944016}@csie.ntu.edu.tw Department of Computer Science and Information Engineering March 31, 2010 Abstract

More information

PRACE WP4 Distributed Systems Management. Riccardo Murri, CSCS Swiss National Supercomputing Centre

PRACE WP4 Distributed Systems Management. Riccardo Murri, CSCS Swiss National Supercomputing Centre PRACE WP4 Distributed Systems Management Riccardo Murri, CSCS Swiss National Supercomputing Centre PRACE WP4 WP4 is the Distributed Systems Management activity User administration and accounting Distributed

More information

Last time. Today. IaaS Providers. Amazon Web Services, overview

Last time. Today. IaaS Providers. Amazon Web Services, overview Last time General overview, motivation, expected outcomes, other formalities, etc. Please register for course Online (if possible), or talk to Yvonne@CS Course evaluation forgotten Please assign one volunteer

More information

HPC Cloud Computing with OpenNebula

HPC Cloud Computing with OpenNebula High Performance Cloud Computing Day BiG Grid - SARA Amsterdam, The Netherland, October 4th, 2011 HPC Cloud Computing with OpenNebula Ignacio M. Llorente Project Director Acknowledgments The research leading

More information

Cloud Computing with Red Hat Solutions. Sivaram Shunmugam Red Hat Asia Pacific Pte Ltd. sivaram@redhat.com

Cloud Computing with Red Hat Solutions. Sivaram Shunmugam Red Hat Asia Pacific Pte Ltd. sivaram@redhat.com Cloud Computing with Red Hat Solutions Sivaram Shunmugam Red Hat Asia Pacific Pte Ltd sivaram@redhat.com Linux Automation Details Red Hat's Linux Automation strategy for next-generation IT infrastructure

More information

Open Source Cloud Computing Management with OpenNebula

Open Source Cloud Computing Management with OpenNebula CloudCamp Campus Party July 2011, Valencia Open Source Cloud Computing Management with OpenNebula Javier Fontán Muiños dsa-research.org Distributed Systems Architecture Research Group Universidad Complutense

More information

The Agile Infrastructure Project. Monitoring. Markus Schulz Pedro Andrade. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.

The Agile Infrastructure Project. Monitoring. Markus Schulz Pedro Andrade. CERN IT Department CH-1211 Genève 23 Switzerland www.cern. The Agile Infrastructure Project Monitoring Markus Schulz Pedro Andrade CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Outline Monitoring WG and AI Today s Monitoring in IT Architecture

More information

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Cloud Computing 159.735 Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Table of Contents Introduction... 3 What is Cloud Computing?... 3 Key Characteristics...

More information

Challenges in Hybrid and Federated Cloud Computing

Challenges in Hybrid and Federated Cloud Computing Cloud Day 2011 KTH-SICS Cloud Innovation Center and EIT ICT Labs Kista, Sweden, September 14th, 2011 Challenges in Hybrid and Federated Cloud Computing Ignacio M. Llorente Project Director Acknowledgments

More information

15 th April 2010 FIA Valencia

15 th April 2010 FIA Valencia Autonomic Internet (AutoI) FP7 STREP Project Management of Virtual Infrastructure http://ist-autoi.eu/ 15 th April 2010 FIA Valencia Alex Galis University College London a.galis@ee.ucl.ac.uk www.ee.ucl.ac.uk/~agalis

More information

Infrastructure as a Service (IaaS)

Infrastructure as a Service (IaaS) Infrastructure as a Service (IaaS) (ENCS 691K Chapter 4) Roch Glitho, PhD Associate Professor and Canada Research Chair My URL - http://users.encs.concordia.ca/~glitho/ References 1. R. Moreno et al.,

More information

StratusLab project. Standards, Interoperability and Asset Exploitation. Vangelis Floros, GRNET

StratusLab project. Standards, Interoperability and Asset Exploitation. Vangelis Floros, GRNET StratusLab project Standards, Interoperability and Asset Exploitation Vangelis Floros, GRNET EGI Technical Forum 2011 19-22 September 2011, Lyon, France StratusLab is co-funded by the European Community

More information

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed

More information

CLOUD COMPUTING. When It's smarter to rent than to buy

CLOUD COMPUTING. When It's smarter to rent than to buy CLOUD COMPUTING When It's smarter to rent than to buy Is it new concept? Nothing new In 1990 s, WWW itself Grid Technologies- Scientific applications Online banking websites More convenience Not to visit

More information

Copyright 11/1/2010 BMC Software, Inc 1

Copyright 11/1/2010 BMC Software, Inc 1 Copyright 11/1/2010 BMC Software, Inc 1 Copyright 11/1/2010 BMC Software, Inc 2 Copyright 11/1/2010 BMC Software, Inc 3 The current state of IT Service How we work today! INCIDENT SERVICE LEVEL DATA SERVICE

More information

PRIVACY PRESERVATION ALGORITHM USING EFFECTIVE DATA LOOKUP ORGANIZATION FOR STORAGE CLOUDS

PRIVACY PRESERVATION ALGORITHM USING EFFECTIVE DATA LOOKUP ORGANIZATION FOR STORAGE CLOUDS PRIVACY PRESERVATION ALGORITHM USING EFFECTIVE DATA LOOKUP ORGANIZATION FOR STORAGE CLOUDS Amar More 1 and Sarang Joshi 2 1 Department of Computer Engineering, Pune Institute of Computer Technology, Maharashtra,

More information

How to Do/Evaluate Cloud Computing Research. Young Choon Lee

How to Do/Evaluate Cloud Computing Research. Young Choon Lee How to Do/Evaluate Cloud Computing Research Young Choon Lee Cloud Computing Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing

More information

Data Sheet Netrounds Control Center

Data Sheet Netrounds Control Center Data Sheet Netrounds Control Center The core component of Netrounds is a unifying cloud-based Control Center, which provides a consolidated GUI for operations staff as well as a cloud API for external

More information

Grid Computing Vs. Cloud Computing

Grid Computing Vs. Cloud Computing International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 6 (2013), pp. 577-582 International Research Publications House http://www. irphouse.com /ijict.htm Grid

More information

Deutsche Börse Cloud Exchange AG First vendor-neutral platform for IaaS cloud computing products. October 2013

Deutsche Börse Cloud Exchange AG First vendor-neutral platform for IaaS cloud computing products. October 2013 Deutsche Börse Cloud Exchange AG First vendor-neutral platform for IaaS cloud computing products October 2013 Deutsche Börse Cloud Exchange AG 1 Deutsche Börse Cloud Exchange AG What if Deutsche Börse

More information

Fast Innovation requires Fast IT

Fast Innovation requires Fast IT Fast Innovation requires Fast IT 2014 Cisco and/or its affiliates. All rights reserved. 2 2014 Cisco and/or its affiliates. All rights reserved. 3 IoT World Forum Architecture Committee 2013 Cisco and/or

More information

IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud

IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud February 25, 2014 1 Agenda v Mapping clients needs to cloud technologies v Addressing your pain

More information

Manjrasoft Market Oriented Cloud Computing Platform

Manjrasoft Market Oriented Cloud Computing Platform Manjrasoft Market Oriented Cloud Computing Platform Aneka Aneka is a market oriented Cloud development and management platform with rapid application development and workload distribution capabilities.

More information

- from Clusters and Grids to Cloud computing

- from Clusters and Grids to Cloud computing Lecture - 1 Moving Ahead - from Clusters and Grids to Cloud computing Salman Toor salman.toor@it.uu.se Basic questions Why Cloud computing? What are the previous technologies? What was missing in the previous

More information

Lavanya Ramakrishnan Phone: (919)-949-2615 Email: lavanya@gmail.com

Lavanya Ramakrishnan Phone: (919)-949-2615 Email: lavanya@gmail.com Lavanya Ramakrishnan Phone: (919)-949-2615 Email: lavanya@gmail.com Education Ph.D. Computer Science, Indiana University, Bloomington, 2009 M.S. Computer Science, Indiana University, Bloomington, 2002

More information

TECHNOLOGY WHITE PAPER Jun 2012

TECHNOLOGY WHITE PAPER Jun 2012 TECHNOLOGY WHITE PAPER Jun 2012 Technology Stack C# Windows Server 2008 PHP Amazon Web Services (AWS) Route 53 Elastic Load Balancing (ELB) Elastic Compute Cloud (EC2) Amazon RDS Amazon S3 Elasticache

More information

MONITORING RED HAT GLUSTER SERVER DEPLOYMENTS With the Nagios IT infrastructure monitoring tool

MONITORING RED HAT GLUSTER SERVER DEPLOYMENTS With the Nagios IT infrastructure monitoring tool TECHNOLOGY DETAIL MONITORING RED HAT GLUSTER SERVER DEPLOYMENTS With the Nagios IT infrastructure monitoring tool INTRODUCTION Storage system monitoring is a fundamental task for a storage administrator.

More information

A SURVEY ON WORKFLOW SCHEDULING IN CLOUD USING ANT COLONY OPTIMIZATION

A SURVEY ON WORKFLOW SCHEDULING IN CLOUD USING ANT COLONY OPTIMIZATION Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 2, February 2014,

More information

C-Meter: A Framework for Performance Analysis of Computing Clouds

C-Meter: A Framework for Performance Analysis of Computing Clouds 9th IEEE/ACM International Symposium on Cluster Computing and the Grid C-Meter: A Framework for Performance Analysis of Computing Clouds Nezih Yigitbasi, Alexandru Iosup, and Dick Epema Delft University

More information