Welcome. Grid on Demand. Willem Toorop and Alain van Hoof. June 30, 2010

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Welcome. Grid on Demand. Willem Toorop and Alain van Hoof. {wtoorop,ahoof}@os3.nl. June 30, 2010"

Transcription

1 Welcome Grid on Demand Willem Toorop and Alain van Hoof June 30, 2010 Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

2 Research Question Introduction Research Question Can Grid computing be offered as a Cloud service and what type of applications benefit from this? Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

3 Introduction The Grid and The Cloud The Grid and The Cloud: Sharing Resources Sharing Resources to minimizing idle resources Resource Capacity Goods Processor Flops Compute Hard-disk memory Bytes Storage Network Bytes/sec Communication Bandwidth Grid Cloud (IaaS) Distribution By Collaboration By Cloud Provider Resource Type Heterogeneous Homogeneous Abstraction Middleware (open standards) Virtualization Sharing Guarantee Security infrastructure Financial infra Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

4 Introduction The Grid and The Cloud The Grid and The Cloud: Dividing the Resources Grid User VO User Cloud Grid Grid User User User VO User User Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

5 A Grid in the Cloud Urgent Computing A Grid in The Cloud: Urgent Computing GRID on Demand Using the elasticity and scalability of the Cloud and providing a Grid interface, current e-science (computationally intensive science) applications can use a Grid on Demand when there is a high demand for resources for a short period of time. Example UrbanFlood (http://urbanflood.eu) wants to calculate flooding predictions when actual flooding happens, Grid on Demand can provide this. Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

6 Grid on Demand A Grid in the Cloud Grid on Demand Job Submission Client OS: Ubuntu Grid: Globus PBS: Torque Compute Nodes Head Node Grid Interface: Globus Toolkit 4.2 (open standard, widely used) Queue Manager (Distribution of jobs to nodes): Torque Amazon EC2 Cloud Provider (de-facto standard with open API) Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

7 Implementation What makes the Grid, the Grid What makes the Grid, the Grid Typical Grid usage example spanning multiple organizations Organization B Organization C Scheduler use 1 st 1 2 of data to compute 1 2 X use my data to compute X Compute Grid User access data store data Organization A Storage use 2 nd 1 2 of data to compute 1 2 X Compute access data Organization D Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

8 Implementation What makes the Grid, the Grid Proxy certificates for delegation The Grid is a collaboration between multiple organizations. Grid users need to be authorized to use remote resources Those resources should be authorized to perform actions on behalf of the Grid user Proxy certificates enable delegation of authority Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

9 Implementation What makes the Grid, the Grid Grid user create proxy Scheduler compute X sign this create cert. request Compute Okay compute X Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

10 Implementation What makes the Grid, the Grid Grid user create proxy Scheduler compute X sign this create cert. request Compute Okay compute X Virtual Organizations are based on delegation by proxy certificates. Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

11 Implementation What makes the Grid, the Grid Grid user create proxy Scheduler Proxy certificates allow for dynamic resource allocation, But the hosts still need a CA signed certificate. compute X sign this create cert. request Compute This is a problem when hostname s are assigned dynamically! Okay compute X Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

12 Implementation What makes the Grid, the Grid Grid user create proxy Scheduler Proxy certificates allow for dynamic resource allocation, But the hosts still need a CA signed certificate. compute X sign this Okay create cert. request compute X Compute This is a problem when hostname s are assigned dynamically! Fixed CA in image? Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

13 Implementation What makes the Grid, the Grid Grid user create proxy Scheduler Proxy certificates allow for dynamic resource allocation, But the hosts still need a CA signed certificate. compute X sign this Okay create cert. request compute X Compute This is a problem when hostname s are assigned dynamically! Fixed CA in image? Dynamic CA on every Launch? Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

14 Implementation Amazon Elastic Block Store Easy Image management with Amazon Elastic Block Store configure (A, S) launch instance I A V S S Pending Running (I A, V S, H) terminate stop Shutting-down Stopping Pending (A, S ) create AMI A S V S Stopped (I A, V S ) start terminate Terminated I A Life cycle of Amazon Machine Images using Elastic Block Storage A = Amazon Machine Image, S = Snapshot, I = Instance, V = Volume, H = Hostname Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

15 Implementation EBS for configuration stages EBS for configuration stages configure configure (A, S) launch instance I A V S S Running (I A, V S, H) provide Amazon Access Credentials (configure cluster) Running (I A, V S, H) shutdown for reuse start Stopped (I A, V S ) shutdown for sharing Stopped (I A, V S ) shutdown for sharing shutdown for reuse launch I A launch I A V S S V S S start Stopped (I A, V S ) create AMI A create AMI A create AMI A S V S S V S S V S (A, S ) (A, S ) (A, S ) Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

16 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

17 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

18 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

19 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

20 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

21 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

22 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

23 Web Interface Launching Grid on Demand Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

24 Web Interface Configuring the cluster Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

25 Web Interface Configuring the cluster Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

26 Web Interface Configuring the cluster Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

27 Web Interface Configuring the cluster Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

28 Web Interface Real Grid certificate usage Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

29 Web Interface Real Grid certificate usage Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

30 Web Interface Real Grid certificate usage Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

31 Web Interface Elasticity of the cluster Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

32 Web Interface Elasticity of the cluster Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

33 Web Interface Elasticity of the cluster Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

34 Web Interface Saving your settings Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

35 Web Interface Saving your settings Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

36 Web Interface Saving your settings Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

37 Real use and Performance Real use and Performance test WS-VLAM and WAVE To test the possibilities and performance of Grid on Demand a Biomedical Application and a workflow manager are used Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

38 Real use and Performance WS-VLAM and WAVE Real use and Performance test To test the possibilities and performance of Grid on Demand a Biomedical Application and a workflow manager are used Biomedical Application: WAVE Parallel (MonteCarlo simulation) application Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

39 Real use and Performance WS-VLAM and WAVE Real use and Performance test To test the possibilities and performance of Grid on Demand a Biomedical Application and a workflow manager are used Biomedical Application: WAVE Parallel (MonteCarlo simulation) application Workflow Manager A (graphical) tool to assist complex e-science application creation Creates a series of jobsubmissions WS-VLAM created by UvA SNE Group Connects to Globus Grid Interface Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

40 Real use and Performance Connecting the Grid on Demand to a Workflow Manager Connecting the Grid on Demand to a Workflow Manager WAVE Biomedical Compute Application OS: Ubuntu Grid: Globus PBS: Torque Compute Nodes Head Node Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

41 Real use and Performance Preliminary performance results: total execution time Preliminary performance results: total execution time Grid on demand compared to a 32 node physical UvA cluster (DAS3) with Globus Grid interface Grid Jobs Total execution time DAS :42:58 Grid on Demand 28 00:46:22 DAS :46:36 Grid on Demand 98 00:52:26 Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

42 Real use and Performance Preliminary performance results: 30 Jobs Preliminary performance results of job pending and running time: 30 jobs Performance on DAS3-30 Jobs Performance on Grid on Demand 30 Jobs 01:20:00 Running time Pending time 01:20:00 Running time Pending time 01:10:00 01:10:00 01:00:00 01:00:00 Time (HH:MM:SS) 00:50:00 00:40:00 00:30:00 Time (HH:MM:SS) 00:50:00 00:40:00 00:30:00 00:20:00 00:20:00 00:10:00 00:10:00 00:00: :00: Job Number Job Number Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

43 Real use and Performance Preliminary performance results: 100 Jobs Preliminary performance results of job pending and running time: 100 jobs Performance on DAS3-100 Jobs Performance on Grid on Demand Jobs 02:30:00 Running time Pending time 02:30:00 Running time Pending time 02:15:00 02:15:00 02:00:00 02:00:00 01:45:00 01:45:00 Time (HH:MM:SS) 01:30:00 01:15:00 01:00:00 Time (HH:MM:SS) 01:30:00 01:15:00 01:00:00 00:45:00 00:45:00 00:30:00 00:30:00 00:15:00 00:15:00 00:00: :00: Job Number Job Number Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

44 Conclusion Conclusion Grid on Demand can offer Grid as a Cloud service Parallel applications run/scale very well on Grid on Demand Grid on Demand can be a solution for Urgent Computing demands Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

45 Questions The End Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, / 39

Under supervision of Dr. Paola Grossa and Drs. Rudolf Strijkers

Under supervision of Dr. Paola Grossa and Drs. Rudolf Strijkers Grid on Demand Alain van Hoof and Willem Toorop Under supervision of Dr. Paola Grossa and Drs. Rudolf Strijkers alain.vanhoof@os3.nl willem.toorop@os3.nl p.grosso@uva.nl strijkers@uva.nl Abstract Currently,

More information

Cluster, Grid, Cloud Concepts

Cluster, Grid, Cloud Concepts Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of

More information

LSKA 2010 Survey Report Job Scheduler

LSKA 2010 Survey Report Job Scheduler LSKA 2010 Survey Report Job Scheduler Graduate Institute of Communication Engineering {r98942067, r98942112}@ntu.edu.tw March 31, 2010 1. Motivation Recently, the computing becomes much more complex. However,

More information

Amazon Web Services Primer. William Strickland COP 6938 Fall 2012 University of Central Florida

Amazon Web Services Primer. William Strickland COP 6938 Fall 2012 University of Central Florida Amazon Web Services Primer William Strickland COP 6938 Fall 2012 University of Central Florida AWS Overview Amazon Web Services (AWS) is a collection of varying remote computing provided by Amazon.com.

More information

Cloud Compu)ng. Adam Belloum Ins)tute of Informa)cs University of Amsterdam a.s.z.belloum@uva.nl

Cloud Compu)ng. Adam Belloum Ins)tute of Informa)cs University of Amsterdam a.s.z.belloum@uva.nl Cloud Compu)ng Adam Belloum Ins)tute of Informa)cs University of Amsterdam a.s.z.belloum@uva.nl High Performance compu)ng Curriculum, Jan 2015 hgp://www.hpc.uva.nl/ UvA- SURFsara What is Cloud Compu)ng?

More information

Cloud Computing and Amazon Web Services

Cloud Computing and Amazon Web Services Cloud Computing and Amazon Web Services Gary A. McGilvary edinburgh data.intensive research 1 OUTLINE 1. An Overview of Cloud Computing 2. Amazon Web Services 3. Amazon EC2 Tutorial 4. Conclusions 2 CLOUD

More information

19.10.11. Amazon Elastic Beanstalk

19.10.11. Amazon Elastic Beanstalk 19.10.11 Amazon Elastic Beanstalk A Short History of AWS Amazon started as an ECommerce startup Original architecture was restructured to be more scalable and easier to maintain Competitive pressure for

More information

Early Cloud Experiences with the Kepler Scientific Workflow System

Early Cloud Experiences with the Kepler Scientific Workflow System Available online at www.sciencedirect.com Procedia Computer Science 9 (2012 ) 1630 1634 International Conference on Computational Science, ICCS 2012 Early Cloud Experiences with the Kepler Scientific Workflow

More information

2) Xen Hypervisor 3) UEC

2) Xen Hypervisor 3) UEC 5. Implementation Implementation of the trust model requires first preparing a test bed. It is a cloud computing environment that is required as the first step towards the implementation. Various tools

More information

CHAPTER 8 CLOUD COMPUTING

CHAPTER 8 CLOUD COMPUTING CHAPTER 8 CLOUD COMPUTING SE 458 SERVICE ORIENTED ARCHITECTURE Assist. Prof. Dr. Volkan TUNALI Faculty of Engineering and Natural Sciences / Maltepe University Topics 2 Cloud Computing Essential Characteristics

More information

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4 Concepts and Architecture of the Grid Summary of Grid 2, Chapter 4 Concepts of Grid Mantra: Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations Allows

More information

Efficient Cloud Management for Parallel Data Processing In Private Cloud

Efficient Cloud Management for Parallel Data Processing In Private Cloud 2012 International Conference on Information and Network Technology (ICINT 2012) IPCSIT vol. 37 (2012) (2012) IACSIT Press, Singapore Efficient Cloud Management for Parallel Data Processing In Private

More information

OSDC PIRE Summer Internship, Sao Paulo, Brazil: Integration of Remote Clusters Under the Same Cloud

OSDC PIRE Summer Internship, Sao Paulo, Brazil: Integration of Remote Clusters Under the Same Cloud OSDC PIRE Summer Internship, Sao Paulo, Brazil: Integration of Remote Clusters Under the Same Cloud Felipe Navarro Florida International University Computer Engineering Master s Student Cloud Computing

More information

Grid Computing Vs. Cloud Computing

Grid Computing Vs. Cloud Computing International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 6 (2013), pp. 577-582 International Research Publications House http://www. irphouse.com /ijict.htm Grid

More information

Dynamic Resource Distribution Across Clouds

Dynamic Resource Distribution Across Clouds University of Victoria Faculty of Engineering Winter 2010 Work Term Report Dynamic Resource Distribution Across Clouds Department of Physics University of Victoria Victoria, BC Michael Paterson V00214440

More information

Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study

Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study Creating Value Delivering Solutions Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study Chris Zajac, NJDOT Bud Luo, Ph.D., Michael Baker Jr., Inc. Overview

More information

XtreemOS and Cloud Computing Alvaro Arenas E-Science Centre Science and Technologies Facilities Council, UK XtreemOS in a Nutshell An open source Linux-based Grid Operating System with native VO support

More information

Cloud Models and Platforms

Cloud Models and Platforms Cloud Models and Platforms Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF A Working Definition of Cloud Computing Cloud computing is a model

More information

The glite File Transfer Service

The glite File Transfer Service Enabling Grids Enabling for E-sciencE Grids for E-sciencE The glite File Transfer Service Paolo Badino On behalf of the JRA1 Data Management team EGEE User Forum - CERN, 2 Mars 2006 www.eu-egee.org Outline

More information

Multilevel Communication Aware Approach for Load Balancing

Multilevel Communication Aware Approach for Load Balancing Multilevel Communication Aware Approach for Load Balancing 1 Dipti Patel, 2 Ashil Patel Department of Information Technology, L.D. College of Engineering, Gujarat Technological University, Ahmedabad 1

More information

Remote sensing information cloud service: research and practice

Remote sensing information cloud service: research and practice Remote sensing information cloud service: research and practice Yang Banghui Dr., Ren Fuhu Prof. and Wang jinnian Prof. yangbh@radi.ac.cn +8613810963452 Content 1 Background 2 Studying and Designing 3

More information

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed

More information

CONDOR CLUSTERS ON EC2

CONDOR CLUSTERS ON EC2 CONDOR CLUSTERS ON EC2 Val Hendrix, Roberto A. Vitillo Lawrence Berkeley National Lab ATLAS Cloud Computing R & D 1 INTRODUCTION This is our initial work on investigating tools for managing clusters and

More information

Cloud Computing. Adam Barker

Cloud Computing. Adam Barker Cloud Computing Adam Barker 1 Overview Introduction to Cloud computing Enabling technologies Different types of cloud: IaaS, PaaS and SaaS Cloud terminology Interacting with a cloud: management consoles

More information

wu.cloud: Insights Gained from Operating a Private Cloud System

wu.cloud: Insights Gained from Operating a Private Cloud System wu.cloud: Insights Gained from Operating a Private Cloud System Stefan Theußl, Institute for Statistics and Mathematics WU Wirtschaftsuniversität Wien March 23, 2011 1 / 14 Introduction In statistics we

More information

Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT

Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT S. Gobinath, S. Saravanan PG Scholar, CSE Dept, M.Kumarasamy College of Engineering, Karur, India 1 Assistant Professor,

More information

A Testing Framework for Assessing Grid and Cloud Infrastructure Interoperability

A Testing Framework for Assessing Grid and Cloud Infrastructure Interoperability 95 A Testing Framework for Assessing Grid and Cloud Infrastructure Interoperability Thomas Rings, Jens Grabowski Institute of Computer Science, University of Göttingen Göttingen, Germany Email: {rings,grabowski}@cs.uni-goettingen.de

More information

Manjrasoft Market Oriented Cloud Computing Platform

Manjrasoft Market Oriented Cloud Computing Platform Manjrasoft Market Oriented Cloud Computing Platform Aneka Aneka is a market oriented Cloud development and management platform with rapid application development and workload distribution capabilities.

More information

THE EUCALYPTUS OPEN-SOURCE PRIVATE CLOUD

THE EUCALYPTUS OPEN-SOURCE PRIVATE CLOUD THE EUCALYPTUS OPEN-SOURCE PRIVATE CLOUD By Yohan Wadia ucalyptus is a Linux-based opensource software architecture that implements efficiencyenhancing private and hybrid clouds within an enterprise s

More information

Eucalyptus: An Open-source Infrastructure for Cloud Computing. Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com

Eucalyptus: An Open-source Infrastructure for Cloud Computing. Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com Eucalyptus: An Open-source Infrastructure for Cloud Computing Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com Exciting Weather Forecasts Commercial Cloud Formation Eucalyptus - Confidential What

More information

Towards a New Model for the Infrastructure Grid

Towards a New Model for the Infrastructure Grid INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS Cetraro (Italy), June 30 - July 4, 2008 Panel: From Grids to Cloud Services Towards a New Model for the Infrastructure Grid

More information

High Performance Applications over the Cloud: Gains and Losses

High Performance Applications over the Cloud: Gains and Losses High Performance Applications over the Cloud: Gains and Losses Dr. Leila Ismail Faculty of Information Technology United Arab Emirates University leila@uaeu.ac.ae http://citweb.uaeu.ac.ae/citweb/profile/leila

More information

Cloud computing - Architecting in the cloud

Cloud computing - Architecting in the cloud Cloud computing - Architecting in the cloud anna.ruokonen@tut.fi 1 Outline Cloud computing What is? Levels of cloud computing: IaaS, PaaS, SaaS Moving to the cloud? Architecting in the cloud Best practices

More information

VMUnify EC2 Gateway Guide

VMUnify EC2 Gateway Guide VMUnify EC2 Gateway Guide Version 2.8.1 Copyright Information This document is the exclusive property of Mindtree limited (Mindtree); the recipient agrees that they may not copy, transmit, use or disclose

More information

AN IMPLEMENTATION OF E- LEARNING SYSTEM IN PRIVATE CLOUD

AN IMPLEMENTATION OF E- LEARNING SYSTEM IN PRIVATE CLOUD AN IMPLEMENTATION OF E- LEARNING SYSTEM IN PRIVATE CLOUD M. Lawanya Shri 1, Dr. S. Subha 2 1 Assistant Professor,School of Information Technology and Engineering, Vellore Institute of Technology, Vellore-632014

More information

An Efficient Use of Virtualization in Grid/Cloud Environments. Supervised by: Elisa Heymann Miquel A. Senar

An Efficient Use of Virtualization in Grid/Cloud Environments. Supervised by: Elisa Heymann Miquel A. Senar An Efficient Use of Virtualization in Grid/Cloud Environments. Arindam Choudhury Supervised by: Elisa Heymann Miquel A. Senar Index Introduction Motivation Objective State of Art Proposed Solution Experimentations

More information

Using ArcGIS for Server in the Amazon Cloud

Using ArcGIS for Server in the Amazon Cloud Federal GIS Conference February 9 10, 2015 Washington, DC Using ArcGIS for Server in the Amazon Cloud Bonnie Stayer, Esri Amy Ramsdell, Blue Raster Session Outline AWS Overview ArcGIS in AWS Cloud Builder

More information

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2 DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing Slide 1 Slide 3 A style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet.

More information

HPC and Grid Concepts

HPC and Grid Concepts HPC and Grid Concepts Divya MG (divyam@cdac.in) CDAC Knowledge Park, Bangalore 16 th Feb 2012 GBC@PRL Ahmedabad 1 Presentation Overview What is HPC Need for HPC HPC Tools Grid Concepts GARUDA Overview

More information

Amazon Web Services. Elastic Compute Cloud (EC2) and more...

Amazon Web Services. Elastic Compute Cloud (EC2) and more... Amazon Web Services Elastic Compute Cloud (EC2) and more... I don t work for Amazon I do however, have a small research grant from Amazon (in AWS$) Portions of this presentation are reproduced from slides

More information

Cloud Computing. Alex Crawford Ben Johnstone

Cloud Computing. Alex Crawford Ben Johnstone Cloud Computing Alex Crawford Ben Johnstone Overview What is cloud computing? Amazon EC2 Performance Conclusions What is the Cloud? A large cluster of machines o Economies of scale [1] Customers use a

More information

Auto-Scaling Model for Cloud Computing System

Auto-Scaling Model for Cloud Computing System Auto-Scaling Model for Cloud Computing System Che-Lun Hung 1*, Yu-Chen Hu 2 and Kuan-Ching Li 3 1 Dept. of Computer Science & Communication Engineering, Providence University 2 Dept. of Computer Science

More information

VM Management for Green Data Centres with the OpenNebula Virtual Infrastructure Engine

VM Management for Green Data Centres with the OpenNebula Virtual Infrastructure Engine OGF-EU: Using IT to reduce Carbon Emissions and Delivering the Potential of Energy Efficient Computing OGF25, Catania, Italy 5 March 2009 VM Management for Green Data Centres with the OpenNebula Virtual

More information

Phylogenetic Code in the Cloud Can it Meet the Expectations?

Phylogenetic Code in the Cloud Can it Meet the Expectations? Phylogenetic Code in the Cloud Can it Meet the Expectations? Adam Kraut 1, Sébastien Moretti 2,3, Marc Robinson-Rechavi 2, Heinz Stockinger 3, and Dean Flanders 4 1) BioTeam Inc., Middleton, MA, USA 2)

More information

Secure Federated Light-weight Web Portals for FusionGrid

Secure Federated Light-weight Web Portals for FusionGrid Secure Federated Light-weight Web Portals for FusionGrid By: D. Aswath, M. Thompson, M. Goode, X. Lee, N. Y. Kim Presented by: Dipti Aswath GCE Workshop 2006 Second International Workshop on Grid Computing

More information

Introduction to Cloud computing. Viet Tran

Introduction to Cloud computing. Viet Tran Introduction to Cloud computing Viet Tran Type of Cloud computing Infrastructure as a Service IaaS: offer full virtual machines via hardware virtualization tech. Amazon EC2, AbiCloud, ElasticHosts, Platform

More information

Hybrid Development and Test USE CASE

Hybrid Development and Test USE CASE Hybrid Development and Test USE CASE CliQr Use Case: Hybrid Development and Test Page 2 Hybrid Development and Test Unlike the production phase, with its typically steady workload, development and test

More information

Use of Cloud Computing for scalable geospatial data processing and access

Use of Cloud Computing for scalable geospatial data processing and access Use of Cloud Computing for scalable geospatial data processing and access Andrew Turner CTO, FortiusOne andrew@fortiusone.com Partner: U.S. Federal Geographic Data Committee What is GeoCommons? A Brief

More information

The ENEA-EGEE site: Access to non-standard platforms

The ENEA-EGEE site: Access to non-standard platforms V INFNGrid Workshop Padova, Italy December 18-20 2006 The ENEA-EGEE site: Access to non-standard platforms C. Sciò**, G. Bracco, P. D'Angelo, L. Giammarino*, S.Migliori, A. Quintiliani, F. Simoni, S. Podda

More information

9/26/2011. What is Virtualization? What are the different types of virtualization.

9/26/2011. What is Virtualization? What are the different types of virtualization. CSE 501 Monday, September 26, 2011 Kevin Cleary kpcleary@buffalo.edu What is Virtualization? What are the different types of virtualization. Practical Uses Popular virtualization products Demo Question,

More information

Introduction to grid technologies, parallel and cloud computing. Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber

Introduction to grid technologies, parallel and cloud computing. Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber Introduction to grid technologies, parallel and cloud computing Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber OUTLINES Grid Computing Parallel programming technologies (MPI- Open MP-Cuda )

More information

Private Clouds with Open Source

Private Clouds with Open Source Private Clouds with Open Source GridKa School 2010 KIT September 7 th 2010 Christian Baun baun@kit.edu http://www.kit.edu Cloud-Computing? Building on compute and storage virtualization, and leveraging

More information

Denis Caromel, CEO Ac.veEon. Orchestrate and Accelerate Applica.ons. Open Source Cloud Solu.ons Hybrid Cloud: Private with Burst Capacity

Denis Caromel, CEO Ac.veEon. Orchestrate and Accelerate Applica.ons. Open Source Cloud Solu.ons Hybrid Cloud: Private with Burst Capacity Cloud computing et Virtualisation : applications au domaine de la Finance Denis Caromel, CEO Ac.veEon Orchestrate and Accelerate Applica.ons Open Source Cloud Solu.ons Hybrid Cloud: Private with Burst

More information

Fundamental Concepts and Models

Fundamental Concepts and Models Fundamental Concepts and Models 1 1. Roles and Boundaries Could provider The organization that provides the cloud based IT resources Cloud consumer An organization (or a human) that has a formal contract

More information

Towards the Future Internet

Towards the Future Internet Towards the Future Internet Dr. Paola Grosso System and Network Engineering research group Informatics Institute Vertices and edges Path finding algorithms Scale-free properties Models Two features of

More information

BPM Architecture Design Based on Cloud Computing

BPM Architecture Design Based on Cloud Computing Intelligent Information Management, 2010, 2, 329-333 doi:10.4236/iim.2010.25039 Published Online May 2010 (http://www.scirp.org/journal/iim) BPM Architecture Design Based on Cloud Computing Abstract Zhenyu

More information

Amazon EC2 Product Details Page 1 of 5

Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Functionality Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of

More information

Manjrasoft Market Oriented Cloud Computing Platform

Manjrasoft Market Oriented Cloud Computing Platform Manjrasoft Market Oriented Cloud Computing Platform Innovative Solutions for 3D Rendering Aneka is a market oriented Cloud development and management platform with rapid application development and workload

More information

GeoCloud Project Report GEOSS Clearinghouse

GeoCloud Project Report GEOSS Clearinghouse GeoCloud Project Report GEOSS Clearinghouse Qunying Huang, Doug Nebert, Chaowei Yang, Kai Liu 2011.12.06 Description of Application GEOSS clearinghouse is a FGDC, GEO, and NASA project that connects directly

More information

Cloud and Virtualization to Support Grid Infrastructures

Cloud and Virtualization to Support Grid Infrastructures ESAC GRID Workshop '08 ESAC, Villafranca del Castillo, Spain 11-12 December 2008 Cloud and Virtualization to Support Grid Infrastructures Distributed Systems Architecture Research Group Universidad Complutense

More information

CLOUD API DOCUMENTATION v2.0. Get list of cloud servers in account

CLOUD API DOCUMENTATION v2.0. Get list of cloud servers in account Get list of cloud servers in account https://my.quickservers.com/api/cloud/?action=list&username=username&password=password ACTION Required API action list 1640 mail.quickservers.com

More information

Performance Analysis of VM Scheduling Algorithm of CloudSim in Cloud Computing

Performance Analysis of VM Scheduling Algorithm of CloudSim in Cloud Computing IJECT Vo l. 6, Is s u e 1, Sp l-1 Ja n - Ma r c h 2015 ISSN : 2230-7109 (Online) ISSN : 2230-9543 (Print) Performance Analysis Scheduling Algorithm CloudSim in Cloud Computing 1 Md. Ashifuddin Mondal,

More information

Scyld Cloud Manager User Guide

Scyld Cloud Manager User Guide Scyld Cloud Manager User Guide Preface This guide describes how to use the Scyld Cloud Manager (SCM) web portal application. Contacting Penguin Computing 45800 Northport Loop West Fremont, CA 94538 1-888-PENGUIN

More information

Deploying Business Virtual Appliances on Open Source Cloud Computing

Deploying Business Virtual Appliances on Open Source Cloud Computing International Journal of Computer Science and Telecommunications [Volume 3, Issue 4, April 2012] 26 ISSN 2047-3338 Deploying Business Virtual Appliances on Open Source Cloud Computing Tran Van Lang 1 and

More information

Eucalyptus: An Open-source Infrastructure for Cloud Computing. Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com

Eucalyptus: An Open-source Infrastructure for Cloud Computing. Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com Eucalyptus: An Open-source Infrastructure for Cloud Computing Rich Wolski Eucalyptus Systems Inc. www.eucalyptus.com Exciting Weather Forecasts Commercial Cloud Formation Eucalyptus - Confidential What

More information

An Introduction to Cloud Computing Concepts

An Introduction to Cloud Computing Concepts Software Engineering Competence Center TUTORIAL An Introduction to Cloud Computing Concepts Practical Steps for Using Amazon EC2 IaaS Technology Ahmed Mohamed Gamaleldin Senior R&D Engineer-SECC ahmed.gamal.eldin@itida.gov.eg

More information

Best Practices for Using MySQL in the Cloud

Best Practices for Using MySQL in the Cloud Best Practices for Using MySQL in the Cloud Luis Soares, Sr. Software Engineer, MySQL Replication, Oracle Lars Thalmann, Director Replication, Backup, Utilities and Connectors THE FOLLOWING IS INTENDED

More information

Infrastructure Clouds for Science and Education: Platform Tools

Infrastructure Clouds for Science and Education: Platform Tools Infrastructure Clouds for Science and Education: Platform Tools Kate Keahey, Renato J. Figueiredo, John Bresnahan, Mike Wilde, David LaBissoniere Argonne National Laboratory Computation Institute, University

More information

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing New Paradigms: Clouds, Virtualization and Co. EGEE08, Istanbul, September 25, 2008 An Introduction to Virtualization and Cloud Technologies to Support Grid Computing Distributed Systems Architecture Research

More information

WHITE PAPER: PAN Cloud Director Technical Overview

WHITE PAPER: PAN Cloud Director Technical Overview WHITE PAPER: PAN Cloud Director Technical Overview July 2015 1. Introduction PAN Cloud Director (PCD) is a comprehensive cloud management solution that lets both, IT organizations and service providers

More information

Elastic Detector on Amazon Web Services (AWS) User Guide v5

Elastic Detector on Amazon Web Services (AWS) User Guide v5 Elastic Detector on Amazon Web Services (AWS) User Guide v5 This guide is intended for Elastic Detector users on AWS. Elastic Detector is available as SaaS or deployed as a virtual appliance through an

More information

The ENEA gateway approach providing EGEE/gLite access to unsupported platforms and operating systems

The ENEA gateway approach providing EGEE/gLite access to unsupported platforms and operating systems EU-IndiaGrid Workshop Taipei, November 2nd 2007 The ENEA gateway approach providing EGEE/gLite access to unsupported platforms and operating systems G. Bracco, S.Migliori, A. Quintiliani, A. Santoro, C.

More information

Scalable Application. Mikalai Alimenkou http://xpinjection.com 11.05.2012

Scalable Application. Mikalai Alimenkou http://xpinjection.com 11.05.2012 Scalable Application Development on AWS Mikalai Alimenkou http://xpinjection.com 11.05.2012 Background Java Technical Lead/Scrum Master at Zoral Labs 7+ years in software development 5+ years of working

More information

PostgreSQL Performance Characteristics on Joyent and Amazon EC2

PostgreSQL Performance Characteristics on Joyent and Amazon EC2 OVERVIEW In today's big data world, high performance databases are not only required but are a major part of any critical business function. With the advent of mobile devices, users are consuming data

More information

A High Performance Computing Scheduling and Resource Management Primer

A High Performance Computing Scheduling and Resource Management Primer LLNL-TR-652476 A High Performance Computing Scheduling and Resource Management Primer D. H. Ahn, J. E. Garlick, M. A. Grondona, D. A. Lipari, R. R. Springmeyer March 31, 2014 Disclaimer This document was

More information

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms Volume 1, Issue 1 ISSN: 2320-5288 International Journal of Engineering Technology & Management Research Journal homepage: www.ijetmr.org Analysis and Research of Cloud Computing System to Comparison of

More information

Distribution transparency. Degree of transparency. Openness of distributed systems

Distribution transparency. Degree of transparency. Openness of distributed systems Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science steen@cs.vu.nl Chapter 01: Version: August 27, 2012 1 / 28 Distributed System: Definition A distributed

More information

Cloud Infrastructure Pattern

Cloud Infrastructure Pattern 1 st LACCEI International Symposium on Software Architecture and Patterns (LACCEI-ISAP-MiniPLoP 2012), July 23-27, 2012, Panama City, Panama. Cloud Infrastructure Pattern Keiko Hashizume Florida Atlantic

More information

Last time. Today. IaaS Providers. Amazon Web Services, overview

Last time. Today. IaaS Providers. Amazon Web Services, overview Last time General overview, motivation, expected outcomes, other formalities, etc. Please register for course Online (if possible), or talk to Yvonne@CS Course evaluation forgotten Please assign one volunteer

More information

A Gentle Introduction to Cloud Computing

A Gentle Introduction to Cloud Computing A Gentle Introduction to Cloud Computing Source: Wikipedia Platform Computing, Inc. Platform Clusters, Grids, Clouds, Whatever Computing The leader in managing large scale shared environments o 18 years

More information

Architectural Implications of Cloud Computing

Architectural Implications of Cloud Computing Architectural Implications of Cloud Computing Grace Lewis Research, Technology and Systems Solutions (RTSS) Program Lewis is a senior member of the technical staff at the SEI in the Research, Technology,

More information

Building Platform as a Service for Scientific Applications

Building Platform as a Service for Scientific Applications Building Platform as a Service for Scientific Applications Moustafa AbdelBaky moustafa@cac.rutgers.edu Rutgers Discovery Informa=cs Ins=tute (RDI 2 ) The NSF Cloud and Autonomic Compu=ng Center Department

More information

EEDC. Scalability Study of web apps in AWS. Execution Environments for Distributed Computing

EEDC. Scalability Study of web apps in AWS. Execution Environments for Distributed Computing EEDC Execution Environments for Distributed Computing 34330 Master in Computer Architecture, Networks and Systems - CANS Scalability Study of web apps in AWS Sergio Mendoza sergio.mendoza@est.fib.upc.edu

More information

for my computation? Stefano Cozzini Which infrastructure Which infrastructure Democrito and SISSA/eLAB - Trieste

for my computation? Stefano Cozzini Which infrastructure Which infrastructure Democrito and SISSA/eLAB - Trieste Which infrastructure Which infrastructure for my computation? Stefano Cozzini Democrito and SISSA/eLAB - Trieste Agenda Introduction:! E-infrastructure and computing infrastructures! What is available

More information

2011 European HyperWorks Technology Conference. Vladi Nosenzo, Roberto Vadori

2011 European HyperWorks Technology Conference. Vladi Nosenzo, Roberto Vadori 2011 European HyperWorks Technology Conference Vladi Nosenzo, Roberto Vadori 20 Novembre, 2010 2011 ABSTRACT The work described below starts from an idea of a previous experience of Reply, developed in

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

Elastic Cloud Computing in the Open Cirrus Testbed implemented via Eucalyptus

Elastic Cloud Computing in the Open Cirrus Testbed implemented via Eucalyptus Elastic Cloud Computing in the Open Cirrus Testbed implemented via Eucalyptus International Symposium on Grid Computing 2009 (Taipei) Christian Baun The cooperation of and Universität Karlsruhe (TH) Agenda

More information

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud CISCO NerdLunch Series November 7, 2008 San Jose, CA New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed Systems Architecture Research

More information

Big Data and Cloud Computing for GHRSST

Big Data and Cloud Computing for GHRSST Big Data and Cloud Computing for GHRSST Jean-Francois Piollé (jfpiolle@ifremer.fr) Frédéric Paul, Olivier Archer CERSAT / Institut Français de Recherche pour l Exploitation de la Mer Facing data deluge

More information

Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012

Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012 Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012 Thomas Hauth,, Günter Quast IEKP KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz

More information

SERVER 101 COMPUTE MEMORY DISK NETWORK

SERVER 101 COMPUTE MEMORY DISK NETWORK Cloud Computing ก ก ก SERVER 101 COMPUTE MEMORY DISK NETWORK SERVER 101 1 GHz = 1,000.000.000 Cycle/Second 1 CPU CYCLE VIRTUALIZATION 101 VIRTUALIZATION 101 VIRTUALIZATION 101 HISTORY YEAR 1800 YEARS LATER

More information

Abstract. 1. Introduction. Ohio State University Columbus, OH 43210 {langella,oster,hastings,kurc,saltz}@bmi.osu.edu

Abstract. 1. Introduction. Ohio State University Columbus, OH 43210 {langella,oster,hastings,kurc,saltz}@bmi.osu.edu Dorian: Grid Service Infrastructure for Identity Management and Federation Stephen Langella 1, Scott Oster 1, Shannon Hastings 1, Frank Siebenlist 2, Tahsin Kurc 1, Joel Saltz 1 1 Department of Biomedical

More information

Elastic Management of Cluster based Services in the Cloud

Elastic Management of Cluster based Services in the Cloud First Workshop on Automated Control for Datacenters and Clouds (ACDC09) June 19th, Barcelona, Spain Elastic Management of Cluster based Services in the Cloud Rafael Moreno Vozmediano, Ruben S. Montero,

More information

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013 Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device

More information

Low-cost Open Data As-a-Service in the Cloud

Low-cost Open Data As-a-Service in the Cloud Low-cost Open Data As-a-Service in the Cloud Marin Dimitrov, Alex Simov, Yavor Petkov Ontotext AD, Bulgaria {first.last}@ontotext.com Abstract. In this paper we present the architecture and prototype of

More information

Provisioning and Resource Management at Large Scale (Kadeploy and OAR)

Provisioning and Resource Management at Large Scale (Kadeploy and OAR) Provisioning and Resource Management at Large Scale (Kadeploy and OAR) Olivier Richard Laboratoire d Informatique de Grenoble (LIG) Projet INRIA Mescal 31 octobre 2007 Olivier Richard ( Laboratoire d Informatique

More information

Automated deployment of virtualization-based research models of distributed computer systems

Automated deployment of virtualization-based research models of distributed computer systems Automated deployment of virtualization-based research models of distributed computer systems Andrey Zenzinov Mechanics and mathematics department, Moscow State University Institute of mechanics, Moscow

More information

III Level Course, 2011 Free Software. Dott. Bertoldo Silvano Ing. Terzo Olivier

III Level Course, 2011 Free Software. Dott. Bertoldo Silvano Ing. Terzo Olivier III Level Course, 2011 Free Software Dott. Bertoldo Silvano Ing. Terzo Olivier 1 1. Introduction to Grid and Cloud Computing 2. Open Source Software in Grid and Cloud Computing 2.1 Hypervisor 2.2 Cloud

More information

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Cloud Computing 159.735 Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Table of Contents Introduction... 3 What is Cloud Computing?... 3 Key Characteristics...

More information

Cloud Computing at CDC Current Status and Future Plans Earl Baum. March, 2014

Cloud Computing at CDC Current Status and Future Plans Earl Baum. March, 2014 Cloud Computing at CDC Current Status and Future Plans Earl Baum March, 2014 1 Background Current Activities Agenda Use Cases, Shared Services and Other Considerations What s Next 2 Background Cloud Definition

More information