Cloud Computing. Up until now

Size: px
Start display at page:

Download "Cloud Computing. Up until now"

Transcription

1 Cloud Computing Lecture 20 Cloud Platform Comparison & Load Up until now Introduction, Definition of Cloud Computing Pre-Cloud Large Scale Computing: Grid Computing Content Distribution Networks Cycle-Sharing Distributed Scheduling Cloud: Map Reduce Storage Execution Monitoring Programming 1

2 Cloud Platform Comparison Load Outline Comparison of Cloud Platform Google / Google App Engine Hadoop Amazon Web Services / Eucalyptus Microsoft Azure 2

3 Computing 3 visions for Cloud Computing: Who will win? AmazonWeb Services x86 Microsoft Azure CLR (VM) Google App Engine Framework Aplicacional (Python, Java) Storage Disk blocks SQL server API BigTable Network BlocksofIP addresses Declarative but automatic (endpoints) 3 level applicational topology»thisistheideal model! Inpractice, the overlap is much larger! Comparison: Storage AWS / Eucalyptus Microsoft Azure Google / Hadoop SQL RDS SQL Azure X Tables SimpleDB Tables (Datastore [BigTable]) / HBase Objects/Blocks S3 Blobs GFS/ HDFS Queues Simple Queue Service(SQS) Queues (Task Queue) 3

4 Comparison: Storage There are two general complaints: Performance (latency). Strict coherency models do not scale. The bottom-line is that the storage scalability problem is not solved. There are no available reliable metrics. The market is still too dynamic. Google services are not accessible remotely. It is always possible to make an intermediary bridge service. Programming languages: Comparison: Programming Model Amazon: Language not relevant. The program is a VM. Google: Java and Python. Azure: Any.NET language - C#, J#, VB.NET, etc... Google (servlet/jsp) has the most restrictive model. It is the simplest choice and will tend to be the first one until limitations are found. 4

5 Comparison: Remote Interaction Model There are little differences/variations. All systems are based on Web Services. Most services support both REST and SOAP protocols. In most cases, applications/machines/services/stores have their own DNS names. Stored objects are identified by type less strings. Comparison: Integration The Amazon VM model permits normal interactions between servers. Google requires that other servers be accessible via Web Services. Azure supports richer integration mechanism with external servers: AppFabric, Access Control e Queues. DryadLINQtransparently integrates local and remote applications. 5

6 Comparison: Price Resource Unit Amazon Google Microsoft Bandwidth (outgoing) GB $ $0.085 $0.12 $0.15 Bandwidth (ingoing) GB $0.10 $0.10 $0.10 Computation Instance hour $ $1.201 $0.10 $0.12 Storage GB per month $0.05 (>5PB) to 0.14 (<1TB) $0.15 $0.15 Storage Calls Each 10k calls $0.01 (GET) $0.10 (others) $0.01 Prices are very similar. AWS, because they use system VMs, has a larger granularity. Scenario Application ported to the cloud Web Application Parallel Processing Mixed Application Characteristics Monolythic application in Java or.net. Web app with load balancer, logic layer and database. Long lasting calculations without GUI. Cloud application integrated with external servers. Platform/Application Match Amazon Normal EC2 instance. System configuration needed. Normal EC2 instance + RDS. Requires system config. and AutoScale. If RDS does not scale, requires port to S3. Many pre-built instances with infra-structure, e.g. MPI. MapReduce instances may be used. EC2 instance may access external servers. Google May require porting and requires data and logic refactoring. Very good match with Google App Engine. Automatic scalability. Requires DB rewrite. No support for larger scale applications. No direct support. Some integration possible using a bridge app to the Datastore. Microsoft If.NET refactor data. Otherwise more complex. Well adapted to the Web Role model. Worker roles + blobs e queues provide some/adequate support. AppFabric ServiceBus supports integration with external applications. 6

7 Hurdles to CC on the 3 Main Platforms 1. Availability: Depends on the SLA and the provider s track record. 2. Lock-In: Stronger with Google App Engine, then Azure, weaker with AWS. 3. Confidentiality and Auditing: In general confidentiality is guaranteed. No open auditing is available. Regarding applications, EC2 provides higher isolation. 4. Data transfer costs: Similar prices. AWS now has bulk transfer services (you can send them your disks). Cost/benefit is application dependent. Must be analyzed. 5. Reliable Performance For general applications, the situation is similar: there are recovery and repetition mechanisms for most services. In the case of MapReducethere is skipping mode to recover tasks. 6. Scalable storage 7. Large-scale software errors 8. Speed of scale-up: Hurdles to Cloud Computing Clearer feedback with EC2 instances. 9. Reputation propagation: Similar situation on all 3 major platforms. Not solved. Less relevant for Google App Engine. 10. Compatible licensing: only relevant at AWS (solved!) 7

8 Conclusions The main difference between the main providers is the applicational model: Google has the most restrictive model. The cost of an easy to program system is more lock-in than lack of functionality. I can do whatever I want on EC2 but a scalable application will require distributed scalable services.. Scalability: What is the Best Approach for Cloud Computing Clients? Handling flash crowds from your garage, USENIX 08 8

9 Flash Crowds! We have seen several examples of scalability in a cloud platform. What about the clients? What if we have a server running an application and need to scale quickly? How do I adapt the front-ends? Three main requirements: The system must scale to a very large size. The system must scale quickly. Off-peek operation must be cheap. Data storage services: Available Tools (i) Pros: they are cheap and they scale transparently for the user. Cons: Only solve the problem of static content. Virtual servers: Before the cloud it was already possible to rent virtual servers at ISP (even at different geographical locations). Cons: It only solves the bandwidth problem. Mostly, the computation of the distributed applications doesn t really scale. 9

10 Available Tools (ii) Cloud computing services. External DNS services: Prevents the service from facing a bottleneck on the DNS requests. MISSING! Scalable relational database service: As we have seen, it s not trivial to scale a classical relational database service. There are many similar services but they always sacrifice some aspect: transactional model, features of the query language, scalability. Scalable Architectures (i) What is the best approach to matching a large set of clients with a multi-server service? Hyp. 1: Use only a storage service. Good for servers with a large percentage of static content. 10

11 Scalable Architectures (ii) Hyp. 2: Cluster with DNS load balancing Rent several machines (e.g. EC2). Add machines to the DNS record. By default, addresses are used in round-robin fashion. Causes delays to the clients who cached the DNS record but in general the issue is the large number of clients and not a large number of requests from the same client. There are commercial implementations (e.g. RightScale). Scalable Architectures (iii) Hyp. 3: Redirection Having a server redirect the initial client request to a set of back-end servers. Subsequent requests don t go through the redirection. Example: Amazon Elastic Load. Hyp. 4: L4 or L7 Rerouting A front-end server analyzes the request source (4 OSI level 4 e.g. TCP) or the content (OSI level 7 e.g. ) and reroutes the request to the corresponding back-end server. Requires a high-performance server or switch, but the client does not see the redirection. There are commercial implementations (e.g. Flexiscale). Hyp. 5: Hybrids of the 4 previous hypothesis. 11

12 M: repl. front-end Redirection M: repl. front-end Static Redirection 12

13 M: repl. front-end Static Redirection Scales very well Client arrival rate M: repl. front-end Static Redir. Scales very well Client arrival rate Redirecting clients (specially if it s done only when a session begins), is very cheap even if the front-server is receiving back-end status reports and running a load balancing algorithm. 13

14 M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited The UDP-based DNS response has only 512 bytes (up to 25 back-end servers). Most ISP complete the request using TCP if there are more than 25. However, some DNS clients only use the first reply. M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited Incoherent, but in the case of L4 there are growing hurdles to success: NAT, proxies,... 14

15 M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited Incoherent Immediate + DNS TTL M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited Incoherent Immediate + DNS TTL Immediate Session duration Session duration Days It is difficult to identify when sessions finish (e.g. webmail). There are DNS clients that ignore DNS records TTL and take days to invalidate their DNS cache. 15

16 M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited Incoherent Immediate + DNS TTL Immediate Session duration Session duration Days It is difficult to identify when sessions finish (e.g. webmail). There are DNS clients that ignore DNS records TTL and take days to invalidate their DNS cache. M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited Incoherent Immediate The front-end VM start-up of the time storage service. Not the web server. Immediate Session duration Session duration + DNS TTL Days Total Failure Total Failure Significant Fault Especially when using low TTL s for good scalling 16

17 M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited If there is load balancing of the redirection servers, Immediate one has to VM wait start-up for the timeclient VM to start-up try time another server. It should take max 2.5 s but in some Linux implementations it takes up to 3 min! Immediate Session duration Session duration Incoherent + DNS TTL Days Total Failure Total Failure Significant Fault Has no effect Total Failure Rare Effect Improbable Longdelayfor 1/m sessions? Longdelayfor 1/m sessions? Small Delay M: repl. front-end Static Redir. Scales very well Client arrival rate Request arrival rate Unlimited Incoherent Immediate + DNS TTL Immediate Session duration Session duration Days e.g., in S3 1% of first write attempts Has no fail, effect but immediate retries succeed. Improbable Improbable Improbable Improbable Total Failure Longdelayfor 1/m sessions? Has no effect Has no effect User recouverable fault Total Failure Total Failure Longdelayfor 1/m sessions? 1/m sessions fail. Has no effect Occasional fault Significant Fault Rare Effect Small Delay Some sessions have small delay. Longdelayfor 1/n sessions. Longdelayfor 1/n sessions. 17

18 M: repl. front-end Redir. Static Scales very well Client arrival rate Request arrival rate Unlimited Incoherent Immediate + DNS TTL Immediate Session duration Session duration Days Total Failure Total Failure Significant Fault Has no effect Total Failure Rare Effect Improbable Longdelayfor 1/m sessions? Longdelayfor 1/m sessions? Small Delay Improbable Has no effect 1/m sessions fail. Some sessions have small delay. Improbable Improbable Has no effect User recoverable fault Has no effect Occasional fault Longdelayfor 1/n sessions. Longdelayfor 1/n sessions. MapCruncher 18

19 Example: MapCruncher Map conversion site. Loaded with 25 GB of interactive demo maps. Flash crowd due to Microsoft publicizing it. The server had theoretical capacity to handle traffic (100 images/sec.), but the lack of reference locality (each client looking at different parts of the maps) made the thrashing unbearable. Moved all the static content to S3: they pay $4/month if there is no traffic. Example 2: Asirra 19

20 Example 2: Assirra CaptchaWeb Service based on distinguishing cats from dogs. EC2 servers + 100GB of images placed on S3. Database of image metadata: SQL server was slow. Nightly transfer of a image key indexed structure (read-only DB) to each of the applicational servers. Example 2: Azirra How can the session state be maintained? Hyp. 1: Inside S3. It s slow. Hyp. 2: On the applicationalservers disks. Since they use DNS load balancing it s not guaranteed that the question and answer to the captchago to the same server. Solution: Forward all session requests to the same server. Server id stored in session id. It s very cheap because it requires no disk accesses and only 10% change servers between request and response. 20

21 Example 2: Azirra Again, a flash crowd after a trade fair appearance requests in 24h. Two interesting observations: requests were from a DoS. Using more instance was cheap. The attacker gave up but it would have been cheap to keep them running until a filter were set up. Example 3: InkBlotPassword.com Website for associating mnemonic images (Rorschach inkblots) to passwords. After the two previous experiences, they simplified the development process. Is it worth optimizing code? If optimizations are only for peek periods, it s better to pay for more machines. The website was mentioned on Slashdot (tech news site) without the authors knowing. They detected a flash crowd (request queue = 130!), started 12 new nodes. 20 min. later, the website was stable. Three days later they were again stable at only 3 servers. Total cost of the flash crowd: $

22 Next Time... Cloud Data Centers 22

Cloud Computing. Up until now

Cloud Computing. Up until now Cloud Computing Lecture 20 Cloud Platform Comparison & Load Balancing 2010-2011 Up until now Introduction, Definition of Cloud Computing Pre-Cloud Large Scale Computing: Grid Computing Content Distribution

More information

Cloud Computing. Lecture 24 Cloud Platform Comparison 2014-2015

Cloud Computing. Lecture 24 Cloud Platform Comparison 2014-2015 Cloud Computing Lecture 24 Cloud Platform Comparison 2014-2015 1 Up until now Introduction, Definition of Cloud Computing Pre-Cloud Large Scale Computing: Grid Computing Content Distribution Networks Cycle-Sharing

More information

Handling Flash Crowds From Your Garage

Handling Flash Crowds From Your Garage Handling Flash Crowds From Your Garage Jeremy Elson and Jon Howell Microsoft Research USENIX ATC 2008 Scaling For Many Users Scaling For Many Users... quickly Scaling For Many Users... quickly... on a

More information

A Survey on Cloud Storage Systems

A Survey on Cloud Storage Systems A Survey on Cloud Storage Systems Team : Xiaoming Xiaogang Adarsh Abhijeet Pranav Motivations No Taxonomy Detailed Survey for users Starting point for researchers Taxonomy Category Definition Example Instance

More information

DNS records. RR format: (name, value, type, TTL) Type=NS

DNS records. RR format: (name, value, type, TTL) Type=NS DNS records RR format: (name, value, type, TTL) Type=A name is hostname value is IP address Type=NS name is domain (e.g. foo.com) value is hostname of authoritative name server for this domain Type=CNAME

More information

Alfresco Enterprise on AWS: Reference Architecture

Alfresco Enterprise on AWS: Reference Architecture Alfresco Enterprise on AWS: Reference Architecture October 2013 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 13 Abstract Amazon Web Services (AWS)

More information

A programming model in Cloud: MapReduce

A programming model in Cloud: MapReduce A programming model in Cloud: MapReduce Programming model and implementation developed by Google for processing large data sets Users specify a map function to generate a set of intermediate key/value

More information

References. Introduction to Database Systems CSE 444. Motivation. Basic Features. Outline: Database in the Cloud. Outline

References. Introduction to Database Systems CSE 444. Motivation. Basic Features. Outline: Database in the Cloud. Outline References Introduction to Database Systems CSE 444 Lecture 24: Databases as a Service YongChul Kwon Amazon SimpleDB Website Part of the Amazon Web services Google App Engine Datastore Website Part of

More information

Introduction to Database Systems CSE 444

Introduction to Database Systems CSE 444 Introduction to Database Systems CSE 444 Lecture 24: Databases as a Service YongChul Kwon References Amazon SimpleDB Website Part of the Amazon Web services Google App Engine Datastore Website Part of

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information

Cloud Computing Trends

Cloud Computing Trends UT DALLAS Erik Jonsson School of Engineering & Computer Science Cloud Computing Trends What is cloud computing? Cloud computing refers to the apps and services delivered over the internet. Software delivered

More information

A Very Brief Introduction To Cloud Computing. Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman

A Very Brief Introduction To Cloud Computing. Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman A Very Brief Introduction To Cloud Computing Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman What is The Cloud Cloud computing refers to logical computational resources accessible via a computer

More information

A Comparison of Clouds: Amazon Web Services, Windows Azure, Google Cloud Platform, VMWare and Others (Fall 2012)

A Comparison of Clouds: Amazon Web Services, Windows Azure, Google Cloud Platform, VMWare and Others (Fall 2012) 1. Computation Amazon Web Services Amazon Elastic Compute Cloud (Amazon EC2) provides basic computation service in AWS. It presents a virtual computing environment and enables resizable compute capacity.

More information

Ø Teaching Evaluations. q Open March 3 through 16. Ø Final Exam. q Thursday, March 19, 4-7PM. Ø 2 flavors: q Public Cloud, available to public

Ø Teaching Evaluations. q Open March 3 through 16. Ø Final Exam. q Thursday, March 19, 4-7PM. Ø 2 flavors: q Public Cloud, available to public Announcements TIM 50 Teaching Evaluations Open March 3 through 16 Final Exam Thursday, March 19, 4-7PM Lecture 19 20 March 12, 2015 Cloud Computing Cloud Computing: refers to both applications delivered

More information

Assignment # 1 (Cloud Computing Security)

Assignment # 1 (Cloud Computing Security) Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual

More information

Putchong Uthayopas, Kasetsart University

Putchong Uthayopas, Kasetsart University Putchong Uthayopas, Kasetsart University Introduction Cloud Computing Explained Cloud Application and Services Moving to the Cloud Trends and Technology Legend: Cluster computing, Grid computing, Cloud

More information

Cloud Infrastructure Planning. Chapter Six

Cloud Infrastructure Planning. Chapter Six Cloud Infrastructure Planning Chapter Six Topics Key to successful cloud service adoption is an understanding of underlying infrastructure. Topics Understanding cloud networks Leveraging automation and

More information

Where We Are. References. Cloud Computing. Levels of Service. Cloud Computing History. Introduction to Data Management CSE 344

Where We Are. References. Cloud Computing. Levels of Service. Cloud Computing History. Introduction to Data Management CSE 344 Where We Are Introduction to Data Management CSE 344 Lecture 25: DBMS-as-a-service and NoSQL We learned quite a bit about data management see course calendar Three topics left: DBMS-as-a-service and NoSQL

More information

Scaling Analysis Services in the Cloud

Scaling Analysis Services in the Cloud Our Sponsors Scaling Analysis Services in the Cloud by Gerhard Brückl gerhard@gbrueckl.at blog.gbrueckl.at About me Gerhard Brückl Working with Microsoft BI since 2006 Windows Azure / Cloud since 2013

More information

Cloud Computing: Meet the Players. Performance Analysis of Cloud Providers

Cloud Computing: Meet the Players. Performance Analysis of Cloud Providers BASEL UNIVERSITY COMPUTER SCIENCE DEPARTMENT Cloud Computing: Meet the Players. Performance Analysis of Cloud Providers Distributed Information Systems (CS341/HS2010) Report based on D.Kassman, T.Kraska,

More information

Distribution transparency. Degree of transparency. Openness of distributed systems

Distribution transparency. Degree of transparency. Openness of distributed systems Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science steen@cs.vu.nl Chapter 01: Version: August 27, 2012 1 / 28 Distributed System: Definition A distributed

More information

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud) Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University

More information

Lecture 6 Cloud Application Development, using Google App Engine as an example

Lecture 6 Cloud Application Development, using Google App Engine as an example Lecture 6 Cloud Application Development, using Google App Engine as an example 922EU3870 Cloud Computing and Mobile Platforms, Autumn 2009 (2009/10/19) http://code.google.com/appengine/ Ping Yeh ( 葉 平

More information

ECE6130 Grid and Cloud Computing

ECE6130 Grid and Cloud Computing ECE6130 Grid and Cloud Computing Howie Huang Department of Electrical and Computer Engineering School of Engineering and Applied Science Cloud Computing Hardware Software Outline Research Challenges 2

More information

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms Volume 1, Issue 1 ISSN: 2320-5288 International Journal of Engineering Technology & Management Research Journal homepage: www.ijetmr.org Analysis and Research of Cloud Computing System to Comparison of

More information

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze

More information

CLOUD COMPUTING. When It's smarter to rent than to buy

CLOUD COMPUTING. When It's smarter to rent than to buy CLOUD COMPUTING When It's smarter to rent than to buy Is it new concept? Nothing new In 1990 s, WWW itself Grid Technologies- Scientific applications Online banking websites More convenience Not to visit

More information

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Cloud Computing 159.735 Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Table of Contents Introduction... 3 What is Cloud Computing?... 3 Key Characteristics...

More information

How To Scale A Server Farm

How To Scale A Server Farm Basics of Cloud Computing Lecture 3 Scaling Applications on the Cloud Satish Srirama Outline Scaling Information Systems Scaling Enterprise Applications in the Cloud Auto Scaling 25/02/2014 Satish Srirama

More information

Cloud computing. Examples

Cloud computing. Examples Cloud computing Cloud computing Web Systems and Algorithms Cloud Computing Chris Brooks Department of Computer Science University of San Francisco What is cloud computing? What separates it from: grid

More information

Cloud Computing. Summary

Cloud Computing. Summary Cloud Computing Lecture 1 2011-2012 https://fenix.ist.utl.pt/disciplinas/cn Summary Teaching Staff. Rooms and Schedule. Goals. Context. Syllabus. Reading Material. Assessment and Grading. Important Dates.

More information

2) Xen Hypervisor 3) UEC

2) Xen Hypervisor 3) UEC 5. Implementation Implementation of the trust model requires first preparing a test bed. It is a cloud computing environment that is required as the first step towards the implementation. Various tools

More information

Mark Bennett. Search and the Virtual Machine

Mark Bennett. Search and the Virtual Machine Mark Bennett Search and the Virtual Machine Agenda Intro / Business Drivers What to do with Search + Virtual What Makes Search Fast (or Slow!) Virtual Platforms Test Results Trends / Wrap Up / Q & A Business

More information

Platforms in the Cloud

Platforms in the Cloud Platforms in the Cloud Where Will Your Next Application Run? Jazoon, Zurich June 2011 Copyright 2011 Chappell & Associates An Organization without Cloud Computing Users A A A VM VM VM A A A Application

More information

Cloud computing - Architecting in the cloud

Cloud computing - Architecting in the cloud Cloud computing - Architecting in the cloud anna.ruokonen@tut.fi 1 Outline Cloud computing What is? Levels of cloud computing: IaaS, PaaS, SaaS Moving to the cloud? Architecting in the cloud Best practices

More information

Cloud Computing: Making the right choices

Cloud Computing: Making the right choices Cloud Computing: Making the right choices Kalpak Shah Clogeny Technologies Pvt Ltd 1 About Me Kalpak Shah Founder & CEO, Clogeny Technologies Passionate about economics and technology evolving through

More information

Hadoop IST 734 SS CHUNG

Hadoop IST 734 SS CHUNG Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to

More information

Francesco.diaz@insight.com. http://francescodiaz.azurewebsites.net

Francesco.diaz@insight.com. http://francescodiaz.azurewebsites.net Francesco.diaz@insight.com http://francescodiaz.azurewebsites.net Agenda SQL workloads on Azure Microsoft Azure IaaS concepts Best Practices for SQL running on Azure VMs HA/DR SQL DB HA/DR Dedicated Higher

More information

What Is It? Business Architecture Research Challenges Bibliography. Cloud Computing. Research Challenges Overview. Carlos Eduardo Moreira dos Santos

What Is It? Business Architecture Research Challenges Bibliography. Cloud Computing. Research Challenges Overview. Carlos Eduardo Moreira dos Santos Research Challenges Overview May 3, 2010 Table of Contents I 1 What Is It? Related Technologies Grid Computing Virtualization Utility Computing Autonomic Computing Is It New? Definition 2 Business Business

More information

Cloud Computing Training

Cloud Computing Training Cloud Computing Training TechAge Labs Pvt. Ltd. Address : C-46, GF, Sector 2, Noida Phone 1 : 0120-4540894 Phone 2 : 0120-6495333 TechAge Labs 2014 version 1.0 Cloud Computing Training Cloud Computing

More information

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 1 Hadoop: A Framework for Data- Intensive Distributed Computing CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 2 What is Hadoop? Hadoop is a software framework for distributed processing of large datasets

More information

Distributed Data Parallel Computing: The Sector Perspective on Big Data

Distributed Data Parallel Computing: The Sector Perspective on Big Data Distributed Data Parallel Computing: The Sector Perspective on Big Data Robert Grossman July 25, 2010 Laboratory for Advanced Computing University of Illinois at Chicago Open Data Group Institute for Genomics

More information

From Internet Data Centers to Data Centers in the Cloud

From Internet Data Centers to Data Centers in the Cloud From Internet Data Centers to Data Centers in the Cloud This case study is a short extract from a keynote address given to the Doctoral Symposium at Middleware 2009 by Lucy Cherkasova of HP Research Labs

More information

Technical Writing - Definition of Cloud A Rational Perspective

Technical Writing - Definition of Cloud A Rational Perspective INTRODUCTIONS Storm Technology Who we are and what we do David Chappell IT strategist and international advisor The Cloud A Rational Perspective The cloud platforms An objective overview of the Windows

More information

Cloud Computing Disaster Recovery (DR)

Cloud Computing Disaster Recovery (DR) Cloud Computing Disaster Recovery (DR) Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Need for Disaster Recovery (DR) What happens when you

More information

Data Centers and Cloud Computing. Data Centers

Data Centers and Cloud Computing. Data Centers Data Centers and Cloud Computing Slides courtesy of Tim Wood 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises for server applications Internet

More information

Cloud Computing Is In Your Future

Cloud Computing Is In Your Future Cloud Computing Is In Your Future Michael Stiefel www.reliablesoftware.com development@reliablesoftware.com http://www.reliablesoftware.com/dasblog/default.aspx Cloud Computing is Utility Computing Illusion

More information

Aspera Direct-to-Cloud Storage WHITE PAPER

Aspera Direct-to-Cloud Storage WHITE PAPER Transport Direct-to-Cloud Storage and Support for Third Party April 2014 WHITE PAPER TABLE OF CONTENTS OVERVIEW 3 1 - THE PROBLEM 3 2 - A FUNDAMENTAL SOLUTION - ASPERA DIRECT-TO-CLOUD TRANSPORT 5 3 - VALIDATION

More information

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 14

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 14 Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases Lecture 14 Big Data Management IV: Big-data Infrastructures (Background, IO, From NFS to HFDS) Chapter 14-15: Abideboul

More information

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Alexandra Carpen-Amarie Diana Moise Bogdan Nicolae KerData Team, INRIA Outline

More information

Cloud computing opens new perspectives for hosting

Cloud computing opens new perspectives for hosting ConPaaS: a Platform for Hosting Elastic Cloud Applications Guillaume Pierre Corina Stratan Vrije Universiteit Amsterdam Cloud computing opens new perspectives for hosting applications. From an application

More information

Building a Highly Available and Scalable Web Farm

Building a Highly Available and Scalable Web Farm Page 1 of 10 MSDN Home > MSDN Library > Deployment Rate this page: 10 users 4.9 out of 5 Building a Highly Available and Scalable Web Farm Duwamish Online Paul Johns and Aaron Ching Microsoft Developer

More information

Amazon EC2 Product Details Page 1 of 5

Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Functionality Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of

More information

Introduction to Cloud Computing

Introduction to Cloud Computing Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services

More information

WINDOWS AZURE EXECUTION MODELS

WINDOWS AZURE EXECUTION MODELS WINDOWS AZURE EXECUTION MODELS Windows Azure provides three different execution models for running applications: Virtual Machines, Web Sites, and Cloud Services. Each one provides a different set of services,

More information

ArcGIS for Server in the Amazon Cloud. Michele Lundeen Esri

ArcGIS for Server in the Amazon Cloud. Michele Lundeen Esri ArcGIS for Server in the Amazon Cloud Michele Lundeen Esri What we will cover ArcGIS for Server in the Amazon Cloud Why How Extras Why do you need ArcGIS Server? Some examples Publish - Dynamic Map Services

More information

Open Source Technologies on Microsoft Azure

Open Source Technologies on Microsoft Azure Open Source Technologies on Microsoft Azure A Survey @DChappellAssoc Copyright 2014 Chappell & Associates The Main Idea i Open source technologies are a fundamental part of Microsoft Azure The Big Questions

More information

DATA SECURITY MODEL FOR CLOUD COMPUTING

DATA SECURITY MODEL FOR CLOUD COMPUTING DATA SECURITY MODEL FOR CLOUD COMPUTING POOJA DHAWAN Assistant Professor, Deptt of Computer Application and Science Hindu Girls College, Jagadhri 135 001 poojadhawan786@gmail.com ABSTRACT Cloud Computing

More information

19.10.11. Amazon Elastic Beanstalk

19.10.11. Amazon Elastic Beanstalk 19.10.11 Amazon Elastic Beanstalk A Short History of AWS Amazon started as an ECommerce startup Original architecture was restructured to be more scalable and easier to maintain Competitive pressure for

More information

Cloud Computing and Amazon Web Services. CJUG March, 2009 Tom Malaher

Cloud Computing and Amazon Web Services. CJUG March, 2009 Tom Malaher Cloud Computing and Amazon Web Services CJUG March, 2009 Tom Malaher Agenda What is Cloud Computing? Amazon Web Services (AWS) Other Offerings Composing AWS Services Use Cases Ecosystem Reality Check Pros&Cons

More information

Cloud Computing with Microsoft Azure

Cloud Computing with Microsoft Azure Cloud Computing with Microsoft Azure Michael Stiefel www.reliablesoftware.com development@reliablesoftware.com http://www.reliablesoftware.com/dasblog/default.aspx Azure's Three Flavors Azure Operating

More information

Cloud computing. Intelligent Services for Energy-Efficient Design and Life Cycle Simulation. as used by the ISES project

Cloud computing. Intelligent Services for Energy-Efficient Design and Life Cycle Simulation. as used by the ISES project Intelligent Services for Energy-Efficient Design and Life Cycle Simulation Project number: 288819 Call identifier: FP7-ICT-2011-7 Project coordinator: Technische Universität Dresden, Germany Website: ises.eu-project.info

More information

Cloud Models and Platforms

Cloud Models and Platforms Cloud Models and Platforms Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF A Working Definition of Cloud Computing Cloud computing is a model

More information

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes

More information

Real Time Big Data Processing

Real Time Big Data Processing Real Time Big Data Processing Cloud Expo 2014 Ian Meyers Amazon Web Services Global Infrastructure Deployment & Administration App Services Analytics Compute Storage Database Networking AWS Global Infrastructure

More information

How To Choose Between A Relational Database Service From Aws.Com

How To Choose Between A Relational Database Service From Aws.Com The following text is partly taken from the Oracle book Middleware and Cloud Computing It is available from Amazon: http://www.amazon.com/dp/0980798000 Cloud Databases and Oracle When designing your cloud

More information

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2 DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing Slide 1 Slide 3 A style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet.

More information

Users VM A A A. Application. Compute/Storage/Network. VM Virtual Machine. On-Premises Data Center

Users VM A A A. Application. Compute/Storage/Network. VM Virtual Machine. On-Premises Data Center Users A A A VM VM VM A A A Application Compute/Storage/Network On-Premises Data Center VM Virtual Machine On-Premises Service Provider Users A A A Applications Compute/Storage/Network The main focus of

More information

Hadoop and Map-Reduce. Swati Gore

Hadoop and Map-Reduce. Swati Gore Hadoop and Map-Reduce Swati Gore Contents Why Hadoop? Hadoop Overview Hadoop Architecture Working Description Fault Tolerance Limitations Why Map-Reduce not MPI Distributed sort Why Hadoop? Existing Data

More information

The full setup includes the server itself, the server control panel, Firebird Database Server, and three sample applications with source code.

The full setup includes the server itself, the server control panel, Firebird Database Server, and three sample applications with source code. Content Introduction... 2 Data Access Server Control Panel... 2 Running the Sample Client Applications... 4 Sample Applications Code... 7 Server Side Objects... 8 Sample Usage of Server Side Objects...

More information

Large-Scale Web Applications

Large-Scale Web Applications Large-Scale Web Applications Mendel Rosenblum Web Application Architecture Web Browser Web Server / Application server Storage System HTTP Internet CS142 Lecture Notes - Intro LAN 2 Large-Scale: Scale-Out

More information

Challenges for Data Driven Systems

Challenges for Data Driven Systems Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Quick History of Data Management 4000 B C Manual recording From tablets to papyrus to paper A. Payberah 2014 2

More information

Written examination in Cloud Computing

Written examination in Cloud Computing Written examination in Cloud Computing February 11th 2014 Last name: First name: Student number: Provide on all sheets (including the cover sheet) your last name, rst name and student number. Use the provided

More information

Research Paper Available online at: www.ijarcsse.com A COMPARATIVE STUDY OF CLOUD COMPUTING SERVICE PROVIDERS

Research Paper Available online at: www.ijarcsse.com A COMPARATIVE STUDY OF CLOUD COMPUTING SERVICE PROVIDERS Volume 2, Issue 2, February 2012 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: A COMPARATIVE STUDY OF CLOUD

More information

SQL Server on Azure An e2e Overview. Nosheen Syed Principal Group Program Manager Microsoft

SQL Server on Azure An e2e Overview. Nosheen Syed Principal Group Program Manager Microsoft SQL Server on Azure An e2e Overview Nosheen Syed Principal Group Program Manager Microsoft Dedicated Higher cost Shared Lower cost SQL Server Cloud Continuum Hybrid SQL Server in Azure VM Virtualized Machines

More information

Scalable Application. Mikalai Alimenkou http://xpinjection.com 11.05.2012

Scalable Application. Mikalai Alimenkou http://xpinjection.com 11.05.2012 Scalable Application Development on AWS Mikalai Alimenkou http://xpinjection.com 11.05.2012 Background Java Technical Lead/Scrum Master at Zoral Labs 7+ years in software development 5+ years of working

More information

Designing Apps for Amazon Web Services

Designing Apps for Amazon Web Services Designing Apps for Amazon Web Services Mathias Meyer, GOTO Aarhus 2011 Montag, 10. Oktober 11 Montag, 10. Oktober 11 Me infrastructure code databases @roidrage www.paperplanes.de Montag, 10. Oktober 11

More information

PV213 Enterprise Information Systems in Practice 07 - Architecture of the EIS in the cloud

PV213 Enterprise Information Systems in Practice 07 - Architecture of the EIS in the cloud PV213 Enterprise Information Systems in Practice 07 - Architecture of the EIS in the cloud PV213 EIS in Practice: 07 - Architecture of the EIS in the cloud 1 PV213 EIS in Practice: 07 - Architecture of

More information

Introduction to Big Data! with Apache Spark" UC#BERKELEY#

Introduction to Big Data! with Apache Spark UC#BERKELEY# Introduction to Big Data! with Apache Spark" UC#BERKELEY# This Lecture" The Big Data Problem" Hardware for Big Data" Distributing Work" Handling Failures and Slow Machines" Map Reduce and Complex Jobs"

More information

Scalable Linux Clusters with LVS

Scalable Linux Clusters with LVS Scalable Linux Clusters with LVS Considerations and Implementation, Part I Eric Searcy Tag1 Consulting, Inc. emsearcy@tag1consulting.com April 2008 Abstract Whether you are perusing mailing lists or reading

More information

Amazon Web Services Primer. William Strickland COP 6938 Fall 2012 University of Central Florida

Amazon Web Services Primer. William Strickland COP 6938 Fall 2012 University of Central Florida Amazon Web Services Primer William Strickland COP 6938 Fall 2012 University of Central Florida AWS Overview Amazon Web Services (AWS) is a collection of varying remote computing provided by Amazon.com.

More information

BlobSeer: Towards efficient data storage management on large-scale, distributed systems

BlobSeer: Towards efficient data storage management on large-scale, distributed systems : Towards efficient data storage management on large-scale, distributed systems Bogdan Nicolae University of Rennes 1, France KerData Team, INRIA Rennes Bretagne-Atlantique PhD Advisors: Gabriel Antoniu

More information

Matchmaking in the Cloud: Amazon EC2 and Apache Hadoop at eharmony

Matchmaking in the Cloud: Amazon EC2 and Apache Hadoop at eharmony Matchmaking in the Cloud: Amazon EC2 and Apache Hadoop at eharmony Speaker logo centered below image Steve Kuo, Software Architect Joshua Tuberville, Software Architect Goal > Leverage EC2 and Hadoop to

More information

Scaling Out With Apache Spark. DTL Meeting 17-04-2015 Slides based on https://www.sics.se/~amir/files/download/dic/spark.pdf

Scaling Out With Apache Spark. DTL Meeting 17-04-2015 Slides based on https://www.sics.se/~amir/files/download/dic/spark.pdf Scaling Out With Apache Spark DTL Meeting 17-04-2015 Slides based on https://www.sics.se/~amir/files/download/dic/spark.pdf Your hosts Mathijs Kattenberg Technical consultant Jeroen Schot Technical consultant

More information

Cloud Design and Implementation. Cheng Li MPI-SWS Nov 9 th, 2010

Cloud Design and Implementation. Cheng Li MPI-SWS Nov 9 th, 2010 Cloud Design and Implementation Cheng Li MPI-SWS Nov 9 th, 2010 1 Modern Computing CPU, Mem, Disk Academic computation Chemistry, Biology Large Data Set Analysis Online service Shopping Website Collaborative

More information

24/11/14. During this course. Internet is everywhere. Frequency barrier hit. Management costs increase. Advanced Distributed Systems Cloud Computing

24/11/14. During this course. Internet is everywhere. Frequency barrier hit. Management costs increase. Advanced Distributed Systems Cloud Computing Advanced Distributed Systems Cristian Klein Department of Computing Science Umeå University During this course Treads in IT Towards a new data center What is Cloud computing? Types of Clouds Making applications

More information

Architecting For Failure Why Cloud Architecture is Different! Michael Stiefel www.reliablesoftware.com development@reliablesoftware.

Architecting For Failure Why Cloud Architecture is Different! Michael Stiefel www.reliablesoftware.com development@reliablesoftware. Architecting For Failure Why Cloud Architecture is Different! Michael Stiefel www.reliablesoftware.com development@reliablesoftware.com Outsource Infrastructure? Traditional Web Application Web Site Virtual

More information

Data Centers and Cloud Computing

Data Centers and Cloud Computing Data Centers and Cloud Computing CS377 Guest Lecture Tian Guo 1 Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Case Study: Amazon EC2 2 Data Centers

More information

Chapter 4 Cloud Computing Applications and Paradigms. Cloud Computing: Theory and Practice. 1

Chapter 4 Cloud Computing Applications and Paradigms. Cloud Computing: Theory and Practice. 1 Chapter 4 Cloud Computing Applications and Paradigms Chapter 4 1 Contents Challenges for cloud computing. Existing cloud applications and new opportunities. Architectural styles for cloud applications.

More information

Jeffrey D. Ullman slides. MapReduce for data intensive computing

Jeffrey D. Ullman slides. MapReduce for data intensive computing Jeffrey D. Ullman slides MapReduce for data intensive computing Single-node architecture CPU Machine Learning, Statistics Memory Classical Data Mining Disk Commodity Clusters Web data sets can be very

More information

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

More information

Massive Cloud Auditing using Data Mining on Hadoop

Massive Cloud Auditing using Data Mining on Hadoop Massive Cloud Auditing using Data Mining on Hadoop Prof. Sachin Shetty CyberBAT Team, AFRL/RIGD AFRL VFRP Tennessee State University Outline Massive Cloud Auditing Traffic Characterization Distributed

More information

Facilitating Communal Data Sharing in Public Clouds

Facilitating Communal Data Sharing in Public Clouds Facilitating Communal Data Sharing in Public Clouds Roxana Geambasu Steve Gribble Hank Levy University of Washington Outline Vision: cloud as a platform for sharing code and data Why now: favorable cloud

More information

Managing large clusters resources

Managing large clusters resources Managing large clusters resources ID2210 Gautier Berthou (SICS) Big Processing with No Locality Job( /crawler/bot/jd.io/1 ) submi t Workflow Manager Compute Grid Node Job This doesn t scale. Bandwidth

More information

Ch. 4 - Topics of Discussion

Ch. 4 - Topics of Discussion CPET 581 Cloud Computing: Technologies and Enterprise IT Strategies Lecture 6 Cloud Platform Architecture over Virtualized Data Centers Part -4 Cloud Security and Trust Management Text Book: Distributed

More information

Implementing Microsoft Azure Infrastructure Solutions

Implementing Microsoft Azure Infrastructure Solutions Course Code: M20533 Vendor: Microsoft Course Overview Duration: 5 RRP: 2,025 Implementing Microsoft Azure Infrastructure Solutions Overview This course is aimed at experienced IT Professionals who currently

More information

Lecture 5: GFS & HDFS! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl

Lecture 5: GFS & HDFS! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl Big Data Processing, 2014/15 Lecture 5: GFS & HDFS!! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl 1 Course content Introduction Data streams 1 & 2 The MapReduce paradigm Looking behind

More information

VM-Series Firewall Deployment Tech Note PAN-OS 5.0

VM-Series Firewall Deployment Tech Note PAN-OS 5.0 VM-Series Firewall Deployment Tech Note PAN-OS 5.0 Revision A 2012, Palo Alto Networks, Inc. www.paloaltonetworks.com Contents Overview... 3 Supported Topologies... 3 Prerequisites... 4 Licensing... 5

More information

Hadoop & Spark Using Amazon EMR

Hadoop & Spark Using Amazon EMR Hadoop & Spark Using Amazon EMR Michael Hanisch, AWS Solutions Architecture 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda Why did we build Amazon EMR? What is Amazon EMR?

More information

Cloud Computing. Technologies and Types

Cloud Computing. Technologies and Types Cloud Computing Cloud Computing Technologies and Types Dell Zhang Birkbeck, University of London 2015/16 The Technological Underpinnings of Cloud Computing Data centres Virtualisation RESTful APIs Cloud

More information