Data Sharing in the Cloud: Scaling to the World, Unleashing Creativity, and Generating Value?

Size: px
Start display at page:

Download "Data Sharing in the Cloud: Scaling to the World, Unleashing Creativity, and Generating Value?"

Transcription

1 Data Sharing in the Cloud: Scaling to the World, Unleashing Creativity, and Generating Value? Marcos Vaz Salles Assistant Professor, University of Copenhagen (DIKU)

2 About the Speaker Marcos Vaz Salles Assistant Professor, University of Copenhagen (DIKU) Postdoc: Cornell University PhD: ETH Zurich Mission: Find creative ways to expand the reach of the 30+ years of top-level R&D invested in database technology, broadly defined Examples: Database techniques for search and integration, games, simulations, geospatial data 2

3 Where does your most important data live? 3

4 Where does your most important data live? DATABASES! 4

5 Historical Justification for Databases 5

6 Historical Justification for Databases Common applications Record maintenance, banking, government Complex implementation Concurrency, integrity, durability, storage, representation, Enough abstraction Operating systems virtualize low-level hardware Competing platforms No virtualization of platform: IBM, DEC, Data-Driven Applications Data Sharing (DBMS) Virtualization (Operating Systems) Platforms (Hardware) 6

7 Historical Justification for Databases Common applications Record maintenance, banking, government Complex implementation Concurrency, integrity, durability, storage, representation, Enough abstraction Operating systems virtualize low-level hardware Competing platforms No virtualization of platform: IBM, DEC, Data-Driven Applications But the Cloud today is completely different?! Data Sharing (DBMS) Virtualization (Operating Systems) Platforms (Hardware) 7

8 The Cloud Today Common applications Web Services, Data Warehousing, Big Data Complex implementation Data consistency and management, distribution, scalability, fault tolerance Enough abstraction Cloud IaaS virtualizes enormous clusters of machines Competing platforms No virtualization of platform: Amazon, Microsoft, Data-Driven Applications Data Sharing (????) Virtualization (Cloud IaaS) Platforms (Cloud Datacenter) 8

9 The Cloud Today Common applications Web Services, Data Warehousing, Big Data Complex implementation Data consistency and management, distribution, scalability, fault tolerance Enough abstraction Cloud IaaS virtualizes enormous clusters of machines Competing platforms No virtualization of platform: Amazon, Microsoft, Data-Driven Applications Challenge: What Data Sharing (????) should be the new Data Sharing Virtualization (Cloud IaaS) Abstraction in the Cloud? Platforms (Cloud Datacenter) 9

10 From Databases to Dataclouds While there were databases in the past, we will have dataclouds in the future Databases à Database Management System (DBMS) Dataclouds à Datacloud Management System (DCMS) Emerging application systems already being built! But at high cost And with less features than desired 10

11 Emerging Datacloud Application Systems Programmable news services Example: Guardian.co.uk Open Platform & MicroApps Programmable social networks Example: Apps on Facebook Programmable CRM Example: Salesforce Platform Far-fetched (?!) future Programmable government Programmable banking Programmable whoever-has-data 11

12 Emerging Datacloud Application Systems Programmable news services Example: Guardian.co.uk Open Platform & MicroApps Programmable social networks Example: Apps on Facebook Programmable CRM Example: Salesforce Platform Far-fetched (?!) future Programmable government Programmable banking Programmable whoever-has-data Data is a new means of production 12

13 Challenges in Dataclouds and DCMS Programming, programming, programming Resources, resources, resources Scale, scale, scale 13

14 Challenges in Dataclouds and DCMS Programming, programming, programming Re-use or create new programming abstractions? How to incorporate data into software engineering? Resources, resources, resources How to deal with virtualized environments and abstract cost? Scale, scale, scale How to scale applications to petabytes automatically? 14 Career Opportunity: DataCloud Administrator (DCA) J

15 ClouDiA: A Cloud Deployment Advisor Initial work on deployment of latency-sensitive data services in public clouds Simulation analytics, e.g., multi-agent simulations Search engines Key-value stores Acknowledgment: Joint work with Tao Zou, Ronan LeBras, Alan Demers, and Johannes Gehrke at Cornell University, to appear at VLDB

16 Latency-sensitive Data Services Distributed, latency-sensitive applications Goal: Time-to-solution Goal: Service response time Communication graph: captures interaction among application nodes 16 grid tree bipartite

17 Running Example: Fish Schools

18 Latency in the Cloud Some links have far worse latency than others Mean link latency is fairly stable over time Mean latency measurement in Amazon EC2 100 large instances, links, every hour, 10 days 18 TCP round-trip times of 1KB messages

19 Key Observations Observation #1: Avoid bad links Typical communication graph requires less links than complete graph Deploy application nodes to instances carefully Observation #2: Over-allocate to get better links Say communication graph has n nodes 19 Allocate, e.g., 1.1n instances Deploy and terminate extra 0.1n instances Why do we care? A) Improve response time B) Spend less money C) Get more bang for the buck

20 Node Deployment by Example Simulation analytics Tick-based, synchronization end of every tick in a grid Objective: Minimize worst link Costs: Communication Graph Instances Source: LeBras, Zou (partial) 20

21 Node Deployment by Example Simulation analytics Tick-based, synchronization end of every tick in a grid Objective: Minimize worst link Costs: Communication Graph Objective function value = Instances 9 Source: LeBras, Zou (partial)

22 Node Deployment by Example Simulation analytics Tick-based, synchronization end of every tick in a grid Objective: Minimize worst link Costs: Communication Graph Objective function value = Instances 9 Source: LeBras, Zou (partial)

23 Summary of Node Deployment Objectives Minimize cost of worst link Minimize cost of longest path Optimization Methods Akin to graph embedding problem, but with minimization goals Mixed-integer programming (MIP) formulation for both objectives Constraint programming (CP) formulation also for worst link Greedy easy to beat Network measurements Staged message exchange to measure costs More details on the paper! 23

24 Experiments with ClouDiA on Amazon EC2 Workloads & Setup Behavioral simulation 24 Fish simulation by Couzin et al., Nature 2D mesh 100 Amazon EC2 large instances Minimize Worst Link objective Synthetic aggregation workload Models search engines, distributed text databases Multi-level aggregation tree 50 Amazon EC2 large instances Minimize Longest Path objective Key-value store workload Bipartite graph of front-end servers and storage servers 100 Amazon EC2 large instances Minimize Worst Link objective used, but not perfect fit

25 Overall Improvement: All Workloads 15%-55% reduction of time Aggregation query largest improvement 25

26 Effect of Over-Allocation: Behavioral Simulation Default uses first 100 instances always Improvements with ClouDiA: 16% without 26 over-allocation, 38% with 50% extra instances

27 Wrap-up Dataclouds and DCMS Programming, programming, programming Resources, resources, resources Scale, scale, scale ClouDiA An initial step in resource optimization in public clouds Next steps: Collaborate with us to build a DCMS! Tons of research challenges open We are already collaborating with Danish Geodata Agency (GST) We are looking for partners J Thank you! 27

ClouDiA: A Deployment Advisor for Public Clouds

ClouDiA: A Deployment Advisor for Public Clouds ClouDiA: A Deployment Advisor for Public Clouds Tao Zou, Ronan Le Bras, Marcos Vaz Salles, Alan Demers, Johannes Gehrke Cornell University University of Copenhagen Ithaca, NY Copenhagen, Denmark {taozou,

More information

ClouDiA: a deployment advisor for public clouds. Tao Zou, Ronan Le Bras, Marcos Vaz Salles, Alan Demers & Johannes Gehrke

ClouDiA: a deployment advisor for public clouds. Tao Zou, Ronan Le Bras, Marcos Vaz Salles, Alan Demers & Johannes Gehrke ClouDiA: a deployment advisor for public clouds Tao Zou, Ronan Le Bras, Marcos Vaz Salles, Alan Demers & Johannes Gehrke The VLDB Journal The International Journal on Very Large Data Bases ISSN 66-8888

More information

OPTIMIZING RESPONSE TIME FOR DISTRIBUTED APPLICATIONS IN PUBLIC CLOUDS

OPTIMIZING RESPONSE TIME FOR DISTRIBUTED APPLICATIONS IN PUBLIC CLOUDS OPTIMIZING RESPONSE TIME FOR DISTRIBUTED APPLICATIONS IN PUBLIC CLOUDS A Dissertation Presented to the Faculty of the Graduate School of Cornell University in Partial Fulfillment of the Requirements for

More information

WOLKEN KOSTEN GELD GUSTAVO ALONSO SYSTEMS GROUP ETH ZURICH WWW.SYSTEMS.ETHZ.CH

WOLKEN KOSTEN GELD GUSTAVO ALONSO SYSTEMS GROUP ETH ZURICH WWW.SYSTEMS.ETHZ.CH WOLKEN KOSTEN GELD GUSTAVO ALONSO SYSTEMS GROUP ETH ZURICH WWW.SYSTEMS.ETHZ.CH ELCA Update June 16, 2010, Gustavo Alonso About the speaker Professor of Computer Science at ETH Zurich Areas of interest:

More information

ECE6130 Grid and Cloud Computing

ECE6130 Grid and Cloud Computing ECE6130 Grid and Cloud Computing Howie Huang Department of Electrical and Computer Engineering School of Engineering and Applied Science Cloud Computing Hardware Software Outline Research Challenges 2

More information

Multilevel Communication Aware Approach for Load Balancing

Multilevel Communication Aware Approach for Load Balancing Multilevel Communication Aware Approach for Load Balancing 1 Dipti Patel, 2 Ashil Patel Department of Information Technology, L.D. College of Engineering, Gujarat Technological University, Ahmedabad 1

More information

From Spark to Ignition:

From Spark to Ignition: From Spark to Ignition: Fueling Your Business on Real-Time Analytics Eric Frenkiel, MemSQL CEO June 29, 2015 San Francisco, CA What s in Store For This Presentation? 1. MemSQL: A real-time database for

More information

Divy Agrawal and Amr El Abbadi Department of Computer Science University of California at Santa Barbara

Divy Agrawal and Amr El Abbadi Department of Computer Science University of California at Santa Barbara Divy Agrawal and Amr El Abbadi Department of Computer Science University of California at Santa Barbara Sudipto Das (Microsoft summer intern) Shyam Antony (Microsoft now) Aaron Elmore (Amazon summer intern)

More information

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Alexandra Carpen-Amarie Diana Moise Bogdan Nicolae KerData Team, INRIA Outline

More information

JOURNAL OF OBJECT TECHNOLOGY

JOURNAL OF OBJECT TECHNOLOGY JOURNAL OF OBJECT TECHNOLOGY Online at http://www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2009 Vol. 8, No. 3, May-June 2009 Cloud Computing Benefits and Challenges! Dave Thomas

More information

Cloud Computing and Internet Services. Wei-Ying Ma ( 马 维 英 博 士 ) Principal Researcher, Research Area Manager Microsoft Research Asia

Cloud Computing and Internet Services. Wei-Ying Ma ( 马 维 英 博 士 ) Principal Researcher, Research Area Manager Microsoft Research Asia Cloud Computing and Internet Services Wei-Ying Ma ( 马 维 英 博 士 ) Principal Researcher, Research Area Manager Microsoft Research Asia Computing as Utility Grid Computing Web Services in the Cloud What is

More information

Cloud Computing Technology

Cloud Computing Technology Cloud Computing Technology The Architecture Overview Danairat T. Certified Java Programmer, TOGAF Silver danairat@gmail.com, +66-81-559-1446 1 Agenda What is Cloud Computing? Case Study Service Model Architectures

More information

Lecture 26 Enterprise Internet Computing 1. Enterprise computing 2. Enterprise Internet computing 3. Natures of enterprise computing 4.

Lecture 26 Enterprise Internet Computing 1. Enterprise computing 2. Enterprise Internet computing 3. Natures of enterprise computing 4. Lecture 26 Enterprise Internet Computing 1. Enterprise computing 2. Enterprise Internet computing 3. Natures of enterprise computing 4. Platforms High end solutions Microsoft.Net Java technology 1 Enterprise

More information

Hadoop. MPDL-Frühstück 9. Dezember 2013 MPDL INTERN

Hadoop. MPDL-Frühstück 9. Dezember 2013 MPDL INTERN Hadoop MPDL-Frühstück 9. Dezember 2013 MPDL INTERN Understanding Hadoop Understanding Hadoop What's Hadoop about? Apache Hadoop project (started 2008) downloadable open-source software library (current

More information

Cloud Computing. Chapter 1 Introducing Cloud Computing

Cloud Computing. Chapter 1 Introducing Cloud Computing Cloud Computing Chapter 1 Introducing Cloud Computing Learning Objectives Understand the abstract nature of cloud computing. Describe evolutionary factors of computing that led to the cloud. Describe virtualization

More information

There Are Clouds In Your Future. Jeff Barr Amazon Web Services jbarr@amazon.com @jeffbarr (Twitter)

There Are Clouds In Your Future. Jeff Barr Amazon Web Services jbarr@amazon.com @jeffbarr (Twitter) There Are Clouds In Your Future Jeff Barr Amazon Web Services jbarr@amazon.com @jeffbarr (Twitter) My Goals For This Talk Introduce you to cloud computing Show you what others are already doing Alert you

More information

bigdata Managing Scale in Ontological Systems

bigdata Managing Scale in Ontological Systems Managing Scale in Ontological Systems 1 This presentation offers a brief look scale in ontological (semantic) systems, tradeoffs in expressivity and data scale, and both information and systems architectural

More information

Scientific and Technical Applications as a Service in the Cloud

Scientific and Technical Applications as a Service in the Cloud Scientific and Technical Applications as a Service in the Cloud University of Bern, 28.11.2011 adapted version Wibke Sudholt CloudBroker GmbH Technoparkstrasse 1, CH-8005 Zurich, Switzerland Phone: +41

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

Dr Markus Hagenbuchner markus@uow.edu.au CSCI319. Introduction to Cloud Computing

Dr Markus Hagenbuchner markus@uow.edu.au CSCI319. Introduction to Cloud Computing Dr Markus Hagenbuchner markus@uow.edu.au CSCI319 Introduction to Cloud Computing CSCI319 Chapter 1 Page: 1 of 10 Content and Objectives 1. Introduce to cloud computing 2. Develop and understanding to how

More information

Alfresco Enterprise on AWS: Reference Architecture

Alfresco Enterprise on AWS: Reference Architecture Alfresco Enterprise on AWS: Reference Architecture October 2013 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 13 Abstract Amazon Web Services (AWS)

More information

Apache Hadoop. Alexandru Costan

Apache Hadoop. Alexandru Costan 1 Apache Hadoop Alexandru Costan Big Data Landscape No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard, except Hadoop 2 Outline What is Hadoop? Who uses it? Architecture HDFS MapReduce Open

More information

Building Out Your Cloud-Ready Solutions. Clark D. Richey, Jr., Principal Technologist, DoD

Building Out Your Cloud-Ready Solutions. Clark D. Richey, Jr., Principal Technologist, DoD Building Out Your Cloud-Ready Solutions Clark D. Richey, Jr., Principal Technologist, DoD Slide 1 Agenda Define the problem Explore important aspects of Cloud deployments Wrap up and questions Slide 2

More information

Networking in the Hadoop Cluster

Networking in the Hadoop Cluster Hadoop and other distributed systems are increasingly the solution of choice for next generation data volumes. A high capacity, any to any, easily manageable networking layer is critical for peak Hadoop

More information

Windows Azure and private cloud

Windows Azure and private cloud Windows Azure and private cloud Joe Chou Senior Program Manager China Cloud Innovation Center Customer Advisory Team Microsoft Asia-Pacific Research and Development Group 1 Agenda Cloud Computing Fundamentals

More information

Using Cloud Services for Test Environments A case study of the use of Amazon EC2

Using Cloud Services for Test Environments A case study of the use of Amazon EC2 Using Cloud Services for Test Environments A case study of the use of Amazon EC2 Lee Hawkins (Quality Architect) Quest Software, Melbourne Copyright 2010 Quest Software We are gathered here today to talk

More information

Cloud Computing and the Data Centre of the Future

Cloud Computing and the Data Centre of the Future Cloud Computing and the Data Centre of the Future Martin Hingley, CEO, ITCandor CLOUD COMPUTING SUMMIT 2014 MILAN Clouds in the Eye of the Beholder Maximise Value Enforce SLAs Manage Suppliers Cap Ex v

More information

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

More information

A Cost-Evaluation of MapReduce Applications in the Cloud

A Cost-Evaluation of MapReduce Applications in the Cloud 1/23 A Cost-Evaluation of MapReduce Applications in the Cloud Diana Moise, Alexandra Carpen-Amarie Gabriel Antoniu, Luc Bougé KerData team 2/23 1 MapReduce applications - case study 2 3 4 5 3/23 MapReduce

More information

Session 1: IT Infrastructure Security Vertica / Hadoop Integration and Analytic Capabilities for Federal Big Data Challenges

Session 1: IT Infrastructure Security Vertica / Hadoop Integration and Analytic Capabilities for Federal Big Data Challenges Session 1: IT Infrastructure Security Vertica / Hadoop Integration and Analytic Capabilities for Federal Big Data Challenges James Campbell Corporate Systems Engineer HP Vertica jcampbell@vertica.com Big

More information

Data Management in the Cloud. Zhen Shi

Data Management in the Cloud. Zhen Shi Data Management in the Cloud Zhen Shi Overview Introduction 3 characteristics of cloud computing 2 types of cloud data management application 2 types of cloud data management architecture Conclusion Introduction

More information

IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures

IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Introduction

More information

The Sierra Clustered Database Engine, the technology at the heart of

The Sierra Clustered Database Engine, the technology at the heart of A New Approach: Clustrix Sierra Database Engine The Sierra Clustered Database Engine, the technology at the heart of the Clustrix solution, is a shared-nothing environment that includes the Sierra Parallel

More information

Challenges for Data Driven Systems

Challenges for Data Driven Systems Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Quick History of Data Management 4000 B C Manual recording From tablets to papyrus to paper A. Payberah 2014 2

More information

Domain driven design, NoSQL and multi-model databases

Domain driven design, NoSQL and multi-model databases Domain driven design, NoSQL and multi-model databases Java Meetup New York, 10 November 2014 Max Neunhöffer www.arangodb.com Max Neunhöffer I am a mathematician Earlier life : Research in Computer Algebra

More information

Chapter 7. Using Hadoop Cluster and MapReduce

Chapter 7. Using Hadoop Cluster and MapReduce Chapter 7 Using Hadoop Cluster and MapReduce Modeling and Prototyping of RMS for QoS Oriented Grid Page 152 7. Using Hadoop Cluster and MapReduce for Big Data Problems The size of the databases used in

More information

Cloud Computing Services and its Application

Cloud Computing Services and its Application Advance in Electronic and Electric Engineering. ISSN 2231-1297, Volume 4, Number 1 (2014), pp. 107-112 Research India Publications http://www.ripublication.com/aeee.htm Cloud Computing Services and its

More information

CLOUD COMPUTING. When It's smarter to rent than to buy

CLOUD COMPUTING. When It's smarter to rent than to buy CLOUD COMPUTING When It's smarter to rent than to buy Is it new concept? Nothing new In 1990 s, WWW itself Grid Technologies- Scientific applications Online banking websites More convenience Not to visit

More information

Introduction to Cloud Computing

Introduction to Cloud Computing Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services

More information

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world Analytics March 2015 White paper Why NoSQL? Your database options in the new non-relational world 2 Why NoSQL? Contents 2 New types of apps are generating new types of data 2 A brief history of NoSQL 3

More information

Advanced Computer Networks. Scheduling

Advanced Computer Networks. Scheduling Oriana Riva, Department of Computer Science ETH Zürich Advanced Computer Networks 263-3501-00 Scheduling Patrick Stuedi, Qin Yin and Timothy Roscoe Spring Semester 2015 Outline Last time Load balancing

More information

CLOUD COMPUTING USING HADOOP TECHNOLOGY

CLOUD COMPUTING USING HADOOP TECHNOLOGY CLOUD COMPUTING USING HADOOP TECHNOLOGY DHIRAJLAL GANDHI COLLEGE OF TECHNOLOGY SALEM B.NARENDRA PRASATH S.PRAVEEN KUMAR 3 rd year CSE Department, 3 rd year CSE Department, Email:narendren.jbk@gmail.com

More information

Report Data Management in the Cloud: Limitations and Opportunities

Report Data Management in the Cloud: Limitations and Opportunities Report Data Management in the Cloud: Limitations and Opportunities Article by Daniel J. Abadi [1] Report by Lukas Probst January 4, 2013 In this report I want to summarize Daniel J. Abadi's article [1]

More information

Keywords Cloud computing, virtual machines, migration approach, deployment modeling

Keywords Cloud computing, virtual machines, migration approach, deployment modeling Volume 3, Issue 8, August 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Effective Scheduling

More information

IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud

IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud February 25, 2014 1 Agenda v Mapping clients needs to cloud technologies v Addressing your pain

More information

Are You Ready for Big Data?

Are You Ready for Big Data? Are You Ready for Big Data? Jim Gallo National Director, Business Analytics February 11, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?

More information

PaaS Cloud Migration Migration Process, Architecture Problems and Solutions. Claus Pahl and Huanhuan Xiong

PaaS Cloud Migration Migration Process, Architecture Problems and Solutions. Claus Pahl and Huanhuan Xiong PaaS Cloud Migration Migration Process, Architecture Problems and Solutions Claus Pahl and Huanhuan Xiong Cloud Migration Motivation HOW TO MIGRATE TO CLOUD IaaS PaaS SaaS Cloud Migration Definition A

More information

Cloud Platforms Today: The Big Picture

Cloud Platforms Today: The Big Picture Cloud Platforms Today: The Big Picture David Chappell Chappell & Associates www.davidchappell.com Mobile Workforce Big Data Cloud Computing Social Enterprise Privacy and Security The traditional world

More information

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms Volume 1, Issue 1 ISSN: 2320-5288 International Journal of Engineering Technology & Management Research Journal homepage: www.ijetmr.org Analysis and Research of Cloud Computing System to Comparison of

More information

Distributed Data Stores

Distributed Data Stores Distributed Data Stores 1 Distributed Persistent State MapReduce addresses distributed processing of aggregation-based queries Persistent state across a large number of machines? Distributed DBMS High

More information

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 1 Hadoop: A Framework for Data- Intensive Distributed Computing CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 2 What is Hadoop? Hadoop is a software framework for distributed processing of large datasets

More information

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING Carlos de Alfonso Andrés García Vicente Hernández 2 INDEX Introduction Our approach Platform design Storage Security

More information

yvette@yvetteagostini.it yvette@yvetteagostini.it

yvette@yvetteagostini.it yvette@yvetteagostini.it 1 The following is merely a collection of notes taken during works, study and just-for-fun activities No copyright infringements intended: all sources are duly listed at the end of the document This work

More information

International Journal of Engineering Research & Management Technology

International Journal of Engineering Research & Management Technology International Journal of Engineering Research & Management Technology March- 2015 Volume 2, Issue-2 Survey paper on cloud computing with load balancing policy Anant Gaur, Kush Garg Department of CSE SRM

More information

A1 and FARM scalable graph database on top of a transactional memory layer

A1 and FARM scalable graph database on top of a transactional memory layer A1 and FARM scalable graph database on top of a transactional memory layer Miguel Castro, Aleksandar Dragojević, Dushyanth Narayanan, Ed Nightingale, Alex Shamis Richie Khanna, Matt Renzelmann Chiranjeeb

More information

QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering

QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering June 2014 Page 1 Contents Introduction... 3 About Amazon Web Services (AWS)... 3 About Amazon Redshift... 3 QlikView on AWS...

More information

Alfresco Enterprise on Azure: Reference Architecture. September 2014

Alfresco Enterprise on Azure: Reference Architecture. September 2014 Alfresco Enterprise on Azure: Reference Architecture Page 1 of 14 Abstract Microsoft Azure provides a set of services for deploying critical enterprise workloads on its highly reliable cloud platform.

More information

A programming model in Cloud: MapReduce

A programming model in Cloud: MapReduce A programming model in Cloud: MapReduce Programming model and implementation developed by Google for processing large data sets Users specify a map function to generate a set of intermediate key/value

More information

2) Xen Hypervisor 3) UEC

2) Xen Hypervisor 3) UEC 5. Implementation Implementation of the trust model requires first preparing a test bed. It is a cloud computing environment that is required as the first step towards the implementation. Various tools

More information

ANALYSIS OF CLOUD VENDORS IN INDIAN ENVIORNMENT

ANALYSIS OF CLOUD VENDORS IN INDIAN ENVIORNMENT ANALYSIS OF CLOUD VENDORS IN INDIAN ENVIORNMENT Mrs. Jeena Thomas Asst. Professor, Department of Computer Science St.Joseph s College of Engineering & Technology, Palai, Kerala,(India) ABSTRACT Grid Computing

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

CLOUD COMPUTING Y SU IMPACTO EN LA INFORMATICA

CLOUD COMPUTING Y SU IMPACTO EN LA INFORMATICA CLOUD COMPUTING Y SU IMPACTO EN LA INFORMATICA Gustavo Alonso Systems Group Department of Computer Science ETH Zurich, Switzerland www.systems.ethz.ch JISBD - 2010 1 Background ETH Zürich Systems Group

More information

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1 Why NoSQL? Your database options in the new non- relational world 2015 IBM Cloudant 1 Table of Contents New types of apps are generating new types of data... 3 A brief history on NoSQL... 3 NoSQL s roots

More information

INTRODUCTION TO CLOUD COMPUTING

INTRODUCTION TO CLOUD COMPUTING INTRODUCTION TO CLOUD COMPUTING EXISTING PROBLEMS Application Platform Hardware CONTENTS What is cloud computing Key technologies enabling cloud computing Hardware Internet technologies Distributed computing

More information

Cloud Computing. Key Considerations for Adoption. Abstract. Ramkumar Dargha

Cloud Computing. Key Considerations for Adoption. Abstract. Ramkumar Dargha Cloud Computing Key Considerations for Adoption Ramkumar Dargha Abstract Cloud Computing technology and services have been witnessing quite a lot of attention for the past couple of years now. We believe

More information

Lecture Data Warehouse Systems

Lecture Data Warehouse Systems Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART C: Novel Approaches in DW NoSQL and MapReduce Stonebraker on Data Warehouses Star and snowflake schemas are a good idea in the DW world C-Stores

More information

Data Consistency on Private Cloud Storage System

Data Consistency on Private Cloud Storage System Volume, Issue, May-June 202 ISS 2278-6856 Data Consistency on Private Cloud Storage System Yin yein Aye University of Computer Studies,Yangon yinnyeinaye.ptn@email.com Abstract: Cloud computing paradigm

More information

Microsoft Private Cloud

Microsoft Private Cloud Microsoft Private Cloud Lorenz Wolf, Solution Specialist Datacenter, Microsoft SoftwareOne @ Au Premier Zürich - 22.03.2011 What is PRIVATE CLOUD Private Public Public Cloud Private Cloud shared resources.

More information

Mining Large Datasets: Case of Mining Graph Data in the Cloud

Mining Large Datasets: Case of Mining Graph Data in the Cloud Mining Large Datasets: Case of Mining Graph Data in the Cloud Sabeur Aridhi PhD in Computer Science with Laurent d Orazio, Mondher Maddouri and Engelbert Mephu Nguifo 16/05/2014 Sabeur Aridhi Mining Large

More information

Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at

Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at distributing load b. QUESTION: What is the context? i. How

More information

Cloud Computing: Meet the Players. Performance Analysis of Cloud Providers

Cloud Computing: Meet the Players. Performance Analysis of Cloud Providers BASEL UNIVERSITY COMPUTER SCIENCE DEPARTMENT Cloud Computing: Meet the Players. Performance Analysis of Cloud Providers Distributed Information Systems (CS341/HS2010) Report based on D.Kassman, T.Kraska,

More information

BASHO DATA PLATFORM SIMPLIFIES BIG DATA, IOT, AND HYBRID CLOUD APPS

BASHO DATA PLATFORM SIMPLIFIES BIG DATA, IOT, AND HYBRID CLOUD APPS WHITEPAPER BASHO DATA PLATFORM BASHO DATA PLATFORM SIMPLIFIES BIG DATA, IOT, AND HYBRID CLOUD APPS INTRODUCTION Big Data applications and the Internet of Things (IoT) are changing and often improving our

More information

An Introduction to Private Cloud

An Introduction to Private Cloud An Introduction to Private Cloud As the word cloud computing becomes more ubiquitous these days, several questions can be raised ranging from basic question like the definitions of a cloud and cloud computing

More information

High Availability with Postgres Plus Advanced Server. An EnterpriseDB White Paper

High Availability with Postgres Plus Advanced Server. An EnterpriseDB White Paper High Availability with Postgres Plus Advanced Server An EnterpriseDB White Paper For DBAs, Database Architects & IT Directors December 2013 Table of Contents Introduction 3 Active/Passive Clustering 4

More information

Distributed System Principles

Distributed System Principles Distributed System Principles 1 What is a Distributed System? Definition: A distributed system consists of a collection of autonomous computers, connected through a network and distribution middleware,

More information

ECHO: Recreating Network Traffic Maps for Datacenters with Tens of Thousands of Servers

ECHO: Recreating Network Traffic Maps for Datacenters with Tens of Thousands of Servers ECHO: Recreating Network Traffic Maps for Datacenters with Tens of Thousands of Servers Christina Delimitrou 1, Sriram Sankar 2, Aman Kansal 3, Christos Kozyrakis 1 1 Stanford University 2 Microsoft 3

More information

Software as a Service (SaaS) and Platform as a Service (PaaS) (ENCS 691K Chapter 1)

Software as a Service (SaaS) and Platform as a Service (PaaS) (ENCS 691K Chapter 1) Roch Glitho, PhD Software as a Service (SaaS) and Platform as a Service (PaaS) (ENCS 691K Chapter 1) Associate Professor and Canada Research Chair My URL - http://users.encs.concordia.ca/~glitho/ Software

More information

Ø Teaching Evaluations. q Open March 3 through 16. Ø Final Exam. q Thursday, March 19, 4-7PM. Ø 2 flavors: q Public Cloud, available to public

Ø Teaching Evaluations. q Open March 3 through 16. Ø Final Exam. q Thursday, March 19, 4-7PM. Ø 2 flavors: q Public Cloud, available to public Announcements TIM 50 Teaching Evaluations Open March 3 through 16 Final Exam Thursday, March 19, 4-7PM Lecture 19 20 March 12, 2015 Cloud Computing Cloud Computing: refers to both applications delivered

More information

Big Data Analytics. Chances and Challenges. Volker Markl

Big Data Analytics. Chances and Challenges. Volker Markl Volker Markl Professor and Chair Database Systems and Information Management (DIMA), Technische Universität Berlin www.dima.tu-berlin.de Big Data Analytics Chances and Challenges Volker Markl DIMA BDOD

More information

Real Time Big Data Processing

Real Time Big Data Processing Real Time Big Data Processing Cloud Expo 2014 Ian Meyers Amazon Web Services Global Infrastructure Deployment & Administration App Services Analytics Compute Storage Database Networking AWS Global Infrastructure

More information

Amazon EC2 XenApp Scalability Analysis

Amazon EC2 XenApp Scalability Analysis WHITE PAPER Citrix XenApp Amazon EC2 XenApp Scalability Analysis www.citrix.com Table of Contents Introduction...3 Results Summary...3 Detailed Results...4 Methods of Determining Results...4 Amazon EC2

More information

BIG DATA SOLUTION DATA SHEET

BIG DATA SOLUTION DATA SHEET BIG DATA SOLUTION DATA SHEET Highlight. DATA SHEET HGrid247 BIG DATA SOLUTION Exploring your BIG DATA, get some deeper insight. It is possible! Another approach to access your BIG DATA with the latest

More information

Can the Elephants Handle the NoSQL Onslaught?

Can the Elephants Handle the NoSQL Onslaught? Can the Elephants Handle the NoSQL Onslaught? Avrilia Floratou, Nikhil Teletia David J. DeWitt, Jignesh M. Patel, Donghui Zhang University of Wisconsin-Madison Microsoft Jim Gray Systems Lab Presented

More information

Webpage: www.ijaret.org Volume 3, Issue XI, Nov. 2015 ISSN 2320-6802

Webpage: www.ijaret.org Volume 3, Issue XI, Nov. 2015 ISSN 2320-6802 An Effective VM scheduling using Hybrid Throttled algorithm for handling resource starvation in Heterogeneous Cloud Environment Er. Navdeep Kaur 1 Er. Pooja Nagpal 2 Dr.Vinay Guatum 3 1 M.Tech Student,

More information

Overview of Cloud Computing (ENCS 691K Chapter 1)

Overview of Cloud Computing (ENCS 691K Chapter 1) Overview of Cloud Computing (ENCS 691K Chapter 1) Roch Glitho, PhD Associate Professor and Canada Research Chair My URL - http://users.encs.concordia.ca/~glitho/ Overview of Cloud Computing Towards a definition

More information

Cloud Computing and the Future of Internet Services. Wei-Ying Ma Principal Researcher, Research Area Manager Microsoft Research Asia

Cloud Computing and the Future of Internet Services. Wei-Ying Ma Principal Researcher, Research Area Manager Microsoft Research Asia Cloud Computing and the Future of Internet Services Wei-Ying Ma Principal Researcher, Research Area Manager Microsoft Research Asia Computing as Utility Grid Computing Web Services in the Cloud What is

More information

EWeb: Highly Scalable Client Transparent Fault Tolerant System for Cloud based Web Applications

EWeb: Highly Scalable Client Transparent Fault Tolerant System for Cloud based Web Applications ECE6102 Dependable Distribute Systems, Fall2010 EWeb: Highly Scalable Client Transparent Fault Tolerant System for Cloud based Web Applications Deepal Jayasinghe, Hyojun Kim, Mohammad M. Hossain, Ali Payani

More information

Cloud Computing. Chapter 1 Introducing Cloud Computing

Cloud Computing. Chapter 1 Introducing Cloud Computing Cloud Computing Chapter 1 Introducing Cloud Computing Learning Objectives Understand the abstract nature of cloud computing. Describe evolutionary factors of computing that led to the cloud. Describe virtualization

More information

Scaling Database Performance in Azure

Scaling Database Performance in Azure Scaling Database Performance in Azure Results of Microsoft-funded Testing Q1 2015 2015 2014 ScaleArc. All Rights Reserved. 1 Test Goals and Background Info Test Goals and Setup Test goals Microsoft commissioned

More information

Cloud Computing and Advanced Relationship Analytics

Cloud Computing and Advanced Relationship Analytics Cloud Computing and Advanced Relationship Analytics Using Objectivity/DB to Discover the Relationships in your Data By Brian Clark Vice President, Product Management Objectivity, Inc. 408 992 7136 brian.clark@objectivity.com

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda 1 Outline Build a cost-efficient Swift cluster with expected performance Background & Problem Solution Experiments

More information

CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level. -ORACLE TIMESTEN 11gR1

CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level. -ORACLE TIMESTEN 11gR1 CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level -ORACLE TIMESTEN 11gR1 CASE STUDY Oracle TimesTen In-Memory Database and Shared Disk HA Implementation

More information

Capacity Management for Cloud Computing

Capacity Management for Cloud Computing Capacity Management for Cloud Computing Chris Molloy Distinguished Engineer Member, IBM Academy of Technology October 2009 1 Is a cloud like touching an elephant? 2 Gartner defines cloud computing as a

More information

Big Data and Industrial Internet

Big Data and Industrial Internet Big Data and Industrial Internet Keijo Heljanko Department of Computer Science and Helsinki Institute for Information Technology HIIT School of Science, Aalto University keijo.heljanko@aalto.fi 16.6-2015

More information

High Performance Cluster Support for NLB on Window

High Performance Cluster Support for NLB on Window High Performance Cluster Support for NLB on Window [1]Arvind Rathi, [2] Kirti, [3] Neelam [1]M.Tech Student, Department of CSE, GITM, Gurgaon Haryana (India) arvindrathi88@gmail.com [2]Asst. Professor,

More information

How to Do/Evaluate Cloud Computing Research. Young Choon Lee

How to Do/Evaluate Cloud Computing Research. Young Choon Lee How to Do/Evaluate Cloud Computing Research Young Choon Lee Cloud Computing Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing

More information

Brian Amedro CTO. Worldwide Customers

Brian Amedro CTO. Worldwide Customers Denis Caromel CEO Brian Amedro CTO Cloud Enterprise Applications (B2B) Reduce Costs (IT) + Reduce Pains (Time) Worldwide Customers 1 1 Software company born of INRIA in 2007 Software Editor, Open Source

More information