Big Data & Cloud. 4 th European Summit on the Future Internet. António Miguel Ferreira, CEO, Lunacloud. Aveiro, 13 to 14th June 2013



Similar documents
Benchmarking Couchbase Server for Interactive Applications. By Alexey Diomin and Kirill Grigorchuk

Hadoop on Windows Azure: Hive vs. JavaScript for Processing Big Data

Managed Services for the Cloud Foundry PaaS

Using Cloud Services for Test Environments A case study of the use of Amazon EC2

STeP-IN SUMMIT June 2014 at Bangalore, Hyderabad, Pune - INDIA. Performance testing Hadoop based big data analytics solutions

The Inside Scoop on Hadoop

On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform

Cloud Computing and Amazon Web Services

BIG DATA USING HADOOP

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)

Scalable Architecture on Amazon AWS Cloud

Accelerating Big Data: Using SanDisk SSDs for MongoDB Workloads

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

Applied Storage Performance For Big Analytics. PRESENTATION TITLE GOES HERE Hubbert Smith LSI

Cloud Computing: How to Solve Challenges and Avoid Typical Mistakes Using Amazon Web Services

Yahoo! Cloud Serving Benchmark

VMware for your hosting services

Big Data Analytics for Cyber

The Future of Data Management

Virtualizing Apache Hadoop. June, 2012

CA Technologies Big Data Infrastructure Management Unified Management and Visibility of Big Data

Use of Hadoop File System for Nuclear Physics Analyses in STAR

Microsoft Azure Cloud oplossing als een extensie op mijn datacenter? Frederik Baert Solution Advisor

Hadoop in the Hybrid Cloud

Can Flash help you ride the Big Data Wave? Steve Fingerhut Vice President, Marketing Enterprise Storage Solutions Corporation

MapReduce with Apache Hadoop Analysing Big Data

Ali Eghlima Ph.D Director of Bioinformatics. A Bioinformatics Research & Consulting Group

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Cloud Connectivity Offense & defense

Cloud Panel Service Evaluation Scenarios

How To Scale Out Of A Nosql Database

Cloud Platforms in the Enterprise

NoSQL Performance Test In-Memory Performance Comparison of SequoiaDB, Cassandra, and MongoDB

Architecting for the next generation of Big Data Hortonworks HDP 2.0 on Red Hat Enterprise Linux 6 with OpenJDK 7

Apache Spark and the future of big data applica5ons. Eric Baldeschwieler

How To Choose Cloud Computing

NoSQL: Going Beyond Structured Data and RDBMS

Big Data and Industrial Internet

#mstrworld. Tapping into Hadoop and NoSQL Data Sources in MicroStrategy. Presented by: Trishla Maru. #mstrworld

Big Data and Data Science: Behind the Buzz Words

Maximizing Hadoop Performance and Storage Capacity with AltraHD TM

Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software

Introduction to Cloud Computing

Introducing EEMBC Cloud and Big Data Server Benchmarks

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 14

Cloud Application Development (SE808, School of Software, Sun Yat-Sen University) Yabo (Arber) Xu

Alternative Deployment Models for Cloud Computing in HPC Applications. Society of HPC Professionals November 9, 2011 Steve Hebert, Nimbix

Ubuntu and Hadoop: the perfect match

Overview. The Cloud. Characteristics and usage of the cloud Realities and risks of the cloud

CSE-E5430 Scalable Cloud Computing Lecture 2

Benchmarking Sahara-based Big-Data-as-a-Service Solutions. Zhidong Yu, Weiting Chen (Intel) Matthew Farrellee (Red Hat) May 2015

LARGE-SCALE DATA STORAGE APPLICATIONS

Accelerating Enterprise Big Data Success. Tim Stevens, VP of Business and Corporate Development Cloudera

Certified Cloud Computing Professional VS-1067

Application Development. A Paradigm Shift

Why NoSQL? Your database options in the new non- relational world IBM Cloudant 1

Benchmarking and Analysis of NoSQL Technologies

Cloud Courses Description

Datenverwaltung im Wandel - Building an Enterprise Data Hub with

The Future of Data Management with Hadoop and the Enterprise Data Hub

Open Source Technologies on Microsoft Azure

Dell Reference Configuration for Hortonworks Data Platform

STORAGE AS. A SERVICE (STaaS) ELASTIC CLOUD STORAGE. global.de/cloud-storage MADE IN GERMANY

Extending Hadoop beyond MapReduce

IBM Software Hadoop in the cloud

LARGE, DISTRIBUTED COMPUTING INFRASTRUCTURES OPPORTUNITIES & CHALLENGES. Dominique A. Heger Ph.D. DHTechnologies, Data Nubes Austin, TX, USA

PaaS - Platform as a Service Google App Engine

Why Private Cloud? Nenad BUNCIC VPSI 29-JUNE-2015 EPFL, SI-EXHEB

WINDOWS AZURE DATA MANAGEMENT

Hadoop IST 734 SS CHUNG

The Total Cost of (Non) Ownership of a NoSQL Database Cloud Service

Microsoft Research Windows Azure for Research Training

Open source large scale distributed data management with Google s MapReduce and Bigtable

QUEST meeting Big Data Analytics

Big Data Streams. Analytics Challenges, Analysis, and Applications. Adel M. Alimi

CS / Cloud Computing

Cloud Computing: Making the right choices

Can the Elephants Handle the NoSQL Onslaught?

Microsoft Research Microsoft Azure for Research Training

Enterprise Operational SQL on Hadoop Trafodion Overview

Hadoop. for Oracle database professionals. Alex Gorbachev Calgary, AB September 2013

Tapping Into Hadoop and NoSQL Data Sources with MicroStrategy. Presented by: Jeffrey Zhang and Trishla Maru

APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS

INTRODUCTION TO CASSANDRA

Elastic Cloud Computing in the Open Cirrus Testbed implemented via Eucalyptus

Viswanath Nandigam Sriram Krishnan Chaitan Baru

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum

Sujee Maniyam, ElephantScale

Introduction to Cloud Computing

CSE-E5430 Scalable Cloud Computing Lecture 7

CLOUD STORAGE USING HADOOP AND PLAY

IBM Spectrum Protect in the Cloud

Proact whitepaper on Big Data

Session 4 Cloud computing for future ICT Knowledge platforms

Where We Are. References. Cloud Computing. Levels of Service. Cloud Computing History. Introduction to Data Management CSE 344

NoSQL Data Base Basics

Cloud Computing and Amazon Web Services. CJUG March, 2009 Tom Malaher

Data Integration Checklist

Cloud Database Demystified to Deliver SaaS Customer Value

Investigation of Cloud Computing: Applications and Challenges

Transcription:

Big Data & Cloud 4 th European Summit on the Future Internet António Miguel Ferreira, CEO, Lunacloud Aveiro, 13 to 14th June 2013

? About

Lunacloud is a cloud infrastructure and platform services provider (IaaS + PaaS), with datacenters in the UK, Portugal, France and Russia (July 2013).

1. The Cloud is a more efficient way of using IT resources. 2. High-end compute & storage resources are now widely available through the global Internet. 3. A whole new set of challenges may be addressed quickly and with lower costs.

Cloud use case 1 IT research

Altoros Product engineering in areas such as implementation of NoSQL and NewSQL storage systems, Hadoop distributed computing, etc. Offices in Silicon Valley (Sunnyvale, California), Norway, Denmark, Switzerland, UK, Eastern Europe (Minsk, Belarus) and South America (Buenos Aires, Santa Fe, Argentina). Challenge #1: NoSQL benchmarking Testing of different NoSQL databases against various types of workloads: Cassandra, MongoDB, Riak, Couchbase, MySQL Cluster, and Hbase. Yahoo! Cloud Serving Benchmark (YCSB) used to evaluate performance. Infrastructure needs 120 virtual machines with a total of 960 GB RAM + 920 CPU Cores + 12 TB local storage... For only 15 days! Traditional IT capex = 100.000 Cloud IaaS cost = 8.000

Altoros Product engineering in areas such as implementation of NoSQL and NewSQL storage systems, Hadoop distributed computing, etc. Offices in Silicon Valley (Sunnyvale, California), Norway, Denmark, Switzerland, UK, Eastern Europe (Minsk, Belarus) and South America (Buenos Aires, Santa Fe, Argentina). Challenge #2: Hadoop benchmarking Testing different Hadoop distribution packages to assess their performance, ease of use, functionality, etc: Apache Hadoop; CDH, Cloudera's Distribution, including Apache Hadoop; HDP, Hortonworks Data Platform; MapR M3 Edition; Intel Hadoop; Pivotal HD. TeraSort benchmark to measure performance. Infrastructure needs 250 virtual machines with a total of 2000 GB RAM + 2000 CPU Cores + 25TB local storage... For only 25 days! Traditional IT capex = 200.000 Cloud IaaS cost = 28.000

Concepts Use case 1: 1.Immediate availability 2.Elasticity 3.Pay per use

Cloud use case 2 Entertainment

Music Stage Social network for independent bands and musicians. Challenge: Multimedia storage 10 s or 100 s of thousands of bands and artists want to upload 10 s or 100 s of musics and videos, each with 4 to 30 MB in size. Need for unlimited storage space with no upfront investment. Infrastructure needs Up to 100.000 bands x (100 musics + 10 videos) = 60TB, spread across the world Growing from 1 band to 100.000 bands, without worrying about storage, resilience and geographical coverage.

Concepts Use case 2 1.Elasticity 2.On-demand self-service (API) 3.Broad network access

Big Data NoSQL Our case

Cloud Storage Unlimited virtual disk accessible through a web interface or an API. Applications can use cloud storage to place any objects. Data is replicated in at least 3 different physical stores. Built over a NoSQL Cassandra database. Cloud Mongo MongoDB as a Service, with single or replicated instances, provisioned through a point-and-click web interface. It allows developers of applications to focus on code, not database management.

The future of the Cloud In our view IaaS is a commodity, a utility service, that is best when infrastructure is closer to customers, in nearby datacenters. PaaS is a differentiator and will accelerate innovation. SaaS is where most of the activity and innovation takes place.

Thank you Email Web Twitter antonio@lunacloud.com www.lunacloud.com @lunacloud