Savanna Hadoop on. OpenStack. Savanna Technical Lead

Size: px
Start display at page:

Download "Savanna Hadoop on. OpenStack. Savanna Technical Lead"

Transcription

1 Savanna Hadoop on OpenStack Sergey Lukjanov Savanna Technical Lead Mirantis, 2013

2 Agenda Savanna Overview Savanna Use Cases Roadmap & Current Status Architecture & Features Overview Hadoop vs. Virtualization

3 Agenda Savanna Overview Savanna Use Cases Roadmap & Current Status Architecture & Features Overview Hadoop vs. Virtualization

4 Savanna - Elastic Hadoop on OpenStack Open source native OpenStack component Supports different Hadoop distributions Solves both bare cluster provisioning use case and "analytics as a service" Managed through REST API Web UI as part of the OpenStack Dashboard Flexible templates of Hadoop configurations

5 Savanna - Elastic Hadoop on OpenStack Project home - https://launchpad.net/savanna bug tracking blueprints answers Code review (gerrit) - https://review.openstack.org Sources - https://github.com/stackforge/savanna Mailing list - CI - https://jenkins.openstack.org and

6 Savanna - Participants Contributors: large core team from Mirantis teams from RedHat, Hortonworks several minor contributors Intel joined recently Several upcoming customers

7 Agenda Savanna Overview Savanna Use Cases Roadmap & Current Status Architecture & Features Overview Hadoop vs. Virtualization

8 Savanna Use Cases Administrators - centralized cluster management and monitoring Dev and QA teams - fast clusters provisioning Data Scientists/Analysts - API to run the analytic jobs with infrastructure provisioning happening under the hood Making resources dedicated to IaaS cloud available for Hadoop workload

9 Administrators Use Case Central point of control over infrastructure Enables self-service capabilities, including choice of Hadoop distribution to be used Integration with vendor tooling: Ambari for Apache/HortonWorks Cloudera Management Console Intel Hadoop Utilization of free IaaS capacity for Hadoop tasks

10 Dev and QA Use Cases Fast on-demand provisioning of the environments Increase agility and speed of innovation Controlled access to data from production

11 Analytics Use Cases Simplified tasks execution - complexity of provisioning and managing cluster hidden under the hood Access to higher level interfaces (e.g. pig, hive) Bursty workload: ad-hoc queries requiring a significant resource only for short time period Utilization of free IaaS capacity for Hadoop tasks

12 Agenda Savanna Overview Savanna Use Cases Roadmap & Current Status Architecture & Features Overview Hadoop vs. Virtualization

13 Roadmap for Hadoop in Cloud Phase 1 Basic cluster provisioning of Apache Hadoop Phase 2 Cluster operation support and integration with tooling, advanced configuration (HDFS, Swift, etc.) Phase 3 "Analytics as a service": job execution framework, support different scripting languages, deeper integration with OS

14 Phase 1 - Basic Cluster Operation Cluster provisioning Deployment Engine implementation for preinstalled images Templates for Hadoop cluster configuration REST API for cluster startup and operations Web UI integrated into OpenStack Dashboard

15 Roadmap for Hadoop in Cloud Phase 1 [Released - April, 10] Basic cluster provisioning of Apache Hadoop Phase 2 Cluster operation support and integration with tooling, advanced configuration (HDFS, Swift, etc.) Phase 3 "Analytics as a service": job execution framework, support different scripting languages, deeper integration with OS

16 Phase 2 - Advanced Configuration Hadoop cluster configuration support: Solutions for HDFS data reliability issue Configurable storage location Configurable topology of, NN, TT, JT Add/remove nodes More Hadoop parameters Integration with vendor deployment/management tooling Basic monitoring support

17 Roadmap for Hadoop in Cloud Phase 1 [Released - April, 10] Basic cluster provisioning of Apache Hadoop Phase 2 [In progress - July 15] Cluster operation support and integration with tooling, advanced configuration (HDFS, Swift, etc.) Phase 3 "Analytics as a service": job execution framework, support different scripting languages, deeper integration with OS

18 Phase 3 - Analytics as a Service API to execute Map/Reduce jobs without exposing details of underlying infrastructure (similar to AWS EMR) User-friendly UI for ad-hoc analytics queries based on Hive or Pig

19 Roadmap for Hadoop in Cloud Phase 1 [Released - April, 10] Basic cluster provisioning of Apache Hadoop Phase 2 [In progress - July 15] Cluster operation support and integration with tooling, advanced configuration (HDFS, Swift, etc.) Phase 3 [Planned - October 15] "Analytics as a service": job execution framework, support different scripting languages, deeper integration with OS

20 Further Roadmap Autoscaling HA for NameNode Deeper HDFS and Swift integration Caching of Swift data on HDFS Integration with logging and error handling HBase support

21 Agenda Savanna Overview Savanna Use Cases Roadmap & Current Status Architecture & Features Overview Hadoop vs. Virtualization

22 Architecture Overview Keystone Horizon Hadoop VM Hadoop VM Hadoop VM Hadoop VM Savanna Pages Auth Savanna Python Client REST API Swift Cluster Configuration Manager DAL Provisioning Plugin Instance Interop Helper Image Registry Nova Glance

23 Hadoop vs. Virtualization HDFS Reliability Data Persistence I/O Performance etc.

24 Hadoop vs. Virtualization HDFS Reliability Data Persistence I/O Performance etc.

25 Hadoop vs. Virtualization HDFS Reliability Data Persistence I/O Performance etc.

26 Hadoop vs. Virtualization HDFS Reliability Data Persistence I/O Performance etc.

27 HDFS Reliability: the issue Data Block Compute Compute

28 HDFS Reliability: the issue Data Block Compute Compute

29 HDFS Reliability: the issue Data Block Compute Compute

30 HDFS Reliability: single per host Compute Compute Compute TT Cluster A Cluster B

31 HDFS Reliability: Hadoop-8468 hypervisor-awareness for HDFS scheduler Compute Compute Compute HDFS Data Block

32 HDFS Reliability: Hadoop-8545 enables Swift for Hadoop t pu n i al i init Swift Hadoop Job #1 Hadoop Job #2 fin al o ut pu t... Hadoop Job #N HDFS

33 Configurable topology of, NN, TT, JT Master node(s) JT NN JT + NN Worker nodes TT TT

34 HDFS Placement Options Ephemeral drive /var/lib/nova/instances/instance-xxx/disk -> /mnt/ephemeral Block storage volume Cinder Volume -> /mnt/volume Bare hard drive support /dev/sdb -> /mnt/sdb

35 Q&A

36 We are hiring!

37 Phase 1 deployment mechanism Provision VMs with pre-installed Hadoop Savanna Configure Hadoop Cluster Hadoop VM Hadoop VM Hadoop VM Hadoop VM

38 Tool usage scenarios Scenario I Tool Manage Hadoop Cluster Hadoop VM Hadoop VM Hadoop VM Hadoop VM VM VM VM VM Scenario II Tool Provision & Manage Hadoop Cluster

39 Extensible Provisioning S a v a n n a Plugin get extra configs validate input launch/terminate cluster add/remove nodes Image registry register image in Savanna add/remove tags get image by tag Instance Interop launch/terminate VMs get VM status ssh/scp to VM

40 Provisioning Interaction get extra parameters for the plugin launch cluster U s e r add/remove nodes S a v a n n a get extra parameters validate cluster parameters launch cluster add/remove nodes P l u g i n launch cluster add/remove nodes

41 Provisioning: Launching a Cluster get image by tag P L U G I N Image Registry launch VMs install and configure Hadoop launch VMs Instance Interop Helper pass commands via ssh, scp Hadoop VM Hadoop VM Hadoop VM Hadoop VM

42 Q&A

43 We are hiring!

Hadoop on OpenStack Cloud. Dmitry Mescheryakov Software Engineer, @MirantisIT

Hadoop on OpenStack Cloud. Dmitry Mescheryakov Software Engineer, @MirantisIT Hadoop on OpenStack Cloud Dmitry Mescheryakov Software Engineer, @MirantisIT Agenda OpenStack Sahara Demo Hadoop Performance on Cloud Conclusion OpenStack Open source cloud computing platform 17,209 commits

More information

A STUDY OF ADOPTING BIG DATA TO CLOUD COMPUTING

A STUDY OF ADOPTING BIG DATA TO CLOUD COMPUTING A STUDY OF ADOPTING BIG DATA TO CLOUD COMPUTING ASMAA IBRAHIM Technology Innovation and Entrepreneurship Center, Egypt aelrehim@itida.gov.eg MOHAMED EL NAWAWY Technology Innovation and Entrepreneurship

More information

TUT5605: Deploying an elastic Hadoop cluster Alejandro Bonilla

TUT5605: Deploying an elastic Hadoop cluster Alejandro Bonilla TUT5605: Deploying an elastic Hadoop cluster Alejandro Bonilla Sales Engineer abonilla@suse.com Agenda Overview Manual Deployment Orchestration Generic workload autoscaling Sahara Dedicated for Hadoop

More information

Sahara. Release 2014.1.rc2. OpenStack Foundation

Sahara. Release 2014.1.rc2. OpenStack Foundation Sahara Release 2014.1.rc2 OpenStack Foundation April 13, 2014 Contents i ii Sahara project aims to provide users with simple means to provision a Hadoop cluster at OpenStack by specifying several parameters

More information

System Administrators, engineers and consultants who will plan and manage OpenStack-based environments.

System Administrators, engineers and consultants who will plan and manage OpenStack-based environments. OpenStack Foundations (HP-H6C68) Course Overview This three day course assists administrators and users to configure, manage, and use the OpenStack cloud services platform. An architectural overview ensures

More information

HP Cloud OS. Платформа OpenStack корпоративного уровня. Иван Кровяков Архитектор облачных решений HP Центральная и Восточная Европа

HP Cloud OS. Платформа OpenStack корпоративного уровня. Иван Кровяков Архитектор облачных решений HP Центральная и Восточная Европа HP Cloud OS Платформа OpenStack корпоративного уровня Иван Кровяков Архитектор облачных решений HP Центральная и Восточная Европа What is OpenStack Software Massively scalable cloud operating system that

More information

Cloud Essentials for Architects using OpenStack

Cloud Essentials for Architects using OpenStack Cloud Essentials for Architects using OpenStack Course Overview Start Date 18th December 2014 Duration 2 Days Location Dublin Course Code SS906 Programme Overview Cloud Computing is gaining increasing

More information

One-click Hadoop Cluster Deployment on OpenPOWER Systems Pradeep K Surisetty IBM. #OpenPOWERSummit

One-click Hadoop Cluster Deployment on OpenPOWER Systems Pradeep K Surisetty IBM. #OpenPOWERSummit One-click Hadoop Cluster Deployment on OpenPOWER Systems Pradeep K Surisetty IBM #OpenPOWERSummit #Whoami l Systems & Infrastructure Engineer l 9 Years + of Linux, Virtualization l Believe in Open Source

More information

Cloudify and OpenStack Heat

Cloudify and OpenStack Heat Cloudify and OpenStack Heat General Cloudify is an application orchestration platform that provides a complete solution for automating and managing application deployment and DevOps processes on top of

More information

HOPS: Hadoop Open Platform-as-a-Service

HOPS: Hadoop Open Platform-as-a-Service HOPS: Hadoop Open Platform-as-a-Service Alberto Lorente, Hamid Afzali, Salman Niazi, Mahmoud Ismail, Kamal Hakimazadeh, Hooman Piero, Jim Dowling jdowling@sics.se Scale Research Laboratory What is Hadoop?

More information

Building Storage as a Service with OpenStack. Greg Elkinbard Senior Technical Director

Building Storage as a Service with OpenStack. Greg Elkinbard Senior Technical Director Building Storage as a Service with OpenStack Greg Elkinbard Senior Technical Director MIRANTIS 2012 PAGE 1 About the Presenter Greg Elkinbard Senior Technical Director at Mirantis Builds on demand IaaS

More information

Benchmarking Sahara-based Big-Data-as-a-Service Solutions. Zhidong Yu, Weiting Chen (Intel) Matthew Farrellee (Red Hat) May 2015

Benchmarking Sahara-based Big-Data-as-a-Service Solutions. Zhidong Yu, Weiting Chen (Intel) Matthew Farrellee (Red Hat) May 2015 Benchmarking Sahara-based Big-Data-as-a-Service Solutions Zhidong Yu, Weiting Chen (Intel) Matthew Farrellee (Red Hat) May 2015 Agenda o Why Sahara o Sahara introduction o Deployment considerations o Performance

More information

Using SUSE Cloud to Orchestrate Multiple Hypervisors and Storage at ADP

Using SUSE Cloud to Orchestrate Multiple Hypervisors and Storage at ADP Using SUSE Cloud to Orchestrate Multiple Hypervisors and Storage at ADP Agenda ADP Cloud Vision and Requirements Introduction to SUSE Cloud Overview Whats New VMWare intergration HyperV intergration ADP

More information

Infomatics. Big-Data and Hadoop Developer Training with Oracle WDP

Infomatics. Big-Data and Hadoop Developer Training with Oracle WDP Big-Data and Hadoop Developer Training with Oracle WDP What is this course about? Big Data is a collection of large and complex data sets that cannot be processed using regular database management tools

More information

OpenStack. Orgad Kimchi. Principal Software Engineer. Oracle ISV Engineering. 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved.

OpenStack. Orgad Kimchi. Principal Software Engineer. Oracle ISV Engineering. 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved. OpenStack Orgad Kimchi Principal Software Engineer Oracle ISV Engineering 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline

More information

Qsoft Inc www.qsoft-inc.com

Qsoft Inc www.qsoft-inc.com Big Data & Hadoop Qsoft Inc www.qsoft-inc.com Course Topics 1 2 3 4 5 6 Week 1: Introduction to Big Data, Hadoop Architecture and HDFS Week 2: Setting up Hadoop Cluster Week 3: MapReduce Part 1 Week 4:

More information

Virtualizing Apache Hadoop. June, 2012

Virtualizing Apache Hadoop. June, 2012 June, 2012 Table of Contents EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 VIRTUALIZING APACHE HADOOP... 4 INTRODUCTION TO VSPHERE TM... 4 USE CASES AND ADVANTAGES OF VIRTUALIZING HADOOP... 4 MYTHS ABOUT RUNNING

More information

Introduction to Big data. Why Big data? Case Studies. Introduction to Hadoop. Understanding Features of Hadoop. Hadoop Architecture.

Introduction to Big data. Why Big data? Case Studies. Introduction to Hadoop. Understanding Features of Hadoop. Hadoop Architecture. Big Data Hadoop Administration and Developer Course This course is designed to understand and implement the concepts of Big data and Hadoop. This will cover right from setting up Hadoop environment in

More information

OpenStack: we drink our own Champagne. Teun Docter Software developer

OpenStack: we drink our own Champagne. Teun Docter Software developer OpenStack: we drink our own Champagne Teun Docter Software developer About Bright Computing Enterprise infrastructure software company Founded in 2009 HQ in Amsterdam & San Jose Bare-metal & cloud provisioning,

More information

Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments

Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments Important Notice 2010-2015 Cloudera, Inc. All rights reserved. Cloudera, the Cloudera logo, Cloudera Impala, Impala, and

More information

Hadoop & Spark Using Amazon EMR

Hadoop & Spark Using Amazon EMR Hadoop & Spark Using Amazon EMR Michael Hanisch, AWS Solutions Architecture 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda Why did we build Amazon EMR? What is Amazon EMR?

More information

Upcoming Announcements

Upcoming Announcements Enterprise Hadoop Enterprise Hadoop Jeff Markham Technical Director, APAC jmarkham@hortonworks.com Page 1 Upcoming Announcements April 2 Hortonworks Platform 2.1 A continued focus on innovation within

More information

OpenStack Introduction. November 4, 2015

OpenStack Introduction. November 4, 2015 OpenStack Introduction November 4, 2015 Application Platforms Undergoing A Major Shift What is OpenStack Open Source Cloud Software Launched by NASA and Rackspace in 2010 Massively scalable Managed by

More information

Cloud Computing #8 - Datacenter OS. Johan Eker

Cloud Computing #8 - Datacenter OS. Johan Eker Cloud Computing #8 - Datacenter OS Johan Eker Outline What is a Datacenter OS? OpenStack Kubernetes Resource Management What is an OS? What is an OS? Manage hardware resources such as CPU, RAM, disk, I/O,

More information

SUSE Cloud 2.0. Pete Chadwick. Douglas Jarvis. Senior Product Manager pchadwick@suse.com. Product Marketing Manager djarvis@suse.

SUSE Cloud 2.0. Pete Chadwick. Douglas Jarvis. Senior Product Manager pchadwick@suse.com. Product Marketing Manager djarvis@suse. SUSE Cloud 2.0 Pete Chadwick Douglas Jarvis Senior Product Manager pchadwick@suse.com Product Marketing Manager djarvis@suse.com SUSE Cloud SUSE Cloud is an open source software solution based on OpenStack

More information

KVM, OpenStack, and the Open Cloud

KVM, OpenStack, and the Open Cloud KVM, OpenStack, and the Open Cloud Adam Jollans, IBM & Mike Kadera, Intel CloudOpen Europe - October 13, 2014 13Oct14 Open VirtualizaGon Alliance 1 Agenda A Brief History of VirtualizaGon KVM Architecture

More information

HADOOP BIG DATA DEVELOPER TRAINING AGENDA

HADOOP BIG DATA DEVELOPER TRAINING AGENDA HADOOP BIG DATA DEVELOPER TRAINING AGENDA About the Course This course is the most advanced course available to Software professionals This has been suitably designed to help Big Data Developers and experts

More information

AMD SEAMICRO OPENSTACK BLUEPRINTS CLOUD- IN- A- BOX OCTOBER 2013

AMD SEAMICRO OPENSTACK BLUEPRINTS CLOUD- IN- A- BOX OCTOBER 2013 AMD SEAMICRO OPENSTACK BLUEPRINTS CLOUD- IN- A- BOX OCTOBER 2013 OpenStack What is OpenStack? OpenStack is a cloud operaeng system that controls large pools of compute, storage, and networking resources

More information

CON8473 Oracle Distribution of OpenStack Making OpenStack an Enterprise Grade Solution

CON8473 Oracle Distribution of OpenStack Making OpenStack an Enterprise Grade Solution CON8473 Oracle Distribution of OpenStack Making OpenStack an Enterprise Grade Solution Ronen Kofman Director of Product Development Oracle OpenStack September, 2014 Safe Harbor Statement The following

More information

Red Hat Enterprise Linux OpenStack Platform 7 OpenStack Data Processing

Red Hat Enterprise Linux OpenStack Platform 7 OpenStack Data Processing Red Hat Enterprise Linux OpenStack Platform 7 OpenStack Data Processing Manually provisioning and scaling Hadoop clusters in Red Hat OpenStack OpenStack Documentation Team Red Hat Enterprise Linux OpenStack

More information

Bright Cluster Manager

Bright Cluster Manager Bright Cluster Manager For HPC, Hadoop and OpenStack Craig Hunneyman Director of Business Development Bright Computing Craig.Hunneyman@BrightComputing.com Agenda Who is Bright Computing? What is Bright

More information

Big data blue print for cloud architecture

Big data blue print for cloud architecture Big data blue print for cloud architecture -COGNIZANT Image Area Prabhu Inbarajan Srinivasan Thiruvengadathan Muralicharan Gurumoorthy Praveen Codur 2012, Cognizant Next 30 minutes Big Data / Cloud challenges

More information

OpenStack The State of the Stack

OpenStack The State of the Stack OpenStack The State of the Stack Heidi Bretz, Dir Business Development, OpenStack Foundation Stefano Maffulli, Director of OpenStack Community Mariano Cunietti, CTO, Enter.it OpenStack Software for Building

More information

Cloud Computing using

Cloud Computing using Cloud Computing using Summary of Content Introduction of Cloud Computing Cloud Computing vs. Server Virtualization Cloud Computing Components Stack Public vs. Private Clouds Open Source Software for Private

More information

Infrastructure as a Service

Infrastructure as a Service Infrastructure as a Service Jose Castro Leon CERN IT/OIS Cloud Computing On-Demand Self-Service Scalability and Efficiency Resource Pooling Rapid elasticity 2 Infrastructure as a Service Objectives 90%

More information

Getting Started with OpenStack. Charles Eckel, Cisco DevNet (eckelcu@cisco.com)

Getting Started with OpenStack. Charles Eckel, Cisco DevNet (eckelcu@cisco.com) Getting Started with OpenStack Charles Eckel, Cisco DevNet (eckelcu@cisco.com) Agenda What is OpenStack? Use cases and work loads Demo: Install and operate OpenStack on your laptop Getting help and additional

More information

Clodoaldo Barrera Chief Technical Strategist IBM System Storage. Making a successful transition to Software Defined Storage

Clodoaldo Barrera Chief Technical Strategist IBM System Storage. Making a successful transition to Software Defined Storage Clodoaldo Barrera Chief Technical Strategist IBM System Storage Making a successful transition to Software Defined Storage Open Server Summit Santa Clara Nov 2014 Data at the core of everything Data is

More information

Deploying Hadoop with Manager

Deploying Hadoop with Manager Deploying Hadoop with Manager SUSE Big Data Made Easier Peter Linnell / Sales Engineer plinnell@suse.com Alejandro Bonilla / Sales Engineer abonilla@suse.com 2 Hadoop Core Components 3 Typical Hadoop Distribution

More information

Multi Provider Cloud. Srinivasa Acharya, Engineering Manager, Hewlett-Packard rsacharya@hp.com

Multi Provider Cloud. Srinivasa Acharya, Engineering Manager, Hewlett-Packard rsacharya@hp.com Multi Provider Cloud Srinivasa Acharya, Engineering Manager, Hewlett-Packard rsacharya@hp.com Agenda Introduction to OpenStack Multi Hypervisor Architecture Use cases for Multi Hypervisor cloud Ironic

More information

Adobe Deploys Hadoop as a Service on VMware vsphere

Adobe Deploys Hadoop as a Service on VMware vsphere Adobe Deploys Hadoop as a Service A TECHNICAL CASE STUDY APRIL 2015 Table of Contents A Technical Case Study.... 3 Background... 3 Why Virtualize Hadoop on vsphere?.... 3 The Adobe Marketing Cloud and

More information

Instance Types. Standard Instances:

Instance Types. Standard Instances: Instance Types Standard Instances: 1EC2CU: equivalent of 1.0-1.2GHz 2007 AMD Opteron or 2007 Intel Xeon processor Small: 1.7GBmem, 1EC2Compute Unit (EC2CU), 160GB local instance storage(lis), 32/64bits.

More information

Building your Big Data Architecture on Amazon Web Services

Building your Big Data Architecture on Amazon Web Services Building your Big Data Architecture on Amazon Web Services Abhishek Sinha @abysinha sinhaar@amazon.com AWS Services Deployment & Administration Application Services Compute Storage Database Networking

More information

CloudStack and Big Data. Sebastien Goasguen @sebgoa May 22nd 2013 LinuxTag, Berlin

CloudStack and Big Data. Sebastien Goasguen @sebgoa May 22nd 2013 LinuxTag, Berlin CloudStack and Big Data Sebastien Goasguen @sebgoa May 22nd 2013 LinuxTag, Berlin Google trends Start of Clouds Cloud computing trending down, while Big Data is booming. Virtualization BigData on the Trigger

More information

Pro Apache Hadoop. Second Edition. Sameer Wadkar. Madhu Siddalingaiah

Pro Apache Hadoop. Second Edition. Sameer Wadkar. Madhu Siddalingaiah Pro Apache Hadoop Second Edition Sameer Wadkar Madhu Siddalingaiah Contents J About the Authors About the Technical Reviewer Acknowledgments Introduction xix xxi xxiii xxv Chapter 1: Motivation for Big

More information

Introduction to CoprHD: An Open Source Software Defined Storage Controller

Introduction to CoprHD: An Open Source Software Defined Storage Controller CoprHD.github.io Introduction to CoprHD: An Open Source Software Defined Storage Controller Anjaneya Reddy Chagam Principal Engineer, Intel Corporation Urayoan Irizarry Consultant Software Engineer, EMC

More information

BIRT ihub 3. 2013 Actuate Customer Days. Wow that looks good! Jeff Morris & Mark Gamble

BIRT ihub 3. 2013 Actuate Customer Days. Wow that looks good! Jeff Morris & Mark Gamble BIRT ihub 3 Wow that looks good! Jeff Morris & Mark Gamble SF Nov7 - UK Nov12 - DE Nov13 - FR Nov14 - SG Nov19 - JP Nov22 - NY Dec4 2013 Actuate Customer Days Actuate BIRT ihub 3 Focus Areas Simplified,

More information

Microsoft Research Windows Azure for Research Training

Microsoft Research Windows Azure for Research Training Copyright 2013 Microsoft Corporation. All rights reserved. Except where otherwise noted, these materials are licensed under the terms of the Apache License, Version 2.0. You may use it according to the

More information

An Introduction to OpenStack and its use of KVM. Daniel P. Berrangé

An Introduction to OpenStack and its use of KVM. Daniel P. Berrangé <berrange@redhat.com> An Introduction to OpenStack and its use of KVM Daniel P. Berrangé About me Contributor to multiple virt projects Libvirt Developer / Architect 8 years OpenStack contributor 1 year

More information

Introduction to OpenStack

Introduction to OpenStack Introduction to OpenStack Carlo Vallati PostDoc Reseracher Dpt. Information Engineering University of Pisa carlo.vallati@iet.unipi.it Cloud Computing - Definition Cloud Computing is a term coined to refer

More information

HYPER-CONVERGED INFRASTRUCTURE STRATEGIES

HYPER-CONVERGED INFRASTRUCTURE STRATEGIES 1 HYPER-CONVERGED INFRASTRUCTURE STRATEGIES MYTH BUSTING & THE FUTURE OF WEB SCALE IT 2 ROADMAP INFORMATION DISCLAIMER EMC makes no representation and undertakes no obligations with regard to product planning

More information

HP OpenStack & Automation

HP OpenStack & Automation HP OpenStack & Automation Where we are heading Thomas Goh Cloud Computing Cloud Computing Cloud computing is a model for enabling ubiquitous network access to a shared pool of configurable computing resources.

More information

Ubuntu OpenStack Fundamentals Training

Ubuntu OpenStack Fundamentals Training Ubuntu OpenStack Fundamentals Training Learn from the best, how to use the best! You ve made the decision to use the most powerful open cloud platform, and now you need to learn how to make the most of

More information

Microsoft Research Microsoft Azure for Research Training

Microsoft Research Microsoft Azure for Research Training Copyright 2014 Microsoft Corporation. All rights reserved. Except where otherwise noted, these materials are licensed under the terms of the Apache License, Version 2.0. You may use it according to the

More information

HP Converged Cloud Cloud Platform Overview. Shane Pearson Vice President, Portfolio & Product Management

HP Converged Cloud Cloud Platform Overview. Shane Pearson Vice President, Portfolio & Product Management HP Converged Cloud Cloud Platform Overview Shane Pearson Vice President, Portfolio & Product Management Cloud is the biggest disruption since the Internet 1970-80s Mainframe 1990s Client/Server 2000s The

More information

How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014

How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014 How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014 2014 年 云 计 算 效 率 与 能 耗 暨 第 一 届 国 际 云 计 算 咨 询 委 员 会 中 国 高 峰 论 坛 Contents Background

More information

RED HAT STORAGE PORTFOLIO OVERVIEW

RED HAT STORAGE PORTFOLIO OVERVIEW RED HAT STORAGE PORTFOLIO OVERVIEW Andrew Hatfield Practice Lead Cloud Storage and Big Data MILCIS November 2015 THE RED HAT STORAGE MISSION To offer a unified, open software-defined storage portfolio

More information

Introduction. Various user groups requiring Hadoop, each with its own diverse needs, include:

Introduction. Various user groups requiring Hadoop, each with its own diverse needs, include: Introduction BIG DATA is a term that s been buzzing around a lot lately, and its use is a trend that s been increasing at a steady pace over the past few years. It s quite likely you ve also encountered

More information

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray ware 2 Agenda The Hadoop Journey Why Virtualize Hadoop? Elasticity and Scalability Performance Tests Storage Reference

More information

Red Hat CloudForms for Cloud Management: Key Features & Roadmap

Red Hat CloudForms for Cloud Management: Key Features & Roadmap Red Hat CloudForms for Cloud Management: Key Features & Roadmap Damon Tepe Sr. Product Marketing Manager Red Hat John Hardy Technical Product Marketing Manager Red Hat Your Speakers Damon Tepe Sr Product

More information

Cloud computing for research

Cloud computing for research & for research Cécile Cavet cecile.cavet at apc.univ-paris7.fr Centre François Arago (FACe), Laboratoire APC, Université Paris Diderot January 11, 2015 Plan & 1 2 & 3 4 What is cloud computing? &??? Provides

More information

OpenStack Alberto Molina Coballes

OpenStack Alberto Molina Coballes OpenStack Alberto Molina Coballes Teacher at IES Gonzalo Nazareno alb.molina@gmail.com @alberto_molina Table of Contents From public to private clouds Open Source Cloud Platforms Why OpenStack? OpenStack

More information

Automating Big Data Benchmarking for Different Architectures with ALOJA

Automating Big Data Benchmarking for Different Architectures with ALOJA www.bsc.es Jan 2016 Automating Big Data Benchmarking for Different Architectures with ALOJA Nicolas Poggi, Postdoc Researcher Agenda 1. Intro on Hadoop performance 1. Current scenario and problematic 2.

More information

Adrian Otto, Rackspace @adrian_otto

Adrian Otto, Rackspace @adrian_otto Adrian Otto, Rackspace @adrian_otto Ancient History RackSpace Cloud Files Swift NASA Nova OpenStack born with 25 participating companies! Mission Statement "To produce the ubiquitous Open Source cloud

More information

HDFS Federation. Sanjay Radia Founder and Architect @ Hortonworks. Page 1

HDFS Federation. Sanjay Radia Founder and Architect @ Hortonworks. Page 1 HDFS Federation Sanjay Radia Founder and Architect @ Hortonworks Page 1 About Me Apache Hadoop Committer and Member of Hadoop PMC Architect of core-hadoop @ Yahoo - Focusing on HDFS, MapReduce scheduler,

More information

Native Connectivity to Big Data Sources in MSTR 10

Native Connectivity to Big Data Sources in MSTR 10 Native Connectivity to Big Data Sources in MSTR 10 Bring All Relevant Data to Decision Makers Support for More Big Data Sources Optimized Access to Your Entire Big Data Ecosystem as If It Were a Single

More information

CloudCIX Bootcamp. The essential IaaS getting started guide. http://www.cix.ie

CloudCIX Bootcamp. The essential IaaS getting started guide. http://www.cix.ie The essential IaaS getting started guide. http://www.cix.ie Revision Date: 17 th August 2015 Contents Acronyms... 2 Table of Figures... 3 1 Welcome... 4 2 Architecture... 5 3 Getting Started... 6 3.1 Login

More information

Implement Hadoop jobs to extract business value from large and varied data sets

Implement Hadoop jobs to extract business value from large and varied data sets Hadoop Development for Big Data Solutions: Hands-On You Will Learn How To: Implement Hadoop jobs to extract business value from large and varied data sets Write, customize and deploy MapReduce jobs to

More information

One click Hadoop clusters - anywhere

One click Hadoop clusters - anywhere One click Hadoop clusters - anywhere Janos Matyas, Senior Director of Engineering October, Page 1 2015 Overview Page 2 Introduction Goals and motivations Technology stack How it works Results/achievements/future

More information

Cloud Computing. Adam Barker

Cloud Computing. Adam Barker Cloud Computing Adam Barker 1 Overview Introduction to Cloud computing Enabling technologies Different types of cloud: IaaS, PaaS and SaaS Cloud terminology Interacting with a cloud: management consoles

More information

Intel Service Assurance Administrator. Product Overview

Intel Service Assurance Administrator. Product Overview Intel Service Assurance Administrator Product Overview Running Enterprise Workloads in the Cloud Enterprise IT wants to Start a private cloud initiative to service internal enterprise customers Find an

More information

Hadoop: Embracing future hardware

Hadoop: Embracing future hardware Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop

More information

Déployer son propre cloud avec OpenStack. GULL 18.11.2014 François Deppierraz francois.deppierraz@nimag.net

Déployer son propre cloud avec OpenStack. GULL 18.11.2014 François Deppierraz francois.deppierraz@nimag.net Déployer son propre cloud avec OpenStack GULL francois.deppierraz@nimag.net Who Am I? System and Network Engineer Stuck in the Linux world for almost 2 decades Sysadmin who doesn't like to type the same

More information

Managing Traditional Workloads Together with Cloud Computing Workloads

Managing Traditional Workloads Together with Cloud Computing Workloads Managing Traditional Workloads Together with Cloud Computing Workloads Table of Contents Introduction... 3 Cloud Management Challenges... 3 Re-thinking of Cloud Management Solution... 4 Teraproc Cloud

More information

STeP-IN SUMMIT 2014. June 2014 at Bangalore, Hyderabad, Pune - INDIA. Performance testing Hadoop based big data analytics solutions

STeP-IN SUMMIT 2014. June 2014 at Bangalore, Hyderabad, Pune - INDIA. Performance testing Hadoop based big data analytics solutions 11 th International Conference on Software Testing June 2014 at Bangalore, Hyderabad, Pune - INDIA Performance testing Hadoop based big data analytics solutions by Mustufa Batterywala, Performance Architect,

More information

Apache Hadoop 1.0 High Availability Solution on VMware vsphere TM

Apache Hadoop 1.0 High Availability Solution on VMware vsphere TM Apache Hadoop 1.0 High Availability Solution on VMware vsphere TM Reference Architecture TECHNICAL WHITE PAPER v 1.0 June 2012 Table of Contents Executive Summary... 3 Introduction... 3 Terminology...

More information

KVM, OpenStack, and the Open Cloud

KVM, OpenStack, and the Open Cloud KVM, OpenStack, and the Open Cloud Adam Jollans, IBM Southern California Linux Expo February 2015 1 Agenda A Brief History of VirtualizaJon KVM Architecture OpenStack Architecture KVM and OpenStack Case

More information

Sales Slide Midokura Enterprise MidoNet V1. July 2015 Fujitsu Limited

Sales Slide Midokura Enterprise MidoNet V1. July 2015 Fujitsu Limited Sales Slide Midokura Enterprise MidoNet V1 July 2015 Fujitsu Limited What Is Midokura Enterprise MidoNet? Network Virtualization Software Coordinated with OpenStack Provides safe & effective virtual networks

More information

Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases

Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases NASA Ames NASA Advanced Supercomputing (NAS) Division California, May 24th, 2012 Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases Ignacio M. Llorente Project Director OpenNebula Project.

More information

Data Governance in the Hadoop Data Lake. Michael Lang May 2015

Data Governance in the Hadoop Data Lake. Michael Lang May 2015 Data Governance in the Hadoop Data Lake Michael Lang May 2015 Introduction Product Manager for Teradata Loom Joined Teradata as part of acquisition of Revelytix, original developer of Loom VP of Sales

More information

The Inside Scoop on Hadoop

The Inside Scoop on Hadoop The Inside Scoop on Hadoop Orion Gebremedhin National Solutions Director BI & Big Data, Neudesic LLC. VTSP Microsoft Corp. Orion.Gebremedhin@Neudesic.COM B-orgebr@Microsoft.com @OrionGM The Inside Scoop

More information

Bringing Big Data to People

Bringing Big Data to People Bringing Big Data to People Microsoft s modern data platform SQL Server 2014 Analytics Platform System Microsoft Azure HDInsight Data Platform Everyone should have access to the data they need. Process

More information

COURSE CONTENT Big Data and Hadoop Training

COURSE CONTENT Big Data and Hadoop Training COURSE CONTENT Big Data and Hadoop Training 1. Meet Hadoop Data! Data Storage and Analysis Comparison with Other Systems RDBMS Grid Computing Volunteer Computing A Brief History of Hadoop Apache Hadoop

More information

Iron Chef: Bare Metal OpenStack

Iron Chef: Bare Metal OpenStack Rebecca Brenton Partner Alliances Manager Rob Hirschfeld Principal Cloud Architect Session Hashtags #chefconf #openstack About the Solution: http://dell.com/openstack http://dell.com/crowbak Iron Chef:

More information

Fundamentals of OpenStack Technology

Fundamentals of OpenStack Technology Fundamentals of OpenStack Technology Education Services course product number H6C68S Course length 3 days Delivery mode Instructor Led Training (ILT) Virtual Instructor Led Training (vilt) Register Click

More information

By Reeshu Patel. Getting Started with OpenStack

By Reeshu Patel. Getting Started with OpenStack Getting Started with OpenStack By Reeshu Patel 1 What is OpenStack OpenStack is a set of softwares tools for building and managing cloud computing platforms for public and personal clouds. Backed by a

More information

How Bigtop Leveraged Docker for Build Automation and One-Click Hadoop Provisioning

How Bigtop Leveraged Docker for Build Automation and One-Click Hadoop Provisioning How Bigtop Leveraged Docker for Build Automation and One-Click Hadoop Provisioning Evans Ye Apache Big Data 2015 Budapest Who am I Apache Bigtop PMC member Software Engineer at Trend Micro Develop Big

More information

Comparing Ganeti to other Private Cloud Platforms. Lance Albertson Director lance@osuosl.org @ramereth

Comparing Ganeti to other Private Cloud Platforms. Lance Albertson Director lance@osuosl.org @ramereth Comparing Ganeti to other Private Cloud Platforms Lance Albertson Director lance@osuosl.org @ramereth About me OSU Open Source Lab Server hosting for Open Source Projects Open Source development projects

More information

Getting Started with OpenStack and VMware vsphere TECHNICAL MARKETING DOCUMENTATION V 0.1/DECEMBER 2013

Getting Started with OpenStack and VMware vsphere TECHNICAL MARKETING DOCUMENTATION V 0.1/DECEMBER 2013 Getting Started with OpenStack and VMware vsphere TECHNICAL MARKETING DOCUMENTATION V 0.1/DECEMBER 2013 Table of Contents Introduction.... 3 1.1 VMware vsphere.... 3 1.2 OpenStack.... 3 1.3 Using OpenStack

More information

Intel IT Cloud Extending OpenStack* IaaS with Cloud Foundry* PaaS

Intel IT Cloud Extending OpenStack* IaaS with Cloud Foundry* PaaS Intel IT Cloud Extending OpenStack* IaaS with Cloud Foundry* PaaS Speaker: Catherine Spence, IT Principal Engineer, Cloud Computing Acknowledgements: Aaron Huber, Jon Price November 2014 Legal Notices

More information

Development in Azure. Dan Gartner Developer Technology Specialist Microsoft

Development in Azure. Dan Gartner Developer Technology Specialist Microsoft Development in Azure Dan Gartner Developer Technology Specialist Microsoft MSDN Azure Benefits Visual Studio / Azure Integration Azure SDK 2.5 Visual Studio Online Build and Load Test Application Insights

More information

Allocation of VMs: A primer Alex Glikson (IBM), John M. Kennedy (Intel), Giovanni Toffetti (IBM) FI-WARE Cloud Hosting Chapter June 6th, 2013

Allocation of VMs: A primer Alex Glikson (IBM), John M. Kennedy (Intel), Giovanni Toffetti (IBM) FI-WARE Cloud Hosting Chapter June 6th, 2013 Allocation of VMs: A primer Alex Glikson (IBM), John M. Kennedy (Intel), Giovanni Toffetti (IBM) FI-WAE Cloud Hosting Chapter June 6th, 2013 http://www.fi-ware.eu http://www.fi-ppp.eu Agenda Overview Web-based

More information

2) Xen Hypervisor 3) UEC

2) Xen Hypervisor 3) UEC 5. Implementation Implementation of the trust model requires first preparing a test bed. It is a cloud computing environment that is required as the first step towards the implementation. Various tools

More information

Virtualization and IaaS management

Virtualization and IaaS management CLOUDFORMS Virtualization and IaaS management Calvin Smith, Senior Solutions Architect calvin@redhat.com VIRTUALIZATION TO CLOUD CONTINUUM Virtual Infrastructure Management Drivers Server Virtualization

More information

Cloud enablement with Flexiant. Cloud Orchestrator

Cloud enablement with Flexiant. Cloud Orchestrator Cloud enablement with Flexiant Copyright 2012, Flexiant Copyright 2015, Flexiant Cloud Orchestrator Innovation in action: FP7, H2020 and Flexiant SummerSOC, Crete Craig Sheridan, 30th June 2015 Copyright

More information

Hadoop Introduction. Olivier Renault Solution Engineer - Hortonworks

Hadoop Introduction. Olivier Renault Solution Engineer - Hortonworks Hadoop Introduction Olivier Renault Solution Engineer - Hortonworks Hortonworks A Brief History of Apache Hadoop Apache Project Established Yahoo! begins to Operate at scale Hortonworks Data Platform 2013

More information

OpenNebula Open Souce Solution for DC Virtualization. C12G Labs. Online Webinar

OpenNebula Open Souce Solution for DC Virtualization. C12G Labs. Online Webinar OpenNebula Open Souce Solution for DC Virtualization C12G Labs Online Webinar What is OpenNebula? Multi-tenancy, Elasticity and Automatic Provision on Virtualized Environments I m using virtualization/cloud,

More information

OpenNebula Open Souce Solution for DC Virtualization

OpenNebula Open Souce Solution for DC Virtualization 13 th LSM 2012 7 th -12 th July, Geneva OpenNebula Open Souce Solution for DC Virtualization Constantino Vázquez Blanco OpenNebula.org What is OpenNebula? Multi-tenancy, Elasticity and Automatic Provision

More information

CLOUDSTACK VS OPENSTACK. Apache CloudStack: It Just Works for Service Providers

CLOUDSTACK VS OPENSTACK. Apache CloudStack: It Just Works for Service Providers Apache CloudStack: It Just Works for Service Providers TABLE OF CONTENTS Introduction 03 Platform History 05 Underlying Technology 09 Commitment to CloudStack 19 What Does this Mean for You? 23 2 INTRODUCTION

More information

Accelerate OpenStack* Together. * OpenStack is a registered trademark of the OpenStack Foundation

Accelerate OpenStack* Together. * OpenStack is a registered trademark of the OpenStack Foundation Accelerate OpenStack* Together * OpenStack is a registered trademark of the OpenStack Foundation Where are your workloads running Ensuring Boundary Control in OpenStack Cloud. Raghu Yeluri Principal Engineer,

More information

Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software. SC13, November, 2013

Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software. SC13, November, 2013 Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software SC13, November, 2013 Agenda Abstract Opportunity: HPC Adoption of Big Data Analytics on Apache

More information