DATA LAKE FOUNDATION 2.0 JEUDI 19 NOVEMBRE Denis FRAVAL-OLIVIER : ISD Presales Manager

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "DATA LAKE FOUNDATION 2.0 JEUDI 19 NOVEMBRE 2015. Denis FRAVAL-OLIVIER : ISD Presales Manager"

Transcription

1 DATA LAKE FOUNDATION 2.0 JEUDI 19 NOVEMBRE 2015 Denis FRAVAL-OLIVIER : ISD Presales Manager

2 EMC Isilon Unifying Workloads in one place Module 4: Horizontal and Vertical Markets

3 ISILON FOR ALL TYPES OF DATA Processes & Transaction Home Directories Sync n Share File shares Surveillance BLOBS Social & Next-Gen Web Content Splunk Hadoop & Analytics Design, Test & Manufacture Consumerization Personalization Demographics Archive & Backup Target

4 EMC Isilon Scale-Out platform Protocols Clients and Applications NFS SMB HTTP FTP HDFS for Hadoop REST for Object RESTful API GET PUT POST DELETE Gig-e 10 Gig-e Network Multi-Protocol Client/Application Layer Ethernet Layer OneFS Operating Environment Intra-cluster Communication

5 DATA LAKE EMC ISILON SCALE-OUT NAS DATA PROTECTION S-Series X-Series Data Lake DATA MANAGEMENT NL-Series HD-Series DATA SECURITY PERFORMANCE MANAGEMENT 5

6 3rd Platform cloud Innovation S - Series X - Series NL-Series Isilon CloudPools FUTURE HD-Series 6

7 Support for Multiple Analytics Applications Node reply NFS SMB NFS SMB SMB, NFS, HTTP, FTP, HDFS name node name node name node data node HDFS NFS name node MAP Reduce MAP Reduce MAP Reduce MAP Reduce MAP Reduce MAP Reduce

8 Splunk Index Architecture

9 The Big, Cold Data Lake

10 The Bottomless Cold Bucket Benefit: Unmatched Scalability = Unmatched Simplicity Single Volume scaling to Cold Isilon ****.gz ****.tsidx 50PB

11 Backupless Bliss Benefit: Snapshots + Splunk Replication = NO BACKUPS ****.gz ****.tsidx ****.gz ****.tsidx Hot/Warm XtremIO Cold Isilon Use Splunk Multi-Site Clustering For Site Protection Use Array Snapshots For Data Protection Snapshots Splunk Multi- Site Clustering

12 Bonus!! Encryption Benefit: Encryption = Piece Of Mind Hot/Warm XtremIO Using Self Encrypting Drives (SED) AES256 Encryption Algorithm Drives Taken Out Are Unreadable Cold Isilon

13 Bonus!! Always Searchable Benefit: Automation = Simplicity Cold Isilon Automated Tiering For High Density Capacity Keep Data In Cold Then Delete Write-Once-Read-Many (WORM) protection SEC Rule 17a-4(f) definition standards

14 Bonus!! HUNK Ready Benefit: Isilon HDFS = SIMPLE Hadoop Analytics Data-in-place analytics Multi-protocol access Efficiency gains of 20% vs 300% overhead Enterprise Features For Hadoop

15 Hadoop Market Leadership #1 Market Leader in Hadoop Shared Storage 250% YoY Growth 700+ Customers

16 HADOOP ARCHITECTURE - TRADITIONAL Map Red Hive Pig Hbase Storm Kafka Knox Spark Tez Sqoop YARN Data Node + Compute Node Data Node + Compute Node Data Node + Compute Node Ethernet NameNode Data Node + Compute Node Data Node + Compute Node Data Node + Compute Node

17 HADOOP ARCHITECTURE WITH ISILON Map Red Hive Pig Hbase Storm Ambari Agent Kafka Knox Spark Tez Sqoop NameNode YARN DataNode Compute Node Compute Node Compute Node Ethernet Compute Node Compute Node Compute Node name node name node name node name node data node

18 Traditional Hadoop - Layers

19 Isilon + Hadoop NO Layers

20

21

22 HDFS: Integrated Isilon and vhadoop Node reply NFS SMB SMB SMB, NFS, HTTP, FTP, HDFS name node name node name node data node HDFS Apache NFS name node

23 Si vous avez des questions sur cette présentation, n hésitez pas à prendre directement contact avec : Denis FRAVAL-OLIVIER

24

ISILON SCALE-OUT NAS OVERVIEW AND FUTURE DIRECTIONS

ISILON SCALE-OUT NAS OVERVIEW AND FUTURE DIRECTIONS 1 ISILON SCALE-OUT NAS OVERVIEW AND FUTURE DIRECTIONS PHIL BULLINGER, SVP, EMC ISILON 2 ROADMAP INFORMATION DISCLAIMER EMC makes no representation and undertakes no obligations with regard to product planning

More information

THE EMC ISILON STORY. Big Data In The Enterprise. Copyright 2012 EMC Corporation. All rights reserved.

THE EMC ISILON STORY. Big Data In The Enterprise. Copyright 2012 EMC Corporation. All rights reserved. THE EMC ISILON STORY Big Data In The Enterprise 2012 1 Big Data In The Enterprise Isilon Overview Isilon Technology Summary 2 What is Big Data? 3 The Big Data Challenge File Shares 90 and Archives 80 Bioinformatics

More information

EMC IRODS RESOURCE DRIVERS

EMC IRODS RESOURCE DRIVERS EMC IRODS RESOURCE DRIVERS PATRICK COMBES: PRINCIPAL SOLUTION ARCHITECT, LIFE SCIENCES 1 QUICK AGENDA Intro to Isilon (~2 hours) Isilon resource driver Intro to ECS (~1.5 hours) ECS Resource driver Possibilities

More information

How to Hadoop Without the Worry: Protecting Big Data at Scale

How to Hadoop Without the Worry: Protecting Big Data at Scale How to Hadoop Without the Worry: Protecting Big Data at Scale SESSION ID: CDS-W06 Davi Ottenheimer Senior Director of Trust EMC Corporation @daviottenheimer Big Data Trust. Redefined Transparency Relevance

More information

EMC ISILON SCALE-OUT STORAGE PRODUCT FAMILY

EMC ISILON SCALE-OUT STORAGE PRODUCT FAMILY SCALE-OUT STORAGE PRODUCT FAMILY Unstructured data storage made simple ESSENTIALS Simple storage management designed for ease of use Massive scalability of capacity and performance Unmatched efficiency

More information

EMC ISILON SCALE-OUT STORAGE PRODUCT FAMILY

EMC ISILON SCALE-OUT STORAGE PRODUCT FAMILY SCALE-OUT STORAGE PRODUCT FAMILY Storage made simple ESSENTIALS Simple storage designed for ease of use Massive scalability with easy, grow-as-you-go flexibility World s fastest-performing NAS Unmatched

More information

EMC SOLUTION FOR SPLUNK

EMC SOLUTION FOR SPLUNK EMC SOLUTION FOR SPLUNK Splunk validation using all-flash EMC XtremIO and EMC Isilon scale-out NAS ABSTRACT This white paper provides details on the validation of functionality and performance of Splunk

More information

EMC ISILON OneFS OPERATING SYSTEM Powering scale-out storage for the new world of Big Data in the enterprise

EMC ISILON OneFS OPERATING SYSTEM Powering scale-out storage for the new world of Big Data in the enterprise EMC ISILON OneFS OPERATING SYSTEM Powering scale-out storage for the new world of Big Data in the enterprise ESSENTIALS Easy-to-use, single volume, single file system architecture Highly scalable with

More information

Storage made simple. Essentials. Expand it... Simply

Storage made simple. Essentials. Expand it... Simply EMC ISILON SCALE-OUT STORAGE PRODUCT FAMILY Storage made simple Essentials Simple storage management, designed for ease of use Massive scalability with easy, grow-as-you-go flexibility World s fastest

More information

UNLEASH THE POWER OF SYNCPLICITY ENTERPRISE FILE SYNC & SHARE ON-PREM WITH ISILON, VNX, & ATMOS STORAGE

UNLEASH THE POWER OF SYNCPLICITY ENTERPRISE FILE SYNC & SHARE ON-PREM WITH ISILON, VNX, & ATMOS STORAGE 1 UNLEASH THE POWER OF SYNCPLICITY ENTERPRISE FILE SYNC & SHARE ON-PREM WITH ISILON, VNX, & ATMOS STORAGE DAVE WILD & VISHNU KODE 2 ROADMAP INFORMATION DISCLAIMER EMC makes no representation and undertakes

More information

HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics

HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics ESSENTIALS EMC ISILON Use the industry's first and only scale-out NAS solution with native Hadoop

More information

The BIG Data Era has. your storage! Bratislava, Slovakia, 21st March 2013

The BIG Data Era has. your storage! Bratislava, Slovakia, 21st March 2013 The BIG Data Era has arrived Re-invent your storage! Bratislava, Slovakia, 21st March 2013 Luka Topic Regional Manager East Europe EMC Isilon Storage Division luka.topic@emc.com 1 What is Big Data? 2 EXABYTES

More information

DIGITAL STORAGE CONCERNS AND CONSIDERATIONS

DIGITAL STORAGE CONCERNS AND CONSIDERATIONS DIGITAL STORAGE CONCERNS AND CONSIDERATIONS JOE HEWES, EMC OEM Copyright 2015 EMC Corporation. All rights reserved. 1 DIGITAL STORAGE & ARCHIVING FOR NDT BUSINESS DRIVERS WHY DO THIS? Improve Product Safety

More information

Implementation of Hadoop Distributed File System Protocol on OneFS Tanuj Khurana EMC Isilon Storage Division

Implementation of Hadoop Distributed File System Protocol on OneFS Tanuj Khurana EMC Isilon Storage Division Implementation of Hadoop Distributed File System Protocol on OneFS Tanuj Khurana EMC Isilon Storage Division Outline HDFS Overview OneFS Overview HDFS protocol on OneFS HDFS protocol server implementation

More information

EMC ISILON ONEFS OPERATING SYSTEM

EMC ISILON ONEFS OPERATING SYSTEM EMC ISILON ONEFS OPERATING SYSTEM Powering scale-out storage for the Big Data and Object workloads of today and tomorrow ESSENTIALS Easy-to-use, single volume, single file system architecture Highly scalable

More information

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE Hadoop Storage-as-a-Service ABSTRACT This White Paper illustrates how EMC Elastic Cloud Storage (ECS ) can be used to streamline the Hadoop data analytics

More information

SOFTWARE DEFINED SOLUTIONS JEUDI 19 NOVEMBRE 2015. Nicolas EHRMAN Sr Presales SDS

SOFTWARE DEFINED SOLUTIONS JEUDI 19 NOVEMBRE 2015. Nicolas EHRMAN Sr Presales SDS SOFTWARE DEFINED SOLUTIONS JEUDI 19 NOVEMBRE 2015 Nicolas EHRMAN Sr Presales SDS Transform your Datacenter to the next level with EMC SDS EMC SOFTWARE DEFINED STORAGE, A SUCCESS STORY 5 ÈME ÉDITEUR MONDIAL

More information

WHITE PAPER. www.fusionstorm.com. Get Ready for Big Data:

WHITE PAPER. www.fusionstorm.com. Get Ready for Big Data: WHitE PaPER: Easing the Way to the cloud: 1 WHITE PAPER Get Ready for Big Data: How Scale-Out NaS Delivers the Scalability, Performance, Resilience and manageability that Big Data Environments Demand 2

More information

L EVOLUTION DES DATACENTERS : LA STRATEGIE EMC JEUDI 19 NOVEMBRE 2015

L EVOLUTION DES DATACENTERS : LA STRATEGIE EMC JEUDI 19 NOVEMBRE 2015 L EVOLUTION DES DATACENTERS : LA STRATEGIE EMC JEUDI 19 NOVEMBRE 2015 Gilles POTARD SAS Business Director APPLICATION CHARACTERISTICS TIERED apps Virtualized Transactional processing PBs data Block & File

More information

Data Security in Hadoop

Data Security in Hadoop Data Security in Hadoop Eric Mizell Director, Solution Engineering Page 1 What is Data Security? Data Security for Hadoop allows you to administer a singular policy for authentication of users, authorize

More information

EMC ISILON MULTITENANCY FOR HADOOP BIG DATA ANALYTICS

EMC ISILON MULTITENANCY FOR HADOOP BIG DATA ANALYTICS EMC ISILON MULTITENANCY FOR HADOOP BIG DATA ANALYTICS ABSTRACT The EMC Isilon scale-out storage platform provides multitenancy through access zones that segregate tenants and their data sets. An access

More information

EMC s Enterprise Hadoop Solution. By Julie Lockner, Senior Analyst, and Terri McClure, Senior Analyst

EMC s Enterprise Hadoop Solution. By Julie Lockner, Senior Analyst, and Terri McClure, Senior Analyst White Paper EMC s Enterprise Hadoop Solution Isilon Scale-out NAS and Greenplum HD By Julie Lockner, Senior Analyst, and Terri McClure, Senior Analyst February 2012 This ESG White Paper was commissioned

More information

HDP Hadoop From concept to deployment.

HDP Hadoop From concept to deployment. HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some

More information

Hortonworks and ODP: Realizing the Future of Big Data, Now Manila, May 13, 2015

Hortonworks and ODP: Realizing the Future of Big Data, Now Manila, May 13, 2015 Hortonworks and ODP: Realizing the Future of Big Data, Now Manila, May 13, 2015 We Do Hadoop Fall 2014 Page 1 HDP delivers a comprehensive data management platform GOVERNANCE Hortonworks Data Platform

More information

Can Storage Fix Hadoop

Can Storage Fix Hadoop Can Storage Fix Hadoop John Webster, Senior Partner 9/18/2013 1 Agenda What is the Internet Data Center and how is it different from Enterprise Data Center? How is the Apache Software Foundation (ASF)

More information

Data Storage. Vendor Neutral Data Archiving. May 2015 Sue Montagna. Imagination at work. GE Proprietary Information

Data Storage. Vendor Neutral Data Archiving. May 2015 Sue Montagna. Imagination at work. GE Proprietary Information Data Storage Vendor Neutral Data Archiving May 2015 Sue Montagna Imagination at work GE Proprietary Information Vendor Neutral Archiving Storing data in a standard format with a standard interface, such

More information

Tactical Advantage for Data Management at Scale and gaining value. Callan Fox, Emerging Technologies Division, EMC.

Tactical Advantage for Data Management at Scale and gaining value. Callan Fox, Emerging Technologies Division, EMC. Tactical Advantage for Data Management at Scale and gaining value Callan Fox, Emerging Technologies Division, EMC. 1 How Much Data Is There? 44 Zettabytes 1 ZB = 1B TBs 7.6B people 200B things 4 Lens 368

More information

Implementing the Hadoop Distributed File System Protocol on OneFS Jeff Hughes EMC Isilon

Implementing the Hadoop Distributed File System Protocol on OneFS Jeff Hughes EMC Isilon Implementing the Hadoop Distributed File System Protocol on OneFS Jeff Hughes EMC Isilon Outline Hadoop Overview OneFS Overview MapReduce + OneFS Details of isi_hdfs_d Wrap up & Questions 2 Hadoop Overview

More information

Protecting Big Data Data Protection Solutions for the Business Data Lake

Protecting Big Data Data Protection Solutions for the Business Data Lake White Paper Protecting Big Data Data Protection Solutions for the Business Data Lake Abstract Big Data use cases are maturing and customers are using Big Data to improve top and bottom line revenues. With

More information

Comprehensive Analytics on the Hortonworks Data Platform

Comprehensive Analytics on the Hortonworks Data Platform Comprehensive Analytics on the Hortonworks Data Platform We do Hadoop. Page 1 Page 2 Back to 2005 Page 3 Vertical Scaling Page 4 Vertical Scaling Page 5 Vertical Scaling Page 6 Horizontal Scaling Page

More information

RED HAT STORAGE PORTFOLIO OVERVIEW

RED HAT STORAGE PORTFOLIO OVERVIEW RED HAT STORAGE PORTFOLIO OVERVIEW Andrew Hatfield Practice Lead Cloud Storage and Big Data MILCIS November 2015 THE RED HAT STORAGE MISSION To offer a unified, open software-defined storage portfolio

More information

EMC SOLUTION FOR AGILE AND ROBUST ANALYTICS ON HADOOP DATA LAKE WITH PIVOTAL HDB

EMC SOLUTION FOR AGILE AND ROBUST ANALYTICS ON HADOOP DATA LAKE WITH PIVOTAL HDB EMC SOLUTION FOR AGILE AND ROBUST ANALYTICS ON HADOOP DATA LAKE WITH PIVOTAL HDB ABSTRACT As companies increasingly adopt data lakes as a platform for storing data from a variety of sources, the need for

More information

EXPLORATION TECHNOLOGY REQUIRES A RADICAL CHANGE IN DATA ANALYSIS

EXPLORATION TECHNOLOGY REQUIRES A RADICAL CHANGE IN DATA ANALYSIS EXPLORATION TECHNOLOGY REQUIRES A RADICAL CHANGE IN DATA ANALYSIS EMC Isilon solutions for oil and gas EMC PERSPECTIVE TABLE OF CONTENTS INTRODUCTION: THE HUNT FOR MORE RESOURCES... 3 KEEPING PACE WITH

More information

HDP Enabling the Modern Data Architecture

HDP Enabling the Modern Data Architecture HDP Enabling the Modern Data Architecture Herb Cunitz President, Hortonworks Page 1 Hortonworks enables adoption of Apache Hadoop through HDP (Hortonworks Data Platform) Founded in 2011 Original 24 architects,

More information

Communicating with the Elephant in the Data Center

Communicating with the Elephant in the Data Center Communicating with the Elephant in the Data Center Who am I? Instructor Consultant Opensource Advocate http://www.laubersoltions.com sml@laubersolutions.com Twitter: @laubersm Freenode: laubersm Outline

More information

Sujee Maniyam, ElephantScale

Sujee Maniyam, ElephantScale Hadoop PRESENTATION 2 : New TITLE and GOES Noteworthy HERE Sujee Maniyam, ElephantScale SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member

More information

Agenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR

Agenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR 1 Agenda Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback 2 A World of Connected Devices Need a new data management architecture for Internet of Things 21% the % of

More information

Upcoming Announcements

Upcoming Announcements Enterprise Hadoop Enterprise Hadoop Jeff Markham Technical Director, APAC jmarkham@hortonworks.com Page 1 Upcoming Announcements April 2 Hortonworks Platform 2.1 A continued focus on innovation within

More information

EMC BIG DATA GIS INFRASTRUCTURE

EMC BIG DATA GIS INFRASTRUCTURE EMC BIG DATA GIS INFRASTRUCTURE EMC Isilon And Scale-Out NAS Sergei Butenko District Manager, Baltic region Sergei.Butenko@emc.com 1 IN A DECADE THE DIGITAL UNIVERSE WILL BE 35 ZETTABYTES 35,000,000,000,000,000,000,000

More information

Big Data Management and Security

Big Data Management and Security Big Data Management and Security Audit Concerns and Business Risks Tami Frankenfield Sr. Director, Analytics and Enterprise Data Mercury Insurance What is Big Data? Velocity + Volume + Variety = Value

More information

SOLUTIONS CLOUD - DPS JEUDI 19 NOVEMBRE 2015

SOLUTIONS CLOUD - DPS JEUDI 19 NOVEMBRE 2015 SOLUTIONS CLOUD - DPS JEUDI 19 NOVEMBRE 2015 Christian LE CORRE - DPS Advisory Systems Engineer DATA PROTECTION EVERYWHERE On Premise Virtual Hybrid Born In The Cloud 30 Retention (LTR) BaaS & DRaaS Archive

More information

Integrated Grid Solutions. and Greenplum

Integrated Grid Solutions. and Greenplum EMC Perspective Integrated Grid Solutions from SAS, EMC Isilon and Greenplum Introduction Intensifying competitive pressure and vast growth in the capabilities of analytic computing platforms are driving

More information

THE EMC ISILON SCALE-OUT DATA LAKE

THE EMC ISILON SCALE-OUT DATA LAKE THE EMC ISILON SCALE-OUT DATA LAKE Key capabilities ABSTRACT This white paper provides an introduction to the EMC Isilon scale-out data lake as the key enabler to store, manage, and protect unstructured

More information

EMC HADOOP AS A SERVICE SOLUTION

EMC HADOOP AS A SERVICE SOLUTION White Paper EMC HADOOP AS A SERVICE SOLUTION EMC Isilon, Pivotal HD, VMware vsphere Big Data Extensions Hadoop for service providers Virtualized and shared infrastructure Global Solutions Sales Abstract

More information

Hadoop Ecosystem B Y R A H I M A.

Hadoop Ecosystem B Y R A H I M A. Hadoop Ecosystem B Y R A H I M A. History of Hadoop Hadoop was created by Doug Cutting, the creator of Apache Lucene, the widely used text search library. Hadoop has its origins in Apache Nutch, an open

More information

Frequently Asked Questions: EMC Isilon Data at Rest Encryption Solution

Frequently Asked Questions: EMC Isilon Data at Rest Encryption Solution 1 Frequently Asked Questions: EMC Isilon Data at Rest Encryption Solution Table of Contents What s New? Target Customers Customer Benefits Competitive Positioning Technical Sales Questions General Sales

More information

EMC Solutions for Oil & Gas. Rune Olsen Senior Systems Engineer

EMC Solutions for Oil & Gas. Rune Olsen Senior Systems Engineer EMC Solutions for Oil & Gas Rune Olsen Senior Systems Engineer 1 Challenges in the Energy Value Chain EXPLORATION DESIGN/ CONSTRUCTION HANDOVER OPERATION/ MAINTENANCE TRANSMISSION/ DISTRIBUTION Survey

More information

There's Plenty of Room in the Cloud

There's Plenty of Room in the Cloud There's Plenty of Room in the Cloud [Shameless reference to Feynman s talk from 1959] Lecturer: Zoran Dimitrijevic Altiscale, Inc. Spring 2015 CS290B -- Cloud Computing 50 Years of Moore

More information

EMC ISILON NL-SERIES. Specifications. EMC Isilon NL400. EMC Isilon NL410 ARCHITECTURE

EMC ISILON NL-SERIES. Specifications. EMC Isilon NL400. EMC Isilon NL410 ARCHITECTURE EMC ISILON NL-SERIES The challenge of cost-effectively storing and managing data is an ever-growing concern. You have to weigh the cost of storing certain aging data sets against the need for quick access.

More information

THE FUTURE OF STORAGE IS SOFTWARE DEFINED. Jasper Geraerts Business Manager Storage Benelux/Red Hat

THE FUTURE OF STORAGE IS SOFTWARE DEFINED. Jasper Geraerts Business Manager Storage Benelux/Red Hat THE FUTURE OF STORAGE IS SOFTWARE DEFINED Jasper Geraerts Business Manager Storage Benelux/Red Hat THE FUTURE OF STORAGE Traditional Storage Complex proprietary silos Open, Software-Defined Storage Standardized,

More information

HADOOP ON EMC ISILON SCALE-OUT NAS

HADOOP ON EMC ISILON SCALE-OUT NAS White Paper HADOOP ON EMC ISILON SCALE-OUT NAS Abstract This white paper details the way EMC Isilon Scale-out NAS can be used to support a Hadoop data analytics workflow for an enterprise. It describes

More information

Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments

Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments Important Notice 2010-2015 Cloudera, Inc. All rights reserved. Cloudera, the Cloudera logo, Cloudera Impala, Impala, and

More information

EMC Federation Big Data Solutions. Copyright 2015 EMC Corporation. All rights reserved.

EMC Federation Big Data Solutions. Copyright 2015 EMC Corporation. All rights reserved. EMC Federation Big Data Solutions 1 Introduction to data analytics Federation offering 2 Traditional Analytics! Traditional type of data analysis, sometimes called Business Intelligence! Type of analytics

More information

VxRACK : L HYPER-CONVERGENCE AVEC L EXPERIENCE VCE JEUDI 19 NOVEMBRE 2015. Jean-Baptiste ROBERJOT - VCE - Software Defined Specialist

VxRACK : L HYPER-CONVERGENCE AVEC L EXPERIENCE VCE JEUDI 19 NOVEMBRE 2015. Jean-Baptiste ROBERJOT - VCE - Software Defined Specialist VxRACK : L HYPER-CONVERGENCE AVEC L EXPERIENCE VCE JEUDI 19 NOVEMBRE 2015 Jean-Baptiste ROBERJOT - VCE - Software Defined Specialist Who is VCE Today? #1 Market Share & Gartner MQ position 96% Customer

More information

HDFS Federation. Sanjay Radia Founder and Architect @ Hortonworks. Page 1

HDFS Federation. Sanjay Radia Founder and Architect @ Hortonworks. Page 1 HDFS Federation Sanjay Radia Founder and Architect @ Hortonworks Page 1 About Me Apache Hadoop Committer and Member of Hadoop PMC Architect of core-hadoop @ Yahoo - Focusing on HDFS, MapReduce scheduler,

More information

Collaborative Big Data Analytics. Copyright 2012 EMC Corporation. All rights reserved.

Collaborative Big Data Analytics. Copyright 2012 EMC Corporation. All rights reserved. Collaborative Big Data Analytics 1 Big Data Is Less About Size, And More About Freedom TechCrunch!!!!!!!!! Total data: bigger than big data 451 Group Findings: Big Data Is More Extreme Than Volume Gartner!!!!!!!!!!!!!!!

More information

EMC ISILON HD-SERIES. Specifications. EMC Isilon HD400 ARCHITECTURE

EMC ISILON HD-SERIES. Specifications. EMC Isilon HD400 ARCHITECTURE EMC ISILON HD-SERIES The rapid growth of unstructured data combined with increasingly stringent compliance requirements is resulting in a growing need for efficient data archiving solutions that can store

More information

Case Study : 3 different hadoop cluster deployments

Case Study : 3 different hadoop cluster deployments Case Study : 3 different hadoop cluster deployments Lee moon soo moon@nflabs.com HDFS as a Storage Last 4 years, our HDFS clusters, stored Customer 1500 TB+ data safely served 375,000 TB+ data to customer

More information

Introduction to Big data. Why Big data? Case Studies. Introduction to Hadoop. Understanding Features of Hadoop. Hadoop Architecture.

Introduction to Big data. Why Big data? Case Studies. Introduction to Hadoop. Understanding Features of Hadoop. Hadoop Architecture. Big Data Hadoop Administration and Developer Course This course is designed to understand and implement the concepts of Big data and Hadoop. This will cover right from setting up Hadoop environment in

More information

EMC ISILON X-SERIES. Specifications. EMC Isilon X200. EMC Isilon X210. EMC Isilon X410 ARCHITECTURE

EMC ISILON X-SERIES. Specifications. EMC Isilon X200. EMC Isilon X210. EMC Isilon X410 ARCHITECTURE EMC ISILON X-SERIES EMC Isilon X200 EMC Isilon X210 The EMC Isilon X-Series, powered by the OneFS operating system, uses a highly versatile yet simple scale-out storage architecture to speed access to

More information

Understanding Enterprise NAS

Understanding Enterprise NAS Anjan Dave, Principal Storage Engineer LSI Corporation Author: Anjan Dave, Principal Storage Engineer, LSI Corporation SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA

More information

Adobe Deploys Hadoop as a Service on VMware vsphere

Adobe Deploys Hadoop as a Service on VMware vsphere Adobe Deploys Hadoop as a Service A TECHNICAL CASE STUDY APRIL 2015 Table of Contents A Technical Case Study.... 3 Background... 3 Why Virtualize Hadoop on vsphere?.... 3 The Adobe Marketing Cloud and

More information

Security. Reliability. Performance. Flexibility. Scalability

Security. Reliability. Performance. Flexibility. Scalability ESG Lab Review VCE Vblock Systems with EMC Isilon for Enterprise Hadoop Date: November 2014 Author: Tony Palmer, Senior ESG Lab Analyst, and Mike Leone, ESG Lab Analyst Abstract: This ESG Lab review documents

More information

TRANSFORM YOUR BUSINESS: BIG DATA AND ANALYTICS WITH VCE AND EMC

TRANSFORM YOUR BUSINESS: BIG DATA AND ANALYTICS WITH VCE AND EMC TRANSFORM YOUR BUSINESS: BIG DATA AND ANALYTICS WITH VCE AND EMC Vision Big data and analytic initiatives within enterprises have been rapidly maturing from experimental efforts to production-ready deployments.

More information

Big + Fast + Safe + Simple = Lowest Technical Risk

Big + Fast + Safe + Simple = Lowest Technical Risk Big + Fast + Safe + Simple = Lowest Technical Risk The Synergy of Greenplum and Isilon Architecture in HP Environments Steffen Thuemmel (Isilon) Andreas Scherbaum (Greenplum) 1 Our problem 2 What is Big

More information

HADOOP. Revised 10/19/2015

HADOOP. Revised 10/19/2015 HADOOP Revised 10/19/2015 This Page Intentionally Left Blank Table of Contents Hortonworks HDP Developer: Java... 1 Hortonworks HDP Developer: Apache Pig and Hive... 2 Hortonworks HDP Developer: Windows...

More information

Introduction to Hadoop HDFS and Ecosystems. Slides credits: Cloudera Academic Partners Program & Prof. De Liu, MSBA 6330 Harvesting Big Data

Introduction to Hadoop HDFS and Ecosystems. Slides credits: Cloudera Academic Partners Program & Prof. De Liu, MSBA 6330 Harvesting Big Data Introduction to Hadoop HDFS and Ecosystems ANSHUL MITTAL Slides credits: Cloudera Academic Partners Program & Prof. De Liu, MSBA 6330 Harvesting Big Data Topics The goal of this presentation is to give

More information

The Evolving Apache Hadoop Eco-System

The Evolving Apache Hadoop Eco-System The Evolving Apache Hadoop Eco-System What it means for Big Data Analytics and Storage Sanjay Radia Architect/Founder, Hortonworks Inc. All Rights Reserved Page 1 Outline Hadoop and Big Data Analytics

More information

EMC ISILON HD-SERIES. Specifications. EMC Isilon HD400 ARCHITECTURE

EMC ISILON HD-SERIES. Specifications. EMC Isilon HD400 ARCHITECTURE EMC ISILON HD-SERIES The rapid growth of unstructured data combined with increasingly stringent compliance requirements is resulting in a growing need for efficient data archiving solutions that can store

More information

Infomatics. Big-Data and Hadoop Developer Training with Oracle WDP

Infomatics. Big-Data and Hadoop Developer Training with Oracle WDP Big-Data and Hadoop Developer Training with Oracle WDP What is this course about? Big Data is a collection of large and complex data sets that cannot be processed using regular database management tools

More information

Hadoop: Embracing future hardware

Hadoop: Embracing future hardware Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop

More information

Deploying Silver Peak VXOA with EMC Isilon SyncIQ. February 2012. www.silver-peak.com

Deploying Silver Peak VXOA with EMC Isilon SyncIQ. February 2012. www.silver-peak.com Deploying Silver Peak VXOA with EMC Isilon SyncIQ February 2012 www.silver-peak.com Table of Contents Table of Contents Overview... 3 Solution Components... 3 EMC Isilon...3 Isilon SyncIQ... 3 Silver Peak

More information

VIRTUALIZING HADOOP IN LARGE-SCALE INFRASTRUCTURES

VIRTUALIZING HADOOP IN LARGE-SCALE INFRASTRUCTURES VIRTUALIZING HADOOP IN LARGE-SCALE INFRASTRUCTURES How Adobe Systems achieved breakthrough results in Big Data analytics with Hadoop-as-a-Service ABSTRACT Large-scale Apache Hadoop analytics have long

More information

Large scale processing using Hadoop. Ján Vaňo

Large scale processing using Hadoop. Ján Vaňo Large scale processing using Hadoop Ján Vaňo What is Hadoop? Software platform that lets one easily write and run applications that process vast amounts of data Includes: MapReduce offline computing engine

More information

(Scale Out NAS System)

(Scale Out NAS System) For Unlimited Capacity & Performance Clustered NAS System (Scale Out NAS System) Copyright 2010 by Netclips, Ltd. All rights reserved -0- 1 2 3 4 5 NAS Storage Trend Scale-Out NAS Solution Scaleway Advantages

More information

Enabling High performance Big Data platform with RDMA

Enabling High performance Big Data platform with RDMA Enabling High performance Big Data platform with RDMA Tong Liu HPC Advisory Council Oct 7 th, 2014 Shortcomings of Hadoop Administration tooling Performance Reliability SQL support Backup and recovery

More information

Hadoop and its Usage at Facebook. Dhruba Borthakur dhruba@apache.org, June 22 rd, 2009

Hadoop and its Usage at Facebook. Dhruba Borthakur dhruba@apache.org, June 22 rd, 2009 Hadoop and its Usage at Facebook Dhruba Borthakur dhruba@apache.org, June 22 rd, 2009 Who Am I? Hadoop Developer Core contributor since Hadoop s infancy Focussed on Hadoop Distributed File System Facebook

More information

EMC ISILON BEST PRACTICES FOR HADOOP DATA STORAGE

EMC ISILON BEST PRACTICES FOR HADOOP DATA STORAGE EMC ISILON BEST PRACTICES FOR HADOOP DATA STORAGE ABSTRACT This paper describes the best practices for setting up and managing the HDFS service on an EMC Isilon cluster to optimize data storage for Hadoop

More information

Modernizing Your Data Warehouse for Hadoop

Modernizing Your Data Warehouse for Hadoop Modernizing Your Data Warehouse for Hadoop Big data. Small data. All data. Audie Wright, DW & Big Data Specialist Audie.Wright@Microsoft.com O 425-538-0044, C 303-324-2860 Unlock Insights on Any Data Taking

More information

Hadoop implementation of MapReduce computational model. Ján Vaňo

Hadoop implementation of MapReduce computational model. Ján Vaňo Hadoop implementation of MapReduce computational model Ján Vaňo What is MapReduce? A computational model published in a paper by Google in 2004 Based on distributed computation Complements Google s distributed

More information

Isilon OneFS. Version 7.2.1. OneFS Migration Tools Guide

Isilon OneFS. Version 7.2.1. OneFS Migration Tools Guide Isilon OneFS Version 7.2.1 OneFS Migration Tools Guide Copyright 2015 EMC Corporation. All rights reserved. Published in USA. Published July, 2015 EMC believes the information in this publication is accurate

More information

How Companies are! Using Spark

How Companies are! Using Spark How Companies are! Using Spark And where the Edge in Big Data will be Matei Zaharia History Decreasing storage costs have led to an explosion of big data Commodity cluster software, like Hadoop, has made

More information

Dominik Wagenknecht Accenture

Dominik Wagenknecht Accenture Dominik Wagenknecht Accenture Improving Mainframe Performance with Hadoop October 17, 2014 Organizers General Partner Top Media Partner Media Partner Supporters About me Dominik Wagenknecht Accenture Vienna

More information

White. Paper. EMC Isilon: A Scalable Storage Platform for Big Data. April 2014

White. Paper. EMC Isilon: A Scalable Storage Platform for Big Data. April 2014 White Paper EMC Isilon: A Scalable Storage Platform for Big Data By Nik Rouda, Senior Analyst and Terri McClure, Senior Analyst April 2014 This ESG White Paper was commissioned by EMC Isilon and is distributed

More information

Apache Hadoop FileSystem and its Usage in Facebook

Apache Hadoop FileSystem and its Usage in Facebook Apache Hadoop FileSystem and its Usage in Facebook Dhruba Borthakur Project Lead, Apache Hadoop Distributed File System dhruba@apache.org Presented at Indian Institute of Technology November, 2010 http://www.facebook.com/hadoopfs

More information

AUTOMATED DATA RETENTION WITH EMC ISILON SMARTLOCK

AUTOMATED DATA RETENTION WITH EMC ISILON SMARTLOCK White Paper AUTOMATED DATA RETENTION WITH EMC ISILON SMARTLOCK Abstract EMC Isilon SmartLock protects critical data against accidental, malicious or premature deletion or alteration. Whether you need to

More information

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84 Index A Amazon Web Services (AWS), 50, 58 Analytics engine, 21 22 Apache Kafka, 38, 131 Apache S4, 38, 131 Apache Sqoop, 37, 131 Appliance pattern, 104 105 Application architecture, big data analytics

More information

EMC Isilon Scale-Out Data Lake Foundation Essential Capabilities for Building Big Data Infrastructure

EMC Isilon Scale-Out Data Lake Foundation Essential Capabilities for Building Big Data Infrastructure Lab Validation Brief EMC Isilon Scale-Out Data Lake Foundation Essential Capabilities for Building Big Data Infrastructure By Ashish Nadkarni, IDC Storage Team Sponsored by EMC Isilon March 2016 Lab Validation

More information

ARCHIVE SOLUTIONS WITH EMC ISILON SCALE-OUT NAS

ARCHIVE SOLUTIONS WITH EMC ISILON SCALE-OUT NAS ARCHIVE SOLUTIONS WITH EMC ISILON SCALE-OUT NAS ABSTRACT This white paper outlines the principles and concepts involved in implementing an enterprise-class archive repository using EMC Isilon scale-out

More information

Hadoop & its Usage at Facebook

Hadoop & its Usage at Facebook Hadoop & its Usage at Facebook Dhruba Borthakur Project Lead, Hadoop Distributed File System dhruba@apache.org Presented at the The Israeli Association of Grid Technologies July 15, 2009 Outline Architecture

More information

MODERNIZE WITH ALL-FLASH

MODERNIZE WITH ALL-FLASH MODERNIZE WITH ALL-FLASH MODERNIZE WITHOUT COMPROMISE In today s lightning-fast digital world, it s critical for businesses to make their move to the Modern Data Center where flash storage reigns supreme.

More information

THE HADOOP DISTRIBUTED FILE SYSTEM

THE HADOOP DISTRIBUTED FILE SYSTEM THE HADOOP DISTRIBUTED FILE SYSTEM Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler Presented by Alexander Pokluda October 7, 2013 Outline Motivation and Overview of Hadoop Architecture,

More information

EMC BACKUP MEETS BIG DATA

EMC BACKUP MEETS BIG DATA EMC BACKUP MEETS BIG DATA Strategies To Protect Greenplum, Isilon And Teradata Systems 1 Agenda Big Data: Overview, Backup and Recovery EMC Big Data Backup Strategy EMC Backup and Recovery Solutions for

More information

Simple. Extensible. Open.

Simple. Extensible. Open. White Paper Simple. Extensible. Open. Unleash the Value of Data with EMC ViPR Global Data Services Abstract The following paper opens with the evolution of enterprise storage infrastructure in the era

More information

Information Builders Mission & Value Proposition

Information Builders Mission & Value Proposition Value 10/06/2015 2015 MapR Technologies 2015 MapR Technologies 1 Information Builders Mission & Value Proposition Economies of Scale & Increasing Returns (Note: Not to be confused with diminishing returns

More information

Hadoop 2.6 Configuration and More Examples

Hadoop 2.6 Configuration and More Examples Hadoop 2.6 Configuration and More Examples Big Data 2015 Apache Hadoop & YARN Apache Hadoop (1.X)! De facto Big Data open source platform Running for about 5 years in production at hundreds of companies

More information

Hadoop Job Oriented Training Agenda

Hadoop Job Oriented Training Agenda 1 Hadoop Job Oriented Training Agenda Kapil CK hdpguru@gmail.com Module 1 M o d u l e 1 Understanding Hadoop This module covers an overview of big data, Hadoop, and the Hortonworks Data Platform. 1.1 Module

More information

Isilon: Scalable solutions using clustered storage

Isilon: Scalable solutions using clustered storage Isilon: Scalable solutions using clustered storage TERENA Storage WG Conference September, 2008 Rob Anderson Systems Engineering Manager, UK & Ireland rob@isilon.com Isilon at HEAnet HEAnet were looking

More information

Chase Wu New Jersey Ins0tute of Technology

Chase Wu New Jersey Ins0tute of Technology CS 698: Special Topics in Big Data Chapter 4. Big Data Analytics Platforms Chase Wu New Jersey Ins0tute of Technology Some of the slides have been provided through the courtesy of Dr. Ching-Yung Lin at

More information

HADOOP ADMINISTATION AND DEVELOPMENT TRAINING CURRICULUM

HADOOP ADMINISTATION AND DEVELOPMENT TRAINING CURRICULUM HADOOP ADMINISTATION AND DEVELOPMENT TRAINING CURRICULUM 1. Introduction 1.1 Big Data Introduction What is Big Data Data Analytics Bigdata Challenges Technologies supported by big data 1.2 Hadoop Introduction

More information