The Elephant in the Cloud. Big Data Summit

Transcription

1 The Elephant in the Cloud Big Data Summit May

2 Changing Shape of Data Confidential and Proprietary, Qubole Inc. page 4

3 Emergence of Hadoop Democratization of Data Processing Scalability on Commodity Hardware Confidential and Proprietary, Qubole Inc. page 4

4 Impediments Investment Risk as upfront investment needed to discover value of data Execution Risk in order to come up to speed on the technology and then integrating it Operationalizing this new technology as use case move into production Enabling Accessibility to this technology in the enterprise Confidential and Proprietary, Qubole Inc. page 4

5 Benefits of the Cloud On-demand and Turn Key without the hassles Flexible in supporting different workloads and use cases and growing as the enterprise moves from PoC to Production Accessible in multiple regions and geographies Confidential and Proprietary, Qubole Inc. page 4

6 Cloud and Big Data Use Hadoop on the Cloud to Discover and Accelerate Big Data Use Cases Confidential and Proprietary, Qubole Inc. page 6 6

7 De-risk Hadoop & Big Data 80 node PoC Cluster to start with that grew to 3000 node Cluster Platform from 2007 to 2011 Took 3 months to get the cluster in place and the base software deployed on it Took another 9 months to make it a true strategic platform

8 Cloud based Hadoop as Service Zero upfront investment in infrastructure Instantaneous scaling because of the cloud Hadoop becomes a strategic platform very quickly

9 De-risk Hadoop & Big Data Monitoring GUI Hive/PIG Scheduler Interfaces (ODBC/JDBC) Data Connectors System Mgmt Hadoop Lots of moving parts and open source technologies needed before getting to a data processing platform

10 Cloud based Hadoop as Service Monitoring GUI(Hue) Interfaces (ODBC/JDBC) Data Connectors (MongoAdaptor..) System Mgmt Hadoop Scheduler(Oozie) Hive/PIG Mahout/Weka Fully Integrated Turn Key Platform that enables you to discover the ROI quickly with low execution and investment risk

11 Challenges in Operationalizing Hadoop Managing Growth placed on the infrastructure as usage grows Managing Unpredictability in today s agile development environment where use cases change leading to different types of demands from the infrastructure Managing Open Source as there is constant innovation and internal operations teams have to continue to monitor open source contributions and distros

12 Cloud based Hadoop as a Service Self Managed the growth of the infrastructure is matched with the growth of usage and data dynamically and in an on demand manner Flexibility and Elasticity of the cloud ensures instant availability of different machine types for different use cases Open Source Innovation is managed by the service provider while enterprises can focus on deriving actionable insights from their data

13 Cloud Accessibility

14 Cloud based Hadoop as a Service Sharing and Collaboration of data and analysis is easily enabled across: Geographies Organizations Supply Chains

15 Objections to a Cloud Service Is TCO higher in the rent model vs pay upfront model? Prices on the cloud keep dropping due to economies of scale and competition Storage: 3 cents/gb/month (70% drop in the last 4 weeks) Accenture study found broadly better performance in the same price on the cloud Hadoop-Deployment-Comparison-Study.pdf It is already more cost effective to run on the cloud and it is just going to get better

16 Objections to a Cloud Service Is security and compliance there? For security use encryption For compliance AWS has many new products such as auditability, Even the CIA runs workloads on the cloud

17 Cloud and Big Data Cloud Based Hadoop Services provide a Quick path to a Big Data Platform that Adapts to the needs of an organization while Reducing Costs and Reducing Failure Risk Confidential and Proprietary, Qubole Inc. page 6 17

18 Thank You Ashish Thusoo