Enterprise Operational SQL on Hadoop Trafodion Overview

Transcription

1 Enterprise Operational SQL on Hadoop Trafodion Overview Rohit Jain Distinguished & Chief Technologist Strategic & Emerging Technologies Enterprise Database Solutions Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

2 Agenda Directions in Hadoop Directions in NoSQL The case for Enterprise Operational SQL on Hadoop HP s Enterprise Operational SQL on Hadoop Solution Trafodion 2 HP PRIVATE Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

3 Directions in Hadoop Characteristics Lower cost BI & Analytics Open Source & proprietary Eco-system Big Data Volume Elastic Scalability Unstructured Lower cost Software Server & storage MapReduce Machine Learning Stinger Text / Document Images Social media Video Audio Mobile Enterprise Integrated Hadoop Data Lake Real-time Analytics Internet of things Storm Enterprise Readiness & Manageability Apache Yarn HP Data Services Manager Now available on the Vertica Marketplace Kiji 3 HP PRIVATE Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

4 and what s missing What s missing Integration of structured, semistructured, and unstructured support Operational transactional workloads Using Hadoop for all operational SQL needs Not Only Big Data Free at last! Capture data directly into open file structures Open distributed HDFS structures HBase & Hive Accessible for reporting & analytics with no latency Structured Semi- structured Unstructured Transaction Item id Description Cost Price TV Book Type Display Size Resolution Brand Model 3D ISBN Author Publish Date Format Dept Image Review Add item BEGIN WORK INSERT item into Trafodion table ITEM (item_id, desc, cost, price, ) INSERT item attributes for TV or book into HBase table ITEM_ATTR as col-value pairs using item_id END WORK Queries SELECT all TVs WHERE Price > 2000 and Type = Plasma and Display Size > 50 and customer sentiment is very positive Orders needing transactional support across Order and Order Detail Backend operational workloads Order tracking, supply chain, inventory control, 4 HP PRIVATE Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

5 Directions in NoSQL and what s missing Characteristics Big Data Volume Elastic Scalability Lower cost Lower cost Software Server & storage Variety: Semi-structured & Unstructured Distributed across data centers Eventual consistency What s missing Full ACID transactions SQL querying capability Joins Data in open HDFS except for HBase Velocity Key-value (Riak, Redis) Document JSON / BSON (MongoDB, Couchbase) Column families (Cassandra, HBase) Graph (Neo4j, Giraph, Titan) Low Latency High Availability Schema Flexibility In-memory 5 HP PRIVATE Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

6 The Case for Enterprise Operational SQL-on-Hadoop Sector Road Map: SQL-on-Hadoop platforms in 2013 Joseph Turian, March 20, 2013 If a strong player or two emerges in the category, it will completely shake up the big data and database landscape. If Hadoop were operational, it could be used to power websites and store transactions. Traditional SQL databases would no longer be necessary. The data stack would be significantly simplified. An operational database offers write access, not just read access, to data. However, there are other key features for an operational database: concurrency, interactive write speed, and distributed transactional support (guarantees about data consistency). Currently no existing SQL-on-Hadoop solution satisfies these requirements. 5 Reasons Hadoop is Kicking Can and Taking Names Mike Gualtieri, October 22, 2013 #5 The future of Hadoop is real-time and transactional. The key commercial vendors are focusing on fast SQL access, real-time streaming, and manageability features that enterprises demand. The groundwork is being laid for an eruption in data management technologies as Hadoop sneaks its way into the transactional database market. The Future of Hadoop: What Happened & What's Possible? Doug Cutting, Oct So I think the prediction we can make here is that it is inevitable that we will see just about every kind of workload be moved to this platform even Online Transaction Processing. 6 HP PRIVATE Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

7 Trafodion Enterprise Operational SQL on Hadoop NonStop SQL/MX Neoview SeaQuest Open distributed HDFS Semi-structured & unstructured support Schema flexibility Elastic scalability Automatic rebalancing Replication for High Availability (k-safety) Disaster Recovery (via MapR) Column level access control Column level encryption Lower cost cheap storage & servers Space quotas (via MapR) Huge open source & proprietary eco-system Versioning snapshot support & incremental data replication Cloud enabled HP Cloud Services OpenStack Industry trend towards Enterprise Hadoop Lake OLTP K/V & document stores OLTP and ODS on Hadoop Unstructured analytics Trafodion Can join Trafodion, HBase, Hive tables in a single statement Structured OLTP through EDW One of the most powerful database engines in the industry for OLTP and EDW Full ANSI SQL support Full ACID transactional support for multirow, multi- table, & multi-region updates Support for nested loop, merge, hash joins Structured tables, indexes, views Incremental equal height histograms for better execution plans Efficient data flow architecture Referential Integrity, Triggers, Grant/Revoke Security support UDFs for Complex Event processing Workload Management Enterprise class Monitoring & Manageability Compound primary keys Encoding column names for compaction Salting to spread updates 7 HP PRIVATE Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

8 Private Beta program launched Seeking early adopters for POCs Available for internal evaluation: Send to for download details 8 HP PRIVATE Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

9 Thank You Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.