SQream Technologies Ltd - Confiden7al

Transcription

1 SQream Technologies Ltd - Confiden7al 1

2 Ge#ng Big Data Done On a GPU- Based Database Ori Netzer VP Product 26- Mar- 14

3 Analy7cs Performance - 3 TB, 18 Billion records SQream Database 400x More Cost Efficient!

4 Big Data 3V s - Volume 4

5 Big Data 3V s Vareity 5

6 Big Data 3V s - velocity 6

7 Structured Data 7

8 Un- Structured Data 8

9 Big Data Challenges Volume Growth rate, history Work Data Modeling, ETL Big Data Performance Batch, In- rest, In- Mo7on Variety Data Types, logs, video Velocity Throughput Transac7on/Seconds Time Work, storage, ETL, computa7on 9

10 Big Data Challenges Hardware Disk Space Processing Power Network Skill Set Large clusters New querying language New methods 10

11 GPU advantages in a nutshell Massive parallel compu7ng power Aggressive data crunching Extreme scalability Low power consump7on

12 GPU Challenges in a nutshell PCI- E Limited GRAM New Algorithms SIMD

13 SQream Top Level Architecture

14 The Product - under the hood 14

15 SQream Data Storage 100T Data in rows + Indices + Views a 1 c 3 d 1 C 4 d 1 c 2 4 a d d c d j 50T Columnar Data a 1 c d 1 C d 1 c 9T Crunched Data + Smart Meta Data a d 1 c

16 Storage Foot Print Before 100T AIer 9T

17 DBMS - Query Run Time No Op7miza7ons Select T, sum(a) from Example group by T having (A) > 1000 Read ALL data Process ALL data Query output A B C D E F 1010 N 1 a CPU 900 N 2 b x = 16 cores Long execu7on 7me 5000 N 3 c N 4 f N 5 a N 6 r

18 SQream - Query Run Time Select T, sum(a) from Example group by T having (A) > 1000 Read Pin Point data Process Relevant data Query output A B C D E F 1010 N 1 a N 2 b N 3 c N 4 f N 5 a x GPU = ~5600 cores GPUs Shortest execu7on 7me CPUs 600 N 6 r

19 SQream as Data Warehouse

20 CEP Cluster Data stream: web logs Real time stream processing Ad hoc query processing Direct GPU SQREAM Web App Direct GPU Data stream: CDR InfiniBand Direct GPU SQREAM SQREAM PBX Apply new query on Data in Motion Apply Real time Ad Hoc query on all data in rest

21 High Availability Cluster

22 Analytics as a Services (AaaS)

23 USE CASES & BENCHMARKS 23 23

24 Lab Results 100GB TPC H Time in Seconds Sqream Infobright MSSQL 24

25 SQream vs. Redshii vs. Hadoop* Query Run 7me based on Hapyrus benchmark SQream vs. Redshii vs. Hadoop Time in Sec Sqream Redshii Hadoop Amount of Data hkp:// amazon- redshii- is- 10x- faster- and- cheaper- than- hadoop- hive- slides 25

26 SQream vs. Redshii vs. Hadoop* Data Load Time based on Hapyrus benchmark 18 SQream vs. Redshii vs. Hadoop Time in Hours Sqream Redshii Amount of Data hkp:// amazon- redshii- is- 10x- faster- and- cheaper- than- hadoop- hive- slides 26

27 Cyber Analysis Pakern Matching Use Case load and analyze massive network traffic The Test Load 6000 IP CDR per second Run queries under 10 seconds

28 Cyber Use Case Exis7ng Status Hardware HP Superdome 32 Cores EMC Storage Total Hardware cost: ~$150,000 SoIware Oracle 11g

29 Cyber Use Case SQream Solu7on Hardware Dell T7600 Total Hardware cost: ~$7,000 SoIware SQream DB Dell T7600

30 Results for 8.5B Records ~17TB Run7me query results for IP Traffic Analysis by query number Time in Seconds Sqream Oracle

31 SQream POC on Databases with Orange Silicon Valley Soumik Sinharoy Sr. Product Manager at Orange

32 SQream s DB Saves ~$6,000,000 on Big Data insights $250,000 Price list $6,300,000 Price list X40 cost performance

33 Data Is Growing World Wide IP Traffic Growth

34 Orange Group

35

36 Our Data Is Growing Orange France IP Traffic Growth Monthly CDR Volume Today : 60 Billion Q : 150 Billion

37 What We Plan To Do With Big Data Airborne Datacenters - SuperCompu_ng Real_me processing of sensor data Massive database consolida_on : 400 X savings in CapEX 37

38 Current Challenges Problem Statement - Massive Data Store - Mul7 Dimensional Queries Billion CDR - Reduce execu7on 7me - Reduce TCO IP CDR IPTV VoD Fixed Mul_ Media

39 Revenue Assurance POC vs. Leading RDBMS Scan & Join 35 Billion CDR and 30M Subscribers Challenges: Require massive investment in Hardware DW system Require have manual process for op7mizing the soiware Doesn t scale Time in minutes Loading Time 300M CDRs

40 Query Run Time 300 Million CDRs Q2 Map load on the network by hour, total call, minutes Q3 Map by hour and call type, break down for the load on the network by call type. Sms, data, voice Q4 Map load on the network in specific 7me Q5 - Map load on the network in specific 7me and specific service: voice, sms, data SQream vs. leading RDBMS run7me 300 million leading RDBMS

41 Mul_ Media Repor_ng IP Traffic Analysis on anonymized IP CDR from Orange France Data coming from MMT are monthly received on MMDM server Data are processed for a dynamic repor7ng applica7on Challenges: Require massive investment in Hardware DW system Require have manual process for op7mizing the soiware Time in minutes Loading Time 300M CDRs

42 ~4,300,000,000 Call records ~30,000,000 Subscribers 4 Months 1 Operator - Orange 14 Servers 168 CPU cores 1,344GB RAM 67,200GB SSD 403,200GB HDD 875 Seconds

43 ~4,300,000,000 Call records ~30,000,000 Subscribers 4 Months 1 Operator - Orange 1x K40 GPU 1 Server 8 CPU cores 128GB RAM 3,200GB SSD 10,000GB HDD $250,000 Price list 636 Seconds 18 month of data was also tested Execu7on Time = 2484 Seconds

44 Less is More SQream vs. Leading EDW Q1 Aggrega7on query for IP u7liza7on by users Q2 Group By query for 1 dimension Time(Sec) Q1 Q2 Q1 Q2 1 Month 4 Months

45 SQream Yes, We Scale Linearly! 3000 Grouping for all costumer segments - linear scaling 1 month to 18 months Seconds Month 4 Months 9 Months 13 Months 18 Months Months

46 QUESTIONS??? 46

47 Why SQream? Technology Data Crunching: Faster compression 7me x20 Faster decompression 7me x50-70 Higher compression ra7o x5-15 Compute: Faster MPP in a node x20 Higher scalability x1 node x3000 cores Lower hardware cost $7,000,000 > $15K

48 Why SQream? Hassle free Solu7on TCO Lower power consump7on 1 node vs. 10 nodes Lower storage and license cost x10 x50 Lower footprint 2U vs. 20U Deployment advantages No indexes, no cubicles, no projec7ons. Built- in full scan ad- hoc queries support. No code changes. Simple SQL.

49 SQream the fastest analy_cs ever Thank you