Ultra-Scalable Real-Time Big Data Analytics. Ricardo Jimenez-Peris Co-Founder

Transcription

1 Ultra-Scalable Real-Time Big Data Analytics Ricardo Jimenez-Peris Co-Founder

2 Major Pains/Gaps in Data Management 1. Lack of scalability of traditional & cloud DBs. 2. Lack of real-time analytics support. 3. NoSQL lacks consistency guarantees. 4. Lack of scalable data consistency across data stores (SQL and NoSQL). 5. Inability to correlate large data/event streams with operational data. Common Factor: Lack of scalable data consistency.

3 1. Lack of scalability of traditional & cloud DBs State of the art in cloud DBs: Sharding (e.g. Amazon Oracle Relational DB) Google spent 2 years to change its sharded MySQL s F1 database LeanXcale would have saved Google 2 years

4 2. Real-Time Analytics Cutting costs by 80% Cost of ETLs 80% of business analytics Go for a dinner of 200 Euros and pay 800 Euros for the taxi!! OLTP Copy data (ETL) OLAP Transactional DB Consistent Updates LeanXcale analytical queries over transactional DB: Real-Time Analytics no more ETLs!! Data Warehouse Fast Analytical Queries

5 3. NoSQL Lacks Data Consistency NoSQL data stores wanted to scale beyond traditional databases. However, they did not know how to scale transactional processing, so they dismissed it. Bitcoin was stolen millions of dollars due to they were using a nontransactional NoSQL solution. LeanXcale brings transactional data consistency to NoSQL.

6 4. Lack of Consistency Across Data Stores (NoSQL & SQL) Enterprises today combine NoSQL, Hadoop data lakes, SQL, CEP to scale in an ad-hoc manner but they miss data consistency when updating across data stores. Querying across data stores is too hard LeanXcale provides consistent (full ACID) updates across data stores SQL, nosql (Neo4J, MongoDB ) and Hadoop Reduction of development and maintenance cost by 3 times

7 5. Inability to correlate large data streams with operational DB Complex Event Processing (CEP) and data streaming are being embraced my many enterprises. Core technology for IoT and M2M. Current technology is unable to correlate large data streams with the operational DB due to the lack of scalability of operational DB. LeanXcale enables to correlate million events per second with the operational DB Enabling IoT/M2M apps

8 What is LeanXcale? OLTP Full SQL Full ACID DB Polyglot Document DB, Graph DB, key-value DS, Hadoop files OLAP Big Data Analytics CEP Complex Event Processing An Ultra-Scalable integral data management platform for Real-Time Big Data Analytics

9 Unique Distinctive Feature We have solved a core problem in data management of the last three decades. How to scale out transactional processing to very large levels (100s of nodes) Unlike any existing technology we can scale updates with full transactional (full ACID) consistency to very high rates Protected by a filed patent at USPTO and extended to Europe.

10 Ultra-Scalable Transactions Traditional systems have a single-node bottleneck LeanXcale Process & commits transactions in parallel Provides a consistent view Time Time VS Traditional transactional DB

11 Based on Hadoop Ecosystem Query Engine Ultra-Scalable Transactions Key-Value Data Store Apache Hadoop File System

12 Scalability All layers scale linearly TPC-C SPEC jenterprise 2010

13 Role of FP7 Funding and EIT ICT Labs FP7 has funded: The research and the creation of a pre-industrial prototype through CumuloNimbo, CoherentPaaS and LeanBigData projects. EIT ICT Labs is coaching LeanXcale due to being one of the winners of the Idea Challenge competition: Contacting VCs (Berlin node has been very helpful). Contacting potential partners and clients (Madrid node has been very helpful). Funding the attendance to relevant startup events (Web Summit Dublin, ).

14 Contact CONTACT DETAILS Ricardo Jiménez-Peris WWW: