Use Your MySQL Knowledge to Become an Instant Cassandra Guru

Transcription

1 Use Your MySQL Knowledge to Become an Instant Cassandra Guru Percona Live Santa Clara 2014 Robert Hodges CEO Continuent Tim Callaghan VP/Engineering Tokutek

2 Who are we? Robert Hodges CEO at Continuent Database nerd since 1982 starting with M204, RDBMS since 1990, NoSQL since 2012; designed Continuent Tungsten Continuent offers clustering and replication for MySQL and other fine DBMS types Tim Callaghan VP/Engineering at Tokutek Long time database consumer (Oracle) and producer (VoltDB, Tokutek) Tokutek offers Fractal Tree indexes in MySQL (TokuDB) and MongoDB (TokuMX)

3 Cassandra! Cassandra, used by NetFlix, ebay, Twitter, Reddit and many others, is one of today's most popular NoSQLdatabases in use. According to the website, the largest known Cassandra setup involves over 300 TB of data on over 400 machines. High Performance Reads and Writes Linear Scalability High Availability

4 One Good Thing about Cassandra Cassandra makes it easy to scale capacity Existing cluster nodes running out of space Start new nodes and let them join Stored data redistribute automatically

5 One Bad Thing about Cassandra cql> create table foo ( id int primary key, customerid int, orderid int, ordervalue double); OK! <Me: I think I m gonna like Cassandra.> cql> create index idx_foo_1 on foo (customerid, orderid); ERROR! - Secondary indexes only support 1 column <Me: I think I just changed my mind. CQL!= SQL> <Me: (much later) secondary indexes aren t that useful>

6 Today s Question How can you use your MySQL knowledge to get up to speed on Cassandra?

7 CQL CQL is not SQL!

8 What is CQL? Cassandra originally used a Thrift RPC-based API CQL was added in 0.8 Simplifies access Smaller learning curve for SQL users So, you ll feel right at home Create table Data types Insert, update, select, delete But Cassandra isn t pretending to be relational...

9 What ISN T CQL? It s familiar, and then it s not! No joins No foreign keys No not null No sum(), group(), min(), Some ORDER BY support Single row UPDATE Limited secondary indexing INSERT == UPDATE but it all behaves like REPLACE INTO

10 The Bottom Line on CQL It s just a language It s very similar to SQL Don t blame CQL What first appears as a limitation in Cassandra can also be a strength CQL enables us to get started quickly Try Cassandra 0.7 for a little while Commit this to memory... You do not just throw data into Cassandra and add later indexes to make it fast.

11 Schema Design Easy to learn and difficult to master - Nolan Bushnell (founder of Atari and Chuck E. Cheese s Pizza-Time)

12 Coming from a Relational World? Tradeoffs are hard Feature RDBMS Cassandra Single Point of Failure Cross Datacenter Linear Scaling Data Modeling *source = Patrick McFadin (@PatrickMcMcFadin)

13 How is My Data Organized? Relational Model Cassandra Model Database Keyspace Table Column Family Primary Key Row Key Column Name Column Name/Key Column Value Column Value

14 What is a BigTable? Cassandra uses it for the data model Supports extremely wide rows Row lookup is fast and easily distributed Columns are sorted by name Up to 2 billion! Row Key Column Name Column Name Column Name Column Name Column Value Column Value Column Value Column Value Timestamp Timestamp Timestamp Timestamp TTL TTL TTL TTL

15 What About Data Types? Unlike some other NoSQL databases, types are included via CQL Usual suspects ascii, bigint, blob, boolean, decimal, varchar, More interesting uuid - global uniqueness timeuuid - global uniqueness, sorted by time portion inet - IPv4 or IPv6 address varint - variable precision integer

16 How Do I Create a Static Table? CREATE TABLE users ( user varchar, These rows look and feel familiar. varchar, state varchar, PRIMARY KEY (user) ); user state timestamp ttl timestamp ttl Primary key is hashed for location/placement (more on that later) Meaning you cannot range scan by PK No not null or varchar sizing (enforce in applications) Reads (on username) and all writes are easily distributed Schema flexibility without downtime alter table users add column password varchar;

17 No Auto-Increment? Auto-increment is extremely difficult in a distributed system Use a natural primary key, if possible Remember, small primary keys aren t important when denormalizing Or, use uuid / timeuuid Generated in your client applications CREATE TABLE payments ( id timeuuid, user varchar, type varchar, amount decimal, PRIMARY KEY (id) );

18 What About Secondary Indexes? CREATE TABLE users ( user varchar, varchar, state varchar, PRIMARY KEY (user)); -- OPTION 1 : create an index CREATE INDEX idxubs on users (state); -- OPTION 2 : create another table (store data twice) CREATE TABLE usersbystate ( state varchar, user varchar, PRIMARY KEY (state, user));

19 Why Not Create Secondary Indexes? select * from users where state = CA ; Secondary index Pro : Index is automatically maintained Con : Above query sent to entire cluster (slows everyone down) Additional table Pro : Above query is directed to single node (important for scalability) Con : Index is manually maintained (insert data twice, which is OK) General tips Low selectivity is bad - (male/female) Extremely high selectivity is bad - (unique) In general, create additional table and don t busy entire cluster for reads

20 How Do I Create a Dynamic Table? CREATE TABLE payments ( user varchar, - Compound PK creates wide row - Remainder of columns are grouped - Still 1 row per user id timeuuid, type varchar, amount decimal, PRIMARY KEY (user, id) ); id1 type timestamp ttl amount id2 type amount timestamp ttl timestamp ttl timestamp ttl user Affects how data is stored/organized Still returned in row format to CSQL Enables ORDER BY id and range query on id for user =? queries Remember, BigTables support up to 2 billion columns

21 What About Relationships? Relational table emp (empname text PK, deptid int,...); index empidx1 on emp (deptid); table dept (deptid int PK, deptname text); Store department name with employee Cassandra table emp (empname text PK, deptid int, deptname text); table dept ( Store employee names with department (up to ~2 billion) deptid int, deptname text, empname text, PRIMARY KEY ((deptid, deptname), empname)); 1, Accounting FrankJones FredJones SamSmith

22 What About Time Series Data? CREATE TABLE dashboard ( dashboardid text, event_time timestamp, event_value double, PRIMARY KEY (dashboardid, event_time)) WITH CLUSTERING ORDER BY (event_time DESC); great for last n, like dashboards WITH CLUSTERING ORDER BY () Data is stored in given order in BigTable row In this case descending by event_time Easy to access most recent data

23 How Do I Remove Time Series Data? CREATE TABLE dashboard ( dashboardid text, event_time timestamp, event_value double, PRIMARY KEY (dashboardid, event_time)) WITH CLUSTERING ORDER BY (event_time DESC); insert into dashboard (dashboardid, event_time, event_value) values (25, now(), ) using ttl 86400; defined at the data level data magically disappears no more cron jobs

24 Are There Other Cool Schemas? collections : sets, lists, maps an alternative to making the row wide set<text> : ordered by CQL Type comparator list<text> : ordered by insertion map<int,text> : unordered support for 64,000 objects per collection (but don t go crazy) create table users ( user varchar, s set<text>, PRIMARY KEY (user)); insert into users (user, s) values ( tmcallaghan, { tim@tokutek.com, timc@tokutek.com });

25 Topic: Transactions

26 MySQL Transactions and Isolation InnoDB creates MVCC view of data; locks updated rows, commits atomically mysql> BEGIN;... mysql> INSERT INTO sample(data) VALUES ( Hello world! ); mysql> INSERT INTO sample(data) VALUES ( Goodbye world! );... MyISAM locks table mysql> COMMIT; and commits each row immediately

27 Does Cassandra Have Transactions? Transactions includes a lot of things so No: Updates on different rows are separate, like MyISAM Failed transactions on replicas might create partial writes (Cassandra does not guarantee clean-up) Yes: Updates to single rows are atomic and isolated Updates to rows are durable (logged) as well (Important: Cassandra rows can be bigger than MySQL)

28 How Does Cassandra Handle Locks? Cassandra uses timestamps instead create columnfamily sample(id int primary key, data varchar, n int); insert into sample(id, data, n) values(1, 'hello', 25); insert into sample(id, data, n) values(2, 'goodbye', 27); cqlsh:cbench> update sample set data='goodbye!' where id=2; cqlsh:cbench> select id, writetime(data),writetime(n) from sample; id writetime(data) writetime(n) Updated together Updated separately Timestamps are the same for columns updated at the same time

29 How Does Cassandra Handle Isolation? Row updates are completely isolated until they are completed Updates can propagate out to replicas at different times

30 Topic: Replication and HA

31 Review of MySQL Replication

32 How Does Cassandra Replication Work? Cassandra is fully multi-master Updates are allowed at any location Updates can happen even when there is a partition Coordinator Client program issues write db1 db2 db3 db4 Writes proxied to other instances

33 How Many Replicas Are There? The number of replicas and how they are distributed are properties of keyspaces CREATE KEYSPACE cbench WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '3' }; Strategy class to distribute replicas Keep 3 copies of data

34 MySQL Partitioning MySQL partitioning breaks a table into <n> tables PARTITION is actually a storage engine Tables can be partitioned by hash or range Hash = random distribution Range = user controlled distribution (date range) Helpful in big data use-cases Partitions can usually be dropped efficiently Unlike delete from table1 where timefield < 12/31/2012 ;

35 How Does Cassandra Partition Data? Cassandra splits data by key across hosts using a ring to assign locations H A G B F C E db1 ABCDEF db2 CDEFGH db3 EFGHAB db4 GHABCD D Virtual node D gets ⅛ of hash range Copy assigned to host db4 by strategy

36 Partitioning in Action Replica placement is based on primary key Primary Key insert into sample(id, data) values (550e8400-e29b-41d4-a , 'hello!'); Run hash function on value, assign to virtual node, from there to actual hosts

37 What About Conflicts? Any client can update rows from any location Cassandra resolves conflicts using timestamps Conflicts are resolved at the cell level The latest timestamp value wins UPDATE sample SET data = hello, age=35 WHERE id=352 UPDATE sample SET data = bye WHERE id=352 LOSE WIN WIN Id Data Age 352 bye 35

38 So Is This Like Galera? Galera uses ACID transactions with optimistic locking It would accept the first transaction and roll back the second completely UPDATE sample SET data = hello, age=35 WHERE id=352 UPDATE sample SET data = bye WHERE id=352 WIN WIN LOSE Id Data Age 352 bye 35

39 How Does Failover Work? Cassandra does not have failover If a node fails or disappears, others continue Writes or reads may stop if too many replicas fail Coordinator Client program issues write db1 db2 X db4 Writes proxied to other instances

40 Tunable Consistency Clients define the level of consistency cqlsh:cbench> consistency all Consistency level set to ALL. cqlsh:cbench> update rtest set k1=4444 where id=3; Unable to complete request: one or more nodes were unavailable. cqlsh:cbench> select * from cbench.rtest where id=3; Unable to complete request: one or more nodes were unavailable. cqlsh:cbench> consistency quorum Consistency level set to QUORUM. cqlsh:cbench> update rtest set k1=4444 where id=3;

41 What Happens To Failed Writes? Cassandra has several repair mechanisms for failures Hinted Handoff - The coordinator remembers the write and replays it when node returns Read Repair - Coordinator for a later read notices that a value is out of sync Node Repair - Run an external process (nodetool) to scan for and fix inconsistencies

42 What Else is There to Learn? A lot would be an understatement, but here are some topics to consider. You need to rethink your data model, so read up, practice, repeat Data consistency is an application problem Storage/Internals : LSM, Compaction, Tombstones, Bloom Filters

43 What Should You Do?

44 Summary 1 We liked CQL + Tables made it easy to get started HA Scaling model Look [out] for Don t bother trying to port your MySQL application Query patterns are critical, model around them Pay attention to version when reading docs/blogs Cassandra is quickly evolving (0.7, 1.0, 1.2, 2.0, )

45 Summary 2 Highly recommended reading Book: Practical Cassandra: A Developer s Approach Datastax has great docs, Presentations Blogs

46 Questions? Robert Hodges CEO, Tim Callaghan VP/Engineering,