NoSQL's biggest secret: SQL went nowhere Matthew Revell Director of Developer Advocacy, Couchbase

Transcription

1 NoSQL's biggest secret: SQL went nowhere Matthew Revell Director of Developer Advocacy, Couchbase 1

2 Meet my toaster 2

3 A toaster Redundancy built-in Balanced input/output Commodity hardware 3

4 A cruster of toasters Redundancy built-in Balanced input/output Commodity hardware Easily clustered 100% NoSQL! 4

5 A brief history of data storage 5

6 This is data 6

7 But what does it mean? 7

8 This is data 8

9 1960: the first commercial database 9

10 The data model determines the query pattern 10

11 Hierarchical The GOTO statement of databases 11

12 Hierarchical 12

13 Hierarchical CEO CTO CFO CMO SVP HR VP Engineering Chief Architect VP of Comms VP of Demand Gen Director of Product Marketing Head of Media Relations Head of Analyst Relations Head of Event Marketing 13

14 Network The programmer as navigator 14

15 Network Employee Employee Name Matthew Revell Name Owen Hughes Office London Office London Title Next Owner Prior Director of Developer Advocacy Owen Hughes Arun Gupta Adam Blackshaw Title Next Owner Prior Head of Pre-Sales Bindi Bhullar Dipti Borkar Matthew Revell 15

16 Relational (with SQL) Heirarchical A declarative Network/ data query CODASYL language Relational with SQL 16

17 Object oriented databases Why are we flattening everything? 17

18 : NoSQL 18

19 Key value { } personal : [email protected], work : [email protected] 19

20 Document London Developer Advocacy { [email protected] "city": "London", [email protected] "glasses": true, [email protected] "team": "Developer Advocacy", [email protected] "music": "METAAAAAL!" [email protected] } [email protected] [email protected] [email protected] 20

21 Document London and Developer Advocacy 21

22 Column and graph 22

23 Context is all 23

24 There's always a trade-off Offload from some other data store (i.e. caching) Computation offload Speed Scalability Availability Flexibility in what you store Query flexibility 24

25 Querying NoSQL 25

26 It's your problem Photo by Donarreiskoffer. CC-by

27 Manual 2i 27

28 Map/Reduce 28

29 Declarative query 29

30 NoSQL declarative query DBMS-specific Bold, new options SQL-derivatives 30

31 MongoDB query db.staff.find({office: 'London'}) Index document contents db.staff.find({office: {$in:['london', 'Amsterdam']}}) db.staff.insert({name: 'Matthew Revell', office: 'London'}) and query natively db.staff.update({name: 'Matthew Revell', office: 'Amsterdam'}) 31

32 JSONiq XQuery for JSON Declarative language for JSON Functional,composable, set-based 32

33 JSONiq for $p in collection('staff') where $p.serviceyears gt 2 let $name := $p.firstname " " $p.lastname group by $p.office order by $p.serviceyears return { $name, $p.office, $p.serviceyears } 33

34 JSONiq for $captain in collection("captains"), $movie in collection("movies") [ try { $$.captain eq $captain.name } catch * { false } ] return { "captain" : $captain.name, "movie" : $movie.name } 34

35 Why SQL? Creative Commons Attribution-Share Alike 2.5 Generic, image by Per Erik Strandberg 35

36 Cassandra's CQL 36

37 Cassandra's CQL Really looks like SQL Schema is back No JOINs, no GROUP BY 37

38 Cassandra's CQL CREATE TABLE authors ( name text, year int, title text, isbn text, publisher text, PRIMARY KEY (name, year, title) ) WITH CLUSTERING ORDER BY (year DESC); 38

39 Cassandra's CQL INSERT INTO books (title, author, year) VALUES ('Patriot Games', 'Tom Clancy', 1987); INSERT INTO books (title, author, year) VALUES ('Without Remorse', 'Tom Clancy', 1993); 39

40 Cassandra's CQL name year title isbn publisher Tom Clancy 1993 Without Remorse Putnam Tom Clancy 1987 Patriot Games Putnam RowKey: Tom Clancy => (name=1993:without Remorse:ISBN, value= ) => (name=1993:without Remorse:publisher, value=putnam) => (name=1987:patriot Games:ISBN, value= ) => (name=1987:patriot Games:publisher, value=putnam) 40

41 Cassandra's CQL SELECT * FROM authors WHERE name = 'Tom Clancy' AND year >= 1993; 41

42 Cassandra's CQL RowKey: Tom Clancy => (name=1996:executive Orders:publisher, value=putnam) => (name=1996:executive Orders:ISBN, value= ) => (name=1994:debt of Honor:publisher, value=putnam) => (name=1994:debt of Honor:ISBN, value= ) => (name=1993:without Remorse:publisher, value=putnam) => (name=1993:without Remorse:ISBN, value= ) => (name=1991:the Sum of All Fears:publisher, value=putnam) => (name=1991:the Sum of All Fears:ISBN, value= )... => (name=1987:patriot Games:publisher, value=putnam) => (name=1987:patriot Games:ISBN, value= ) 42

43 Cassandra's CQL Not an ad-hoc query language 43

44 This story is mostly about JSON 44

45 SQL++ 45

46 SQL++ Non-relational data is semi-structured Non-relational data is heterogenous JSON in, JSON out! 46

47 What must happen to SQL to make it JSON friendly? Data is nested Sometimes data is missing Data is likely to be found in more than one place JOINs need thinking about 47

48 SQL++ Superset of SQL for semi-structured data Handles missing data gracefully and/or explicitly Can query inside nested data Nests and unnests data JOINs between documents 48

49 N1QL: SQL++ in practice 49

50 Couchbase Server 4.0 High availability cache Key-value store Document database N1QL SQL-like query for JSON 50

51 A profile { " ": "[email protected]", "office": "London", "title": "Director of Developer Advocacy", "team": "Developer Advocacy", "manager": "Arun Gupta", "start-date": " ", "meet-up-groups": ["London", "Dublin", "Manchester"], "conferences": [ { "name": "OSCON Europe", "location": "Amsterdam", "roles": ["booth", "speaker"], "start-date": " ", "end-date": " " }, { "name": "Topconf", "location": "Talinn", "roles": "speaker", "start-date": " ", "end-date": " " }, { "name": "Big Data Strategy", "location": "Vilnius", "roles": "speaker", "start-date": " ", "end-date": " " } ] } 51

52 N1QL N1QL implements much of SQL++ Dive into arrays and objects NEST data from JOINs UNNEST data Gracefully handles MISSING data 52

53 SELECT SELECT FROM `default` WHERE office = "London"; 53

54 ARRAY ELEMENTS SELECT conferences[0].name AS event_name FROM `default`; 54

55 REMOVE MISSING ITEMS SELECT DISTINCT conferences[0].name AS event_name FROM `default` WHERE conferences IS NOT MISSING; 55

56 WHO IS GOING TO DROIDCON SWEDEN? SELECT AS person, conferences[0].name AS event FROM `default` WHERE ANY event in conferences SATISFIES event.name = "Droidcon Sweden" END; 56

57 Updating and deleting DELETE: provide the key to delete the document INSERT: provide a key and some JSON to create a new document UPSERT: as INSERT but will overwrite existing docs UPDATE: change individual values inside existing docs 57

58 A larger data-set: travel-sample 58

59 TRAVEL SAMPLE DATA 59

60 JOINs 60

61 JOINs Retrieve data from multiple documents in a single query Join within a keyspace/bucket Join across keyspaces/buckets 61

62 IN A RELATIONAL DATABASE AIRLINES id country iata icao name callsign 5209 United States UA UAL United Airlines UNITED 1355 United Kingdom BA BAW British Airways SPEEDBIRD AIRPORTS id airportname city country alt lat lon icao tz 507 Heathrow London United Kingdom EGLL Europe/London 3469 San Francisco Intl San Francisco United States KSFO America/Los_Angeles FLIGHTS id airline source destination equipment day flight utc stops UA LHR SFO UA894 02:32:

63 IN JSON { } "callsign": "UNITED", "country": "United States", "iata": "UA", "icao": "UAL", "id": 5209, "name": "United Airlines", "type": "airline" { } "airline": "UA", "airlineid": "airline_5209", "destinationairport": "SFO", "equipment": "777", "id": 57047, "schedule": [ { "day": 0, "flight": "UA894", "utc": "02:32:00" },... ], "sourceairport": "LHR", "stops": 0, "type": "route" { } "airportname": "Heathrow", "city": "London", "country": "United Kingdom", "faa": "LHR", "geo": { "alt": 83, "lat": , "lon": }, "icao": "EGLL", "id": 507, "type": "airport", "tz": "Europe/London" 63

64 A SIMPLE JOIN SELECT * FROM `travel-sample` r JOIN `travel-sample` a ON KEYS r.airlineid WHERE r.sourceairport="lhr" AND r.destinationairport = "SFO"; 64

65 WHO FLIES LHR->SFO? SELECT DISTINCT a.name FROM `travel-sample` r JOIN `travel-sample` a ON KEYS r.airlineid WHERE r.sourceairport="lhr" AND r.destinationairport = "SFO"; 65

66 UNNEST Breaks out nested JSON from the results 66

67 SOMETHING USEFUL SELECT a.name, s.flight, s.utc, r.sourceairport, r.destinationairport, r.equipment FROM `travel-sample` r UNNEST r.schedule s JOIN `travel-sample` a ON KEYS r.airlineid WHERE r.sourceairport="lhr" AND r.destinationairport = "SFO" AND s.day=1 ORDER BY s.utc; 67

68 Next Steps

69 Couchbase Developer Portal developer.couchbase.com 69

70 SQL++ paper 70

71 Forums 71