nosql and Non Relational Databases

Size: px

Start display at page:

Download "nosql and Non Relational Databases"

Joella Carpenter
8 years ago
Views:

1 nosql and Non Relational Databases Image src: Matthias Lee Johns Hopkins University

2 What NoSQL? Yes no SQL.. Atleast not only SQL Large class of Non Relaltional Databases trading Consistancy for Availability Easily Scalable (Partitioning) Highly fault tolerent Google, Facebook, Amazon, Twitter et al.

3 What? Why Non Relational? No complicated Relationships Schema light/free Less inter dependencies Easier scaling Higher fault tolerance Distributed Computing Store and search Hashing & MapReduce

4 CAP Theorem Choose 2 and work around the other. Eric Brewer UC Berkley Consistency Availability Patritioning

5 Dropping some of ACID Compromises must be made ACID Atomic All or nothing? partial Consitency eventually consistent Isolation revision history Durability written in stone sometimes

6 BASE BASE Basically Available Soft State Eventual consistency ASYNC conflict resolution and repair

8 ID <unique id> <unique id> <unique id> <unique id> Content Key1, Value Key2, Value Key3, Value Key4, Value Key2, Value Key3, Value Key8, Value Key9, Value Key5, Value Key4, Value Key1, Value Key2, Value Key3, Value Key4, Value Key5, Value Key6, Value Key7, Value Key8, Value

Key8, Value Key9, Value Key5, Value Key4, Value Key1, Value Key2,

9 Things these DBs do... Easily Highlights Fast processing/specific tasks Usage of distributed queries and operations MapReduce/Hadoop Async reads and writes Fire & Forget Flexible schema (often) And its scalable / easy replecation

10 Things these DBs do... Easily Distributed storage (performance/fault tolerance) Increased response time and fault tolerance User Request User Request Master Master Master Slave 1 Slave 2 Slave 3 Slave 4

11 Things these DBs do... Server 2 Distributed storage (locality) North America Server 4 Server 1 Europe Australia Server 3

12 Things these DBs *dont* do... Easily Issues and challenges ACID goes out of the windows No direct translation SQL< >nosql Relatively new field many similar solutions All solutions have different trade offs

13 nosql means mostly no SQL Querying nosql Map/Reduce Various query languages RQL rasdaman CQL Cassandra

14 Map/Reduce Easily distributed method of processing data Fault tolarant map() and reduce() Input reader/partitioner map() Sort and partition reduce() Output writer

15 Map/Reduce Feburary 14th 2013

16 Map/Reduce INPUT map() sorting reduce() Chunk_1 map_out_1 Chunk_2 map_out_2 red_out_1 Chunk_3 Chunk_4 map_out_3 map_out_4 magic red_out_2 Chunk_5 map_out_5 Chunk_6 map_out_6 red_out_3 Chunk_7 map_out_7 Chunk_8 map_out_8

17 Types of nosql DBs Document Databases Key/Value Stores Array Databases Column Oriented Datastores Graph Databases(they exist)

18 Types of nosql DBs Column Oriented Datastores Indexing over Column families Fast aggregation & searching Inline compression Easy sharding Column Families id Fname Lname Zip Street 1 Joe Shmoe Cedar 2 Ralph Peters Birch 3 Mary Lewis Green Name { Location { Joe,Ralph,Mary Shmoe,Peters,Lewis 32818,65636,10337 Cedar,Birch,Green

Lname Zip Street 1 Joe Shmoe 32818 Cedar 2 Ralph Peters 65636 Birch 3 Mary Lewis

19 Types of nosql DBs Column Oriented Datastores Big Table and its clones Hbase Google Hypertable Facebook, Hulu and StumbleUpon Baidu and Rediff

20 Types of nosql DBs Key/Value Stores (simple) Some of the earliest nosql early 90's Easily distributed Storage and Searching Hashtable like structure MapReduce Often used as caching engine O(1) ave lookup time [hash] : bytes[n]

Searching Hashtable like structure MapReduce Often used

21 Types of nosql DBs Document stores (mostly structured K,V store) MongoDB FourSquare, Shutterfly, Intuit, Github & more CouchDB BBC, Canonical, Cern, Android apps & more Redis Digg, Flicker, StackOverflow, Craigslist & more

22 Types of nosql DBs Key/Value Stores BerkleyDB Redis MySQL, Bitcoin, MemcachedDB, SVN & more Digg, Flicker, StackOverflow, Craigslist & more Cassandra (CQL) Facebook, Reddit, Twitter, Netflix & many more

23 Types of nosql DBs Document stores (mostly structured K,V store) Versitile Dynamic schema Eventual consistency Highly Parallelizable Easy replication "_id": "4eea98de1550e2cc ": { "lastmodified": " :03:26", "name": "Peter Lustig", "avatar": "4eea61a11550e26f7d000000", " ": "Peter.Lustig@void.net", hobbies : sleeping }

24 Types of nosql DBs Document stores (mostly structured K,V store) MongoDB FourSquare, Shutterfly, Intuit, Github & more CouchDB BBC, Canonical, Cern, Android apps & more Redis Digg, Flicker, StackOverflow, Craigslist & more

25 Distributed TinyURL crawler CouchDB set it up and relax cluster of unreliable commodity hardware RESTful JSON API Document Store with easy replication Eventual consistency Light weight (runs on phones) Easy replication

26 Distributed TinyURL crawler TinyURL crawler Quick deploy TinyURL resolver Master Slave architecture Replicating Databases Amazon EC2 spot instances

27 Distributed TinyURL crawler TinyURL crawler resolver TinyURL google.com Feburary 14th 2013

28 Distributed TinyURL crawler TinyURL crawler Master R Amazon EC2 R R R R R R

29 Any questions? Comments? Thanks for listening Interested in this? Want to know more? #jhuacm on irc.freenode.net +Matthias Lee github.com/madmaze

Lecture Data Warehouse Systems

Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART C: Novel Approaches in DW NoSQL and MapReduce Stonebraker on Data Warehouses Star and snowflake schemas are a good idea in the DW world C-Stores