Session 9 Scalable Data Management: NoSQL & Map/Reduce

Transcription

1 Session 9 Scalable Data Management: NoSQL & Map/Reduce Sistemas Distribuidos Diego Sevilla Ruiz DITEC Facultad de Informática Murcia, 2012 Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

2 Contents 1 Cache: memcached 2 NoSQL DBs 3 MapReduce 4 Real REST-based Architectures Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

3 Contents 1 Cache: memcached 2 NoSQL DBs 3 MapReduce 4 Real REST-based Architectures Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

4 Cache memcached Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

5 Cache memcached (ii) 1 function get_foo ( foo_id ) foo = memcached_get (" foo :". foo_id ) 3 return foo if defined foo 5 foo = fetch_ foo_ from_ database ( foo_id ) memcached_set (" foo :". foo_id, foo ) 7 return foo end Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

6 Cache memcached (iii) $ telnet localhost Trying Connected to localhost. 4 Escape character is ^]. get foo 6 VALUE foo 0 2 hi 8 END Simple text-based protocol Values represented as strings (images?) Several servers can be used (sync?) Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

7 Memcache + Erlang Usando merle: > merle : connect (). 2 {ok, < > } > merle : set (key1, [1,2,3]). 4 ok > merle : getkey ( key1 ). 6 [1,2,3] % With versioning 8 > [CASV, V] = merle : getskey ( key1 ). [1,[1,2,3]] merle uses term_to_binary and binary_to_term in the values Allows setting a time of expiration Content has to include headers also Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

8 Memcache + Erlang (ii) Expiration time set: 1 % 5 seconds % 0 is an integer for client flags 3 > merle : set (key3, 0, 5, [1,2,3]). ok 5 94 > merle : getkey ( key3 ). [1,2,3] 7 95 > merle : getkey ( key3 ). [1,2,3] 9 96 > merle : getkey ( key3 ). [1,2,3] > merle : getkey ( key3 ). [1,2,3] > merle : getkey ( key3 ). [1,2,3] > merle : getkey ( key3 ). % after 5 seconds undefined Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

9 Memcache + Erlang (iii) Test & Set, versioning control, transactions (ETag equivalent) % Get the resource and the actual version 2 > [CASV, V] = merle : getskey ( key1 ). [1,[1,2,3]] 4 % Send to client % Get update, and update if CASV is still valid 6 > merle : cas (key1, CASV, 0, [1,2,3,4,5]). ok 8 > merle : getskey ( key1 ). [16,[1,2,3,4,5]] % Now the version is 16. Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

10 Memcache + Erlang (iv) Cache key and value should include also the representation: The key can be hashed with the media type The value can include several representations (for instance, an image in JPEG and PNG formats) 1 -type url () :: string (). -type media_type () :: string (). 3 -type header () :: { string (), string ()}. -type headers () :: [ header () ]. 5 -type key () :: term (). -type value () :: term (). 7 -type map_key () :: { url (), media_type () } -type map_value () :: { headers (), value () } Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

11 Memcache + Erlang (v) % Comes from the HTTP library 2 > Request = { get, "/ yyy / image. png ", [{" Host ", " ZZZ " },{" Accept ", " image / png "}] }, { Method, URI, Headers } = Request, 4 [ ContentType ] = [ V {K, V} <- Headers, V =:= " Accept "], % simplified case merle : getkey ( { URI, ContentType } ) of 6 {H, V} -> { response, H, V}; % headers, value undefined -> 8 {ok, B} = read_file ( URI ), { response, build_ headers (), B } 10 end. Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

12 Contents 1 Cache: memcached 2 NoSQL DBs 3 MapReduce 4 Real REST-based Architectures Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

13 NoSQL DBs MongoDB, CouchDB Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

14 CouchDB Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

15 CouchDB (ii) Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

16 CouchDB (iii) Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

17 CouchDB (iv) Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

18 CouchDB (v) Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

19 CouchDB (vi) $ curl - XPOST - HContent - type : application / json localhost :5984/ watches -- data {" brand " : " Seiko ", " model ":" skx007k1 " } 2 {" ok ": true," id ":" cfe517a13ace79a06fd d5 "," rev ":"1 - d2908ae e45c37f782ce "} 4 $ curl - XGET - HContent - type : application / json localhost :5984/ watches / cfe517a13ace79a06fd d5 {" _id ":" cfe517a13ace79a06fd d5 "," _rev ":"1 - d2908ae e45c37f782ce "," brand ":" Seiko "," model ":" skx007k1 "} Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

20 Contents 1 Cache: memcached 2 NoSQL DBs 3 MapReduce 4 Real REST-based Architectures Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

21 Contents 1 Cache: memcached 2 NoSQL DBs 3 MapReduce 4 Real REST-based Architectures Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

22 References C. Pautasso Some REST Design Patterns (and Anti-Patterns) InfoQ (varios autores) InfoQ Explores REST emag rest/en/pdf/rest%20emag.pdf S. Tilkov REST Anti-Patterns y REST doubts M. Paternostro, K. Hussey Building RESTful Java Applications with EMF building-restful-java-applications-with-emf Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23

23 References (ii) L. Richardson, S. Ruby RESTful Web Services O Reilly, 2007, ISBN: R. Balasubramanian, B. Carlyle, T. Erl, C. Pautasso SOA with REST: Principles, Patterns & Constraints Prentice Hall, Pearson, 2012, ISBN: Diego Sevilla Ruiz (DITEC Facultad de Informática) Session 9 Scalable Data Management: NoSQL & Map/Reduce Murcia, / 23