Dr. Chuck Cartledge. 15 Oct. 2015

Transcription

1 CS-695 NoSQL Database MongoDB (part 2 of 2) Dr. Chuck Cartledge 15 Oct /17

2 Table of contents I 1 Miscellanea 2 Assignment #4 3 DB comparisons 4 Extensions 6 Midterm 7 Conclusion 8 References 5 Summary 2/17

3 Corrections and additions since last lecture. Assignment #04 is available Corrected typos in lecture #007 3/17

4 Words of explanation. The full text is available at: ~ccartled/ Teaching/2015-Fall/NoSQL/ Assignments/04/ In general terms: 1 Parse data 2 Create document database 3 Update documents based on number of movies filmed in each city 4 Query database 5 Create list of 10 most used locations 6 Solve TSP with 10 cities 7 Plot route on a map 8 List locations and movies 9 What about when a popular location doesn t have a city?? 4/17

5 How different DBs compare to a RDBMS We have some terms to compare now[2] RDBMS K/V Columnar Doc. DB. instance cluster cluster instance database namespace table bucket table collection row key-value row document rowid key id col. col. fam. schema database join DBRef 5/17

6 Aggregations Aggregation pipeline gives you a way to transform and combine documents in your collection. You do it by passing the documents through a pipeline thats somewhat analogous to the Unix pipe where you send output from one command to another to a third, etc. Karl Seguin [3] General format of the aggregate function: db.collection.aggregate ([< stage >,...]) 6/17

7 Aggregations A partial list of aggregation operators: 1 $match filters the stream $limit passes first n docs $skip skips first n docs $group groups docs based on id and applies accumulator(s) $geonear returns docs close to a location $and, $or, $not typical Boolean operators for Note: see how all operators start with a $. arrays of comparison operators $eq, $gt, $gte, $lt, $lte usual math comparisons $add, $subtract, $multiply, $divide, $mod math operators $cmp, $ne compares and returns -1, 0, /17

8 Aggregations Examples[1]: db.employees.aggregate(...{... "$project" : {... "totalpay" : {... "$subtract" : [{"$add" : ["$salary", "$bonus"]}, "$401k"]... }}}) db.employees.aggregate(... {... "$project" : {... " " : {... "$concat" : [... "$substr" : ["$firstname", 0, 1],... ".",... "$lastname",... "@example.com"... ]}}}) 8/17

9 Indices MongoDB has indices. They can be created: db.unicorns.ensureindex({name: 1}); Note: the 1 means ascending, -1 is descending. They can be dropped: db.unicorns.dropindex({name: 1}); They can be unique: db.unicorns.ensureindex({name: 1},{unique: true}); They can be on embedded fields, arrays, and compounded. db.unicorns.ensureindex({name: 1,vampires: -1}); 9/17

10 Indices Special indices[5]: Capped collections collections act like circular queues TTL indices for caches documents older than TTL are purged (automatically) Full-text indices all strings are split, stemmed, and stored (heavyweight operations) Geospatial (2d and 2dsphere) supports: inclusion, intersection, and proximity GridFS for large files virtual file system, supports sharding, large files (2G block size) Image from [4]. 10/17

11 Indices Classic (a.k.a, Hadoop) MapReduce The classic Map Reduce ecosystem operations: 1 Split the input file into chunks 2 Present each split to its own map function 3 Map function emits 0 or more <key, value> sets 4 Ecosystem collects are all <key, value> sets 5 Ecosystem merges <key, values> sets 6 Ecosystem presents <key, values> set to reducers 7 Ecosystem collects output Image from [6]. 11/17

12 Indices MongoDB MapReduce: 12/17

13 Strengths and weaknesses Good and not so good Strengths: Handling huge amounts of data by replication and horizontal scaling Very flexible data model Ease of use (object oriented bent) Weaknesses: Discourages normalization Items can be inserted anywhere (lack of schema) May require large infrastructure 13/17

14 Applicabilities Good for, and not so good for Good fit; Event logging, especially where the data changes rapidly Anything that can communicate via JSON documents (CRM, publishing websites, user comments, profiles, web-facing documents,...) Web-analytics detailing transactions and changes Intermediate database when transitioning from one db to another Not so good fit: Anything requiring complex transactions spanning multiple documents Anything that required normalized data 14/17

15 General notes and ideas It will be about things we covered in class. Including: Information and ideas Various lectures/presentations Different database technologies Open book, open notes (not open neighbor) It will be about Sarah and Hasta. Sarah will have an idea for Hasta. Your task will be to discuss which DB technology to use and why. 15/17

16 What have we covered? Reviewed assignment #04 Covered some of the mongdb querying capabilities Remember Assignment #04 due before next class Next time: CouchDB 16/17

17 References I [1] Kristina Chodorow, Mongodb, the definitive guide, O Reilly Media, Inc., [2] Eric Redmond and Jim R Wilson, Seven databases in seven weeks, Pragmatic Bookshelf, [3] Karl Seguin, The little mongodb book, [4] Alessandro Siletto, A quick start with mongodb geospatial queries, [5] MongoDB Staff, Mongodb documentation, MongoDB Documentation Project, [6] Tom White, Hadoop: The lay of the land, /17