How To Store Data In Mongodb

Size: px

Start display at page:

Download "How To Store Data In Mongodb"

Brook Banks
3 years ago
Views:

1 What is MongoDB? The webinar will begin at 3pm You now have a menu in the top right corner of your screen. The red button with a white arrow allows you to expand and contract the webinar menu, in which you can write questions/comments. We won t have time to answer questions while we are presenting, but will answer them at the end You will be on mute throughout we can t hear you.

The red button with a white arrow allows you to expand and contract the webinar menu, in which

2 What is MongoDB? Webinar 30 June 2016 Peter Smyth UK Data Service

3 Can you hear us?

4 Can you hear us? If Not: Check your volume, and that your speaker/headset is plugged in. Your invitation also included a phone number, you can call that to listen in. UK +44 (0) US +1 (646) We are recording this webinar, so you can always listen to it later.

in. Your invitation also included a phone number, you can call that

5 Overview of this webinar MongoDB background Glossary of terms Simple comparison between storing data in MongoDB and a Relational database system Examples of using MongoDB Using MongoDB in R and Python

6 MongoDB Background First developed in 2007, publicly available in 2009 A document based database system The most popular non relational database in use today Open source (free to use) Schema-less NoSQL queries Excellent horizontal scalability (can handle very large collections of documents => Big Data) More at

to use) Schema-less NoSQL queries Excellent horizontal scalability (can handle very

7 Database Rankings (db-engines.com)

8 Glossary Relational A database system which uses tables to store data and defines relationships between tables to relate data in different tables Non-Relational No predefined relationships between the data. If appropriate, they can be defined by the user in code SQL Structured Query Language, used to write queries to extract data from relational databases NoSQL The query language used is not based on SQL Structured data Typically data which can easily be recorded in a table structure Unstructured data Inappropriate to be stored in a table Semi-structured data Not easily stored in a table structure

If appropriate, they can be defined by the user in code SQL Structured Query Language, used to write queries to extract data from relational databases

9 Terms MongoDB Database Collection Document Field Relational Database Database Table Row Column Shard Partition NoSQL Semi-structured data Non-relational SQL Structured data Relational

10 Systems being used Windows commandline MongoDB Shell NoSQL Manager for MongoDB

11 NoSQL Manager for MongoDB Available from: Free 30 day trial Licences from $79.00

12 Installing MongoDB in Windows Instructions for installing MongoDB in a Windows desktop will be provided guide or video on the UK Data Service web site

13 Demonstration Loading data into a table in a Relational database system (MS SQL Server) Loading data as a set of documents into a MongoDB collection

14 A csv file

15 Loading a csv file into a relational database A two step process. 1. First, define the Schema (structure) of the table to be used 2. Second, load the data into the table

16 Create table (SQL) CREATE TABLE SalesOrderDetails ( SalesOrderID int, SalesOrderDetailID int, CarrierTrackingNumber varchar(50), OrderQty int, ProductID int, SpecialOfferID int, UnitPrice decimal(10,4), UnitPriceDiscount decimal(10,4), LineTotal decimal(10,4), rowguid varchar(50), ModifiedDate date ) Table Name Column Names Data Types

int, SpecialOfferID int, UnitPrice decimal(10,4), UnitPriceDiscount decimal(10,4),

17 A loaded table (SQL)

18 Loading a csv file into MongoDB MongoDB provides a commandline utility to import data in either JSON or csv/tsv formats (There is a mongoexport utility as well) mongoimport --collection salesorderdetails --db sales --type csv --headerline --file "D:\Webinar - MongoDB\salesOrderDetails.csv" The database and the collection will be created automatically if they do not exist

$--collection salesorderdetails --db sales --type csv --headerline --file "D:\Webinar -$

19 A document in MongoDB This is in a data format called JSON

20 So why use MongoDB? So far the MongoDB storage strategy of using JSON objects seems to make the overall size of the data bigger For each document, every Field name is being repeated for each field. Not all data is as structured as the csv file we have used For example - the data associated with a tweet

overall size of the data bigger For each document, every Field name is

21 What part of a tweet looks like in JSON Start of a sub-document An array of subdocuments, including another array End of sub-document

22 Tweet data You cannot easily store a tweet in a single table structure Each tweet might have a different structure - different fields, missing fields You would need many tables with relationships defined between them. This would be complex to set up It would be slow to actually place actual tweet data into all of the relevant tables. In MongoDB you store the whole tweet as a single document You query the tweet data in the same way as simpler documents

23 Demonstrations The MongoDB shell (using NoSQL Manager) Accessing MongoDB from an R script Accessing MongoDB from a Python script

24 Simple shell commands use database Which database within the MongoDB system do you wish to use? show collections Within the database which collections are available findone() Return the first document in the collection find() Return a set of documents based on the criteria contained in the command aggregate() Includes aggregations over field values but can be arbitrarily complex structures using a pipeline arrangement

25 Getting data from two different collections 1. Read data into a cursor 2. Select a field 3. Iterate through the cursor using the field value as selection criteria for a find on a second table 4. Store the results of the second find into a second cursor 5. Iterate through the second cursor to extract fields from the second table 6. Print out the value from the first table alongside the value of the second table

26 Joining collections

27 New for v3.2 - Joins Uses the $lookup function within aggregate() Results in an array of sub documents being added to the documents of one collection The results can be stored (using code) into a separate collection

28 Accessing MongoDB from R There is a package RMongo which can be installed It allows the user to connect to a MongoDb system and return the results of a query as an R dataframe It has limitations anything other than a simple structure can produce erroneous results In fairness the package does warn you of this

29 Accessing MongoDB from Python A python package Pymongo can be installed It allows the user to connect to a MongoDb system and return the results of a query as an Python Dictionary object A python discionary object is a set of key:value pairs which makes it ideal for holding JSON formatted data The set of commands available in pymongo, in terms of names and functions map closely with the functions available in the MongoDB Shell

30 More Information o The MongoDB official site - this includes links to the official documentation and also to o o o o which has a variety of self study courses. - allows you to find user groups in your area has a list of community support forums The usual suspects stackoverflow or just google your question

31 MongoDB books There are many books available they may be based on older versions of MongoDB Introductory examples might include;

32 Questions Peter Smyth ukdataservice.ac.uk/help/ Subscribe to the UK Data Service news list at Follow us on Twitter or Facebook

Perl & NoSQL Focus on MongoDB. Jean-Marie Gouarné http://jean.marie.gouarne.online.fr jmgdoc@cpan.org

Perl & NoSQL Focus on MongoDB. Jean-Marie Gouarné http://jean.marie.gouarne.online.fr jmgdoc@cpan.org Perl & NoSQL Focus on MongoDB Jean-Marie Gouarné http://jean.marie.gouarne.online.fr jmgdoc@cpan.org AGENDA A NoIntroduction to NoSQL The document store data model MongoDB at a glance MongoDB Perl API