The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg. Adam Marcus MIT CSAIL
|
|
|
- William Jackson
- 8 years ago
- Views:
Transcription
1 The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg Adam Marcus MIT CSAIL
2 About Me Social Computing + Database Systems Easily Distracted: Wrote The NoSQL Ecosystem in The Architecture of Open Source Applications 1 1
3 Preliminaries Almost always, use PostgreSQL or MySQL Most problems can be solved by a single (large) machine Consider paying a DB vendor to solve problem
4 But sometimes, need >1 machines Analytics on MapReduce or Column Stores Facebook famously stores 1B+ users on ~10K machines
5 Scaling transactional workloads Google spends thousands of person-hours on Megastore/Spanner Everyone else jumps through hoops to stick to relational model and SQL Partition data to avoid multinode transactions (2PC) Avoid multirow transactions to prevent locking
6 History, Compressed Late 1990s Today
7 History, Compressed Buy RAM Late 1990s Today
8 History, Compressed Buy RAM Wrap RDBMSs Late 1990s Today
9 History, Compressed Buy RAM Wrap RDBMSs NoSQL Late 1990s Today
10 History, Compressed Not Only SQL Buy RAM Wrap RDBMSs NoSQL Late 1990s Today
11 History, Compressed Buy RAM Wrap RDBMSs NoSQL Commercialize Late 1990s Today
12 History, Compressed Buy RAM Wrap RDBMSs NoSQL Commercialize Late 1990s Today
13 The List, So You Don't Yell at Me HBase CouchDB Neo4j Cassandra Riak InfoGrid BerkeleyDB Voldemort HyperTable Redis MongoDB Sones AllegroGraph HyperGraphDB DEX FlockDB VertexDB Oracle NoSQL Tokyo Cabinet MemcacheDB
14 The List, So You Don't Yell at Me Cassandra HBase Riak BerkeleyDB Voldemort Neo4j Marcus' Law of Databases: HyperTable The number of persistence CouchDB InfoGrid Redis MongoDB options doubles every 1.5 years Sones AllegroGraph HyperGraphDB DEX FlockDB VertexDB Oracle NoSQL Tokyo Cabinet MemcacheDB
15 history the parents of nosql tradeoffs + demos scaling with snoop dogg real-world usage discussion
16 BigTable Google, 2006 Column store with sloppy schemas Strong consistency Open source: HBase
17 Dynamo Amazon, 2007 Key-value store Allows eventual consistency Open source: Voldemort, Riak
18 history the parents of nosql tradeoffs + demos scaling with snoop dogg real-world usage discussion
19 understanding NoSQL = understanding a long series of tradeoffs
20 Standards-compliant SQL Systems Relational model Powerful query language Transactional semantics Predefined schemas Strong consistency between replicas*
21 Standards-compliant SQL Systems Relational model Powerful query language Transactional semantics Predefined schemas Strong consistency between replicas* NoSQL: Maybe you don't need all of these?
22 NoSQL Systems are a Buffet
23 NoSQL Systems are a Buffet (of progressively larger grenades)
24 NoSQL Systems are a Buffet (of progressively larger grenades) Data model Query model Durability Transactional consistency Partitioning Replica consistency
25 Prof. Madden will not quiz you on specific features of NoSQL systems (i.e., don't memorize that MongoDB is a document store)
26 Data Model Usually key-based... Binary blob: Voldemort Documents: MongoDB, CouchDB, Riak Data structures: Redis Column-families: HBase, Cassandra
27 Data Model Usually key-based... Binary blob: Voldemort Documents: MongoDB, CouchDB, Riak Data structures: Redis Column-families: HBase, Cassandra...but not always Graph stores
28 Query Model Redis: data structure-specific operations CouchDB, Riak: MapReduce Cassandra, MongoDB: SQL-like languages, no joins or transactions
29 Query Model Redis: data structure-specific operations CouchDB, Riak: MapReduce Cassandra, MongoDB: SQL-like languages, no joins or transactions Third-party High-level: PigLatin, HiveQL Library: Cascading, Crunch Streaming: Flume, Kafka, S4, Scribe
30 Transactions Full ACID for single key Redis: multi-key single-node transactions
31 demo!
32 history the parents of nosql tradeoffs + demos scaling with snoop dogg real-world usage discussion
33 Single-server durability Memory only: memcached Single-server durability: the rest fsync every N seconds: most Write-ahead logging: Cassandra, HBase, Redis, Riak Group commit: Cassandra, HBase/HDFS
34 When one server is not enough Replicate Partition
35 When one server is not enough Replicate Performance K-safety Partition
36 When one server is not enough Replicate Performance K-safety Partition Vertical Horizontal
37 When one server is not enough Replicate Performance K-safety Consistency/Availability Partition Vertical Horizontal
38 When one server is not enough Replicate Performance K-safety Consistency/Availability Partition Vertical Horizontal Partitioning Scheme
39 Eventual vs. Strong Consistency: FIGHT!
40 Eventual vs. Strong Consistency: FIGHT! N = # replicas W = # write acknowledgements R = # read acknowledgements
41 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3
42 N = # replicas W = # write acknowledgements R = # read acknowledgements Give Sally a raise! N = 3
43 N = # replicas W = # write acknowledgements R = # read acknowledgements Give Sally a raise! N = 3
44 N = # replicas W = # write acknowledgements R = # read acknowledgements Give Sally a raise! N = 3, W = 1, R = 1
45 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3, W = 1, R = 1
46 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3, W = 1, R = 1
47 N = # replicas W = # write acknowledgements R = # read acknowledgements Sally's Salary?? N = 3, W = 1, R = 1
48 N = # replicas W = # write acknowledgements R = # read acknowledgements Sally's Salary? N = 3, W = 1, R = 1
49 N = # replicas W = # write acknowledgements R = # read acknowledgements Give Sally a raise! N = 3
50 N = # replicas W = # write acknowledgements R = # read acknowledgements Give Sally a raise! N = 3, W = 3, R = 1
51 N = # replicas W = # write acknowledgements R = # read acknowledgements Waste of time, Waste of time, Waste of time N = 3, W = 3, R = 1
52 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3, W = 3, R = 1
53 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3, W = 3, R = 1
54 N = # replicas W = # write acknowledgements R = # read acknowledgements Sally's Salary?? N = 3, W = 3, R = 1
55 N = # replicas W = # write acknowledgements R = # read acknowledgements Sally's Salary? N = 3, W = 3, R = 1
56 N = # replicas W = # write acknowledgements R = # read acknowledgements Give Sally a raise! N = 3
57 N = # replicas W = # write acknowledgements R = # read acknowledgements Give Sally a raise! N = 3, W = 2, R = 2
58 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3, W = 2, R = 2
59 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3, W = 2, R = 2
60 N = # replicas W = # write acknowledgements R = # read acknowledgements Sally's Salary?? N = 3, W = 2, R = 2
61 N = # replicas W = # write acknowledgements R = # read acknowledgements QUORUM! N = 3, W = 2, R = 2
62 N = # replicas W = # write acknowledgements R = # read acknowledgements N = 3, W = 2, R = 2
63 Consistency wild west Available, inconsistent on failure N = 3, W = 1, R = 1
64 Consistency wild west Available, inconsistent on failure N = 3, W = 1, R = 1 Consistent, unavailable writes on failure N = 3, W = 3, R = 1
65 Consistency wild west Available, inconsistent on failure N = 3, W = 1, R = 1 Consistent, unavailable writes on failure N = 3, W = 3, R = 1 Available, consistent with 1 failure N = 3, W = 2, R = 2
66 Consistency wild west Available, inconsistent on failure N = 3, W = 1, R = 1 Consistent, unavailable writes on failure N = 3, W = 3, R = 1 Available, consistent with 1 failure N = 3, W = 2, R = 2
67 Consistency
68 Consistency (not acid)
69 Consistency (not acid)
70 Consistency Strong: Appears that all replicas see all writes Eventual: If left alone, replicas eventually converge Weak: Replicas have different, divergent versions
71 Consistency Strong: Appears that all replicas see all writes Eventual: If left alone, replicas eventually converge Weak: Replicas have different, divergent versions The magical CAP theorem line
72 Consistency math
73 Consistency math Strong: N < R + W
74 Consistency math Strong: N < R + W Easy: W = N Tricky: W < N
75 Consistency math Strong: N < R + W Easy: W = N Tricky: W < N Eventual/Weak: N >= R + W
76 Consistency Options BigTable (HBase): Strong Dynamo (Voldemort/Riak/Cassandra): tunable strong or eventual
77 Consistency Options BigTable (HBase): Strong Dynamo (Voldemort/Riak/Cassandra): tunable strong or eventual...and many others (Yahoo! PNUTs has timeline consistency)
78 How do we get consistent (eventually)?
79 How does Dynamo go from weak to eventual consistency? Version w/ vector clocks, read repair Hinted handoff on failure Anti-entropy Gossip-based membership
80 Version w/ vector clocks?
81 How replicas get out of sync
82 How replicas get out of sync Give Sally a raise! N = 3, W = 1, R = 1
83 How replicas get out of sync N = 3, W = 1, R = 1
84 How replicas get out of sync N = 3, W = 1, R = 1
85 How replicas get out of sync MOAR RAISE! 130K N = 3, W = 1, R = 1
86 How replicas get out of sync N = 3, W = 1, R = 1 130K
87 How replicas get out of sync Sally Salary?? N = 3, W = 1, R = 1 130K
88 Simple versioning doesn't work! Version: 20 Version: 20 Version: 20
89 Simple versioning doesn't work! Version: 21 Version: 21 Version: 20
90 Simple versioning doesn't work! 130K Version: 21 Version: 21 Version: 21
91 Vector clocks to the rescue Maintain vector with version per replica Each client request sent to one replica, which forwards to others Routing replica updates its vector entry by 1
92 Vector clocks to the rescue Maintain vector with version per replica Each client request sent to one replica, which forwards to others Routing replica updates its vector entry by 1 Version: <1,1,1> Version: <1,1,1> Version: <1,1,1>
93 Vector clocks to the rescue Maintain vector with version per replica Each client request sent to one replica, which forwards to others Routing replica updates its vector entry by 1 Version: <1,1,1> Version: <1,1,1> Version: <1,1,1>
94 Vector clocks to the rescue Maintain vector with version per replica Each client request sent to one replica, which forwards to others Routing replica updates its vector entry by 1 Version: <2,1,1> Version: <1,1,1> Version: <1,1,1>
95 Vector clocks to the rescue Maintain vector with version per replica Each client request sent to one replica, which forwards to others Routing replica updates its vector entry by 1 Version: <2,1,1> Version: <2,1,1> Version: <1,1,1>
96 Vector clocks to the rescue Maintain vector with version per replica Each client request sent to one replica, which forwards to others Routing replica updates its vector entry by 1 Version: <2,1,1> Version: <2,1,1> Version: <1,1,1>
97 Read repair Client reads a value from multiple replicas Compares vector clocks <2,1,1> = <2,1,1> (in sync) <2,1,1> > <1,1,1> (update <1,1,1> before conflict) <2,1,1>!! <1,1,2> (conflict)
98 How does Dynamo go from weak to eventual consistency? Version w/ vector clocks, read repair Hinted handoff on failure Anti-entropy Gossip-based membership
99 How does Dynamo go from weak to eventual consistency? Version w/ vector clocks, read repair Hinted handoff on failure Anti-entropy Gossip-based membership Proactively stay in sync
100 How does Dynamo go from weak to eventual consistency? Version w/ vector clocks, read repair Hinted handoff on failure Anti-entropy Gossip-based membership Proactively stay in sync Track replicas symmetrically
101 Replication Consistency Availability Playing catchup
102 Partitioning Consistent hashing: Voldemort, Riak Range partitioning: HBase, MongoDB Both: Cassandra
103 Your partitioning scheme matters Partition so all requests go to one machine Ideally, all joins happen on one machine Example: map all GMail for a user to one machine
104 Your partitioning scheme matters Partition so all requests go to one machine Ideally, all joins happen on one machine Example: map all GMail for a user to one machine Highly networked data (Facebook News) hard to partition: roughly two options Each request floods many partitions Denormalize data (replicate), partition by user
105 history the parents of nosql tradeoffs + demos scaling with snoop dogg real-world usage discussion
106 NoSQL Use-cases Cassandra at Netflix HBase at Facebook MongoDB at Craigslist
107 Cassandra BigTable data model: key column family Dynamo sharding model: consistent hashing Eventual or strong consistency
108 Cassandra at Netflix Transitioned from Oracle Store customer profiles, customer:movie watch log, and detailed usage logging Note: no multi-record locking (e.g., bank transfer) In-datacenter: 3 replicas, per-app consistency read/write quorum = 1, ~1ms latency read/write quorum = 2, ~3ms latency Replicating Datacenter Oracle with Global Apache Cassandra on AWS by Adrian Cockcroft
109 Cassandra at Netflix Transitioned from Oracle Store customer profiles, customer:movie watch log, and detailed usage logging In-datacenter: 3 replicas, per-app consistency Replicating Datacenter Oracle with Global Apache Cassandra on AWS by Adrian Cockcroft
110 Cassandra at Netflix (cont'd) Benefit: async inter-datacenter replication Benefit: no downtime for schema changes Benefit: hooks for live backups
111 HBase Data model: key column family Sharding model: range partitioning Strong consistency Applications Logging events/crawls, storing analytics Twitter: replicate data from MySQL, Hadoop analytics Facebook Messages
112 HBase Data model: key column family Sharding model: range partitioning Strong consistency Applications Logging events/crawls, storing analytics Twitter: replicate data from MySQL, Hadoop analytics Facebook Messages
113 HBase for Facebook Messages Cassandra/Dynamo eventual consistency was difficult to program against Benefit: simple consistency model Benefit: flexible data model Benefit: simple sharding, load balancing, replication
114 HBase for Facebook Messages Cassandra/Dynamo eventual consistency was difficult to program against Benefit: simple consistency model Benefit: flexible data model Benefit: simple partitioning, load balancing, replication
115 MongoDB Document-based data model Range-based partitioning Consistency depends on how you use it
116 MongoDB: Two use-cases Archiving at Craigslist 2.2B historical posts, semi-structured Relatively large blobs: avg 2KB, max > 4 MB
117 MongoDB: Two use-cases Archiving at Craigslist 2.2B historical posts, semi-structured Relatively large blobs: avg 2KB, max > 4 MB Checkins at Foursquare Geospatial indexing Small location-based updates, partitioned on user
118 history the parents of nosql tradeoffs + demos scaling with snoop dogg real-world usage discussion
119 Polyglot Persistence Account data in MySQL Cache in memcached Counters + queues in redis Statistics in MongoDB Event logging in HDFS/HBase/Cassandra
120 When you see NoSQL, think tradeoffs Data model Query Model Transactions Consistency vs. Availability Partitioning schemes Adam
Structured Data Storage
Structured Data Storage Xgen Congress Short Course 2010 Adam Kraut BioTeam Inc. Independent Consulting Shop: Vendor/technology agnostic Staffed by: Scientists forced to learn High Performance IT to conduct
Cloud Scale Distributed Data Storage. Jürmo Mehine
Cloud Scale Distributed Data Storage Jürmo Mehine 2014 Outline Background Relational model Database scaling Keys, values and aggregates The NoSQL landscape Non-relational data models Key-value Document-oriented
NoSQL Databases. Nikos Parlavantzas
!!!! NoSQL Databases Nikos Parlavantzas Lecture overview 2 Objective! Present the main concepts necessary for understanding NoSQL databases! Provide an overview of current NoSQL technologies Outline 3!
extensible record stores document stores key-value stores Rick Cattel s clustering from Scalable SQL and NoSQL Data Stores SIGMOD Record, 2010
System/ Scale to Primary Secondary Joins/ Integrity Language/ Data Year Paper 1000s Index Indexes Transactions Analytics Constraints Views Algebra model my label 1971 RDBMS O tables sql-like 2003 memcached
SQL VS. NO-SQL. Adapted Slides from Dr. Jennifer Widom from Stanford
SQL VS. NO-SQL Adapted Slides from Dr. Jennifer Widom from Stanford 55 Traditional Databases SQL = Traditional relational DBMS Hugely popular among data analysts Widely adopted for transaction systems
Introduction to NOSQL
Introduction to NOSQL Université Paris-Est Marne la Vallée, LIGM UMR CNRS 8049, France January 31, 2014 Motivations NOSQL stands for Not Only SQL Motivations Exponential growth of data set size (161Eo
MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15
MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15 1 MongoDB in the NoSQL and SQL world. NoSQL What? Why? - How? Say goodbye to ACID, hello BASE You
nosql and Non Relational Databases
nosql and Non Relational Databases Image src: http://www.pentaho.com/big-data/nosql/ Matthias Lee Johns Hopkins University What NoSQL? Yes no SQL.. Atleast not only SQL Large class of Non Relaltional Databases
Preparing Your Data For Cloud
Preparing Your Data For Cloud Narinder Kumar Inphina Technologies 1 Agenda Relational DBMS's : Pros & Cons Non-Relational DBMS's : Pros & Cons Types of Non-Relational DBMS's Current Market State Applicability
BRAC. Investigating Cloud Data Storage UNIVERSITY SCHOOL OF ENGINEERING. SUPERVISOR: Dr. Mumit Khan DEPARTMENT OF COMPUTER SCIENCE AND ENGEENIRING
BRAC UNIVERSITY SCHOOL OF ENGINEERING DEPARTMENT OF COMPUTER SCIENCE AND ENGEENIRING 12-12-2012 Investigating Cloud Data Storage Sumaiya Binte Mostafa (ID 08301001) Firoza Tabassum (ID 09101028) BRAC University
wow CPSC350 relational schemas table normalization practical use of relational algebraic operators tuple relational calculus and their expression in a declarative query language relational schemas CPSC350
NoSQL Systems for Big Data Management
NoSQL Systems for Big Data Management Venkat N Gudivada East Carolina University Greenville, North Carolina USA Venkat Gudivada NoSQL Systems for Big Data Management 1/28 Outline 1 An Overview of NoSQL
NoSQL Data Base Basics
NoSQL Data Base Basics Course Notes in Transparency Format Cloud Computing MIRI (CLC-MIRI) UPC Master in Innovation & Research in Informatics Spring- 2013 Jordi Torres, UPC - BSC www.jorditorres.eu HDFS
Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world
Analytics March 2015 White paper Why NoSQL? Your database options in the new non-relational world 2 Why NoSQL? Contents 2 New types of apps are generating new types of data 2 A brief history of NoSQL 3
Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1
Why NoSQL? Your database options in the new non- relational world 2015 IBM Cloudant 1 Table of Contents New types of apps are generating new types of data... 3 A brief history on NoSQL... 3 NoSQL s roots
Benchmarking and Analysis of NoSQL Technologies
Benchmarking and Analysis of NoSQL Technologies Suman Kashyap 1, Shruti Zamwar 2, Tanvi Bhavsar 3, Snigdha Singh 4 1,2,3,4 Cummins College of Engineering for Women, Karvenagar, Pune 411052 Abstract The
Applications for Big Data Analytics
Smarter Healthcare Applications for Big Data Analytics Multi-channel sales Finance Log Analysis Homeland Security Traffic Control Telecom Search Quality Manufacturing Trading Analytics Fraud and Risk Retail:
The Quest for Extreme Scalability
The Quest for Extreme Scalability In times of a growing audience, very successful internet applications have all been facing the same database issue: while web servers can be multiplied without too many
NoSQL in der Cloud Why? Andreas Hartmann
NoSQL in der Cloud Why? Andreas Hartmann 17.04.2013 17.04.2013 2 NoSQL in der Cloud Why? Quelle: http://res.sys-con.com/story/mar12/2188748/cloudbigdata_0_0.jpg Why Cloud??? 17.04.2013 3 NoSQL in der Cloud
Lecture Data Warehouse Systems
Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART C: Novel Approaches in DW NoSQL and MapReduce Stonebraker on Data Warehouses Star and snowflake schemas are a good idea in the DW world C-Stores
LARGE-SCALE DATA STORAGE APPLICATIONS
BENCHMARKING AVAILABILITY AND FAILOVER PERFORMANCE OF LARGE-SCALE DATA STORAGE APPLICATIONS Wei Sun and Alexander Pokluda December 2, 2013 Outline Goal and Motivation Overview of Cassandra and Voldemort
Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014. Apologies to Martin Fowler ( NoSQL Distilled )
Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014 Rise of Web and cluster-based computing NoSQL Movement Relationships vs. Aggregates Key-value store XML or JSON
How To Write A Database Program
SQL, NoSQL, and Next Generation DBMSs Shahram Ghandeharizadeh Director of the USC Database Lab Outline A brief history of DBMSs. OSs SQL NoSQL 1960/70 1980+ 2000+ Before Computers Database DBMS/Data Store
Can the Elephants Handle the NoSQL Onslaught?
Can the Elephants Handle the NoSQL Onslaught? Avrilia Floratou, Nikhil Teletia David J. DeWitt, Jignesh M. Patel, Donghui Zhang University of Wisconsin-Madison Microsoft Jim Gray Systems Lab Presented
High Throughput Computing on P2P Networks. Carlos Pérez Miguel carlos.perezm@ehu.es
High Throughput Computing on P2P Networks Carlos Pérez Miguel carlos.perezm@ehu.es Overview High Throughput Computing Motivation All things distributed: Peer-to-peer Non structured overlays Structured
NoSQL. What Is NoSQL? Why NoSQL?
SYSADMIN NoSQL GREG BURD Greg Burd is a Developer Advocate for Basho Technologies, makers of Riak. Before Basho, Greg spent nearly ten years as the product manager for Berkeley DB at Sleepycat Software
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases Dave Dykstra dwd@fnal.gov Fermilab is operated by the Fermi Research Alliance, LLC under contract No. DE-AC02-07CH11359
Choosing the right NoSQL database for the job: a quality attribute evaluation
Lourenço et al. Journal of Big Data (2015) 2:18 DOI 10.1186/s40537-015-0025-0 RESEARCH Choosing the right NoSQL database for the job: a quality attribute evaluation João Ricardo Lourenço 1*, Bruno Cabral
NoSQL: Going Beyond Structured Data and RDBMS
NoSQL: Going Beyond Structured Data and RDBMS Scenario Size of data >> disk or memory space on a single machine Store data across many machines Retrieve data from many machines Machine = Commodity machine
NoSQL Database Options
NoSQL Database Options Introduction For this report, I chose to look at MongoDB, Cassandra, and Riak. I chose MongoDB because it is quite commonly used in the industry. I chose Cassandra because it has
Big Data Management. Big Data Management. (BDM) Autumn 2013. Povl Koch September 30, 2013 29-09-2013 1
Big Data Management Big Data Management (BDM) Autumn 2013 Povl Koch September 30, 2013 29-09-2013 1 Overview Today s program 1. Little more practical details about this course 2. Recap from last time 3.
NoSQL Databases. Polyglot Persistence
The future is: NoSQL Databases Polyglot Persistence a note on the future of data storage in the enterprise, written primarily for those involved in the management of application development. Martin Fowler
GRAPH DATABASE SYSTEMS. h_da Prof. Dr. Uta Störl Big Data Technologies: Graph Database Systems - SoSe 2016 1
GRAPH DATABASE SYSTEMS h_da Prof. Dr. Uta Störl Big Data Technologies: Graph Database Systems - SoSe 2016 1 Use Case: Route Finding Source: Neo Technology, Inc. h_da Prof. Dr. Uta Störl Big Data Technologies:
NOSQL INTRODUCTION WITH MONGODB AND RUBY GEOFF LANE <GEOFF@ZORCHED.NET> @GEOFFLANE
NOSQL INTRODUCTION WITH MONGODB AND RUBY GEOFF LANE @GEOFFLANE WHAT IS NOSQL? NON-RELATIONAL DATA STORAGE USUALLY SCHEMA-FREE ACCESS DATA WITHOUT SQL (THUS... NOSQL) WIDE-COLUMN / TABULAR
Introduction to NoSQL
Introduction to NoSQL NoSQL Seminar 2012 @ TUT Arto Salminen What is NoSQL? Class of database management systems (DBMS) "Not only SQL" Does not use SQL as querying language Distributed, fault-tolerant
Distributed Systems. Tutorial 12 Cassandra
Distributed Systems Tutorial 12 Cassandra written by Alex Libov Based on FOSDEM 2010 presentation winter semester, 2013-2014 Cassandra In Greek mythology, Cassandra had the power of prophecy and the curse
CS 4604: Introduc0on to Database Management Systems. B. Aditya Prakash Lecture #13: NoSQL and MapReduce
CS 4604: Introduc0on to Database Management Systems B. Aditya Prakash Lecture #13: NoSQL and MapReduce Announcements HW4 is out You have to use the PGSQL server START EARLY!! We can not help if everyone
NoSQL and Graph Database
NoSQL and Graph Database Biswanath Dutta DRTC, Indian Statistical Institute 8th Mile Mysore Road R. V. College Post Bangalore 560059 International Conference on Big Data, Bangalore, 9-20 March 2015 Outlines
What Next for DBAs in the Big Data Era
What Next for DBAs in the Big Data Era February 21 st, 2015 Copyright 2013. Apps Associates LLC. 1 Satyendra Kumar Pasalapudi Associate Practice Director IMS @ Apps Associates Co Founder & President of
So What s the Big Deal?
So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data
Practical Cassandra. Vitalii Tymchyshyn tivv00@gmail.com @tivv00
Practical Cassandra NoSQL key-value vs RDBMS why and when Cassandra architecture Cassandra data model Life without joins or HDD space is cheap today Hardware requirements & deployment hints Vitalii Tymchyshyn
Big Systems, Big Data
Big Systems, Big Data When considering Big Distributed Systems, it can be noted that a major concern is dealing with data, and in particular, Big Data Have general data issues (such as latency, availability,
Facebook: Cassandra. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Overview Design Evaluation
Facebook: Cassandra Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Leader Election 1/24 Outline 1 2 3 Smruti R. Sarangi Leader Election
You should have a working knowledge of the Microsoft Windows platform. A basic knowledge of programming is helpful but not required.
What is this course about? This course is an overview of Big Data tools and technologies. It establishes a strong working knowledge of the concepts, techniques, and products associated with Big Data. Attendees
Introduction to Big Data Training
Introduction to Big Data Training The quickest way to be introduce with NOSQL/BIG DATA offerings Learn and experience Big Data Solutions including Hadoop HDFS, Map Reduce, NoSQL DBs: Document Based DB
NoSQL. Thomas Neumann 1 / 22
NoSQL Thomas Neumann 1 / 22 What are NoSQL databases? hard to say more a theme than a well defined thing Usually some or all of the following: no SQL interface no relational model / no schema no joins,
CS435 Introduction to Big Data
CS435 Introduction to Big Data Final Exam Date: May 11 6:20PM 8:20PM Location: CSB 130 Closed Book, NO cheat sheets Topics covered *Note: Final exam is NOT comprehensive. 1. NoSQL Impedance mismatch Scale-up
Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015
Big Data Technologies Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015 Situation: Bigger and Bigger Volumes of Data Big Data Use Cases Log Analytics (Web Logs, Sensor
Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace
Introduction to Polyglot Persistence Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace FOSSCOMM 2016 Background - 14 years in databases and system engineering - NoSQL DBA @ ObjectRocket
Introduction to NoSQL
Introduction to NoSQL Gabriele Pozzani April 24, 2013 Outline NoSQL Definition Categories Related concepts Modeling NoSQL stores Properties Performance Relational vs NoSQL DBMSs Applications What does
Advanced Data Management Technologies
ADMT 2014/15 Unit 15 J. Gamper 1/44 Advanced Data Management Technologies Unit 15 Introduction to NoSQL J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE ADMT 2014/15 Unit 15
Distributed Data Stores
Distributed Data Stores 1 Distributed Persistent State MapReduce addresses distributed processing of aggregation-based queries Persistent state across a large number of machines? Distributed DBMS High
NewSQL: Towards Next-Generation Scalable RDBMS for Online Transaction Processing (OLTP) for Big Data Management
NewSQL: Towards Next-Generation Scalable RDBMS for Online Transaction Processing (OLTP) for Big Data Management A B M Moniruzzaman Department of Computer Science and Engineering, Daffodil International
Overview of Databases On MacOS. Karl Kuehn Automation Engineer RethinkDB
Overview of Databases On MacOS Karl Kuehn Automation Engineer RethinkDB Session Goals Introduce Database concepts Show example players Not Goals: Cover non-macos systems (Oracle) Teach you SQL Answer what
Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing
Evaluating NoSQL for Enterprise Applications Dirk Bartels VP Strategy & Marketing Agenda The Real Time Enterprise The Data Gold Rush Managing The Data Tsunami Analytics and Data Case Studies Where to go
Cluster Computing. ! Fault tolerance. ! Stateless. ! Throughput. ! Stateful. ! Response time. Architectures. Stateless vs. Stateful.
Architectures Cluster Computing Job Parallelism Request Parallelism 2 2010 VMware Inc. All rights reserved Replication Stateless vs. Stateful! Fault tolerance High availability despite failures If one
Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores
Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores Composite Software October 2010 TABLE OF CONTENTS INTRODUCTION... 3 BUSINESS AND IT DRIVERS... 4 NOSQL DATA STORES LANDSCAPE...
Hands-on Cassandra. OSCON July 20, 2010. Eric Evans eevans@rackspace.com @jericevans http://blog.sym-link.com
Hands-on Cassandra OSCON July 20, 2010 Eric Evans eevans@rackspace.com @jericevans http://blog.sym-link.com 2 Background Influential Papers BigTable Strong consistency Sparse map data model GFS, Chubby,
Yahoo! Cloud Serving Benchmark
Yahoo! Cloud Serving Benchmark Overview and results March 31, 2010 Brian F. Cooper cooperb@yahoo-inc.com Joint work with Adam Silberstein, Erwin Tam, Raghu Ramakrishnan and Russell Sears System setup and
A Survey of Distributed Database Management Systems
Brady Kyle CSC-557 4-27-14 A Survey of Distributed Database Management Systems Big data has been described as having some or all of the following characteristics: high velocity, heterogeneous structure,
Enterprise Operational SQL on Hadoop Trafodion Overview
Enterprise Operational SQL on Hadoop Trafodion Overview Rohit Jain Distinguished & Chief Technologist Strategic & Emerging Technologies Enterprise Database Solutions Copyright 2012 Hewlett-Packard Development
Scalable SQL and NoSQL Data Stores
Scalable SQL and NoSQL Data Stores Rick Cattell Cattell.Net Software Email: rick@cattell.net ABSTRACT In this paper, we examine a number of SQL and socalled NoSQL data stores designed to scale simple OLTP-style
NoSQL Evaluation. A Use Case Oriented Survey
2011 International Conference on Cloud and Service Computing NoSQL Evaluation A Use Case Oriented Survey Robin Hecht Chair of Applied Computer Science IV University ofbayreuth Bayreuth, Germany robin.hecht@uni
Introduction to Hadoop. New York Oracle User Group Vikas Sawhney
Introduction to Hadoop New York Oracle User Group Vikas Sawhney GENERAL AGENDA Driving Factors behind BIG-DATA NOSQL Database 2014 Database Landscape Hadoop Architecture Map/Reduce Hadoop Eco-system Hadoop
Databases 2 (VU) (707.030)
Databases 2 (VU) (707.030) Introduction to NoSQL Denis Helic KMI, TU Graz Oct 14, 2013 Denis Helic (KMI, TU Graz) NoSQL Oct 14, 2013 1 / 37 Outline 1 NoSQL Motivation 2 NoSQL Systems 3 NoSQL Examples 4
Open Source Technologies on Microsoft Azure
Open Source Technologies on Microsoft Azure A Survey @DChappellAssoc Copyright 2014 Chappell & Associates The Main Idea i Open source technologies are a fundamental part of Microsoft Azure The Big Questions
Making Sense of NoSQL Dan McCreary Wednesday, Nov. 13 th 2014
Making Sense of NoSQL Dan McCreary Wednesday, Nov. 13 th 2014 Agenda Why NoSQL? What are the key NoSQL architectures? How are they different from traditional RDBMS Systems? What types of problems do they
Institutionen för datavetenskap Department of Computer and Information Science
Institutionen för datavetenskap Department of Computer and Information Science Final thesis Exploration of NoSQL Technologies for managing hotel reservations by Sylvain Coulombel LiTH-IDA/ERASMUS-A--15/001--SE
Cloud data store services and NoSQL databases. Ricardo Vilaça Universidade do Minho Portugal
Cloud data store services and NoSQL databases Ricardo Vilaça Universidade do Minho Portugal Context Introduction Traditional RDBMS were not designed for massive scale. Storage of digital data has reached
Big Data Management and NoSQL Databases
NDBI040 Big Data Management and NoSQL Databases Lecture 4. Basic Principles Doc. RNDr. Irena Holubova, Ph.D. holubova@ksi.mff.cuni.cz http://www.ksi.mff.cuni.cz/~holubova/ndbi040/ NoSQL Overview Main objective:
Lecture 10: HBase! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl
Big Data Processing, 2014/15 Lecture 10: HBase!! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl 1 Course content Introduction Data streams 1 & 2 The MapReduce paradigm Looking behind the
Design Patterns for Distributed Non-Relational Databases
Design Patterns for Distributed Non-Relational Databases aka Just Enough Distributed Systems To Be Dangerous (in 40 minutes) Todd Lipcon (@tlipcon) Cloudera June 11, 2009 Introduction Common Underlying
NOSQL DATABASES AND CASSANDRA
NOSQL DATABASES AND CASSANDRA Semester Project: Advanced Databases DECEMBER 14, 2015 WANG CAN, EVABRIGHT BERTHA Université Libre de Bruxelles 0 Preface The goal of this report is to introduce the new evolving
.NET User Group Bern
.NET User Group Bern Roger Rudin bbv Software Services AG roger.rudin@bbv.ch Agenda What is NoSQL Understanding the Motivation behind NoSQL MongoDB: A Document Oriented Database NoSQL Use Cases What is
A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA
A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA Ompal Singh Assistant Professor, Computer Science & Engineering, Sharda University, (India) ABSTRACT In the new era of distributed system where
Scalable SQL and NoSQL Data Stores
Scalable SQL and NoSQL Data Stores Rick Cattell Originally published in 2010, last revised December 2011 ABSTRACT In this paper, we examine a number of SQL and socalled NoSQL data stores designed to scale
Olivier Caudron. Big Data and NoSQL
Olivier Caudron Big Data and NoSQL "Big" Data? "Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional
INTRODUCTION TO CASSANDRA
INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open
Comparing SQL and NOSQL databases
COSC 6397 Big Data Analytics Data Formats (II) HBase Edgar Gabriel Spring 2015 Comparing SQL and NOSQL databases Types Development History Data Storage Model SQL One type (SQL database) with minor variations
Scalable Architecture on Amazon AWS Cloud
Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect
Transactions and ACID in MongoDB
Transactions and ACID in MongoDB Kevin Swingler Contents Recap of ACID transactions in RDBMSs Transactions and ACID in MongoDB 1 Concurrency Databases are almost always accessed by multiple users concurrently
How To Scale Out Of A Nosql Database
Firebird meets NoSQL (Apache HBase) Case Study Firebird Conference 2011 Luxembourg 25.11.2011 26.11.2011 Thomas Steinmaurer DI +43 7236 3343 896 thomas.steinmaurer@scch.at www.scch.at Michael Zwick DI
Amr El Abbadi. Computer Science, UC Santa Barbara amr@cs.ucsb.edu
Amr El Abbadi Computer Science, UC Santa Barbara amr@cs.ucsb.edu Collaborators: Divy Agrawal, Sudipto Das, Aaron Elmore, Hatem Mahmoud, Faisal Nawab, and Stacy Patterson. Client Site Client Site Client
Benchmarking Couchbase Server for Interactive Applications. By Alexey Diomin and Kirill Grigorchuk
Benchmarking Couchbase Server for Interactive Applications By Alexey Diomin and Kirill Grigorchuk Contents 1. Introduction... 3 2. A brief overview of Cassandra, MongoDB, and Couchbase... 3 3. Key criteria
Cassandra A Decentralized Structured Storage System
Cassandra A Decentralized Structured Storage System Avinash Lakshman, Prashant Malik LADIS 2009 Anand Iyer CS 294-110, Fall 2015 Historic Context Early & mid 2000: Web applicaoons grow at tremendous rates
INFO5011 Advanced Topics in IT: Cloud Computing
INFO5011 Advanced Topics in IT: Cloud Computing Week 5: Distributed Data Management: From 2PC to Dynamo Dr. Uwe Röhm School of Information Technologies Outline Distributed Data Processing Data Partitioning
The Cloud Trade Off IBM Haifa Research Storage Systems
The Cloud Trade Off IBM Haifa Research Storage Systems 1 Fundamental Requirements form Cloud Storage Systems The Google File System first design consideration: component failures are the norm rather than
Cloud DBMS: An Overview. Shan-Hung Wu, NetDB CS, NTHU Spring, 2015
Cloud DBMS: An Overview Shan-Hung Wu, NetDB CS, NTHU Spring, 2015 Outline Definition and requirements S through partitioning A through replication Problems of traditional DDBMS Usage analysis: operational
Study and Comparison of Elastic Cloud Databases : Myth or Reality?
Université Catholique de Louvain Ecole Polytechnique de Louvain Computer Engineering Department Study and Comparison of Elastic Cloud Databases : Myth or Reality? Promoters: Peter Van Roy Sabri Skhiri
Hadoop Ecosystem Overview. CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook
Hadoop Ecosystem Overview CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook Agenda Introduce Hadoop projects to prepare you for your group work Intimate detail will be provided in future
An Open Source NoSQL solution for Internet Access Logs Analysis
An Open Source NoSQL solution for Internet Access Logs Analysis A practical case of why, what and how to use a NoSQL Database Management System instead of a relational one José Manuel Ciges Regueiro
MongoDB Developer and Administrator Certification Course Agenda
MongoDB Developer and Administrator Certification Course Agenda Lesson 1: NoSQL Database Introduction What is NoSQL? Why NoSQL? Difference Between RDBMS and NoSQL Databases Benefits of NoSQL Types of NoSQL
NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015
NoSQL Databases Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 Database Landscape Source: H. Lim, Y. Han, and S. Babu, How to Fit when No One Size Fits., in CIDR,
Challenges for Data Driven Systems
Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Quick History of Data Management 4000 B C Manual recording From tablets to papyrus to paper A. Payberah 2014 2
Introduction to Apache Cassandra
Introduction to Apache Cassandra White Paper BY DATASTAX CORPORATION JULY 2013 1 Table of Contents Abstract 3 Introduction 3 Built by Necessity 3 The Architecture of Cassandra 4 Distributing and Replicating
Вовченко Алексей, к.т.н., с.н.с. ВМК МГУ ИПИ РАН
Вовченко Алексей, к.т.н., с.н.с. ВМК МГУ ИПИ РАН Zettabytes Petabytes ABC Sharding A B C Id Fn Ln Addr 1 Fred Jones Liberty, NY 2 John Smith?????? 122+ NoSQL Database
Realtime Apache Hadoop at Facebook. Jonathan Gray & Dhruba Borthakur June 14, 2011 at SIGMOD, Athens
Realtime Apache Hadoop at Facebook Jonathan Gray & Dhruba Borthakur June 14, 2011 at SIGMOD, Athens Agenda 1 Why Apache Hadoop and HBase? 2 Quick Introduction to Apache HBase 3 Applications of HBase at
Database Management System Choices. Introduction To Database Systems CSE 373 Spring 2013
Database Management System Choices Introduction To Database Systems CSE 373 Spring 2013 Outline Introduction PostgreSQL MySQL Microsoft SQL Server Choosing A DBMS NoSQL Introduction There a lot of options
The CAP theorem and the design of large scale distributed systems: Part I
The CAP theorem and the design of large scale distributed systems: Part I Silvia Bonomi University of Rome La Sapienza www.dis.uniroma1.it/~bonomi Great Ideas in Computer Science & Engineering A.A. 2012/2013
Mobile + HA + Cloud. Eugene Ciurana! ! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin! irc.oftc.net: #tor, #tor-dev, #tails!
Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin! irc.oftc.net: #tor, #tor-dev, #tails!! qcon2014@cime.net About Eugene... 15+ years building