Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace"

Transcription

1 Introduction to Polyglot Persistence Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace FOSSCOMM 2016

2 Background - 14 years in databases and system engineering - NoSQL ObjectRocket by Rackspace - Passionate about MongoDB & Cassandra

3 What is Polyglot Persistence? A set of applications that use several core database technologies Application Layer Relational Database Key/Value Store Column-family Database

4 What is Polyglot Persistence? Using the right tool for the right use case

5 Let there be RDBMS Monoglot was (and still is) fine for simple application (one type of workload) But applications become complex A simple E-commerce platform must have: - Session data (Add to Basket) - Search Engine (Search for products) - Recommendation engine (Customers Who Bought This Item Also Bought) - Payment platform - Geo Location service

6 Applications growing rapidly

7 A RDBMS tale In the good old days monoglot = RDBMS Once upon a time there was an RDBMS with performance issues - Vertical scaling - Secondary indexes - Partitioning - Denormalize - Read-Only Slaves - Shading - Start separating workloads But The more we scale the more features we miss

8 The 3V Era Volume: Amount of data Velocity: Speed of data processing Variety: Number of types of data New databases introduced for Big Data General-purpose DB is no longer on-trend Per use-case Datastores becoming more popular The raise of Polyglot Persistence

9 CAP theorem

10 Picking the right tools Data Structure - Relational databases (Oracle, MySQL) - Key-value stores (Redis, Riak) - Column Family stores (Cassandra, Hbase) - Document databases (MongoDB, CouchDB) - Graph databases (Neo4J)

11 Relational Databases Based on the relational model (Codd) Data are organized in tables (Row and columns) Using SQL (Structured Query Language)

12 Relational Databases Use when: - Your dataset is relational - Strong consistency needed - Access patterns are unknown But Doesn t scale well horizontally Use Cases: - Due to early adoption are everywhere - Payment systems

13 Key/Value stores A big hash map associative array - Very simple, One key <-> One value - Very fast read/write - No secondary indexes { Key : (VRN)} => {value (car facts)} [make#ford {YYY0000} => model#fiesta year#2010]

14 Key/Value stores Use when: - Operations are based on the key - Data is not highly related - Basic CRUD needed But Complex queries are painful Use cases: - Session Data - User Profile/Preferences - Shopping cards

15 Document Databases Nested structures of keys and their values - Very flexible schema (JSON, XML) - One key One value but value is visible to queries - Supports hierarchical data - Supports secondary indexes { id (VRN)} => { document (car facts)} { make : Ford, {YYY0000} => model : Fiesta, year :2010 }

16 Document Databases Use when: - You don t know much for the schema - Unstructured and Heterogeneous data But Joins and references are tricky De-normalization requires more space Use cases: - Product Catalog - CMS - Event logging from different sources

17 Column family In a table, data of the same column is stored together (K-V that V is K-V) - Data organized as columns - Great for sparse tables - Very fast column operation including aggregation { id (VRN)} => { column families (car facts)} { car :{ make : Ford, model : Fiesta }, {YYY0000} => parts :{ }, service : { } }

18 Use when: - Big Data (Huge write volumes) - Versioning (Time-series data) But Know your statements in advance Schema design is not trivial Use cases: - Time series data - Bidding platforms - Playlists Column family

19 Graph Databases Inspired by the graph theory Nodes hold Data/Entities Each node connect to others using attribute(s)

20 Graph Databases Use when: - Highly interconnected data - Define explicit relationships and need traversal queries But Doesn t scale well horizontally Use Cases: - Where 3 rd degree (or higher) relationship needed - Social Media - Queries like friend of a friend

21 What about Hadoop? Framework for distributed storing and processing large sets of data Use for: - ETL - Read raw data, apply filters create structured summary - Exploration engine / Data discovery - Data archive / Massive Storage But Do not use Hadoop as a database replacement

22 Picking the right tools - Questions Data structure: - Does it have a natural structure? Is unstructured? - How it is connected to each other? - How is it distributed? - How much data? Access Patterns: - Read/Write ratio? - Uniform or random? - What is more important?

23 Picking the right tools - Questions Organization needs: - Do I need Authentication? What type? - Do I need Encryption? - Do I need Backups? - Do I need a Disaster Site? - What level of Monitoring? - Drivers, Languages? Tools - Third Party tools - Add-ons/Plugins

24 Picking the right tools - Questions Maturity - How long is in the market? Documentation - Books, tutorials - Training Type of Support - Community - Commercial support

25 Challenges Define the architecture - Decide which datastore will use to store certain data - Wrong decision can lead to painful migration(s) Deployment complexity - Provision different type of HW, OS, patches - Backup/Restore - Control configuration changes - Monitor all different components

26 Challenges Application complexity - Different connection per datastore - Handle different type of errors - Map different results on the application layer - Keep Datastores in-sync (cross-database consistency) - Active/Passive topology - Active/Active topology Training for Devs and Ops - Develop new skills for your teams - Support for a period of time

27 Don ts and anti-patterns Over engineering - Keep it simple - Remove pieces that don t add value Conform to stereotypes - Use cases are general guidelines - Benchmarks are indicators Stay static - Try new technologies - Being on-trend but cautious

28 Use Case: Mongo ElasticSearch MongoDB is using B-Tree indexes for everything B-Tree is great but not excel for all use-cases Start getting hard limits for certain use cases like FullText search Lucene engine is better option for Full text search (using inverted indexes) We already had the ElasticSearch to our portfolio so we just connected the two products

29 Use Case: Mongo Elastic Search - Connector is using the Active/Passive model - Writes must go through MongoDB and then propagated to ElasticSearch - Flexible, user can pick what to propagate: - Database(s) - Collection(s) - Field mapping - Indexing/analyzing

30 Use Case: Mongo Elastic Search - An initial sync is needed (bulk API) - Connector is using a tailable cursor that reads MongoDB oplog and propagate the changes - Similar to Extract, Transform, Load (ETL) - Application is responsible to properly direct the read requests to the most suitable datastore

31 Call me Polyglot, Call me Multi-Model MongoDB (version 3 or higher) - MMAPv1 - WiredTiger - In-Memory Percona Server for MongoDB - MMAPv1 - WiredTiger - RocksDB - PerconaFT

32 The future(?) Multi Model databases Support different models within the engine -OR- Offer different layers on top of the engine OrientDB supporting graph, document and key/value models Relationships are managed as in graph databases with direct connections between records FoundationDB feature layers on top of a key-value store

33 Keep in touch : I iamantonios We are hiring!!! Data Engineers, DevOps, DBAs and more https://www.rackspace.com/talent/

34 Questions? Thank you!!!

Cloud Scale Distributed Data Storage. Jürmo Mehine

Cloud Scale Distributed Data Storage. Jürmo Mehine Cloud Scale Distributed Data Storage Jürmo Mehine 2014 Outline Background Relational model Database scaling Keys, values and aggregates The NoSQL landscape Non-relational data models Key-value Document-oriented

More information

Data Services Advisory

Data Services Advisory Data Services Advisory Modern Datastores An Introduction Created by: Strategy and Transformation Services Modified Date: 8/27/2014 Classification: DRAFT SAFE HARBOR STATEMENT This presentation contains

More information

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world Analytics March 2015 White paper Why NoSQL? Your database options in the new non-relational world 2 Why NoSQL? Contents 2 New types of apps are generating new types of data 2 A brief history of NoSQL 3

More information

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1 Why NoSQL? Your database options in the new non- relational world 2015 IBM Cloudant 1 Table of Contents New types of apps are generating new types of data... 3 A brief history on NoSQL... 3 NoSQL s roots

More information

Search and Real-Time Analytics on Big Data

Search and Real-Time Analytics on Big Data Search and Real-Time Analytics on Big Data Sewook Wee, Ryan Tabora, Jason Rutherglen Accenture & Think Big Analytics Strata New York October, 2012 Big Data: data becomes your core asset. It realizes its

More information

MongoDB Developer and Administrator Certification Course Agenda

MongoDB Developer and Administrator Certification Course Agenda MongoDB Developer and Administrator Certification Course Agenda Lesson 1: NoSQL Database Introduction What is NoSQL? Why NoSQL? Difference Between RDBMS and NoSQL Databases Benefits of NoSQL Types of NoSQL

More information

NoSQL Databases. Nikos Parlavantzas

NoSQL Databases. Nikos Parlavantzas !!!! NoSQL Databases Nikos Parlavantzas Lecture overview 2 Objective! Present the main concepts necessary for understanding NoSQL databases! Provide an overview of current NoSQL technologies Outline 3!

More information

MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15

MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15 MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15 1 MongoDB in the NoSQL and SQL world. NoSQL What? Why? - How? Say goodbye to ACID, hello BASE You

More information

extensible record stores document stores key-value stores Rick Cattel s clustering from Scalable SQL and NoSQL Data Stores SIGMOD Record, 2010

extensible record stores document stores key-value stores Rick Cattel s clustering from Scalable SQL and NoSQL Data Stores SIGMOD Record, 2010 System/ Scale to Primary Secondary Joins/ Integrity Language/ Data Year Paper 1000s Index Indexes Transactions Analytics Constraints Views Algebra model my label 1971 RDBMS O tables sql-like 2003 memcached

More information

Mom, I so wish Hibernate for my NoSQL database... Speaker : Alexey Zinoviev

Mom, I so wish Hibernate for my NoSQL database... Speaker : Alexey Zinoviev Mom, I so wish Hibernate for my NoSQL database... Speaker : Alexey Zinoviev About I am a scientist. The area of my interests includes machine learning, traffic jams prediction, BigData algorythms. But

More information

CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3

CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3 INTRODUCTION xvii PART I: GETTING STARTED CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3 Definition and Introduction 4 Context and a Bit of History 4 Big Data 7 Scalability 9 Defi nition and Introduction

More information

Lecture Data Warehouse Systems

Lecture Data Warehouse Systems Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART C: Novel Approaches in DW NoSQL and MapReduce Stonebraker on Data Warehouses Star and snowflake schemas are a good idea in the DW world C-Stores

More information

Preparing Your Data For Cloud

Preparing Your Data For Cloud Preparing Your Data For Cloud Narinder Kumar Inphina Technologies 1 Agenda Relational DBMS's : Pros & Cons Non-Relational DBMS's : Pros & Cons Types of Non-Relational DBMS's Current Market State Applicability

More information

On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform

On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform Page 1 of 16 Table of Contents Table of Contents... 2 Introduction... 3 NoSQL Databases... 3 CumuLogic NoSQL Database Service...

More information

Structured Data Storage

Structured Data Storage Structured Data Storage Xgen Congress Short Course 2010 Adam Kraut BioTeam Inc. Independent Consulting Shop: Vendor/technology agnostic Staffed by: Scientists forced to learn High Performance IT to conduct

More information

Study concluded that success rate for penetration from outside threats higher in corporate data centers

Study concluded that success rate for penetration from outside threats higher in corporate data centers Auditing in the cloud Ownership of data Historically, with the company Company responsible to secure data Firewall, infrastructure hardening, database security Auditing Performed on site by inspecting

More information

NoSQL Databases. Polyglot Persistence

NoSQL Databases. Polyglot Persistence The future is: NoSQL Databases Polyglot Persistence a note on the future of data storage in the enterprise, written primarily for those involved in the management of application development. Martin Fowler

More information

Overview of Databases On MacOS. Karl Kuehn Automation Engineer RethinkDB

Overview of Databases On MacOS. Karl Kuehn Automation Engineer RethinkDB Overview of Databases On MacOS Karl Kuehn Automation Engineer RethinkDB Session Goals Introduce Database concepts Show example players Not Goals: Cover non-macos systems (Oracle) Teach you SQL Answer what

More information

NoSQL Roadshow Berlin Kai Spichale

NoSQL Roadshow Berlin Kai Spichale Full-text Search with NoSQL Technologies NoSQL Roadshow Berlin Kai Spichale 25.04.2013 About me Kai Spichale Software Engineer at adesso AG Author in professional journals, conference speaker adesso is

More information

NoSQL in der Cloud Why? Andreas Hartmann

NoSQL in der Cloud Why? Andreas Hartmann NoSQL in der Cloud Why? Andreas Hartmann 17.04.2013 17.04.2013 2 NoSQL in der Cloud Why? Quelle: http://res.sys-con.com/story/mar12/2188748/cloudbigdata_0_0.jpg Why Cloud??? 17.04.2013 3 NoSQL in der Cloud

More information

Introduction to new high performance storage engines in MongoDB 2.8 3.0

Introduction to new high performance storage engines in MongoDB 2.8 3.0 Introduction to new high performance storage engines in MongoDB 2.8 3.0 Henrik Ingo Solutions Architect, MongoDB Hi, I am Henrik Ingo @h_ingo 2 Introduction to new high performance storage engines in MongoDB

More information

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 NoSQL Databases Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 Database Landscape Source: H. Lim, Y. Han, and S. Babu, How to Fit when No One Size Fits., in CIDR,

More information

Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014. Apologies to Martin Fowler ( NoSQL Distilled )

Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014. Apologies to Martin Fowler ( NoSQL Distilled ) Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014 Rise of Web and cluster-based computing NoSQL Movement Relationships vs. Aggregates Key-value store XML or JSON

More information

Enterprise Operational SQL on Hadoop Trafodion Overview

Enterprise Operational SQL on Hadoop Trafodion Overview Enterprise Operational SQL on Hadoop Trafodion Overview Rohit Jain Distinguished & Chief Technologist Strategic & Emerging Technologies Enterprise Database Solutions Copyright 2012 Hewlett-Packard Development

More information

Introduction to Big Data Training

Introduction to Big Data Training Introduction to Big Data Training The quickest way to be introduce with NOSQL/BIG DATA offerings Learn and experience Big Data Solutions including Hadoop HDFS, Map Reduce, NoSQL DBs: Document Based DB

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

Application-specific databases have always been with us...

Application-specific databases have always been with us... Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014 Rise of Web and cluster-based computing NoSQL Movement Relationships vs. Aggregates Key-value store XML or JSON

More information

Comparing SQL and NOSQL databases

Comparing SQL and NOSQL databases COSC 6397 Big Data Analytics Data Formats (II) HBase Edgar Gabriel Spring 2015 Comparing SQL and NOSQL databases Types Development History Data Storage Model SQL One type (SQL database) with minor variations

More information

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com Lambda Architecture Near Real-Time Big Data Analytics Using Hadoop January 2015 Contents Overview... 3 Lambda Architecture: A Quick Introduction... 4 Batch Layer... 4 Serving Layer... 4 Speed Layer...

More information

SQL VS. NO-SQL. Adapted Slides from Dr. Jennifer Widom from Stanford

SQL VS. NO-SQL. Adapted Slides from Dr. Jennifer Widom from Stanford SQL VS. NO-SQL Adapted Slides from Dr. Jennifer Widom from Stanford 55 Traditional Databases SQL = Traditional relational DBMS Hugely popular among data analysts Widely adopted for transaction systems

More information

X4-2 Exadata announced (well actually around Jan 1) OEM/Grid control 12c R4 just released

X4-2 Exadata announced (well actually around Jan 1) OEM/Grid control 12c R4 just released General announcements In-Memory is available next month http://www.oracle.com/us/corporate/events/dbim/index.html X4-2 Exadata announced (well actually around Jan 1) OEM/Grid control 12c R4 just released

More information

NoSQL Database Systems and their Security Challenges

NoSQL Database Systems and their Security Challenges NoSQL Database Systems and their Security Challenges Morteza Amini amini@sharif.edu Data & Network Security Lab (DNSL) Department of Computer Engineering Sharif University of Technology September 25 2

More information

How graph databases started the multi-model revolution

How graph databases started the multi-model revolution How graph databases started the multi-model revolution Luca Garulli Author and CEO @OrientDB QCon Sao Paulo - March 26, 2015 Welcome to Big Data 90% of the data in the world today has been created in the

More information

Introduction to Hadoop. New York Oracle User Group Vikas Sawhney

Introduction to Hadoop. New York Oracle User Group Vikas Sawhney Introduction to Hadoop New York Oracle User Group Vikas Sawhney GENERAL AGENDA Driving Factors behind BIG-DATA NOSQL Database 2014 Database Landscape Hadoop Architecture Map/Reduce Hadoop Eco-system Hadoop

More information

Big Data Analytics. 6. NoSQL Databases. Lars Schmidt-Thieme

Big Data Analytics. 6. NoSQL Databases. Lars Schmidt-Thieme Big Data Analytics 6. NoSQL Databases Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany original slides by Lucas Rego

More information

Big Data Management. Big Data Management. (BDM) Autumn 2013. Povl Koch September 30, 2013 29-09-2013 1

Big Data Management. Big Data Management. (BDM) Autumn 2013. Povl Koch September 30, 2013 29-09-2013 1 Big Data Management Big Data Management (BDM) Autumn 2013 Povl Koch September 30, 2013 29-09-2013 1 Overview Today s program 1. Little more practical details about this course 2. Recap from last time 3.

More information

The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg. Adam Marcus MIT CSAIL marcua@csail.mit.edu / @marcua

The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg. Adam Marcus MIT CSAIL marcua@csail.mit.edu / @marcua The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg Adam Marcus MIT CSAIL marcua@csail.mit.edu / @marcua About Me Social Computing + Database Systems Easily Distracted: Wrote The NoSQL Ecosystem in

More information

A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA

A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA Ompal Singh Assistant Professor, Computer Science & Engineering, Sharda University, (India) ABSTRACT In the new era of distributed system where

More information

Objectives. Introduce some key concepts behind the NoSQL family of databases

Objectives. Introduce some key concepts behind the NoSQL family of databases NoSQL Source: Pramod J. Sadalage and Martin Fowler NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence, Pearson Education, 2013 Objectives Introduce some key concepts behind the

More information

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2016

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2016 Big Data Technologies Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2016 Situation: Bigger and Bigger Volumes of Data Big Data Use Cases Log Analytics (Web Logs, Sensor

More information

Hybrid Solutions Combining In-Memory & SSD

Hybrid Solutions Combining In-Memory & SSD Hybrid Solutions Combining In-Memory & SSD Author: christos@gigaspaces.com Agenda 1 2 3 4 Overview of the big data technology landscape Building a high-speed SSD-backed data store Complex & compound queries

More information

Performance and Scalability Overview

Performance and Scalability Overview Performance and Scalability Overview This guide provides an overview of some of the performance and scalability capabilities of the Pentaho Business Analytics Platform. Contents Pentaho Scalability and

More information

Integrating Big Data into the Computing Curricula

Integrating Big Data into the Computing Curricula Integrating Big Data into the Computing Curricula Yasin Silva, Suzanne Dietrich, Jason Reed, Lisa Tsosie Arizona State University http://www.public.asu.edu/~ynsilva/ibigdata/ 1 Overview Motivation Big

More information

NoSQL Database Options

NoSQL Database Options NoSQL Database Options Introduction For this report, I chose to look at MongoDB, Cassandra, and Riak. I chose MongoDB because it is quite commonly used in the industry. I chose Cassandra because it has

More information

Scalable Architecture on Amazon AWS Cloud

Scalable Architecture on Amazon AWS Cloud Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect

More information

Open Source Technologies on Microsoft Azure

Open Source Technologies on Microsoft Azure Open Source Technologies on Microsoft Azure A Survey @DChappellAssoc Copyright 2014 Chappell & Associates The Main Idea i Open source technologies are a fundamental part of Microsoft Azure The Big Questions

More information

The Multi-Model Database Cloud Applications in a Complex World

The Multi-Model Database Cloud Applications in a Complex World The Multi-Model Database Cloud Applications in a Complex World Table of Contents INTRODUCTION MULTI-MODEL: AN EVOLUTIONARY TALE FROM RDBMS TO NOSQL TO MULTI-MODEL DATASTAX ENTERPRISE AND MULTI-MODEL DECIDING

More information

Open Source Technologies on Microsoft Azure

Open Source Technologies on Microsoft Azure Open Source Technologies on Microsoft Azure A Survey @DChappellAssoc Copyright 2014 Chappell & Associates The Main Idea i Open source technologies are a fundamental part of Microsoft Azure The Big Questions

More information

Cloud Big Data Architectures

Cloud Big Data Architectures Cloud Big Data Architectures Lynn Langit QCon Sao Paulo, Brazil 2016 About this Workshop Real-world Cloud Scenarios w/aws, Azure and GCP 1. Big Data Solution Types 2. Data Pipelines 3. ETL and Visualization

More information

NoSQL - What we ve learned with mongodb. Paul Pedersen, Deputy CTO paul@10gen.com DAMA SF December 15, 2011

NoSQL - What we ve learned with mongodb. Paul Pedersen, Deputy CTO paul@10gen.com DAMA SF December 15, 2011 NoSQL - What we ve learned with mongodb Paul Pedersen, Deputy CTO paul@10gen.com DAMA SF December 15, 2011 DW2.0 and NoSQL management decision support intgrated access - local v. global - structured v.

More information

What Next for DBAs in the Big Data Era

What Next for DBAs in the Big Data Era What Next for DBAs in the Big Data Era February 21 st, 2015 Copyright 2013. Apps Associates LLC. 1 Satyendra Kumar Pasalapudi Associate Practice Director IMS @ Apps Associates Co Founder & President of

More information

Big Data Analytics Platform @ Nokia

Big Data Analytics Platform @ Nokia Big Data Analytics Platform @ Nokia 1 Selecting the Right Tool for the Right Workload Yekesa Kosuru Nokia Location & Commerce Strata + Hadoop World NY - Oct 25, 2012 Agenda Big Data Analytics Platform

More information

Dr. Chuck Cartledge. 27 Aug. 2015

Dr. Chuck Cartledge. 27 Aug. 2015 CS-695 NoSQL Database Polyglot Persistence; Or, The Many Ways We Store Data Dr. Chuck Cartledge 27 Aug. 2015 1/37 Table of contents I 4 CRUDy stuff 1 A little history 2 A change in the air 3 Database layouts

More information

Oracle Database 12c Plug In. Switch On. Get SMART.

Oracle Database 12c Plug In. Switch On. Get SMART. Oracle Database 12c Plug In. Switch On. Get SMART. Duncan Harvey Head of Core Technology, Oracle EMEA March 2015 Safe Harbor Statement The following is intended to outline our general product direction.

More information

Performance Evaluation of NoSQL Systems Using YCSB in a resource Austere Environment

Performance Evaluation of NoSQL Systems Using YCSB in a resource Austere Environment International Journal of Applied Information Systems (IJAIS) ISSN : 2249-868 Performance Evaluation of NoSQL Systems Using YCSB in a resource Austere Environment Yusuf Abubakar Department of Computer Science

More information

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013 Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device

More information

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015 Big Data Technologies Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015 Situation: Bigger and Bigger Volumes of Data Big Data Use Cases Log Analytics (Web Logs, Sensor

More information

Big Data Approaches. Making Sense of Big Data. Ian Crosland. Jan 2016

Big Data Approaches. Making Sense of Big Data. Ian Crosland. Jan 2016 Big Data Approaches Making Sense of Big Data Ian Crosland Jan 2016 Accelerate Big Data ROI Even firms that are investing in Big Data are still struggling to get the most from it. Make Big Data Accessible

More information

SQL, NoSQL, and Next Generation DBMSs. Shahram Ghandeharizadeh Director of the USC Database Lab

SQL, NoSQL, and Next Generation DBMSs. Shahram Ghandeharizadeh Director of the USC Database Lab SQL, NoSQL, and Next Generation DBMSs Shahram Ghandeharizadeh Director of the USC Database Lab Outline A brief history of DBMSs. OSs SQL NoSQL 1960/70 1980+ 2000+ Before Computers Database DBMS/Data Store

More information

Applications for Big Data Analytics

Applications for Big Data Analytics Smarter Healthcare Applications for Big Data Analytics Multi-channel sales Finance Log Analysis Homeland Security Traffic Control Telecom Search Quality Manufacturing Trading Analytics Fraud and Risk Retail:

More information

Mobile + HA + Cloud. Eugene Ciurana! ! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin! irc.oftc.net: #tor, #tor-dev, #tails!

Mobile + HA + Cloud. Eugene Ciurana! ! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin! irc.oftc.net: #tor, #tor-dev, #tails! Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin! irc.oftc.net: #tor, #tor-dev, #tails!! qcon2014@cime.net About Eugene... 15+ years building

More information

Oracle s Big Data solutions. Roger Wullschleger.

Oracle s Big Data solutions. Roger Wullschleger. <Insert Picture Here> s Big Data solutions Roger Wullschleger DBTA Workshop on Big Data, Cloud Data Management and NoSQL 10. October 2012, Stade de Suisse, Berne 1 The following is intended to outline

More information

Database Design Service

Database Design Service Database Design Service Contents 1 Introduction to Database Design...2 2 Service Definition...4 2.1 Technology Options... 4 2.2 Database Modelling... 5 2.3 Database Architecture... 5 3 Commercials...6

More information

Large Scale/Big Data Federation & Virtualization: A Case Study

Large Scale/Big Data Federation & Virtualization: A Case Study Large Scale/Big Data Federation & Virtualization: A Case Study Vamsi Chemitiganti, Chief Solution Architect Derrick Kittler, Senior Solution Architect Bill Kemp, Senior Solution Architect Red Hat 06.29.12

More information

Database Scalability {Patterns} / Robert Treat

Database Scalability {Patterns} / Robert Treat Database Scalability {Patterns} / Robert Treat robert treat omniti postgres oracle - mysql mssql - sqlite - nosql What are Database Scalability Patterns? Part Design Patterns Part Application Life-Cycle

More information

Understanding NoSQL on Microsoft Azure

Understanding NoSQL on Microsoft Azure David Chappell Understanding NoSQL on Microsoft Azure Sponsored by Microsoft Corporation Copyright 2014 Chappell & Associates Contents Data on Azure: The Big Picture... 3 Relational Technology: A Quick

More information

Study and Comparison of Elastic Cloud Databases : Myth or Reality?

Study and Comparison of Elastic Cloud Databases : Myth or Reality? Université Catholique de Louvain Ecole Polytechnique de Louvain Computer Engineering Department Study and Comparison of Elastic Cloud Databases : Myth or Reality? Promoters: Peter Van Roy Sabri Skhiri

More information

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach Nic Caine NoSQL Matters, April 2013 Overview The Problem Current Big Data Analytics Relationship Analytics Leveraging

More information

OPERATIONAL SCENARIOS USING THE MICROSOFT DATA PLATFORM

OPERATIONAL SCENARIOS USING THE MICROSOFT DATA PLATFORM David Chappell OPERATIONAL SCENARIOS USING THE MICROSOFT DATA PLATFORM A GUIDE FOR IT LEADERS Sponsored by Microsoft Corporation Copyright 2016 Chappell & Associates Contents Microsoft s Data Platform:

More information

NoSQL Drill-Down: So What s a Graph Database? NoCOUG Aug 2013. Philip Rathle Sr. Director of Products for Neo4j philip@neotechnology.

NoSQL Drill-Down: So What s a Graph Database? NoCOUG Aug 2013. Philip Rathle Sr. Director of Products for Neo4j philip@neotechnology. NoSQL Drill-Down: So What s a Graph Database? NoCOUG Aug 2013 Philip Rathle Sr. Director of Products for Neo4j philip@neotechnology.com @prathle 143 Philip 143 326 326 725 Big Data Fremont Neo4j San Francisco

More information

Big Data. Facebook Wall Data using Graph API. Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715

Big Data. Facebook Wall Data using Graph API. Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715 Big Data Facebook Wall Data using Graph API Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715 Outline Data Source Processing tools for processing our data Big Data Processing System: Mongodb

More information

ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA

ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA David Vanderfeesten, Bell Labs Belgium ANNO 2012 YOUR DATA IS MONEY BIG MONEY! Your click stream, your activity stream, your electricity consumption, your call

More information

Department of Software Systems. Presenter: Saira Shaheen, 227233 saira.shaheen@tut.fi 0417016438 Dated: 02-10-2012

Department of Software Systems. Presenter: Saira Shaheen, 227233 saira.shaheen@tut.fi 0417016438 Dated: 02-10-2012 1 MongoDB Department of Software Systems Presenter: Saira Shaheen, 227233 saira.shaheen@tut.fi 0417016438 Dated: 02-10-2012 2 Contents Motivation : Why nosql? Introduction : What does NoSQL means?? Applications

More information

InfiniteGraph: The Distributed Graph Database

InfiniteGraph: The Distributed Graph Database A Performance and Distributed Performance Benchmark of InfiniteGraph and a Leading Open Source Graph Database Using Synthetic Data Objectivity, Inc. 640 West California Ave. Suite 240 Sunnyvale, CA 94086

More information

An Open Source NoSQL solution for Internet Access Logs Analysis

An Open Source NoSQL solution for Internet Access Logs Analysis An Open Source NoSQL solution for Internet Access Logs Analysis A practical case of why, what and how to use a NoSQL Database Management System instead of a relational one José Manuel Ciges Regueiro

More information

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research &

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & Innovation 04-08-2011 to the EC 8 th February, Luxembourg Your Atos business Research technologists. and Innovation

More information

wow CPSC350 relational schemas table normalization practical use of relational algebraic operators tuple relational calculus and their expression in a declarative query language relational schemas CPSC350

More information

Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2

Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2 Volume 6, Issue 3, March 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Special Issue

More information

Вовченко Алексей, к.т.н., с.н.с. ВМК МГУ ИПИ РАН

Вовченко Алексей, к.т.н., с.н.с. ВМК МГУ ИПИ РАН Вовченко Алексей, к.т.н., с.н.с. ВМК МГУ ИПИ РАН Zettabytes Petabytes ABC Sharding A B C Id Fn Ln Addr 1 Fred Jones Liberty, NY 2 John Smith?????? 122+ NoSQL Database

More information

NoSQL Systems for Big Data Management

NoSQL Systems for Big Data Management NoSQL Systems for Big Data Management Venkat N Gudivada East Carolina University Greenville, North Carolina USA Venkat Gudivada NoSQL Systems for Big Data Management 1/28 Outline 1 An Overview of NoSQL

More information

Performance Analysis for NoSQL and SQL

Performance Analysis for NoSQL and SQL Available online at www.ijiere.com International Journal of Innovative and Emerging Research in Engineering e-issn: 2394-3343 p-issn: 2394-5494 Performance Analysis for NoSQL and SQL Ms. Megha Katkar ME

More information

Offload Historical Data to Big Data Lake. Ample White Paper

Offload Historical Data to Big Data Lake. Ample White Paper Offload Historical Data to Big Data Lake The Need to Offload Historical Data for Compliance Queries How often have heard that the legal or compliance department group needs to have access to your company

More information

Databases 2 (VU) (707.030)

Databases 2 (VU) (707.030) Databases 2 (VU) (707.030) Introduction to NoSQL Denis Helic KMI, TU Graz Oct 14, 2013 Denis Helic (KMI, TU Graz) NoSQL Oct 14, 2013 1 / 37 Outline 1 NoSQL Motivation 2 NoSQL Systems 3 NoSQL Examples 4

More information

Performance and Scalability Overview

Performance and Scalability Overview Performance and Scalability Overview This guide provides an overview of some of the performance and scalability capabilities of the Pentaho Business Analytics platform. PENTAHO PERFORMANCE ENGINEERING

More information

NOSQL DATABASE SYSTEMS

NOSQL DATABASE SYSTEMS NOSQL DATABASE SYSTEMS Big Data Technologies: NoSQL DBMS - SoSe 2015 1 Categorization NoSQL Data Model Storage Layout Query Models Solution Architectures NoSQL Database Systems Data Modeling id ti Application

More information

A Selection Method of Database System in Bigdata Environment: A Case Study From Smart Education Service in Korea

A Selection Method of Database System in Bigdata Environment: A Case Study From Smart Education Service in Korea Int. J. Advance Soft Compu. Appl, Vol. 7, No. 1, March 2015 ISSN 2074-8523 A Selection Method of Database System in Bigdata Environment: A Case Study From Smart Education Service in Korea Jong Sung Hwang

More information

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing Evaluating NoSQL for Enterprise Applications Dirk Bartels VP Strategy & Marketing Agenda The Real Time Enterprise The Data Gold Rush Managing The Data Tsunami Analytics and Data Case Studies Where to go

More information

Advanced Data Management Technologies

Advanced Data Management Technologies ADMT 2014/15 Unit 15 J. Gamper 1/44 Advanced Data Management Technologies Unit 15 Introduction to NoSQL J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE ADMT 2014/15 Unit 15

More information

Realtime Apache Hadoop at Facebook. Jonathan Gray & Dhruba Borthakur June 14, 2011 at SIGMOD, Athens

Realtime Apache Hadoop at Facebook. Jonathan Gray & Dhruba Borthakur June 14, 2011 at SIGMOD, Athens Realtime Apache Hadoop at Facebook Jonathan Gray & Dhruba Borthakur June 14, 2011 at SIGMOD, Athens Agenda 1 Why Apache Hadoop and HBase? 2 Quick Introduction to Apache HBase 3 Applications of HBase at

More information

CS 4604: Introduc0on to Database Management Systems. B. Aditya Prakash Lecture #13: NoSQL and MapReduce

CS 4604: Introduc0on to Database Management Systems. B. Aditya Prakash Lecture #13: NoSQL and MapReduce CS 4604: Introduc0on to Database Management Systems B. Aditya Prakash Lecture #13: NoSQL and MapReduce Announcements HW4 is out You have to use the PGSQL server START EARLY!! We can not help if everyone

More information

Benchmarking Couchbase Server for Interactive Applications. By Alexey Diomin and Kirill Grigorchuk

Benchmarking Couchbase Server for Interactive Applications. By Alexey Diomin and Kirill Grigorchuk Benchmarking Couchbase Server for Interactive Applications By Alexey Diomin and Kirill Grigorchuk Contents 1. Introduction... 3 2. A brief overview of Cassandra, MongoDB, and Couchbase... 3 3. Key criteria

More information

Tobias.Trelle@codecentric.de @tobiastrelle. codecentric AG 1

Tobias.Trelle@codecentric.de @tobiastrelle. codecentric AG 1 NoSQL Unit & Travis CI Test Automation for NoSQL Databases Tobias.Trelle@codecentric.de @tobiastrelle codecentric AG 1 Tobias Trelle Senior IT Consultant @ codecentric AG Organizer of MongoDB User Group

More information

Architecting Open source solutions on Azure. Nicholas Dritsas Senior Director, Microsoft Singapore

Architecting Open source solutions on Azure. Nicholas Dritsas Senior Director, Microsoft Singapore Learn. Connect. Explore. Architecting Open source solutions on Azure Nicholas Dritsas Senior Director, Microsoft Singapore Agenda Developing OSS Apps on Azure Customer case with OSS Apps Hadoop on Azure

More information

An Approach to Implement Map Reduce with NoSQL Databases

An Approach to Implement Map Reduce with NoSQL Databases www.ijecs.in International Journal Of Engineering And Computer Science ISSN: 2319-7242 Volume 4 Issue 8 Aug 2015, Page No. 13635-13639 An Approach to Implement Map Reduce with NoSQL Databases Ashutosh

More information

CloudDB: A Data Store for all Sizes in the Cloud

CloudDB: A Data Store for all Sizes in the Cloud CloudDB: A Data Store for all Sizes in the Cloud Hakan Hacigumus Data Management Research NEC Laboratories America http://www.nec-labs.com/dm www.nec-labs.com What I will try to cover Historical perspective

More information

Open source, high performance database

Open source, high performance database Open source, high performance database Anti-social Databases: NoSQL and MongoDB Will LaForest Senior Director of 10gen Federal will@10gen.com @WLaForest 1 SQL invented Dynamic Web Content released IBM

More information

Tungsten Replicator, more open than ever!

Tungsten Replicator, more open than ever! Tungsten Replicator, more open than ever! MC Brown, Senior Product Line Manager September, 2015 2014 VMware Inc. All rights reserved. We Face An Age Old Problem BRS/Search 2 It s Gotten Worse 3 Much Worse

More information

Table of Contents. Développement logiciel pour le Cloud (TLC) Table of Contents. 5. NoSQL data models. Guillaume Pierre

Table of Contents. Développement logiciel pour le Cloud (TLC) Table of Contents. 5. NoSQL data models. Guillaume Pierre Table of Contents Développement logiciel pour le Cloud (TLC) 5. NoSQL data models Guillaume Pierre Université de Rennes 1 Fall 2012 http://www.globule.org/~gpierre/ Développement logiciel pour le Cloud

More information

Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY MANNING ANN KELLY. Shelter Island

Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY MANNING ANN KELLY. Shelter Island Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY ANN KELLY II MANNING Shelter Island contents foreword preface xvii xix acknowledgments xxi about this book xxii Part 1 Introduction

More information