Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2016

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2016"

Transcription

1 Big Data Technologies Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2016

2 Situation: Bigger and Bigger Volumes of Data Big Data Use Cases Log Analytics (Web Logs, Sensor Logs, Event Logs etc.) RFID Tracking and Analytics Fraud and Risk Management Churn Management Gaming Data Streaming Data Social Network Data Volumes of Data Eric Schmidt (Google CEO, 2010): 5 exabytes of information created between the dawn of civilization through 2003 now created every 2 days, and the pace is increasing. h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

3 Characteristics of Big Data Source: L. Haas, IBM h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

4 Emerging Technologies Hype Cycle 2013 Source: Gartner, July 2013 h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

5 Emerging Technologies Hype Cycle 2014 Source: Gartner, August 2014 h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

6 Emerging Technologies Hype Cycle 2015 Source: Gartner, August 2015 h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

7 Scale up vs. Scale out Scale up: big single node system Scale out: many small nodes Source: ibm.com Source: eggmusic.com h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

8 Scale up vs. Scale out Scale up (scale vertically) Advantages: Transparent for database system Management complexity constant Disadvantages: Expensive hardware Limited scalability Scale out (scale horizontally) Advantages: Less expensive hardware High scalability Disadvantages: Load balancing necessary Data distribution necessary Increased error rate Increased management complexity Distributed lock and replication protocols h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

9 NoSQL Database Systems: Idea Need for databases being distributed and horizontally scalable. NoSQL Definition: Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open-source and horizontally scalable. The original intention has been modern web-scale databases. The movement began early 2009 and is growing rapidly. Often more characteristics apply such as: schema-free, easy replication support, simple API, eventually consistent / BASE (not ACID), a huge amount of data and more. So the misleading term "nosql" (the community now translates it mostly with "not only sql") should be seen as an alias to something like the definition above. Source: S. Edlich, nosql-database.org Detailed discussion in chapter NoSQL Database Systems h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

10 NoSQL Database Systems: Classification 1(2) Key-Value Database Systems Key-value pairs Systems: Redis, Memcached, Amazon DynamoDB, Redis, Oracle Berkeley DB, key key key key key value value value value value Document Store Database Systems Key-value pairs with documents as value Document format: JSON, BSON Systems: MongoDB, Couchbase, CouchDB, { } "id": 1, "name": football boot", "price": 199, "stock": { "warehouse": 120, "retail": 10 } h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

11 NoSQL Database Systems: Classification 2(2) Column Familiy Database Systems (Wide Column Stores) Data loosely structured by column families and columns Systems: Cassandra, HBase, Google Bigtable (Cloud Bigtable), Row Key Time Stamp ColumnFamily contents: ColumnFamily anchor "com.cnn.www" t9 anchor:cnnsi.com "CNN" (Graph Database Systems) Graph structures with nodes, edges, and properties Systems: Neo4J, Infinite Graph OrientDB (Multi-Model) t8 anchor:my.look.ch "CNN.com" t6 t5 "<html> " "<html> " Source: h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

12 NoSQL Database Systems: Topics of Interest Categorization Data Modeling Application Development Scalability, Availability and Consistency Select the Right DBMS h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

13 Big Data: Other Approaches Will everything be NoSQL in the future? Definitely not! h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

14 Column Store Database Systems: Idea Goal: Reduce the number of disc access / amount of data to read + easy to insert/update a record + only need to read in relevant data might read in unnecessary data + higher compression ratio insert/update require multiple accesses expensive reads on entire records suitable for read-mostly, read-intensive, large data repositories (OLAP) Source: Harizopoulos/Abadi/Boncz: VLDB2009 h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

15 Storage Layout Column Store Database Systems: Topics of Interest Columnar Storage Compression Execution Engine Operating on columns (insert / update / read) Operating on compressed data (early vs. late materialization) h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

16 In-Memory Database Systems: Idea Traditionell OLTP & OLAP OLTP ETL OLAP Vision: Information in Real Time: Anything, Anytime, Anywhere OLTP & Real Time OLAP Hardware Trends Steadily declining memory price Many-core processor In-Memory Database Management Systems h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

17 In-Memory Database Systems: Topics of Interest In-Memory DBMS Disk-Based DBMS + Tons of Memory Architecture and Implementation Storage Layout: Row, Column, Hybrid? Persistence Design of operations Application Development h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

18 Emerging Technologies Hype Cycle 2014 Source: Gartner, August 2014 h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

19 Course Goals Introduction to the big data technology landscape NoSQL database systems (distributed and horizontally scalable non-relational database systems) Column store database systems In-memory database systems Graph database systems h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

20 Literature Tiwari, S. Professional NoSQL, Wiley, 2011 Sadalage, P. J., Fowler, M. NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence, Addison Wesley, 2012 Edlich, S., Friedland, A., Hampe, J., Brauer, B., Brückner, M. NoSQL Einstieg in die Welt nichtrelationaler Web 2.0 Datenbanken, Carl Hanser Verlag, 2011 (2 nd ed.) Kudraß, Th. (Ed.), Taschenbuch Datenbanken, Störl, U.: Kap. 12 NoSQL- Datenbanksysteme, Ritter, N.: Kap. 13 Verteilte und föderierte Datenbanksysteme, Carl Hanser Verlag, 2015 (2 nd ed.) Abadi, D., Boncz, P., Harizopoulus, S, Idreos, S., Madden, S. The design and implementation of modern column-oriented database, Transatlantic Publishers, 2013 Harizopoulus, S., Abadi, D., Boncz, P. Column-Oriented Database Systems, VLDB Tutorial, Plattner, H., Zeier, A. In-memory data management: technology and applications, Springer Verlag, 2012 (2 nd ed.) h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

21 Big Data Technologies Introduction NoSQL Database Systems Column Store Database Systems In-Memory Database Systems Graph Database Systems Conclusion & Outlook h_da Prof. Dr. Uta Störl Big Data Technologies: Introduction - SoSe

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015 Big Data Technologies Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015 Situation: Bigger and Bigger Volumes of Data Big Data Use Cases Log Analytics (Web Logs, Sensor

More information

GRAPH DATABASE SYSTEMS. h_da Prof. Dr. Uta Störl Big Data Technologies: Graph Database Systems - SoSe 2016 1

GRAPH DATABASE SYSTEMS. h_da Prof. Dr. Uta Störl Big Data Technologies: Graph Database Systems - SoSe 2016 1 GRAPH DATABASE SYSTEMS h_da Prof. Dr. Uta Störl Big Data Technologies: Graph Database Systems - SoSe 2016 1 Use Case: Route Finding Source: Neo Technology, Inc. h_da Prof. Dr. Uta Störl Big Data Technologies:

More information

NOSQL DATABASE SYSTEMS

NOSQL DATABASE SYSTEMS NOSQL DATABASE SYSTEMS Big Data Technologies: NoSQL DBMS - SoSe 2015 1 Categorization NoSQL Data Model Storage Layout Query Models Solution Architectures NoSQL Database Systems Data Modeling id ti Application

More information

Cloud Scale Distributed Data Storage. Jürmo Mehine

Cloud Scale Distributed Data Storage. Jürmo Mehine Cloud Scale Distributed Data Storage Jürmo Mehine 2014 Outline Background Relational model Database scaling Keys, values and aggregates The NoSQL landscape Non-relational data models Key-value Document-oriented

More information

Objectives. Introduce some key concepts behind the NoSQL family of databases

Objectives. Introduce some key concepts behind the NoSQL family of databases NoSQL Source: Pramod J. Sadalage and Martin Fowler NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence, Pearson Education, 2013 Objectives Introduce some key concepts behind the

More information

Applications for Big Data Analytics

Applications for Big Data Analytics Smarter Healthcare Applications for Big Data Analytics Multi-channel sales Finance Log Analysis Homeland Security Traffic Control Telecom Search Quality Manufacturing Trading Analytics Fraud and Risk Retail:

More information

Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores

Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores Composite Software October 2010 TABLE OF CONTENTS INTRODUCTION... 3 BUSINESS AND IT DRIVERS... 4 NOSQL DATA STORES LANDSCAPE...

More information

IN-MEMORY DATABASE SYSTEMS. Prof. Dr. Uta Störl Big Data Technologies: In-Memory DBMS - SoSe 2015 1

IN-MEMORY DATABASE SYSTEMS. Prof. Dr. Uta Störl Big Data Technologies: In-Memory DBMS - SoSe 2015 1 IN-MEMORY DATABASE SYSTEMS Prof. Dr. Uta Störl Big Data Technologies: In-Memory DBMS - SoSe 2015 1 Analytical Processing Today Separation of OLTP and OLAP Motivation Online Transaction Processing (OLTP)

More information

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing Evaluating NoSQL for Enterprise Applications Dirk Bartels VP Strategy & Marketing Agenda The Real Time Enterprise The Data Gold Rush Managing The Data Tsunami Analytics and Data Case Studies Where to go

More information

Lecture Data Warehouse Systems

Lecture Data Warehouse Systems Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART C: Novel Approaches in DW NoSQL and MapReduce Stonebraker on Data Warehouses Star and snowflake schemas are a good idea in the DW world C-Stores

More information

CENG 553 DATABASE. BIG DATA:NOSQL SYSTEMS PRESENTATION Okan Yaman/Eray Ölgün/Emin İnal

CENG 553 DATABASE. BIG DATA:NOSQL SYSTEMS PRESENTATION Okan Yaman/Eray Ölgün/Emin İnal CENG 553 DATABASE BIG DATA:NOSQL SYSTEMS PRESENTATION Okan Yaman/Eray Ölgün/Emin İnal CONTENTS INTRODUCTION TYPES OF DATABASES (ARCHITECTURES) 1. GRAPH DB 2. KEY VALUE DB 3. COLUMN BASED DB INTRODUCTION

More information

NoSQL Databases. Nikos Parlavantzas

NoSQL Databases. Nikos Parlavantzas !!!! NoSQL Databases Nikos Parlavantzas Lecture overview 2 Objective! Present the main concepts necessary for understanding NoSQL databases! Provide an overview of current NoSQL technologies Outline 3!

More information

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world Analytics March 2015 White paper Why NoSQL? Your database options in the new non-relational world 2 Why NoSQL? Contents 2 New types of apps are generating new types of data 2 A brief history of NoSQL 3

More information

Preparing Your Data For Cloud

Preparing Your Data For Cloud Preparing Your Data For Cloud Narinder Kumar Inphina Technologies 1 Agenda Relational DBMS's : Pros & Cons Non-Relational DBMS's : Pros & Cons Types of Non-Relational DBMS's Current Market State Applicability

More information

NoSQL Databases. Polyglot Persistence

NoSQL Databases. Polyglot Persistence The future is: NoSQL Databases Polyglot Persistence a note on the future of data storage in the enterprise, written primarily for those involved in the management of application development. Martin Fowler

More information

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre NoSQL systems: introduction and data models Riccardo Torlone Università Roma Tre Why NoSQL? In the last thirty years relational databases have been the default choice for serious data storage. An architect

More information

MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15

MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15 MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15 1 MongoDB in the NoSQL and SQL world. NoSQL What? Why? - How? Say goodbye to ACID, hello BASE You

More information

Big Data Management in the Clouds. Alexandru Costan IRISA / INSA Rennes (KerData team)

Big Data Management in the Clouds. Alexandru Costan IRISA / INSA Rennes (KerData team) Big Data Management in the Clouds Alexandru Costan IRISA / INSA Rennes (KerData team) Cumulo NumBio 2015, Aussois, June 4, 2015 After this talk Realize the potential: Data vs. Big Data Understand why we

More information

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1 Why NoSQL? Your database options in the new non- relational world 2015 IBM Cloudant 1 Table of Contents New types of apps are generating new types of data... 3 A brief history on NoSQL... 3 NoSQL s roots

More information

Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace

Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace Introduction to Polyglot Persistence Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace FOSSCOMM 2016 Background - 14 years in databases and system engineering - NoSQL DBA @ ObjectRocket

More information

Structured Data Storage

Structured Data Storage Structured Data Storage Xgen Congress Short Course 2010 Adam Kraut BioTeam Inc. Independent Consulting Shop: Vendor/technology agnostic Staffed by: Scientists forced to learn High Performance IT to conduct

More information

NOSQL INTRODUCTION WITH MONGODB AND RUBY GEOFF LANE @GEOFFLANE

NOSQL INTRODUCTION WITH MONGODB AND RUBY GEOFF LANE <GEOFF@ZORCHED.NET> @GEOFFLANE NOSQL INTRODUCTION WITH MONGODB AND RUBY GEOFF LANE @GEOFFLANE WHAT IS NOSQL? NON-RELATIONAL DATA STORAGE USUALLY SCHEMA-FREE ACCESS DATA WITHOUT SQL (THUS... NOSQL) WIDE-COLUMN / TABULAR

More information

CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3

CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3 INTRODUCTION xvii PART I: GETTING STARTED CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3 Definition and Introduction 4 Context and a Bit of History 4 Big Data 7 Scalability 9 Defi nition and Introduction

More information

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84 Index A Amazon Web Services (AWS), 50, 58 Analytics engine, 21 22 Apache Kafka, 38, 131 Apache S4, 38, 131 Apache Sqoop, 37, 131 Appliance pattern, 104 105 Application architecture, big data analytics

More information

Challenges for Data Driven Systems

Challenges for Data Driven Systems Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Quick History of Data Management 4000 B C Manual recording From tablets to papyrus to paper A. Payberah 2014 2

More information

Data Services Advisory

Data Services Advisory Data Services Advisory Modern Datastores An Introduction Created by: Strategy and Transformation Services Modified Date: 8/27/2014 Classification: DRAFT SAFE HARBOR STATEMENT This presentation contains

More information

Slave. Master. Research Scholar, Bharathiar University

Slave. Master. Research Scholar, Bharathiar University Volume 3, Issue 7, July 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper online at: www.ijarcsse.com Study on Basically, and Eventually

More information

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords From A to Z By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords Big data is one of the, well, biggest trends in IT today, and it has spawned a whole new generation

More information

extensible record stores document stores key-value stores Rick Cattel s clustering from Scalable SQL and NoSQL Data Stores SIGMOD Record, 2010

extensible record stores document stores key-value stores Rick Cattel s clustering from Scalable SQL and NoSQL Data Stores SIGMOD Record, 2010 System/ Scale to Primary Secondary Joins/ Integrity Language/ Data Year Paper 1000s Index Indexes Transactions Analytics Constraints Views Algebra model my label 1971 RDBMS O tables sql-like 2003 memcached

More information

Can the Elephants Handle the NoSQL Onslaught?

Can the Elephants Handle the NoSQL Onslaught? Can the Elephants Handle the NoSQL Onslaught? Avrilia Floratou, Nikhil Teletia David J. DeWitt, Jignesh M. Patel, Donghui Zhang University of Wisconsin-Madison Microsoft Jim Gray Systems Lab Presented

More information

BUSINESS INTELLIGENCE AND NOSQL DATABASES

BUSINESS INTELLIGENCE AND NOSQL DATABASES INFORMATION SYSTEMS IN MANAGEMENT Information Systems in Management (2012) Vol. 1 (1) 25 37 BUSINESS INTELLIGENCE AND NOSQL DATABASES JERZY DUDA Department of Applied Computer Science, Faculty of Management,

More information

NoSQL Data Base Basics

NoSQL Data Base Basics NoSQL Data Base Basics Course Notes in Transparency Format Cloud Computing MIRI (CLC-MIRI) UPC Master in Innovation & Research in Informatics Spring- 2013 Jordi Torres, UPC - BSC www.jorditorres.eu HDFS

More information

SQL VS. NO-SQL. Adapted Slides from Dr. Jennifer Widom from Stanford

SQL VS. NO-SQL. Adapted Slides from Dr. Jennifer Widom from Stanford SQL VS. NO-SQL Adapted Slides from Dr. Jennifer Widom from Stanford 55 Traditional Databases SQL = Traditional relational DBMS Hugely popular among data analysts Widely adopted for transaction systems

More information

A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA

A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA Ompal Singh Assistant Professor, Computer Science & Engineering, Sharda University, (India) ABSTRACT In the new era of distributed system where

More information

NewSQL. Andy Pavlo February 6, 2012

NewSQL. Andy Pavlo February 6, 2012 NewSQL Andy Pavlo February 6, 2012 Outline The Last Decade of Databases NewSQL Introduction H-Store Early-2000s All the big players were heavyweight and expensive. Oracle, DB2, Sybase, SQL Server, etc.

More information

NoSQL Systems for Big Data Management

NoSQL Systems for Big Data Management NoSQL Systems for Big Data Management Venkat N Gudivada East Carolina University Greenville, North Carolina USA Venkat Gudivada NoSQL Systems for Big Data Management 1/28 Outline 1 An Overview of NoSQL

More information

How graph databases started the multi-model revolution

How graph databases started the multi-model revolution How graph databases started the multi-model revolution Luca Garulli Author and CEO @OrientDB QCon Sao Paulo - March 26, 2015 Welcome to Big Data 90% of the data in the world today has been created in the

More information

Enterprise Operational SQL on Hadoop Trafodion Overview

Enterprise Operational SQL on Hadoop Trafodion Overview Enterprise Operational SQL on Hadoop Trafodion Overview Rohit Jain Distinguished & Chief Technologist Strategic & Emerging Technologies Enterprise Database Solutions Copyright 2012 Hewlett-Packard Development

More information

Introduction to NOSQL

Introduction to NOSQL Introduction to NOSQL Université Paris-Est Marne la Vallée, LIGM UMR CNRS 8049, France January 31, 2014 Motivations NOSQL stands for Not Only SQL Motivations Exponential growth of data set size (161Eo

More information

X4-2 Exadata announced (well actually around Jan 1) OEM/Grid control 12c R4 just released

X4-2 Exadata announced (well actually around Jan 1) OEM/Grid control 12c R4 just released General announcements In-Memory is available next month http://www.oracle.com/us/corporate/events/dbim/index.html X4-2 Exadata announced (well actually around Jan 1) OEM/Grid control 12c R4 just released

More information

Open source large scale distributed data management with Google s MapReduce and Bigtable

Open source large scale distributed data management with Google s MapReduce and Bigtable Open source large scale distributed data management with Google s MapReduce and Bigtable Ioannis Konstantinou Email: ikons@cslab.ece.ntua.gr Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory

More information

NoSQL for SQL Professionals William McKnight

NoSQL for SQL Professionals William McKnight NoSQL for SQL Professionals William McKnight Session Code BD03 About your Speaker, William McKnight President, McKnight Consulting Group Frequent keynote speaker and trainer internationally Consulted to

More information

Ultra-Scalable Real-Time Big Data Analytics. Ricardo Jimenez-Peris Co-Founder

Ultra-Scalable Real-Time Big Data Analytics. Ricardo Jimenez-Peris Co-Founder Ultra-Scalable Real-Time Big Data Analytics Ricardo Jimenez-Peris Co-Founder Major Pains/Gaps in Data Management 1. Lack of scalability of traditional & cloud DBs. 2. Lack of real-time analytics support.

More information

Cloud & Big Data a perfect marriage? Patrick Valduriez

Cloud & Big Data a perfect marriage? Patrick Valduriez Cloud & Big Data a perfect marriage? Patrick Valduriez Cloud & Big Data: the hype! 2 Cloud & Big Data: the hype! 3 Behind the Hype? Every one who wants to make big money Intel, IBM, Microsoft, Oracle,

More information

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research &

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & Innovation 04-08-2011 to the EC 8 th February, Luxembourg Your Atos business Research technologists. and Innovation

More information

Comparing SQL and NOSQL databases

Comparing SQL and NOSQL databases COSC 6397 Big Data Analytics Data Formats (II) HBase Edgar Gabriel Spring 2015 Comparing SQL and NOSQL databases Types Development History Data Storage Model SQL One type (SQL database) with minor variations

More information

A Selection Method of Database System in Bigdata Environment: A Case Study From Smart Education Service in Korea

A Selection Method of Database System in Bigdata Environment: A Case Study From Smart Education Service in Korea Int. J. Advance Soft Compu. Appl, Vol. 7, No. 1, March 2015 ISSN 2074-8523 A Selection Method of Database System in Bigdata Environment: A Case Study From Smart Education Service in Korea Jong Sung Hwang

More information

Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014. Apologies to Martin Fowler ( NoSQL Distilled )

Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014. Apologies to Martin Fowler ( NoSQL Distilled ) Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014 Rise of Web and cluster-based computing NoSQL Movement Relationships vs. Aggregates Key-value store XML or JSON

More information

ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA

ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA David Vanderfeesten, Bell Labs Belgium ANNO 2012 YOUR DATA IS MONEY BIG MONEY! Your click stream, your activity stream, your electricity consumption, your call

More information

Elastic NoSQL databases over the Cloud

Elastic NoSQL databases over the Cloud Elastic NoSQL databases over the Cloud I. Konstantinou, E. Angelou, C. Boumpouka, D. Tsoumakos, N. Koziris Computing Systems Laboratory School of Electrical and Computer Engineering National Technical

More information

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 NoSQL Databases Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 Database Landscape Source: H. Lim, Y. Han, and S. Babu, How to Fit when No One Size Fits., in CIDR,

More information

Application-specific databases have always been with us...

Application-specific databases have always been with us... Databases : Lecture 11 : Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2014 Rise of Web and cluster-based computing NoSQL Movement Relationships vs. Aggregates Key-value store XML or JSON

More information

The evolution of database technology (II) Huibert Aalbers Senior Certified Executive IT Architect

The evolution of database technology (II) Huibert Aalbers Senior Certified Executive IT Architect The evolution of database technology (II) Huibert Aalbers Senior Certified Executive IT Architect IT Insight podcast This podcast belongs to the IT Insight series You can subscribe to the podcast through

More information

Databases 2 (VU) (707.030)

Databases 2 (VU) (707.030) Databases 2 (VU) (707.030) Introduction to NoSQL Denis Helic KMI, TU Graz Oct 14, 2013 Denis Helic (KMI, TU Graz) NoSQL Oct 14, 2013 1 / 37 Outline 1 NoSQL Motivation 2 NoSQL Systems 3 NoSQL Examples 4

More information

Dr. Chuck Cartledge. 27 Aug. 2015

Dr. Chuck Cartledge. 27 Aug. 2015 CS-695 NoSQL Database Polyglot Persistence; Or, The Many Ways We Store Data Dr. Chuck Cartledge 27 Aug. 2015 1/37 Table of contents I 4 CRUDy stuff 1 A little history 2 A change in the air 3 Database layouts

More information

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK OVERVIEW ON BIG DATA SYSTEMATIC TOOLS MR. SACHIN D. CHAVHAN 1, PROF. S. A. BHURA

More information

REAL-TIME BIG DATA ANALYTICS

REAL-TIME BIG DATA ANALYTICS www.leanxcale.com info@leanxcale.com REAL-TIME BIG DATA ANALYTICS Blending Transactional and Analytical Processing Delivers Real-Time Big Data Analytics 2 ULTRA-SCALABLE FULL ACID FULL SQL DATABASE LeanXcale

More information

The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg. Adam Marcus MIT CSAIL marcua@csail.mit.edu / @marcua

The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg. Adam Marcus MIT CSAIL marcua@csail.mit.edu / @marcua The NoSQL Ecosystem, Relaxed Consistency, and Snoop Dogg Adam Marcus MIT CSAIL marcua@csail.mit.edu / @marcua About Me Social Computing + Database Systems Easily Distracted: Wrote The NoSQL Ecosystem in

More information

Journal of Cloud Computing: Advances, Systems and Applications

Journal of Cloud Computing: Advances, Systems and Applications Journal of Cloud Computing: Advances, Systems and Applications This Provisional PDF corresponds to the article as it appeared upon acceptance. Fully formatted PDF and full text (HTML) versions will be

More information

A REVIEW ON DOCUMENT ORIENTED AND COLUMN ORIENTED DATABASES

A REVIEW ON DOCUMENT ORIENTED AND COLUMN ORIENTED DATABASES A REVIEW ON DOCUMENT ORIENTED AND COLUMN ORIENTED DATABASES Jaspreet kaur Harpreet Kaur Kamaljeet kaur Student of MTech (CSE) Student of MTech (CSE) Assit. Prof. in CSE dept. Sri Guru Granth Sahib World

More information

NOSQL, BIG DATA AND GRAPHS. Technology Choices for Today s Mission- Critical Applications

NOSQL, BIG DATA AND GRAPHS. Technology Choices for Today s Mission- Critical Applications NOSQL, BIG DATA AND GRAPHS Technology Choices for Today s Mission- Critical Applications 2 NOSQL, BIG DATA AND GRAPHS NOSQL, BIG DATA AND GRAPHS TECHNOLOGY CHOICES FOR TODAY S MISSION- CRITICAL APPLICATIONS

More information

TRAINING PROGRAM ON BIGDATA/HADOOP

TRAINING PROGRAM ON BIGDATA/HADOOP Course: Training on Bigdata/Hadoop with Hands-on Course Duration / Dates / Time: 4 Days / 24th - 27th June 2015 / 9:30-17:30 Hrs Venue: Eagle Photonics Pvt Ltd First Floor, Plot No 31, Sector 19C, Vashi,

More information

Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY MANNING ANN KELLY. Shelter Island

Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY MANNING ANN KELLY. Shelter Island Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY ANN KELLY II MANNING Shelter Island contents foreword preface xvii xix acknowledgments xxi about this book xxii Part 1 Introduction

More information

Firebird meets NoSQL (Apache HBase) Case Study

Firebird meets NoSQL (Apache HBase) Case Study Firebird meets NoSQL (Apache HBase) Case Study Firebird Conference 2011 Luxembourg 25.11.2011 26.11.2011 Thomas Steinmaurer DI +43 7236 3343 896 thomas.steinmaurer@scch.at www.scch.at Michael Zwick DI

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

Performance investigation of selected SQL and NoSQL databases

Performance investigation of selected SQL and NoSQL databases Performance investigation of selected SQL and NoSQL databases Stephan Schmid stephan.schmid@unibw.de Eszter Galicz eszter.galicz@unibw.de Wolfgang Reinhardt wolfgang.reinhardt@unibw.de Abstract In the

More information

Big Data Management. Big Data Management. (BDM) Autumn 2013. Povl Koch September 30, 2013 29-09-2013 1

Big Data Management. Big Data Management. (BDM) Autumn 2013. Povl Koch September 30, 2013 29-09-2013 1 Big Data Management Big Data Management (BDM) Autumn 2013 Povl Koch September 30, 2013 29-09-2013 1 Overview Today s program 1. Little more practical details about this course 2. Recap from last time 3.

More information

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 美 國 13 歲 學 生 用 Big Data 找 出 霸 淩 熱 點 Puri 架 設 網 站 Bullyvention, 藉 由 分 析 Twitter 上 找 出 提 到 跟 霸 凌 相 關 的 詞, 搭 配 地 理 位 置

More information

Making Sense of NoSQL Dan McCreary Wednesday, Nov. 13 th 2014

Making Sense of NoSQL Dan McCreary Wednesday, Nov. 13 th 2014 Making Sense of NoSQL Dan McCreary Wednesday, Nov. 13 th 2014 Agenda Why NoSQL? What are the key NoSQL architectures? How are they different from traditional RDBMS Systems? What types of problems do they

More information

Big Data Analytics - Accelerated. stream-horizon.com

Big Data Analytics - Accelerated. stream-horizon.com Big Data Analytics - Accelerated stream-horizon.com Legacy ETL platforms & conventional Data Integration approach Unable to meet latency & data throughput demands of Big Data integration challenges Based

More information

Big Data Architectures. Tom Cahill, Vice President Worldwide Channels, Jaspersoft

Big Data Architectures. Tom Cahill, Vice President Worldwide Channels, Jaspersoft Big Data Architectures Tom Cahill, Vice President Worldwide Channels, Jaspersoft Jaspersoft + Big Data = Fast Insights Success in the Big Data era is more than about size. It s about getting insight from

More information

Olivier Caudron. Big Data and NoSQL

Olivier Caudron. Big Data and NoSQL Olivier Caudron Big Data and NoSQL "Big" Data? "Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional

More information

BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS

BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS WHAT IS BIG DATA? describes any voluminous amount of structured, semi-structured and unstructured data that has the potential to be mined for information

More information

Advanced Data Management Technologies

Advanced Data Management Technologies ADMT 2014/15 Unit 15 J. Gamper 1/44 Advanced Data Management Technologies Unit 15 Introduction to NoSQL J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE ADMT 2014/15 Unit 15

More information

NoSQL and Graph Database

NoSQL and Graph Database NoSQL and Graph Database Biswanath Dutta DRTC, Indian Statistical Institute 8th Mile Mysore Road R. V. College Post Bangalore 560059 International Conference on Big Data, Bangalore, 9-20 March 2015 Outlines

More information

NoSQL Database Systems and their Security Challenges

NoSQL Database Systems and their Security Challenges NoSQL Database Systems and their Security Challenges Morteza Amini amini@sharif.edu Data & Network Security Lab (DNSL) Department of Computer Engineering Sharif University of Technology September 25 2

More information

A Study of NoSQL and NewSQL databases for data aggregation on Big Data

A Study of NoSQL and NewSQL databases for data aggregation on Big Data A Study of NoSQL and NewSQL databases for data aggregation on Big Data ANANDA SENTRAYA PERUMAL MURUGAN Master s Degree Project Stockholm, Sweden 2013 TRITA-ICT-EX-2013:256 A Study of NoSQL and NewSQL

More information

SQL, NoSQL, and Next Generation DBMSs. Shahram Ghandeharizadeh Director of the USC Database Lab

SQL, NoSQL, and Next Generation DBMSs. Shahram Ghandeharizadeh Director of the USC Database Lab SQL, NoSQL, and Next Generation DBMSs Shahram Ghandeharizadeh Director of the USC Database Lab Outline A brief history of DBMSs. OSs SQL NoSQL 1960/70 1980+ 2000+ Before Computers Database DBMS/Data Store

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

Big Data Technologies Compared June 2014

Big Data Technologies Compared June 2014 Big Data Technologies Compared June 2014 Agenda What is Big Data Big Data Technology Comparison Summary Other Big Data Technologies Questions 2 What is Big Data by Example The SKA Telescope is a new development

More information

Hadoop vs Apache Spark

Hadoop vs Apache Spark Innovate, Integrate, Transform Hadoop vs Apache Spark www.altencalsoftlabs.com Introduction Any sufficiently advanced technology is indistinguishable from magic. said Arthur C. Clark. Big data technologies

More information

In-Memory Columnar Databases HyPer. Arto Kärki University of Helsinki 30.11.2012

In-Memory Columnar Databases HyPer. Arto Kärki University of Helsinki 30.11.2012 In-Memory Columnar Databases HyPer Arto Kärki University of Helsinki 30.11.2012 1 Introduction Columnar Databases Design Choices Data Clustering and Compression Conclusion 2 Introduction The relational

More information

The Quest for Extreme Scalability

The Quest for Extreme Scalability The Quest for Extreme Scalability In times of a growing audience, very successful internet applications have all been facing the same database issue: while web servers can be multiplied without too many

More information

Sentimental Analysis using Hadoop Phase 2: Week 2

Sentimental Analysis using Hadoop Phase 2: Week 2 Sentimental Analysis using Hadoop Phase 2: Week 2 MARKET / INDUSTRY, FUTURE SCOPE BY ANKUR UPRIT The key value type basically, uses a hash table in which there exists a unique key and a pointer to a particular

More information

NoSQL storage and management of geospatial data with emphasis on serving geospatial data using standard geospatial web services

NoSQL storage and management of geospatial data with emphasis on serving geospatial data using standard geospatial web services NoSQL storage and management of geospatial data with emphasis on serving geospatial data using standard geospatial web services Pouria Amirian, Adam Winstanley, Anahid Basiri Department of Computer Science,

More information

Cloud Database Emergence

Cloud Database Emergence Abstract RDBMS technology is favorable in software based organizations for more than three decades. The corporate organizations had been transformed over the years with respect to adoption of information

More information

NewSQL: Towards Next-Generation Scalable RDBMS for Online Transaction Processing (OLTP) for Big Data Management

NewSQL: Towards Next-Generation Scalable RDBMS for Online Transaction Processing (OLTP) for Big Data Management NewSQL: Towards Next-Generation Scalable RDBMS for Online Transaction Processing (OLTP) for Big Data Management A B M Moniruzzaman Department of Computer Science and Engineering, Daffodil International

More information

Big Data. Facebook Wall Data using Graph API. Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715

Big Data. Facebook Wall Data using Graph API. Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715 Big Data Facebook Wall Data using Graph API Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715 Outline Data Source Processing tools for processing our data Big Data Processing System: Mongodb

More information

Introduction to NoSQL

Introduction to NoSQL Introduction to NoSQL NoSQL Seminar 2012 @ TUT Arto Salminen What is NoSQL? Class of database management systems (DBMS) "Not only SQL" Does not use SQL as querying language Distributed, fault-tolerant

More information

BIG DATA TOOLS. Top 10 open source technologies for Big Data

BIG DATA TOOLS. Top 10 open source technologies for Big Data BIG DATA TOOLS Top 10 open source technologies for Big Data We are in an ever expanding marketplace!!! With shorter product lifecycles, evolving customer behavior and an economy that travels at the speed

More information

TECHNOLOGY TRANSFER PRESENTS RICK VAN DER LANS APRIL 18 2016 APRIL 19-20 2016 RESIDENZA DI RIPETTA - VIA DI RIPETTA, 231 ROMA

TECHNOLOGY TRANSFER PRESENTS RICK VAN DER LANS APRIL 18 2016 APRIL 19-20 2016 RESIDENZA DI RIPETTA - VIA DI RIPETTA, 231 ROMA TECHNOLOGY TRANSFER PRESENTS RICK VAN DER LANS The Logical Data Warehouse Design, Architecture, and Technology Incorporating Big Data, Hadoop and NoSQL in Data Warehouse and Business Intelligence Systems

More information

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach Nic Caine NoSQL Matters, April 2013 Overview The Problem Current Big Data Analytics Relationship Analytics Leveraging

More information

Data Modeling for NoSQL Document-Oriented Databases

Data Modeling for NoSQL Document-Oriented Databases Data Modeling for NoSQL Document-Oriented Databases Harley Vera, Wagner Boaventura, Maristela Holanda, Valeria Guimarães, Fernanda Hondo Department of Computer Science University of Brasília Brasília,

More information

Big Data Analytics. Rasoul Karimi

Big Data Analytics. Rasoul Karimi Big Data Analytics Rasoul Karimi Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany Big Data Analytics Big Data Analytics 1 / 1 Introduction

More information

NOSQL DATABASE SYSTEMS

NOSQL DATABASE SYSTEMS NOSQL DATABASE SYSTEMS Big Data Technologies: NoSQL DBMS - SoSe 2015 1 Categorization NoSQL Data Model Storage Layout Query Models Solution Architectures NoSQL Database Systems Data Modeling id ti Application

More information

Scalable Architecture on Amazon AWS Cloud

Scalable Architecture on Amazon AWS Cloud Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect

More information

MongoDB. An introduction and performance analysis. Seminar Thesis

MongoDB. An introduction and performance analysis. Seminar Thesis MongoDB An introduction and performance analysis Seminar Thesis Master of Science in Engineering Major Software and Systems HSR Hochschule für Technik Rapperswil www.hsr.ch/mse Advisor: Author: Prof. Stefan

More information

Infrastructures for big data

Infrastructures for big data Infrastructures for big data Rasmus Pagh 1 Today s lecture Three technologies for handling big data: MapReduce (Hadoop) BigTable (and descendants) Data stream algorithms Alternatives to (some uses of)

More information

Big Data Analytics: Hadoop-Map Reduce & NoSQL Databases

Big Data Analytics: Hadoop-Map Reduce & NoSQL Databases Big Data Analytics: Hadoop-Map Reduce & NoSQL Databases Abinav Pothuganti Computer Science and Engineering, CBIT,Hyderabad, Telangana, India Abstract Today, we are surrounded by data like oxygen. The exponential

More information

.NET User Group Bern

.NET User Group Bern .NET User Group Bern Roger Rudin bbv Software Services AG roger.rudin@bbv.ch Agenda What is NoSQL Understanding the Motivation behind NoSQL MongoDB: A Document Oriented Database NoSQL Use Cases What is

More information