NoSQL Database Systems and their Security Challenges

Size: px
Start display at page:

Download "NoSQL Database Systems and their Security Challenges"

Transcription

1 NoSQL Database Systems and their Security Challenges Morteza Amini Data & Network Security Lab (DNSL) Department of Computer Engineering Sharif University of Technology September 25 2 th ISC International ISCISC 25 Conference on Information NoSQL Database Security Systems and Cryptology and their Security (ISCISC 5) Challenges

2 Talk Outline Introduction NoSQL vs. Relational Databases Types of NoSQL Databases NoSQL Security Challenges 2 / 59 ISCISC 25

3 Introduction 3 / 59 ISCISC 25

4 Current Trends The new generation of applications like cloud or Grid apps, Business Intelligence, Web 2., Social networking requires storing and processing of terabytes and even petabytes of data 4 / 59 ISCISC 25

5 ISCISC 25 We have More users More data Interactive apps Today 5 / 59

6 ISCISC 25 The requirements of storage database systems is changed Today Relational Database is not suitable Distributed Storage and Processing NoSQL = Not Only SQL 6 / 59

7 NoSQL vs. Relational Databases 7 / 59 ISCISC 25

8 Why relational database is not suitable? A relational database is a data structure that allows you to link information from different tables 8 / 59 ISCISC 25

9 Why relational database is not suitable? Pros Have been well-developed to meet confidentiality, availability and integrity Work best with structured data Use standard query language ACID Very good for small dataset 9 / 59 ISCISC 25

10 Why relational database is not suitable? Cons Scaling Relied on scale up rather than scale out Large feature set Non-linear query execution time Static schema / 59 ISCISC 25

11 Reasons for Distributed Storage and Processing Take advantage of multiple systems as well as multi-core CPU architectures Servers have to be globally distributed for low latency and failover / 59 ISCISC 25

12 Characteristics of NoSQL Databases NoSQL databases have been designed for solving the Big Data issue by utilizing distributed, collaborating hosts to achieve satisfactory performance in data storage and retrieval. Mostly being non-relational No join / Unstructured data Provide great performance, availability, scalability and flexibility Distribution, Replication, Failover 2 / 59 ISCISC 25

13 NoSQL Trend The term NoSQL was first used in 998. Relational databases As name of file-based database that omitted the use of SQL. The term was picked up again in 29. NoSQL databases It became a serious competitor to the term RDB. Google Trend 3 / 59 ISCISC 25

14 Characteristics of NoSQL Databases Provide BASE (Basically Available, Soft state, Eventual consistent) system, but not ACID as a Relational Database Management System. Schema-free Easy replication support and running well on clusters 4 / 59 Simple API ISCISC 25

15 CAP Theorem Any shared-data system can have at most two of these properties AP Voldemart (Key-value) CouchDB (Document), Riak(Document) CA Relational databases Vertica (column-oriented) GreenPlum (Relational) CP BigTable (Column Oriented), MongoDB(Document) All nodes always have the same view of the data. Consistency Partition Tolerance Every request (read and write) receives response. Availability The system works well despite physical network partitions. 5 / 59 ISCISC 25

16 Types of NoSQL Databases 6 / 59 ISCISC 25

17 NoSQL Data Models There are more than 5 NoSQL databases 7 / 59 ISCISC 25

18 Major Companies using NoSQL Databases Fidelis Cybersecurity, 24 8 / 59 ISCISC 25

19 NoSQL Data Models Graph Key-value Data Model document Columwide 9 / 59 ISCISC 25

20 Key-value Stores Work by matching keys with values, similar to a dictionary very fast very scalable simple model able to distribute horizontally Cons: many data structures (objects) can't be easily modeled key value pairs 2 / 59 ISCISC 25

21 Key-value Stores 2 / 59 ISCISC 25

22 NoSQL Data Models Graph Key-value Data Model document Columwide 22 / 59 ISCISC 25

23 Column-Wide Stores Similar to relational database but not alike. Each KEY is associated with one or more columns (Row key) A Column stores their data in such way that can be efficiently aggregated, reducing the I/O activity. it is suitable for data mining and analytic systems / 59 ISCISC 25

24 NoSQL Data Models Graph Key-value Data Model document Columwide 24 / 59 ISCISC 25

25 Document Stores The data is stored in the form of documents in a standard format (xml,pdf, json, etc). More flexible because of their lack of schema The documents may have only the filled and important fields, letting the empty and null out, saving some storage space 25 / 59 ISCISC 25

26 NoSQL Data Models Graph Key-value Data model document Columwide 26 / 59 ISCISC 25

27 Graph Stores Save the data in a form of graph The nodes are object and the edges relations between the objects. The main goal of graph store is to know the relations between nodes and how the nodes are inter-connected / 59 ISCISC 25

28 Which one is the best? It depends on the application requirements Size of data Complexity CAP theory Format of data Fidelis Cybersecurity, / 59 ISCISC 25

29 NoSQL Security Challenges 29 / 59 ISCISC 25

30 NoSQL Security Most of NoSQL databases do not provide any feature of embedding security in the database itself. Developers need to impose security in the middleware. Security issues that affected RDBMSs were also inherited in the NoSQL databases as well as new ones imposed by their new features. 3 / 59 ISCISC 25

31 NoSQL Security Security may be difficult Owing to the unstructured (dynamic) nature of the data stored in these databases Distributed environment Cost of security in contrast to prformance No strong consistency 3 / 59 ISCISC 25

32 NoSQL Major Security Challenges Threats Posed By Distributed Environments Authorization and Access Control Safeguarding Integrity Protection of Data at Rest User Data Privacy 32 / 59 ISCISC 25

33 NoSQL Major Security Challenges Threats Posed By Distributed Environments Authorization and Access Control Safeguarding Integrity Protection of Data at Rest User Data Privacy 33 / 59 ISCISC 25

34 Threats Posed By Distributed Environments Distributed Environments increase attack surface across several distributed nodes Compromised Clients Malicious data gets propagated from a single compromised location Protecting nodes, name servers and those clients becomes difficult especially when there is no central management security point. Vulnerabilities of Gossip based membership protocol in Cassandra and Dynamo [Aniello, et al. 23] Zombie node Ghost node 34 / 59 ISCISC 25

35 NoSQL Major Security Challenges Threats Posed By Distributed Environments Authorization and Access Control Safeguarding Integrity Protection of Data at Rest User Data Privacy 35 / 59 ISCISC 25

36 Authorization and Access Control Two important challenges: Possibility: how to define security policies for schema-less or dynamic-schema databases? Performance: availability vs. access control overhead: how to manage cost of access control? 36 / 59 ISCISC 25

37 Authorization and Access Control Fine-grained (row or column level) access control: heterogeneous data is stored together in one database as opposed to relational models which conform to defined schemas and tables that store only related data. Schema-less nature of NoSQL DBs does not allow finegrained access control. We need Looking Forward Security Most of them allow Column Family level authorization. NoSQL DBMS Granularity Explanation BigTable Column Family Using ACL Cassandra Column Family Using IAuthorizer API HBase Column Family / Cell Group-based authorization Accumulo Cell Using Visibility field 37 / 59 ISCISC 25

38 Authorization and Access Control Fine-grained (row or column level) access control: heterogeneous data is stored together Secret in one database as opposed to relational models which conform to defined schemas Unclassified and tables that store only related data. Schema-less nature of NoSQL DBs does not Confidential allow finegrained access control. We need Looking Forward Security Most of them allow Column Family level authorization. Using top-level ontology for access policy specification Confidential NoSQL DBMS Granularity Explanation Unclassified BigTable Column Family Using ACL Cassandra Column Family Secret Using IAuthorizer API HBase Column Family / Cell Group-based authorization Accumulo Cell Using Visibility field 38 / 59 ISCISC 25

39 Authorization and Access Control Grouping data with the same security level (node level). Fine-grained (row or column level) access control: heterogeneous data is stored together Secret in one database as opposed to relational models which conform to defined schemas Unclassified and tables that store only related data. Schema-less nature of NoSQL DBs does not allow finegrained access control. We need Looking Forward Confidential Security Most of them allow Column Family level authorization. Confidential NoSQL DBMS Granularity Explanation BigTable Column Family Using ACL Unclassified Cassandra Column Family Using IAuthorizer API HBase Column Family / Cell Group-based Secret authorization Accumulo Cell Using Visibility field 39 / 59 ISCISC 25

40 Authorization and Access Control Inference Control: Access control on aggregated data, especially in Column-Wide databases and Time-Series databases.,22 4 / 59 ISCISC 25

41 Authorization and Access Control Administration / Access Control Management: how and where to grant database accesses Local vs. Global access policies and their possible conflicts. Centralized approach: single-point-of-failure, availability issues Distributed approach: consistency of distributed access rules Semidistributed approach: 4 / 59 ISCISC 25

42 Authorization and Access Control By default, there is no authorization. Privileged admins can grant the privileges on resources to a selected user. By default, there is no authorization. Provisions authorization on a per- database level by using a role- based approach. 42 / 59 ISCISC 25

43 NoSQL Major Security Challenges Threats Posed By Distributed Environments Authorization and Access Control Safeguarding Integrity Protection of Data at Rest User Data Privacy 43 / 59 ISCISC 25

44 Safeguarding Integrity Enforcing integrity constraints is much harder in NoSQL database system Consistency is in contrast with availability and performance Transactional integrity is in contrast with its soft nature How to define integrity constraints? [its schema-less nature] Which types of integrity constraints can be defined? How to control? [there is absence of central control/ performance and availability issues] 44 / 59 ISCISC 25

45 NoSQL Major Security Challenges Threats Posed By Distributed Environments Authorization and Access Control Safeguarding Integrity Protection of Data at Rest User Data Privacy 45 / 59 ISCISC 25

46 Protection of Data at Rest Encryption is widely regarded as the defacto standard for safeguarding data in storage. Most industry solutions offering encryption services lack horizontal scaling and transparency required in the NoSQL environment. Only a few categories of NoSQL databases provide mechanisms to protect data at rest by employing encryption techniques. We need Light Weight Cryptography! 46 / 59 ISCISC 25

47 Protection of Data at Rest Use Transparent Data Encryption (TDE) to protect data that is written to disk. The commit log is not encrypted at all. Data files in MongoDB are never encrypted. 47 / 59 ISCISC 25

48 NoSQL Major Security Challenges Threats Posed By Distributed Environments Authorization and Access Control Safeguarding Integrity Protection of Data at Rest User Data Privacy 48 / 59 ISCISC 25

49 Users Data Privacy Privacy, main challenge of Web 2. and Virtual Social Networks. Large amounts of user- related sensitive information in NoSQL databases. Which kinds of methods is applicable in practice for NoSQL databases? Access Control Encryption Anonymization / 59 ISCISC 25

50 NoSQL Minor Security Challenges Authentication (Users and Clients) Audit And Logging Protection of Data at Motion API Security 5 / 59 ISCISC 25

51 Authentication Some NoSQL databases enforce authentication By default, there is no authentication. mechanism at local node level, but fail to enforce Has Password Authenticator. authentication across all commodity servers. Can further provide Kerberos authentication. By default, there is no authentication. Support for authentication on a per- database level. 5 / 59 ISCISC 25

52 Audit and Logging NoSQL databases has poor logging and log analysis methods Auditing is available in Enterprise Cassandra. Filters are available for logging MongoDB is far behind in implementing the desired security logging and monitoring. 52 / 59 ISCISC 25

53 Protection of Data in Motion Communication between clients and nodes (traditional issue) Communication between nodes RPC over TCP/IP 53 / 59 ISCISC 25

54 Protection of Data in Motion Communications should be encrypted Client-Node Communications: By default, is not encrypted. SSL can be configured. Client-Node Communications Inter-Node Communications: By default, is not encrypted. Inter-Node SSL can Communications be configured. Client-Node Communications: it is required to either recompile MongoDB with the "- ssl" option. Inter-Node Communications: is not supported. 54 / 59 ISCISC 25

55 API Security APIs can be subjected to several attacks such as Code injection, buffer over flows, command injection as they access the NoSQL databases. Server Side JavaScript Injection (SSJS) Schema injection / Query injection / JSON injection In PHP: $query = 'function() {var search_year = \''.$_GET['year']. '\';'. 'return this.publicationyear == search_year '. ' this.filmingyear == search_year '. ' this.recordingyear == search_year;}'; $cursor = $collection->find(array('$where' => $query)); DoS Attacks 55 / 59 ISCISC 25

56 API Security Is vulnerable to injection MongoDB, $where operator can be used for injection. 56 / 59 ISCISC 25

57 Summary NoSQL Database Systems for unstructured and big data Main features: Performance, Availability, Scalability NoSQL Security Challenges: Threats posed by their distributed nature Fine-grained authorization and inference control Integrity constraint definition and control Light weight transparent encryption of data in rest Users privacy / 59 ISCISC 25

58 Some References [Aniello, et al. 23] L. Aniello, S. Bonomi, M. Breno, R. Baldoni, Assessing Data Availability of Cassandra in the Presence of non-accurate Membership, The 2nd International Workshop on Dependability Issues in Cloud Computing, 23. [Kadebu, et al. 24] P. Kadebu, I. Mapanga, A Security Requirements Perspective towards a Secured NOSQL Database Environment, International Conference of Advance Research and Innovation, 24. [Noiumkar, et al. 24] P. Noiumkar, and T. Chomsiri, A Comparison the Level of Security on Top 5 Open Source NoSQL Databases, The 9th International Conference on Information Technology and Applications, 24. [Fidelis Cybersecurity 24] Current Data Security Issues of NoSQL Databases, Fidelis Cybersecurity, 24. [Okman, et al. 2] L. Okman, N. Gal-Oz, Y. Gonen, E. Gudes, and J. Abramov, Security Issues in NoSQL Databases, International Joint Conference of IEEE TrustCom-/IEEE ICESS-/FCST-, 2. [Shermin 23] M. Shermin, An Access Control Model for NoSQL Databases, M.Sc. thesis, The University of Western Ontario, 23. [Ron, et al. 25] A. Ron, A. Shulman-Peleg, E. Bronshtein, No SQL, No Injection? Examining NoSQL Security, The 9th Workshop on Web 2. Security and Privacy, 25. [Rong, et al. 23] C. Rong, Z. Quan, A. Chakravorty, On Access Control Schemes for Hadoop Data Storage, International Conference on Cloud Computing and Big Data, / 59 ISCISC 25

59 Thanks for your attention... Any Question? Thank Ms Dolatnezhad for helping in preparing this presentation. 59 / 59 ISCISC 25

Current Data Security Issues of NoSQL Databases

Current Data Security Issues of NoSQL Databases 1 Current Data Security Issues of NoSQL Databases January 2014 PAGE 1 PAGE 1 1 Fidelis Cybersecurity 1601 Trapelo Road, Suite 270 Waltham, MA 02451 Abstract NoSQL databases, sometimes referred as Not--

More information

SQL VS. NO-SQL. Adapted Slides from Dr. Jennifer Widom from Stanford

SQL VS. NO-SQL. Adapted Slides from Dr. Jennifer Widom from Stanford SQL VS. NO-SQL Adapted Slides from Dr. Jennifer Widom from Stanford 55 Traditional Databases SQL = Traditional relational DBMS Hugely popular among data analysts Widely adopted for transaction systems

More information

Preparing Your Data For Cloud

Preparing Your Data For Cloud Preparing Your Data For Cloud Narinder Kumar Inphina Technologies 1 Agenda Relational DBMS's : Pros & Cons Non-Relational DBMS's : Pros & Cons Types of Non-Relational DBMS's Current Market State Applicability

More information

Cloud Scale Distributed Data Storage. Jürmo Mehine

Cloud Scale Distributed Data Storage. Jürmo Mehine Cloud Scale Distributed Data Storage Jürmo Mehine 2014 Outline Background Relational model Database scaling Keys, values and aggregates The NoSQL landscape Non-relational data models Key-value Document-oriented

More information

Structured Data Storage

Structured Data Storage Structured Data Storage Xgen Congress Short Course 2010 Adam Kraut BioTeam Inc. Independent Consulting Shop: Vendor/technology agnostic Staffed by: Scientists forced to learn High Performance IT to conduct

More information

Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY MANNING ANN KELLY. Shelter Island

Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY MANNING ANN KELLY. Shelter Island Making Sense ofnosql A GUIDE FOR MANAGERS AND THE REST OF US DAN MCCREARY ANN KELLY II MANNING Shelter Island contents foreword preface xvii xix acknowledgments xxi about this book xxii Part 1 Introduction

More information

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre NoSQL systems: introduction and data models Riccardo Torlone Università Roma Tre Why NoSQL? In the last thirty years relational databases have been the default choice for serious data storage. An architect

More information

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1 Why NoSQL? Your database options in the new non- relational world 2015 IBM Cloudant 1 Table of Contents New types of apps are generating new types of data... 3 A brief history on NoSQL... 3 NoSQL s roots

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

Lecture Data Warehouse Systems

Lecture Data Warehouse Systems Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART C: Novel Approaches in DW NoSQL and MapReduce Stonebraker on Data Warehouses Star and snowflake schemas are a good idea in the DW world C-Stores

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

Overview of Databases On MacOS. Karl Kuehn Automation Engineer RethinkDB

Overview of Databases On MacOS. Karl Kuehn Automation Engineer RethinkDB Overview of Databases On MacOS Karl Kuehn Automation Engineer RethinkDB Session Goals Introduce Database concepts Show example players Not Goals: Cover non-macos systems (Oracle) Teach you SQL Answer what

More information

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world Analytics March 2015 White paper Why NoSQL? Your database options in the new non-relational world 2 Why NoSQL? Contents 2 New types of apps are generating new types of data 2 A brief history of NoSQL 3

More information

extensible record stores document stores key-value stores Rick Cattel s clustering from Scalable SQL and NoSQL Data Stores SIGMOD Record, 2010

extensible record stores document stores key-value stores Rick Cattel s clustering from Scalable SQL and NoSQL Data Stores SIGMOD Record, 2010 System/ Scale to Primary Secondary Joins/ Integrity Language/ Data Year Paper 1000s Index Indexes Transactions Analytics Constraints Views Algebra model my label 1971 RDBMS O tables sql-like 2003 memcached

More information

Integrating Big Data into the Computing Curricula

Integrating Big Data into the Computing Curricula Integrating Big Data into the Computing Curricula Yasin Silva, Suzanne Dietrich, Jason Reed, Lisa Tsosie Arizona State University http://www.public.asu.edu/~ynsilva/ibigdata/ 1 Overview Motivation Big

More information

Towards Privacy aware Big Data analytics

Towards Privacy aware Big Data analytics Towards Privacy aware Big Data analytics Pietro Colombo, Barbara Carminati, and Elena Ferrari Department of Theoretical and Applied Sciences, University of Insubria, Via Mazzini 5, 21100 - Varese, Italy

More information

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing Evaluating NoSQL for Enterprise Applications Dirk Bartels VP Strategy & Marketing Agenda The Real Time Enterprise The Data Gold Rush Managing The Data Tsunami Analytics and Data Case Studies Where to go

More information

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 NoSQL Databases Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 Database Landscape Source: H. Lim, Y. Han, and S. Babu, How to Fit when No One Size Fits., in CIDR,

More information

How To Scale Out Of A Nosql Database

How To Scale Out Of A Nosql Database Firebird meets NoSQL (Apache HBase) Case Study Firebird Conference 2011 Luxembourg 25.11.2011 26.11.2011 Thomas Steinmaurer DI +43 7236 3343 896 [email protected] www.scch.at Michael Zwick DI

More information

NoSQL replacement for SQLite (for Beatstream) Antti-Jussi Kovalainen Seminar OHJ-1860: NoSQL databases

NoSQL replacement for SQLite (for Beatstream) Antti-Jussi Kovalainen Seminar OHJ-1860: NoSQL databases NoSQL replacement for SQLite (for Beatstream) Antti-Jussi Kovalainen Seminar OHJ-1860: NoSQL databases Background Inspiration: postgresapp.com demo.beatstream.fi (modern desktop browsers without

More information

Big Systems, Big Data

Big Systems, Big Data Big Systems, Big Data When considering Big Distributed Systems, it can be noted that a major concern is dealing with data, and in particular, Big Data Have general data issues (such as latency, availability,

More information

Can the Elephants Handle the NoSQL Onslaught?

Can the Elephants Handle the NoSQL Onslaught? Can the Elephants Handle the NoSQL Onslaught? Avrilia Floratou, Nikhil Teletia David J. DeWitt, Jignesh M. Patel, Donghui Zhang University of Wisconsin-Madison Microsoft Jim Gray Systems Lab Presented

More information

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK BIG DATA HOLDS BIG PROMISE FOR SECURITY NEHA S. PAWAR, PROF. S. P. AKARTE Computer

More information

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach Nic Caine NoSQL Matters, April 2013 Overview The Problem Current Big Data Analytics Relationship Analytics Leveraging

More information

A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA

A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA A COMPARATIVE STUDY OF NOSQL DATA STORAGE MODELS FOR BIG DATA Ompal Singh Assistant Professor, Computer Science & Engineering, Sharda University, (India) ABSTRACT In the new era of distributed system where

More information

NoSQL Data Base Basics

NoSQL Data Base Basics NoSQL Data Base Basics Course Notes in Transparency Format Cloud Computing MIRI (CLC-MIRI) UPC Master in Innovation & Research in Informatics Spring- 2013 Jordi Torres, UPC - BSC www.jorditorres.eu HDFS

More information

How To Handle Big Data With A Data Scientist

How To Handle Big Data With A Data Scientist III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Top Ten Security and Privacy Challenges for Big Data and Smartgrids. Arnab Roy Fujitsu Laboratories of America

Top Ten Security and Privacy Challenges for Big Data and Smartgrids. Arnab Roy Fujitsu Laboratories of America 1 Top Ten Security and Privacy Challenges for Big Data and Smartgrids Arnab Roy Fujitsu Laboratories of America 2 User Roles and Security Concerns [SKCP11] Users and Security Concerns [SKCP10] Utilities:

More information

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 美 國 13 歲 學 生 用 Big Data 找 出 霸 淩 熱 點 Puri 架 設 網 站 Bullyvention, 藉 由 分 析 Twitter 上 找 出 提 到 跟 霸 凌 相 關 的 詞, 搭 配 地 理 位 置

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

An Approach to Implement Map Reduce with NoSQL Databases

An Approach to Implement Map Reduce with NoSQL Databases www.ijecs.in International Journal Of Engineering And Computer Science ISSN: 2319-7242 Volume 4 Issue 8 Aug 2015, Page No. 13635-13639 An Approach to Implement Map Reduce with NoSQL Databases Ashutosh

More information

Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace

Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace Introduction to Polyglot Persistence Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace FOSSCOMM 2016 Background - 14 years in databases and system engineering - NoSQL DBA @ ObjectRocket

More information

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research &

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & Innovation 04-08-2011 to the EC 8 th February, Luxembourg Your Atos business Research technologists. and Innovation

More information

Challenges for Data Driven Systems

Challenges for Data Driven Systems Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Quick History of Data Management 4000 B C Manual recording From tablets to papyrus to paper A. Payberah 2014 2

More information

Big Data Management and Security

Big Data Management and Security Big Data Management and Security Audit Concerns and Business Risks Tami Frankenfield Sr. Director, Analytics and Enterprise Data Mercury Insurance What is Big Data? Velocity + Volume + Variety = Value

More information

Advanced Data Management Technologies

Advanced Data Management Technologies ADMT 2014/15 Unit 15 J. Gamper 1/44 Advanced Data Management Technologies Unit 15 Introduction to NoSQL J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE ADMT 2014/15 Unit 15

More information

Introduction to NOSQL

Introduction to NOSQL Introduction to NOSQL Université Paris-Est Marne la Vallée, LIGM UMR CNRS 8049, France January 31, 2014 Motivations NOSQL stands for Not Only SQL Motivations Exponential growth of data set size (161Eo

More information

wow CPSC350 relational schemas table normalization practical use of relational algebraic operators tuple relational calculus and their expression in a declarative query language relational schemas CPSC350

More information

Arnab Roy Fujitsu Laboratories of America and CSA Big Data WG

Arnab Roy Fujitsu Laboratories of America and CSA Big Data WG Arnab Roy Fujitsu Laboratories of America and CSA Big Data WG 1 The Big Data Working Group (BDWG) will be identifying scalable techniques for data-centric security and privacy problems. BDWG s investigation

More information

bigdata Managing Scale in Ontological Systems

bigdata Managing Scale in Ontological Systems Managing Scale in Ontological Systems 1 This presentation offers a brief look scale in ontological (semantic) systems, tradeoffs in expressivity and data scale, and both information and systems architectural

More information

Comparison of the Frontier Distributed Database Caching System with NoSQL Databases

Comparison of the Frontier Distributed Database Caching System with NoSQL Databases Comparison of the Frontier Distributed Database Caching System with NoSQL Databases Dave Dykstra [email protected] Fermilab is operated by the Fermi Research Alliance, LLC under contract No. DE-AC02-07CH11359

More information

MongoDB in the NoSQL and SQL world. Horst Rechner [email protected] Berlin, 2012-05-15

MongoDB in the NoSQL and SQL world. Horst Rechner horst.rechner@fokus.fraunhofer.de Berlin, 2012-05-15 MongoDB in the NoSQL and SQL world. Horst Rechner [email protected] Berlin, 2012-05-15 1 MongoDB in the NoSQL and SQL world. NoSQL What? Why? - How? Say goodbye to ACID, hello BASE You

More information

MongoDB Developer and Administrator Certification Course Agenda

MongoDB Developer and Administrator Certification Course Agenda MongoDB Developer and Administrator Certification Course Agenda Lesson 1: NoSQL Database Introduction What is NoSQL? Why NoSQL? Difference Between RDBMS and NoSQL Databases Benefits of NoSQL Types of NoSQL

More information

BIG DATA TOOLS. Top 10 open source technologies for Big Data

BIG DATA TOOLS. Top 10 open source technologies for Big Data BIG DATA TOOLS Top 10 open source technologies for Big Data We are in an ever expanding marketplace!!! With shorter product lifecycles, evolving customer behavior and an economy that travels at the speed

More information

NoSQL Databases. Nikos Parlavantzas

NoSQL Databases. Nikos Parlavantzas !!!! NoSQL Databases Nikos Parlavantzas Lecture overview 2 Objective! Present the main concepts necessary for understanding NoSQL databases! Provide an overview of current NoSQL technologies Outline 3!

More information

Not Relational Models For The Management of Large Amount of Astronomical Data. Bruno Martino (IASI/CNR), Memmo Federici (IAPS/INAF)

Not Relational Models For The Management of Large Amount of Astronomical Data. Bruno Martino (IASI/CNR), Memmo Federici (IAPS/INAF) Not Relational Models For The Management of Large Amount of Astronomical Data Bruno Martino (IASI/CNR), Memmo Federici (IAPS/INAF) What is a DBMS A Data Base Management System is a software infrastructure

More information

Slave. Master. Research Scholar, Bharathiar University

Slave. Master. Research Scholar, Bharathiar University Volume 3, Issue 7, July 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper online at: www.ijarcsse.com Study on Basically, and Eventually

More information

The evolution of database technology (II) Huibert Aalbers Senior Certified Executive IT Architect

The evolution of database technology (II) Huibert Aalbers Senior Certified Executive IT Architect The evolution of database technology (II) Huibert Aalbers Senior Certified Executive IT Architect IT Insight podcast This podcast belongs to the IT Insight series You can subscribe to the podcast through

More information

A survey of big data architectures for handling massive data

A survey of big data architectures for handling massive data CSIT 6910 Independent Project A survey of big data architectures for handling massive data Jordy Domingos - [email protected] Supervisor : Dr David Rossiter Content Table 1 - Introduction a - Context

More information

Big Data and Data Science: Behind the Buzz Words

Big Data and Data Science: Behind the Buzz Words Big Data and Data Science: Behind the Buzz Words Peggy Brinkmann, FCAS, MAAA Actuary Milliman, Inc. April 1, 2014 Contents Big data: from hype to value Deconstructing data science Managing big data Analyzing

More information

Cloud Computing and Advanced Relationship Analytics

Cloud Computing and Advanced Relationship Analytics Cloud Computing and Advanced Relationship Analytics Using Objectivity/DB to Discover the Relationships in your Data By Brian Clark Vice President, Product Management Objectivity, Inc. 408 992 7136 [email protected]

More information

Study and Comparison of Elastic Cloud Databases : Myth or Reality?

Study and Comparison of Elastic Cloud Databases : Myth or Reality? Université Catholique de Louvain Ecole Polytechnique de Louvain Computer Engineering Department Study and Comparison of Elastic Cloud Databases : Myth or Reality? Promoters: Peter Van Roy Sabri Skhiri

More information

Applications for Big Data Analytics

Applications for Big Data Analytics Smarter Healthcare Applications for Big Data Analytics Multi-channel sales Finance Log Analysis Homeland Security Traffic Control Telecom Search Quality Manufacturing Trading Analytics Fraud and Risk Retail:

More information

<Insert Picture Here> Oracle NoSQL Database A Distributed Key-Value Store

<Insert Picture Here> Oracle NoSQL Database A Distributed Key-Value Store Oracle NoSQL Database A Distributed Key-Value Store Charles Lamb, Consulting MTS The following is intended to outline our general product direction. It is intended for information

More information

Referential Integrity in Cloud NoSQL Databases

Referential Integrity in Cloud NoSQL Databases Referential Integrity in Cloud NoSQL Databases by Harsha Raja A thesis submitted to the Victoria University of Wellington in partial fulfilment of the requirements for the degree of Master of Engineering

More information

F1: A Distributed SQL Database That Scales. Presentation by: Alex Degtiar ([email protected]) 15-799 10/21/2013

F1: A Distributed SQL Database That Scales. Presentation by: Alex Degtiar (adegtiar@cmu.edu) 15-799 10/21/2013 F1: A Distributed SQL Database That Scales Presentation by: Alex Degtiar ([email protected]) 15-799 10/21/2013 What is F1? Distributed relational database Built to replace sharded MySQL back-end of AdWords

More information

FIS GT.M Multi-purpose Universal NoSQL Database. K.S. Bhaskar Development Director, FIS [email protected] +1 (610) 578-4265

FIS GT.M Multi-purpose Universal NoSQL Database. K.S. Bhaskar Development Director, FIS ks.bhaskar@fisglobal.com +1 (610) 578-4265 FIS GT.M Multi-purpose Universal NoSQL Database K.S. Bhaskar Development Director, FIS [email protected] +1 (610) 578-4265 Outline M a Universal NoSQL database Using GT.M as a Multi-purpose NoSQL

More information

Comparing SQL and NOSQL databases

Comparing SQL and NOSQL databases COSC 6397 Big Data Analytics Data Formats (II) HBase Edgar Gabriel Spring 2015 Comparing SQL and NOSQL databases Types Development History Data Storage Model SQL One type (SQL database) with minor variations

More information

How To Use Big Data For Telco (For A Telco)

How To Use Big Data For Telco (For A Telco) ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA David Vanderfeesten, Bell Labs Belgium ANNO 2012 YOUR DATA IS MONEY BIG MONEY! Your click stream, your activity stream, your electricity consumption, your call

More information

Compliance & Data Protection in the Big Data Age - MongoDB Security Architecture

Compliance & Data Protection in the Big Data Age - MongoDB Security Architecture Compliance & Data Protection in the Big Data Age - MongoDB Security Architecture Mat Keep MongoDB Product Management & Marketing [email protected] @matkeep Agenda Data Security Landscape and Challenges

More information

nosql and Non Relational Databases

nosql and Non Relational Databases nosql and Non Relational Databases Image src: http://www.pentaho.com/big-data/nosql/ Matthias Lee Johns Hopkins University What NoSQL? Yes no SQL.. Atleast not only SQL Large class of Non Relaltional Databases

More information

Big Data. Facebook Wall Data using Graph API. Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715

Big Data. Facebook Wall Data using Graph API. Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715 Big Data Facebook Wall Data using Graph API Presented by: Prashant Patel-2556219 Jaykrushna Patel-2619715 Outline Data Source Processing tools for processing our data Big Data Processing System: Mongodb

More information

Introduction to Hadoop. New York Oracle User Group Vikas Sawhney

Introduction to Hadoop. New York Oracle User Group Vikas Sawhney Introduction to Hadoop New York Oracle User Group Vikas Sawhney GENERAL AGENDA Driving Factors behind BIG-DATA NOSQL Database 2014 Database Landscape Hadoop Architecture Map/Reduce Hadoop Eco-system Hadoop

More information

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015

Big Data Technologies. Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015 Big Data Technologies Prof. Dr. Uta Störl Hochschule Darmstadt Fachbereich Informatik Sommersemester 2015 Situation: Bigger and Bigger Volumes of Data Big Data Use Cases Log Analytics (Web Logs, Sensor

More information

Understanding NoSQL Technologies on Windows Azure

Understanding NoSQL Technologies on Windows Azure David Chappell Understanding NoSQL Technologies on Windows Azure Sponsored by Microsoft Corporation Copyright 2013 Chappell & Associates Contents Data on Windows Azure: The Big Picture... 3 Windows Azure

More information

Highly available, scalable and secure data with Cassandra and DataStax Enterprise. GOTO Berlin 27 th February 2014

Highly available, scalable and secure data with Cassandra and DataStax Enterprise. GOTO Berlin 27 th February 2014 Highly available, scalable and secure data with Cassandra and DataStax Enterprise GOTO Berlin 27 th February 2014 About Us Steve van den Berg Johnny Miller Solutions Architect Regional Director Western

More information

An Access Control Model for NoSQL Databases

An Access Control Model for NoSQL Databases Western University Scholarship@Western Electronic Thesis and Dissertation Repository December 2013 An Access Control Model for NoSQL Databases Motahera Shermin The University of Western Ontario Supervisor

More information

Cassandra A Decentralized, Structured Storage System

Cassandra A Decentralized, Structured Storage System Cassandra A Decentralized, Structured Storage System Avinash Lakshman and Prashant Malik Facebook Published: April 2010, Volume 44, Issue 2 Communications of the ACM http://dl.acm.org/citation.cfm?id=1773922

More information

NoSQL in der Cloud Why? Andreas Hartmann

NoSQL in der Cloud Why? Andreas Hartmann NoSQL in der Cloud Why? Andreas Hartmann 17.04.2013 17.04.2013 2 NoSQL in der Cloud Why? Quelle: http://res.sys-con.com/story/mar12/2188748/cloudbigdata_0_0.jpg Why Cloud??? 17.04.2013 3 NoSQL in der Cloud

More information

Databases 2 (VU) (707.030)

Databases 2 (VU) (707.030) Databases 2 (VU) (707.030) Introduction to NoSQL Denis Helic KMI, TU Graz Oct 14, 2013 Denis Helic (KMI, TU Graz) NoSQL Oct 14, 2013 1 / 37 Outline 1 NoSQL Motivation 2 NoSQL Systems 3 NoSQL Examples 4

More information

these three NoSQL databases because I wanted to see a the two different sides of the CAP

these three NoSQL databases because I wanted to see a the two different sides of the CAP Michael Sharp Big Data CS401r Lab 3 For this paper I decided to do research on MongoDB, Cassandra, and Dynamo. I chose these three NoSQL databases because I wanted to see a the two different sides of the

More information

Open Source Technologies on Microsoft Azure

Open Source Technologies on Microsoft Azure Open Source Technologies on Microsoft Azure A Survey @DChappellAssoc Copyright 2014 Chappell & Associates The Main Idea i Open source technologies are a fundamental part of Microsoft Azure The Big Questions

More information

Big Data Technologies Compared June 2014

Big Data Technologies Compared June 2014 Big Data Technologies Compared June 2014 Agenda What is Big Data Big Data Technology Comparison Summary Other Big Data Technologies Questions 2 What is Big Data by Example The SKA Telescope is a new development

More information

Introduction to NoSQL Databases. Tore Risch Information Technology Uppsala University 2013-03-05

Introduction to NoSQL Databases. Tore Risch Information Technology Uppsala University 2013-03-05 Introduction to NoSQL Databases Tore Risch Information Technology Uppsala University 2013-03-05 UDBL Tore Risch Uppsala University, Sweden Evolution of DBMS technology Distributed databases SQL 1960 1970

More information

InfiniteGraph: The Distributed Graph Database

InfiniteGraph: The Distributed Graph Database A Performance and Distributed Performance Benchmark of InfiniteGraph and a Leading Open Source Graph Database Using Synthetic Data Objectivity, Inc. 640 West California Ave. Suite 240 Sunnyvale, CA 94086

More information

The CAP theorem and the design of large scale distributed systems: Part I

The CAP theorem and the design of large scale distributed systems: Part I The CAP theorem and the design of large scale distributed systems: Part I Silvia Bonomi University of Rome La Sapienza www.dis.uniroma1.it/~bonomi Great Ideas in Computer Science & Engineering A.A. 2012/2013

More information

Introduction to Big Data Training

Introduction to Big Data Training Introduction to Big Data Training The quickest way to be introduce with NOSQL/BIG DATA offerings Learn and experience Big Data Solutions including Hadoop HDFS, Map Reduce, NoSQL DBs: Document Based DB

More information

MS-55096: Securing Data on Microsoft SQL Server 2012

MS-55096: Securing Data on Microsoft SQL Server 2012 MS-55096: Securing Data on Microsoft SQL Server 2012 Description The goal of this two-day instructor-led course is to provide students with the database and SQL server security knowledge and skills necessary

More information

Understanding NoSQL on Microsoft Azure

Understanding NoSQL on Microsoft Azure David Chappell Understanding NoSQL on Microsoft Azure Sponsored by Microsoft Corporation Copyright 2014 Chappell & Associates Contents Data on Azure: The Big Picture... 3 Relational Technology: A Quick

More information

How to Choose Between Hadoop, NoSQL and RDBMS

How to Choose Between Hadoop, NoSQL and RDBMS How to Choose Between Hadoop, NoSQL and RDBMS Keywords: Jean-Pierre Dijcks Oracle Redwood City, CA, USA Big Data, Hadoop, NoSQL Database, Relational Database, SQL, Security, Performance Introduction A

More information

Server-Side JavaScript Injection Bryan Sullivan, Senior Security Researcher, Adobe Secure Software Engineering Team July 2011

Server-Side JavaScript Injection Bryan Sullivan, Senior Security Researcher, Adobe Secure Software Engineering Team July 2011 Server-Side JavaScript Injection Bryan Sullivan, Senior Security Researcher, Adobe Secure Software Engineering Team July 2011 Abstract This whitepaper is presented in support of the BlackHat USA 2011 talk,

More information

Cloud & Big Data a perfect marriage? Patrick Valduriez

Cloud & Big Data a perfect marriage? Patrick Valduriez Cloud & Big Data a perfect marriage? Patrick Valduriez Cloud & Big Data: the hype! 2 Cloud & Big Data: the hype! 3 Behind the Hype? Every one who wants to make big money Intel, IBM, Microsoft, Oracle,

More information

How to Hadoop Without the Worry: Protecting Big Data at Scale

How to Hadoop Without the Worry: Protecting Big Data at Scale How to Hadoop Without the Worry: Protecting Big Data at Scale SESSION ID: CDS-W06 Davi Ottenheimer Senior Director of Trust EMC Corporation @daviottenheimer Big Data Trust. Redefined Transparency Relevance

More information

Benchmarking and Analysis of NoSQL Technologies

Benchmarking and Analysis of NoSQL Technologies Benchmarking and Analysis of NoSQL Technologies Suman Kashyap 1, Shruti Zamwar 2, Tanvi Bhavsar 3, Snigdha Singh 4 1,2,3,4 Cummins College of Engineering for Women, Karvenagar, Pune 411052 Abstract The

More information

Data Modeling for Big Data

Data Modeling for Big Data Data Modeling for Big Data by Jinbao Zhu, Principal Software Engineer, and Allen Wang, Manager, Software Engineering, CA Technologies In the Internet era, the volume of data we deal with has grown to terabytes

More information

BRAC. Investigating Cloud Data Storage UNIVERSITY SCHOOL OF ENGINEERING. SUPERVISOR: Dr. Mumit Khan DEPARTMENT OF COMPUTER SCIENCE AND ENGEENIRING

BRAC. Investigating Cloud Data Storage UNIVERSITY SCHOOL OF ENGINEERING. SUPERVISOR: Dr. Mumit Khan DEPARTMENT OF COMPUTER SCIENCE AND ENGEENIRING BRAC UNIVERSITY SCHOOL OF ENGINEERING DEPARTMENT OF COMPUTER SCIENCE AND ENGEENIRING 12-12-2012 Investigating Cloud Data Storage Sumaiya Binte Mostafa (ID 08301001) Firoza Tabassum (ID 09101028) BRAC University

More information

Using Object Database db4o as Storage Provider in Voldemort

Using Object Database db4o as Storage Provider in Voldemort Using Object Database db4o as Storage Provider in Voldemort by German Viscuso db4objects (a division of Versant Corporation) September 2010 Abstract: In this article I will show you how

More information

Big Data Management. Big Data Management. (BDM) Autumn 2013. Povl Koch September 2, 2013 01-09-2013 1

Big Data Management. Big Data Management. (BDM) Autumn 2013. Povl Koch September 2, 2013 01-09-2013 1 Big Data Management Big Data Management (BDM) Autumn 2013 Povl Koch September 2, 2013 01-09-2013 1 Overview Today s program 1. Little more practical details about this course 2. Chapter 2 & 3 in NoSQL

More information

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES Vincent Garonne, Mario Lassnig, Martin Barisits, Thomas Beermann, Ralph Vigne, Cedric Serfon [email protected] [email protected] XLDB

More information

Domain driven design, NoSQL and multi-model databases

Domain driven design, NoSQL and multi-model databases Domain driven design, NoSQL and multi-model databases Java Meetup New York, 10 November 2014 Max Neunhöffer www.arangodb.com Max Neunhöffer I am a mathematician Earlier life : Research in Computer Algebra

More information

White paper. The Big Data Security Gap: Protecting the Hadoop Cluster

White paper. The Big Data Security Gap: Protecting the Hadoop Cluster The Big Data Security Gap: Protecting the Hadoop Cluster Introduction While the open source framework has enabled the footprint of Hadoop to logically expand, enterprise organizations face deployment and

More information

Architectural patterns for building real time applications with Apache HBase. Andrew Purtell Committer and PMC, Apache HBase

Architectural patterns for building real time applications with Apache HBase. Andrew Purtell Committer and PMC, Apache HBase Architectural patterns for building real time applications with Apache HBase Andrew Purtell Committer and PMC, Apache HBase Who am I? Distributed systems engineer Principal Architect in the Big Data Platform

More information

Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam [email protected]

Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam sastry.vedantam@oracle.com Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam [email protected] Agenda The rise of Big Data & Hadoop MySQL in the Big Data Lifecycle MySQL Solutions for Big Data Q&A

More information

A Study on Security and Privacy in Big Data Processing

A Study on Security and Privacy in Big Data Processing A Study on Security and Privacy in Big Data Processing C.Yosepu P Srinivasulu Bathala Subbarayudu Assistant Professor, Dept of CSE, St.Martin's Engineering College, Hyderabad, India Assistant Professor,

More information