DICE: Developing Data-Intensive Cloud Applications with Iterative Quality Enhancements

Size: px
Start display at page:

Download "DICE: Developing Data-Intensive Cloud Applications with Iterative Quality Enhancements"

Transcription

1 DICE: Developing Data-Intensive Cloud Applications with Iterative Quality Enhancements Giuliano Casale Imperial College London Project Coordinator DICE Horizon 2020 Project Grant Agreement no Funded by the Horizon 2020 Framework Programme of the European Union

2 DICE Project o Horizon 2020 Research & Innovation Action Quality-Aware Development for Big Data applications Feb Jan 2018, 4M Euros budget 9 partners (Academia & SMEs), 7 EU countries DICE 3/27/2015 2

3 Motivation o Software market rapidly shifting to Big Data 32% compound annual growth rate in EU through % Big data projects are successful [CapGemini 2015] o European call for software quality assurance (QA) ISTAG: call to define environments for understanding the consequences of different implementation alternatives (e.g. quality, robustness, performance, maintenance, evolvability,...) o QA evolving too slowly compared to the trends in software development (Big data, Cloud, DevOps...) Still crucial for competiveness! DICE 3/27/2015 3

4 DICE 3/27/ Quality-Aware MDE Today Platform-Indep. Model Domain Models Analytical Models MARTE Architecture Model QA Models Platform Description Platform-Specific Model Cost-Quality Models Code generation C++ Java C#

5 DICE 3/27/ Challenge 1: QA for Big Data o 5Vs: o Volume, o Velocity, o Variety, o Veracity, o Value o Problem: today no QA toolchain can reason on the quality of complex Big Data applications o Heteregeous Big Data Technologies o NoSQL, Spark, Hadoop/MapReduce, Storm, CEP,... o Cloud infrastructure adds complexity o Cloud storage, auto-scaling, private/public/hybrid,...

6 Challenge 2: Embracing DevOps o QA must become lean as well Continuous quality checks and model versioning o Modelling of the operations Dev needs awareness of infrastructure and costs o Continuous feedback Forward and backward model synchronisation Tracking of self-adaptation events (e.g. auto-scaling) o Big data coming from continuous monitoring QA has its own Big data, use machine learning? DICE 3/27/2015 6

7 An Holistic Approach: DICE DICE DICE IDE Platform Description MARTE Deployment & Continuous Integration Platform-Indep. Model Architecture Model Platform-Specific Model Domain Models Data Awareness Continuous Validation QA Models Big Data Continuous Monitoring DICE 3/27/2015 7

8 Benefits o Tackling skill shortage and steep learning curves Data-aware methods, models, and OSS tools o Shorter time to market for Big Data applications Cost reduction, without sacrificing product quality o Decrease development and testing costs Select optimal architectures that can meet SLAs o Reduce number and severity of quality incidents Iterative refinement of application design DICE 3/27/2015 8

9 DICE QA: Quality Dimensions o Reliability o Efficiency Availability Fault-tolerance Performance Time behaviour Costs o Safety & Privacy Risk of harm Privacy & data protection DICE 3/27/2015 9

10 DICE 3/27/2015 Footer 10 DICE Platform Independent Model (DPIM)

11 DICE Profile: PIM Level o Functional approach to data to be expanded o Data dependencies graph relationships between data, archives and streams o QA focuses on quantitative aspects of data o Static characteristics of data volumes, value, storage location, replication pattern, consistency policies, data access costs, known schedules of data transfers, data access control / privacy,... o Dynamic characteristics of data cache hit/miss probabilities, read/write/update rates, burstiness,... DICE 11

12 DICE 3/27/2015 Footer 12 DICE Platform and Technology Specific Model (DTSM)

13 DICE Platform, Technology and Deployment Specific Model (DDSM) DICE 3/27/2015 Footer 13

14 DICE 14 DICE Profile: PSM Level o Need for technology-specific abstractions Hadoop: Number of mappers and reducers,... In-memory DBs: Peak memory and variable threading Streaming: merge/split/operators, networking,... Storage: Supported operations, cost/byte,... NoSQL: Consistency policies,... o Generation of deployment plan Proposed Chef + TOSCA extension o Interest is both on private and public clouds Private clouds more relevant for batch processing Public clouds more relevant for streaming

15 DICE 3/27/ Demonstrators Case study Domain Features & Challenges Distributed dataintensive media system (ATC) Big Data for e- Government (Netfective) Geo-fencing (Prodevelop) News & Media Social media E-Gov application Maritime sector Large-scale software Data velocities Data volumes Data granularity Multiple data sources and channels Privacy Data volumes Legacy data Data consolidation Data stores Privacy Forecasting and data analysis Vessels movements Safety requirements Streaming & CEP Geographical information

16 DICE 16 Thanks!

DICE: Quality- Driven Development of Data- Intensive Cloud ApplicaPons

DICE: Quality- Driven Development of Data- Intensive Cloud ApplicaPons DICE: Quality- Driven Development of Data- Intensive Cloud ApplicaPons G. Casale, D. Ardagna, M. Artac, F. Barbier, E. Di Ni6o, A. Henry, G. Iuhasz, C. Joubert, J. Merseguer, V. I. Munteanu, J. F. Pérez,

More information

Requirement Specification

Requirement Specification Developing Data-Intensive Cloud Applications with Iterative Quality Enhancements Specification Deliverable 1.2 Deliverable 1.2. s Specification. Deliverable: D1.2 Title: Specification Editor(s): Ilias

More information

Hadoop Evolution In Organizations. Mark Vervuurt Cluster Data Science & Analytics

Hadoop Evolution In Organizations. Mark Vervuurt Cluster Data Science & Analytics In Organizations Mark Vervuurt Cluster Data Science & Analytics AGENDA 1. Yellow Elephant 2. Data Ingestion & Complex Event Processing 3. SQL on Hadoop 4. NoSQL 5. InMemory 6. Data Science & Machine Learning

More information

Reference Architecture, Requirements, Gaps, Roles

Reference Architecture, Requirements, Gaps, Roles Reference Architecture, Requirements, Gaps, Roles The contents of this document are an excerpt from the brainstorming document M0014. The purpose is to show how a detailed Big Data Reference Architecture

More information

HDP Hadoop From concept to deployment.

HDP Hadoop From concept to deployment. HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some

More information

Conjugating data mood and tenses: Simple past, infinite present, fast continuous, simpler imperative, conditional future perfect

Conjugating data mood and tenses: Simple past, infinite present, fast continuous, simpler imperative, conditional future perfect Matteo Migliavacca (mm53@kent) School of Computing Conjugating data mood and tenses: Simple past, infinite present, fast continuous, simpler imperative, conditional future perfect Simple past - Traditional

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS

BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS WHAT IS BIG DATA? describes any voluminous amount of structured, semi-structured and unstructured data that has the potential to be mined for information

More information

ON-PREMISE OR IN THE CLOUD, A SINGLE JAVA EE APPLICATION PLATFORM

ON-PREMISE OR IN THE CLOUD, A SINGLE JAVA EE APPLICATION PLATFORM ON-PREMISE OR IN THE CLOUD, A SINGLE JAVA EE APPLICATION PLATFORM TECHNOLOGY OVERVIEW FEATURES Fully certified Java EE 6 container Full web services stack Modular architecture optimized for cloud and virtual

More information

Demystifying Big Data Government Agencies & The Big Data Phenomenon

Demystifying Big Data Government Agencies & The Big Data Phenomenon Demystifying Big Data Government Agencies & The Big Data Phenomenon Today s Discussion If you only remember four things 1 Intensifying business challenges coupled with an explosion in data have pushed

More information

Unified Big Data Analytics Pipeline. 连 城 lian@databricks.com

Unified Big Data Analytics Pipeline. 连 城 lian@databricks.com Unified Big Data Analytics Pipeline 连 城 lian@databricks.com What is A fast and general engine for large-scale data processing An open source implementation of Resilient Distributed Datasets (RDD) Has an

More information

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016 Machina Research Where is the value in IoT? IoT data and analytics may have an answer Emil Berthelsen, Principal Analyst April 28, 2016 About Machina Research Machina Research is the world s leading provider

More information

Big Data Driven Knowledge Discovery for Autonomic Future Internet

Big Data Driven Knowledge Discovery for Autonomic Future Internet Big Data Driven Knowledge Discovery for Autonomic Future Internet Professor Geyong Min Chair in High Performance Computing and Networking Department of Mathematics and Computer Science College of Engineering,

More information

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP Operates more like a search engine than a database Scoring and ranking IP allows for fuzzy searching Best-result candidate sets returned Contextual analytics to correctly disambiguate entities Embedded

More information

3 Reasons Enterprises Struggle with Storm & Spark Streaming and Adopt DataTorrent RTS

3 Reasons Enterprises Struggle with Storm & Spark Streaming and Adopt DataTorrent RTS . 3 Reasons Enterprises Struggle with Storm & Spark Streaming and Adopt DataTorrent RTS Deliver fast actionable business insights for data scientists, rapid application creation for developers and enterprise-grade

More information

HPC ABDS: The Case for an Integrating Apache Big Data Stack

HPC ABDS: The Case for an Integrating Apache Big Data Stack HPC ABDS: The Case for an Integrating Apache Big Data Stack with HPC 1st JTC 1 SGBD Meeting SDSC San Diego March 19 2014 Judy Qiu Shantenu Jha (Rutgers) Geoffrey Fox gcf@indiana.edu http://www.infomall.org

More information

Talend Real-Time Big Data Sandbox. Big Data Insights Cookbook

Talend Real-Time Big Data Sandbox. Big Data Insights Cookbook Talend Real-Time Big Data Talend Real-Time Big Data Overview of Real-time Big Data Pre-requisites to run Setup & Talend License Talend Real-Time Big Data Big Data Setup & About this cookbook What is the

More information

Pulsar Realtime Analytics At Scale. Tony Ng April 14, 2015

Pulsar Realtime Analytics At Scale. Tony Ng April 14, 2015 Pulsar Realtime Analytics At Scale Tony Ng April 14, 2015 Big Data Trends Bigger data volumes More data sources DBs, logs, behavioral & business event streams, sensors Faster analysis Next day to hours

More information

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84 Index A Amazon Web Services (AWS), 50, 58 Analytics engine, 21 22 Apache Kafka, 38, 131 Apache S4, 38, 131 Apache Sqoop, 37, 131 Appliance pattern, 104 105 Application architecture, big data analytics

More information

BIG DATA TOOLS. Top 10 open source technologies for Big Data

BIG DATA TOOLS. Top 10 open source technologies for Big Data BIG DATA TOOLS Top 10 open source technologies for Big Data We are in an ever expanding marketplace!!! With shorter product lifecycles, evolving customer behavior and an economy that travels at the speed

More information

SAP HANA From Relational OLAP Database to Big Data Infrastructure

SAP HANA From Relational OLAP Database to Big Data Infrastructure SAP HANA From Relational OLAP Database to Big Data Infrastructure Anil K Goel VP & Chief Architect, SAP HANA Data Platform WBDB 2015, June 16, 2015 Toronto SAP Big Data Story Data Lifecycle Management

More information

1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India

1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India 1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India Call for Papers Colossal Data Analysis and Networking has emerged as a de facto

More information

Testing Big data is one of the biggest

Testing Big data is one of the biggest Infosys Labs Briefings VOL 11 NO 1 2013 Big Data: Testing Approach to Overcome Quality Challenges By Mahesh Gudipati, Shanthi Rao, Naju D. Mohan and Naveen Kumar Gajja Validate data quality by employing

More information

From Raw Data to. Actionable Insights with. MATLAB Analytics. Learn more. Develop predictive models. 1Access and explore data

From Raw Data to. Actionable Insights with. MATLAB Analytics. Learn more. Develop predictive models. 1Access and explore data 100 001 010 111 From Raw Data to 10011100 Actionable Insights with 00100111 MATLAB Analytics 01011100 11100001 1 Access and Explore Data For scientists the problem is not a lack of available but a deluge.

More information

3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India

3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India 3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India Call for Papers Cloud computing has emerged as a de facto computing

More information

The Enterprise Data Hub and The Modern Information Architecture

The Enterprise Data Hub and The Modern Information Architecture The Enterprise Data Hub and The Modern Information Architecture Dr. Amr Awadallah CTO & Co-Founder, Cloudera Twitter: @awadallah 1 2013 Cloudera, Inc. All rights reserved. Cloudera Overview The Leader

More information

Fault-Tolerant Computer System Design ECE 695/CS 590. Putting it All Together

Fault-Tolerant Computer System Design ECE 695/CS 590. Putting it All Together Fault-Tolerant Computer System Design ECE 695/CS 590 Putting it All Together Saurabh Bagchi ECE/CS Purdue University ECE 695/CS 590 1 Outline Looking at some practical systems that integrate multiple techniques

More information

Real Time Fraud Detection With Sequence Mining on Big Data Platform. Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA

Real Time Fraud Detection With Sequence Mining on Big Data Platform. Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA Real Time Fraud Detection With Sequence Mining on Big Data Platform Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA Open Source Big Data Eco System Query (NOSQL) : Cassandra,

More information

HDP Enabling the Modern Data Architecture

HDP Enabling the Modern Data Architecture HDP Enabling the Modern Data Architecture Herb Cunitz President, Hortonworks Page 1 Hortonworks enables adoption of Apache Hadoop through HDP (Hortonworks Data Platform) Founded in 2011 Original 24 architects,

More information

Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop

Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop Volume 4, Issue 1, January 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Transitioning

More information

Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control

Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control EP/K006487/1 UK PI: Prof Gareth Taylor (BU) China PI: Prof Yong-Hua Song (THU) Consortium UK Members: Brunel University

More information

Data-intensive HPC: opportunities and challenges. Patrick Valduriez

Data-intensive HPC: opportunities and challenges. Patrick Valduriez Data-intensive HPC: opportunities and challenges Patrick Valduriez Big Data Landscape Multi-$billion market! Big data = Hadoop = MapReduce? No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard,

More information

IAN MASSINGHAM. Technical Evangelist Amazon Web Services

IAN MASSINGHAM. Technical Evangelist Amazon Web Services IAN MASSINGHAM Technical Evangelist Amazon Web Services From 2014: Cloud computing has become the new normal Deploying new applications to the cloud by default Migrating existing applications as quickly

More information

A Big Data Storage Architecture for the Second Wave David Sunny Sundstrom Principle Product Director, Storage Oracle

A Big Data Storage Architecture for the Second Wave David Sunny Sundstrom Principle Product Director, Storage Oracle A Big Data Storage Architecture for the Second Wave David Sunny Sundstrom Principle Product Director, Storage Oracle Growth in Data Diversity and Usage 1.8 Zettabytes of Data in 2011, 20x Growth by 2020

More information

LONDON. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

LONDON. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved LONDON 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Best Practices for Building Partner Managed Services on AWS Kelly Hartman, Global Segment Leader, MSPs Kyle Lichtenberg, Solutions

More information

NoSQL for SQL Professionals William McKnight

NoSQL for SQL Professionals William McKnight NoSQL for SQL Professionals William McKnight Session Code BD03 About your Speaker, William McKnight President, McKnight Consulting Group Frequent keynote speaker and trainer internationally Consulted to

More information

Big Data Frameworks Course. Prof. Sasu Tarkoma 10.3.2015

Big Data Frameworks Course. Prof. Sasu Tarkoma 10.3.2015 Big Data Frameworks Course Prof. Sasu Tarkoma 10.3.2015 Contents Course Overview Lectures Assignments/Exercises Course Overview This course examines current and emerging Big Data frameworks with focus

More information

Simplifying Big Data Analytics: Unifying Batch and Stream Processing. John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!!

Simplifying Big Data Analytics: Unifying Batch and Stream Processing. John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!! Simplifying Big Data Analytics: Unifying Batch and Stream Processing John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!! Streaming Analy.cs S S S Scale- up Database Data And Compute Grid

More information

Future-Proofing MySQL for the Worldwide Data Revolution

Future-Proofing MySQL for the Worldwide Data Revolution Future-Proofing MySQL for the Worldwide Data Revolution Robert Hodges, CEO. What is Future-Proo!ng? Future-proo!ng = creating systems that last while parts change and improve MySQL is not losing out to

More information

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Siva Ravada Senior Director of Development Oracle Spatial and MapViewer 2 Evolving Technology Platforms

More information

Hybrid Software Architectures for Big Data. Laurence.Hubert@hurence.com @hurence http://www.hurence.com

Hybrid Software Architectures for Big Data. Laurence.Hubert@hurence.com @hurence http://www.hurence.com Hybrid Software Architectures for Big Data Laurence.Hubert@hurence.com @hurence http://www.hurence.com Headquarters : Grenoble Pure player Expert level consulting Training R&D Big Data X-data hot-line

More information

White Paper. How Streaming Data Analytics Enables Real-Time Decisions

White Paper. How Streaming Data Analytics Enables Real-Time Decisions White Paper How Streaming Data Analytics Enables Real-Time Decisions Contents Introduction... 1 What Is Streaming Analytics?... 1 How Does SAS Event Stream Processing Work?... 2 Overview...2 Event Stream

More information

Securing Big Data Learning and Differences from Cloud Security

Securing Big Data Learning and Differences from Cloud Security Securing Big Data Learning and Differences from Cloud Security Samir Saklikar RSA, The Security Division of EMC Session ID: DAS-108 Session Classification: Advanced Agenda Cloud Computing & Big Data Similarities

More information

Cloud Big Data Architectures

Cloud Big Data Architectures Cloud Big Data Architectures Lynn Langit QCon Sao Paulo, Brazil 2016 About this Workshop Real-world Cloud Scenarios w/aws, Azure and GCP 1. Big Data Solution Types 2. Data Pipelines 3. ETL and Visualization

More information

Hadoop for Enterprises:

Hadoop for Enterprises: Hadoop for Enterprises: Overcoming the Major Challenges Introduction to Big Data Big Data are information assets that are high volume, velocity, and variety. Big Data demands cost-effective, innovative

More information

COMP9321 Web Application Engineering

COMP9321 Web Application Engineering COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411

More information

The Internet of Things and Big Data: Intro

The Internet of Things and Big Data: Intro The Internet of Things and Big Data: Intro John Berns, Solutions Architect, APAC - MapR Technologies April 22 nd, 2014 1 What This Is; What This Is Not It s not specific to IoT It s not about any specific

More information

REAL-TIME BIG DATA ANALYTICS

REAL-TIME BIG DATA ANALYTICS www.leanxcale.com info@leanxcale.com REAL-TIME BIG DATA ANALYTICS Blending Transactional and Analytical Processing Delivers Real-Time Big Data Analytics 2 ULTRA-SCALABLE FULL ACID FULL SQL DATABASE LeanXcale

More information

Oracle Big Data Strategy Simplified Infrastrcuture

Oracle Big Data Strategy Simplified Infrastrcuture Big Data Oracle Big Data Strategy Simplified Infrastrcuture Selim Burduroğlu Global Innovation Evangelist & Architect Education & Research Industry Business Unit Oracle Confidential Internal/Restricted/Highly

More information

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems DATA WAREHOUSING RESEARCH TRENDS Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Data source heterogeneity and incongruence Filtering out uncorrelated data Strongly unstructured

More information

FINANCIAL SERVICES: FRAUD MANAGEMENT A solution showcase

FINANCIAL SERVICES: FRAUD MANAGEMENT A solution showcase FINANCIAL SERVICES: FRAUD MANAGEMENT A solution showcase TECHNOLOGY OVERVIEW FRAUD MANAGE- MENT REFERENCE ARCHITECTURE This technology overview describes a complete infrastructure and application re-architecture

More information

Big Data Use Cases. To Start Today. Paul Scholey Sales Director, EMEA. 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555

Big Data Use Cases. To Start Today. Paul Scholey Sales Director, EMEA. 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 Big Use Cases To Start Today Paul Scholey Sales Director, EMEA 1 Exabytes of We all know the amount of data in the world is growing exponentially 40000 30000 YOU ARE HERE 20000 FROM 2010 TO 2015 77% of

More information

and NoSQL Data Governance for Regulated Industries Using Hadoop Justin Makeig, Director Product Management, MarkLogic October 2013

and NoSQL Data Governance for Regulated Industries Using Hadoop Justin Makeig, Director Product Management, MarkLogic October 2013 Data Governance for Regulated Industries Using Hadoop and NoSQL Justin Makeig, Director Product Management, MarkLogic October 2013 Who am I? Product Manager for 6 years at MarkLogic Background in FinServ

More information

How Big Data Transforms Data Protection and Storage

How Big Data Transforms Data Protection and Storage I D C E X E C U T I V E B R I E F How Big Data Transforms Data Protection and Storage August 2012 Written by Carla Arend Sponsored by CommVault Introduction: How Big Data Transforms Storage Omøgade 8 P.O.Box

More information

Big Data & the Cloud: The Sum Is Greater Than the Parts

Big Data & the Cloud: The Sum Is Greater Than the Parts E-PAPER March 2014 Big Data & the Cloud: The Sum Is Greater Than the Parts Learn how to accelerate your move to the cloud and use big data to discover new hidden value for your business and your users.

More information

BIG DATA IN BUSINESS ENVIRONMENT

BIG DATA IN BUSINESS ENVIRONMENT Scientific Bulletin Economic Sciences, Volume 14/ Issue 1 BIG DATA IN BUSINESS ENVIRONMENT Logica BANICA 1, Alina HAGIU 2 1 Faculty of Economics, University of Pitesti, Romania olga.banica@upit.ro 2 Faculty

More information

Better Decision Making

Better Decision Making Better Decision Making Big Data Analytics Webinar, November 2013 Dr. Wolfgang Martin Analyst and Member of the Boulder BI Brain Trust Better Decision Making Process Oriented Businesses. Decision Making:

More information

Accelerate your Big Data Strategy. Execute faster with Capgemini and Cloudera s Enterprise Data Hub Accelerator

Accelerate your Big Data Strategy. Execute faster with Capgemini and Cloudera s Enterprise Data Hub Accelerator Accelerate your Big Data Strategy Execute faster with Capgemini and Cloudera s Enterprise Data Hub Accelerator Enterprise Data Hub Accelerator enables you to get started rapidly and cost-effectively with

More information

DevOps - en inngang til den Digtale Transformasjon Rapidly Deliver Exceptional Software

DevOps - en inngang til den Digtale Transformasjon Rapidly Deliver Exceptional Software DevOps - en inngang til den Digtale Transformasjon Rapidly Deliver Exceptional Software Pål Dragseth June 2015 Applications Are Changing the World of Business 84% Video US internet traffic by 2018 $142B

More information

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com Lambda Architecture Near Real-Time Big Data Analytics Using Hadoop January 2015 Contents Overview... 3 Lambda Architecture: A Quick Introduction... 4 Batch Layer... 4 Serving Layer... 4 Speed Layer...

More information

TRANSITIONING TO BIG DATA:

TRANSITIONING TO BIG DATA: TRANSITIONING TO BIG DATA: A Checklist for Operational Readiness Moving to a Big Data platform: Key recommendations to ensure operational readiness Overview Many factors can drive the decision to augment

More information

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data

More information

Big Data for Big Value @ Intel

Big Data for Big Value @ Intel Big Data for Big Value @ Intel Moty Fania, PE Big data Analytics Assaf Araki, Sr. Arch. Big data Analytics Advanced Analytics team @ Intel IT Corporate ownership of advanced analytics Team charter Solve

More information

Hadoop Ecosystem Overview. CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook

Hadoop Ecosystem Overview. CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook Hadoop Ecosystem Overview CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook Agenda Introduce Hadoop projects to prepare you for your group work Intimate detail will be provided in future

More information

Comprehensive Analytics on the Hortonworks Data Platform

Comprehensive Analytics on the Hortonworks Data Platform Comprehensive Analytics on the Hortonworks Data Platform We do Hadoop. Page 1 Page 2 Back to 2005 Page 3 Vertical Scaling Page 4 Vertical Scaling Page 5 Vertical Scaling Page 6 Horizontal Scaling Page

More information

[Hadoop, Storm and Couchbase: Faster Big Data]

[Hadoop, Storm and Couchbase: Faster Big Data] [Hadoop, Storm and Couchbase: Faster Big Data] With over 8,500 clients, LivePerson is the global leader in intelligent online customer engagement. With an increasing amount of agent/customer engagements,

More information

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing

Evaluating NoSQL for Enterprise Applications. Dirk Bartels VP Strategy & Marketing Evaluating NoSQL for Enterprise Applications Dirk Bartels VP Strategy & Marketing Agenda The Real Time Enterprise The Data Gold Rush Managing The Data Tsunami Analytics and Data Case Studies Where to go

More information

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 美 國 13 歲 學 生 用 Big Data 找 出 霸 淩 熱 點 Puri 架 設 網 站 Bullyvention, 藉 由 分 析 Twitter 上 找 出 提 到 跟 霸 凌 相 關 的 詞, 搭 配 地 理 位 置

More information

Amazon Web Services. 18.11.2015 Yu Xiao

Amazon Web Services. 18.11.2015 Yu Xiao Amazon Web Services 18.11.2015 Yu Xiao Agenda Introduction to Amazon Web Services(AWS) 7 Steps to Select the Right Architecture for Your Web Applications Private, Public or Hybrid Cloud? AWS Case Study

More information

Big Data Analytics - Accelerated. stream-horizon.com

Big Data Analytics - Accelerated. stream-horizon.com Big Data Analytics - Accelerated stream-horizon.com Legacy ETL platforms & conventional Data Integration approach Unable to meet latency & data throughput demands of Big Data integration challenges Based

More information

ICT 10: Software Technologies

ICT 10: Software Technologies Technologies Jorge GASOS DG CONNECT Jorge.Gasos@ec.europa.eu Odysseas I. Pyrovolakis DG CONNECT Odysseas.Pyrovolakis@ec.europa.eu Software related activities in WP2016-17 Innovating in software: topics

More information

APPROACHABLE ANALYTICS MAKING SENSE OF DATA

APPROACHABLE ANALYTICS MAKING SENSE OF DATA APPROACHABLE ANALYTICS MAKING SENSE OF DATA AGENDA SAS DELIVERS PROVEN SOLUTIONS THAT DRIVE INNOVATION AND IMPROVE PERFORMANCE. About SAS SAS Business Analytics Framework Approachable Analytics SAS for

More information

Towards a Thriving Data Economy: Open Data, Big Data, and Data Ecosystems

Towards a Thriving Data Economy: Open Data, Big Data, and Data Ecosystems Towards a Thriving Data Economy: Open Data, Big Data, and Data Ecosystems Volker Markl volker.markl@tu-berlin.de dima.tu-berlin.de dfki.de/web/research/iam/ bbdc.berlin Based on my 2014 Vision Paper On

More information

In-memory computing with SAP HANA

In-memory computing with SAP HANA In-memory computing with SAP HANA June 2015 Amit Satoor, SAP @asatoor 2015 SAP SE or an SAP affiliate company. All rights reserved. 1 Hyperconnectivity across people, business, and devices give rise to

More information

Big Data and Analytics: Challenges and Opportunities

Big Data and Analytics: Challenges and Opportunities Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif

More information

G-Cloud Big Data Suite Powered by Pivotal. December 2014. G-Cloud. service definitions

G-Cloud Big Data Suite Powered by Pivotal. December 2014. G-Cloud. service definitions G-Cloud Big Data Suite Powered by Pivotal December 2014 G-Cloud service definitions TABLE OF CONTENTS Service Overview... 3 Business Need... 6 Our Approach... 7 Service Management... 7 Vendor Accreditations/Awards...

More information

Continuous Delivery at SAP: From dinosaur to spaceship. Darren Hague / SAP Global IT November 1st, 2013 Public

Continuous Delivery at SAP: From dinosaur to spaceship. Darren Hague / SAP Global IT November 1st, 2013 Public Continuous Delivery at SAP: From dinosaur to spaceship Darren Hague / SAP Global IT November 1st, 2013 Public Agenda The 6 stages of SAP IT s journey to Continuous Delivery: Dinosaur Age Stone Age Agricultural

More information

Why continuous delivery needs devops, and why devops needs infrastructure-as-code. Sriram Narayan @sriramnarayan 25-Oct-2012

Why continuous delivery needs devops, and why devops needs infrastructure-as-code. Sriram Narayan @sriramnarayan 25-Oct-2012 Why continuous delivery needs devops, and why devops needs infrastructure-as-code Sriram Narayan @sriramnarayan 25-Oct-2012 about me Part of ThoughtWorks Studios Go team Have consulted as Tech Principal,

More information

Educational Opportunities in Big Data

Educational Opportunities in Big Data Educational Opportunities in Big Data Could current Big Gaps in Talent fill the void and Big Market Demand? Dr. KRS Murthy Dr.Sri.Murthy@Gmail.Com BigDataExpert@Gmail.Com (408)-464-3333 Big Gaps in Big

More information

Oracle Big Data SQL Technical Update

Oracle Big Data SQL Technical Update Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical

More information

How to construct a world class SAP Center of Excellence in the era of Cloud. Vipin Singh NTTDATA Inc SESSION CODE: SS1354

How to construct a world class SAP Center of Excellence in the era of Cloud. Vipin Singh NTTDATA Inc SESSION CODE: SS1354 How to construct a world class SAP Center of Excellence in the era of Cloud Vipin Singh NTTDATA Inc SESSION CODE: SS1354 LEARNING POINTS Unique challenges that SAP Cloud applications impose on customer

More information

Getting Started & Successful with Big Data

Getting Started & Successful with Big Data Getting Started & Successful with Big Data @Pentaho #BigDataWebSeries 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 Your Hosts Today Davy Nys VP EMEA & APAC Pentaho Paul

More information

Big Data Zurich, November 23. September 2011

Big Data Zurich, November 23. September 2011 Institute of Technology Management Big Data Projektskizze «Competence Center Automotive Intelligence» Zurich, November 11th 23. September 2011 Felix Wortmann Assistant Professor Technology Management,

More information

Introducing Oracle Exalytics In-Memory Machine

Introducing Oracle Exalytics In-Memory Machine Introducing Oracle Exalytics In-Memory Machine Jon Ainsworth Director of Business Development Oracle EMEA Business Analytics 1 Copyright 2011, Oracle and/or its affiliates. All rights Agenda Topics Oracle

More information

High Availability of VistA EHR in Cloud. ViSolve Inc. White Paper February 2015. www.visolve.com

High Availability of VistA EHR in Cloud. ViSolve Inc. White Paper February 2015. www.visolve.com High Availability of VistA EHR in Cloud ViSolve Inc. White Paper February 2015 1 Abstract Inspite of the accelerating migration to cloud computing in the Healthcare Industry, high availability and uptime

More information

INTRODUCING SOFTWARE PERFORMANCE ANTIPATTERNS IN CLOUD COMPUTING ENVIRONMENTS: DOES IT HELP OR HURT? [VISION PAPER]

INTRODUCING SOFTWARE PERFORMANCE ANTIPATTERNS IN CLOUD COMPUTING ENVIRONMENTS: DOES IT HELP OR HURT? [VISION PAPER] INTRODUCING SOFTWARE PERFORMANCE ANTIPATTERNS IN CLOUD COMPUTING ENVIRONMENTS: DOES IT HELP OR HURT? [VISION PAPER] Catia Trubiani Gran Sasso Science Institute (L Aquila, Italy) http://cs.gssi.infn.it/catia.trubiani

More information

Big Data: Overview and Roadmap. 2015 eglobaltech. All rights reserved.

Big Data: Overview and Roadmap. 2015 eglobaltech. All rights reserved. Big Data: Overview and Roadmap 2015 eglobaltech. All rights reserved. What is Big Data? Large volumes of complex and variable data that require advanced techniques and technologies to enable capture, storage,

More information

Introduction to Big Data Training

Introduction to Big Data Training Introduction to Big Data Training The quickest way to be introduce with NOSQL/BIG DATA offerings Learn and experience Big Data Solutions including Hadoop HDFS, Map Reduce, NoSQL DBs: Document Based DB

More information

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP Business Analytics for All Amsterdam - 2015 Value of Big Data is Being Recognized Executives beginning to see the path from data insights to revenue

More information

Big Data Course Highlights

Big Data Course Highlights Big Data Course Highlights The Big Data course will start with the basics of Linux which are required to get started with Big Data and then slowly progress from some of the basics of Hadoop/Big Data (like

More information

CSE-E5430 Scalable Cloud Computing Lecture 11

CSE-E5430 Scalable Cloud Computing Lecture 11 CSE-E5430 Scalable Cloud Computing Lecture 11 Keijo Heljanko Department of Computer Science School of Science Aalto University keijo.heljanko@aalto.fi 30.11-2015 1/24 Distributed Coordination Systems Consensus

More information

Cloudify and OpenStack Heat

Cloudify and OpenStack Heat Cloudify and OpenStack Heat General Cloudify is an application orchestration platform that provides a complete solution for automating and managing application deployment and DevOps processes on top of

More information

Oracle Database 12c Plug In. Switch On. Get SMART.

Oracle Database 12c Plug In. Switch On. Get SMART. Oracle Database 12c Plug In. Switch On. Get SMART. Duncan Harvey Head of Core Technology, Oracle EMEA March 2015 Safe Harbor Statement The following is intended to outline our general product direction.

More information

Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges

Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges Prerita Gupta Research Scholar, DAV College, Chandigarh Dr. Harmunish Taneja Department of Computer Science and

More information

SPARK USE CASE IN TELCO. Apache Spark Night 9-2-2014! Chance Coble!

SPARK USE CASE IN TELCO. Apache Spark Night 9-2-2014! Chance Coble! SPARK USE CASE IN TELCO Apache Spark Night 9-2-2014! Chance Coble! Use Case Profile Telecommunications company Shared business problems/pain Scalable analytics infrastructure is a problem Pushing infrastructure

More information

Up Your R Game. James Taylor, Decision Management Solutions Bill Franks, Teradata

Up Your R Game. James Taylor, Decision Management Solutions Bill Franks, Teradata Up Your R Game James Taylor, Decision Management Solutions Bill Franks, Teradata Today s Speakers James Taylor Bill Franks CEO Chief Analytics Officer Decision Management Solutions Teradata 7/28/14 3 Polling

More information

Workprogramme 2013 objective 1.2. Sandro D Elia. Software & Service Architectures and Infrastructures

Workprogramme 2013 objective 1.2. Sandro D Elia. Software & Service Architectures and Infrastructures Workprogramme 2013 objective 1.2 Sandro D Elia Software & Service Architectures and Infrastructures Target Outcomes Delivering services in an effective, efficient and reliable manner across the future

More information

Data Services Advisory

Data Services Advisory Data Services Advisory Modern Datastores An Introduction Created by: Strategy and Transformation Services Modified Date: 8/27/2014 Classification: DRAFT SAFE HARBOR STATEMENT This presentation contains

More information

OpenStack. Orgad Kimchi. Principal Software Engineer. Oracle ISV Engineering. 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved.

OpenStack. Orgad Kimchi. Principal Software Engineer. Oracle ISV Engineering. 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved. OpenStack Orgad Kimchi Principal Software Engineer Oracle ISV Engineering 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline

More information