1 BITKOM& NIK - Big Data Wo liegen die Chancen für den Mittelstand?
2 The Big Data Buzz
3 big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. (Wikipedia)
4 Apache Hadoop the leading big data analytics technology won the Media Guardian Innovator of the Year Award
5 Big Data Analytics
6 Is there a comprehensive strategy for Big Data in your company? Please estimate the yearly growth of data for reporting and analysis No 63% % 66% 19% 7% Planned 23% % 54% 35% 8% % 48% 36% 13% Yes 14% 0% 20% 40% 60% 80% 0% 20% 40% 60% 80% 100% Negative growth / No growth 1-25% growth 25%-50% growth Over 50% growth Survey with 274 participants from DACH, France, Nordics, Netherlands,
7 What problems have you encountered when using Big data? In which areas does your company use Big data analysis? Controlling 24% Inadequate technical know-how 46% Marketing 19% Inadequate analytical know-how 44% Sales 18% Lack of compelling business case 36% IT 18% Technical problems 34% Production 17% Cost 33% Research and development 14% Data privacy issues 25% Supply Chain 7% Can not make Big data usable for end-users 15% 0% 20% 40% 0% 20% 40% 60%
8 The database journey continuous: Big Data
9 Scale-Up (SMP) Scale-out (MPP) Up to 256 Cores in Windows today Parallel Data Warehouse
10 but is this already Big Data?
13 The large hadron collider produces 15 PB/year*
14 But what if my customer doesn t own a large hadron collider
15 Large scale plants Vehicle fleets Smart Grids Green Energy Stock Exchanges Host Protocols Computer Centers Web Farms Twitter Facebook Google Analytics
16 Source: The Importance of 'Big Data': A Definition, Mark Beyer, Douglas Laney, G "Big data" is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.
18 200 Mio Feeds 100 PB 20 PB
19 Social analytics different sources data structures sophisticated algorithms
21 Up to 75 control units in one car Up to possible special equipments About 15 GB data on board (incl. navigational data) Up to stores for onboard diagnosis More than car diagnoses happening each day
22 How to deal with the 3 Vs?
27 The Hadoop Ecosystem (simplified) Quelle: Tom White s Hadoop: The Definitive Guide
28 Scalable machine learning library that leverages the Hadoop infrastructure Key use cases: Recommendation mining Examine user behavior, build recommendation model Clustering Grouping data into related topics Classification Learn from classified documents to assign categories to unlabeled data Algorithmns: K-means Clustering, Naïve Bayes, Decision Tree, Neural network, Hierarchical Clustering, Positive Matrix Factorization and more
34 180 PB raw data in > computers (polystructured)* Biggest Hadoop cluster: nodes (2x4 CPUs, 4x1 TB disks, 16 GB RAM) Ad Impressions: Cube with 207 Measures 24 Dimensions 247 Attributes Desktop Clients (Excel & Tableau): < 6s ad hoc query time
35 Query engine for SQL & Hadoop Cost base optimizer. Decides on: Rendering operators in Map/Reduce-Jobs or Moving HDFS data into RDBMS storage HDFS-Bridge for parallelized Data Transport Regular T-SQL Results PDW V2 & HDFS Data Nodes
36 maturing Not every problem questions simple
37 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
OPEN DATA CENTER ALLIANCE : sm Big Data Consumer Guide SM Table of Contents Legal Notice...3 Executive Summary...4 Introduction...5 Objective...5 Big Data 101...5 Defining Big Data...5 Big Data Evolution...7
white paper Boosting Retail Revenue and Efficiency with Big Data Analytics A Simplified, Automated Approach to Big Data Applications: StackIQ Enterprise Data Management and Monitoring Abstract Contents
COULD VS. SHOULD: BALANCING BIG DATA AND ANALYTICS TECHNOLOGY The business world is abuzz with the potential of data. In fact, most businesses have so much data that it is difficult for them to process
Big Data and Advanced Analytics Technologies and Use Cases" Colin White President, BI Research DAMA Portland February 2013" Agenda There is considerable interest at present on the topic of big data. Much
1 Contents Introduction. 1 View Point Phil Shelley, CTO, Sears Holdings Making it Real Industry Use Cases Retail Extreme Personalization. 6 Airlines Smart Pricing. 9 Auto Warranty and Insurance Efficiency.
Infosys Labs Briefings VOL 11 NO 1 2013 Big Data: Testing Approach to Overcome Quality Challenges By Mahesh Gudipati, Shanthi Rao, Naju D. Mohan and Naveen Kumar Gajja Validate data quality by employing
White Paper Big Data Analytics Pilot, Internet of Things Optimizing with the Internet of Things Intel manufacturing advances operational efficiencies and boosts the bottom line with an IoT and big data
1 Modern Data Architecture for Retail with Apache Hadoop on Windows A Hortonworks and Microsoft White Paper JUNE 2014 2 Executive Summary Retailers have a long history of investing in data and analytics
Big Data Analytics and Optimization C e r t i f i c a t e P r o g r a m i n E n g i n e e r i n g E x c e l l e n c e e.edu.in http://www.insof LIST OF COURSES Essential Business Skills for a Data Scientist...
QLIKVIEW AND BIG DATA: HAVE IT YOUR WAY A QlikView White Paper November 2012 qlikview.com Table of Contents Executive Summary 3 Introduction 3 The Two Sides of Big Data Analytics 3 How Big Data Flows from
Solution Brief Big Data in the Cloud: Converging Technologies How to Create Competitive Advantage Using Cloud-Based Big Data Analytics Why You Should Read This Document This paper describes how cloud and
An Oracle White Paper June 2013 Oracle: Big Data for the Enterprise Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure
For: Application Development & Delivery Professionals The Forrester Wave : Big Data Hadoop Solutions, Q1 2014 by Mike Gualtieri and Noel Yuhanna, February 27, 2014 Key Takeaways Hadoop s Momentum Is Unstoppable
May 2014, HAPPIEST MINDS TECHNOLOGIES Big Data: Why should enterprises adopt it Author Manish Kumar 1 S HARING. MINDFUL. INTEGRITY. LEARNING. EXCELLENCE. SOCIAL RESPONSIBILITY. Copyright Information This
INTELLIGENT BUSINESS STRATEGIES W H I T E P A P E R Architecting A Big Data Platform for Analytics By Mike Ferguson Intelligent Business Strategies October 2012 Prepared for: Table of Contents Introduction...
IBM Software Thought Leadership White Paper June 2013 The top five ways to get started with big data 2 The top five ways to get started with big data Big data: A high-stakes opportunity Remember what life
Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments Important Notice 2010-2015 Cloudera, Inc. All rights reserved. Cloudera, the Cloudera logo, Cloudera Impala, Impala, and
Database Systems Journal vol. IV, no. 3/2013 31 Big Data Challenges Alexandru Adrian TOLE Romanian American University, Bucharest, Romania email@example.com The amount of data that is traveling across
Securing Your Big Data Environment Ajit Gaddam firstname.lastname@example.org Abstract Security and privacy issues are magnified by the volume, variety, and velocity of Big Data. The diversity of data sources, formats,
White Paper BIG DATA-AS-A-SERVICE What Big Data is about What service providers can do with Big Data What EMC can do to help EMC Solutions Group Abstract This white paper looks at what service providers
View Point Use of Big Data Technologies in Capital Markets - Ruchi Verma, Sathyan R Mani Abstract Data is growing at a tremendous rate with an increase in digital universe from 281 Exabyte s (year 2007)
Big Data: Challenges and Opportunities Roberto V. Zicari Goethe University Frankfurt This is Big Data. Every day, 2.5 quintillion bytes of data are created. This data comes from digital pictures, videos,