DBI 312 Microsoft Big Data 解決方案與案例分享 Rich Ho Technical Architect 微軟技術中心
Agenda What is Big Data? Microsoft Big Data Strategy Key Benefits of Microsoft Big Data Demo Case Study
What is Big Data?
The world of Data is changing 10x increase every five years Data explosion 85% from new data types Volume Velocity Variety Hadoop Easy Accessibility of External Data Cheap, Distributed Storage & Processing Cloud By 2015, organizations that build a modern information management system will outperform their peers financially by 20 percent. Gartner, Mark Beyer, Information Management in the 21st Century
Three V in Big Data 10x increase every 5 4.3 connected VOLUME VELOCITY years devices per adult Relational Data VARIETY 85% from new data types Big Data
We are moving to Big Data Terabytes Data Volume Petabytes+ Structured Data Variety Unstructured Batch Data Velocity Streaming ----- Garbage? +++++ Traditional Data Big Data Any Data, Any Size, Anywhere
With Big Data, says: 以前我們將尿布和啤酒 放在一起而創造新業績 現在我們將比老爸更早 一步知道他女兒懷孕了!
Traditional data flow
New exploratory data flow
What is Big Data? Increasing Data Volumes Increasing Data & Analysis Complexing Emerging Technologies
Microsoft Big Data Strategy
Microsoft Big Data Stacks SELF-SERVICE INSIGHTS MOBILE REAL-TIME PREDICTIVE COLLABORATIVE DATA ENRICHMENT DISCOVER AND RECOMMEND TRANSFORM AND CLEAN SHARE AND GOVERN DATA MANAGEMENT 1 011 01 RELATIONAL NON-RELATIONAL MULTIDIMENSIONAL STREAMING MARKETPLACE External Data and Services OPERATIONAL
Enterprise Big Data Blueprint Big Data Sources (Raw, Unstructured) Data & Compute Intensive Application Business Insights Data Marts Sensors Load Summarize & Load Devices Fast Reporting Distributed environments for Non-structured data Bots Enterprise Data Warehouse Historical Data (Beyond Active Window) Interactive Reports Integrate/Enrich Multi-dimensional Analysis Crawlers Performance Scorecards Enterprise ETL with SSIS, DQS, MDS ERP CRM LOB Source Systems APPS Online Transaction
Hadoop on Windows Hadoop on Azure
Microsoft Big Data Platform Big Data Sources (Raw, Unstructured) Data & Compute Intensive Application Business Insights Data Marts Sensors Load Summarize & Load Devices Fast Reporting Hadoop on Windows Azure Bots Hadoop on Windows Server Enterprise Data Warehouse Historical Data (Beyond Active Window) Interactive Reports Integrate/Enrich Multi-dimensional Analysis Crawlers Performance Scorecards Enterprise ETL with SSIS, DQS, MDS ERP CRM LOB Source Systems APPS Online Transaction
Microsoft Big Data Platform SQL Server PDW Big Data Sources (Raw, Unstructured) SQL Server FT Data & Compute Intensive Application Business Insights Data Marts Sensors Load Summarize & Load Devices Fast Reporting Hadoop on Windows Azure Bots Hadoop on Windows Server Enterprise Data Warehouse Historical Data (Beyond Active Window) Interactive Reports Integrate/Enrich Multi-dimensional Analysis Crawlers Performance Scorecards Enterprise ETL with SSIS, DQS, MDS SQL Server EE ERP CRM LOB Source Systems APPS Online Transaction
Microsoft Big Data on Public Cloud Hadoop on Windows Azure Big Data Sources (Raw, Unstructured) SQL Azure Data & Compute Intensive Application Business Insights Data Marts Sensors Load Summarize & Load Devices Fast Reporting Hadoop on Windows Server Bots Enterprise Data Warehouse Historical Data (Beyond Active Window) Interactive Reports Integrate/Enrich Multi-dimensional Analysis Crawlers Performance Scorecards Enterprise ETL with SSIS, DQS, MDS ERP CRM LOB Source Systems APPS Online Transaction
Hadoop on Azure (CTP)
Request a New Cluster (4 nodes, 2 TB)
Allocating NN & DN
Starting NN & DN
NN & DN ready! (after 9 mins)
Hadoop on Azure ready! (after 13 min s)
Access Hadoop on Azure through RDP
Key Benefits of Microsoft Big Data
Microsoft Big Data Benefits Breakthrough Insights Broader Access of Hadoop Enterprise Ready Analyze Big Data with familiar tools (Excel, PowerPivot, Power View) New Hadoop-based distribution on Windows / Azure System Center Integration JavaScript based simple programming Hive ODBC driver Hive add-in for Excel Active Directory Integration
Breakthrough Insights Benefits Breakthrough Insights Key Features Hive ODBC Driver integrates Hadoop to SQL Server Analysis Ser vices, PowerPivot, and Power View
Broader Access of Hadoop Benefits Broader Access of Hadoop Key Features Hive add-in for Excel Hive ODBC driver
Enterprise Ready (SCOM Integration) Benefits Enterprise Ready Key Features System Center to monitor performances of Head nodes and Data nodes
1. Hadoop on Azure - JavaScript Console - Job Execution 2. Excel Hive Connectivity 3. Windows Marketplace & Hadoop on Azure Integration
Case Study: Mining Big Mobility Data
Microsoft Big Data Case Study: T-Drive
Microsoft Big Data Case Study: T-Drive
Video Microsoft Research Asia Big Data Case Study
Thank You
Resources Connect. Share. Discusss http://www.microsoft.com/taiwan/techdays2012/ Microsoft Certification & Training Resources http://www.microsoft.com/learning/zh/tw/ Resources for IT Professionals Resources for Developers http://social.msdn.microsoft.com/forums/zh-tw/categories http://social.technet.microsoft.com/forums/zh-tw/categories / /
請協助完成 本課程問卷 並在離開 教室時交給工作人員 填妥大會背包中的大會問卷 可於活動 第三天兌換問卷禮哦 感謝您的合作