1 Microsoft Big Data Solutions Anar Taghiyev P-TSP
2 Why/What is Big Data and Why Microsoft? Options of storage and big data processing in Microsoft Azure. Real Impact of Big Data
3 Why/What is Big Data?
4 Data is rapidly changing GROWING VOLUME (Large) Paper conversion to digital form, more information is being gathered, and greater detail of existing information is being captured. MORE VARIETY (Unstructured) Characterized by text based documents, image, audio, and video files, new sources of data and a lack of standard data models. RAPID VELOCITY (Complex) Fueled by real-time data collection, constantly changing information, and the growth of social networks.
6 Storage Reliable, economical cloud storage for data big and small Highly scalable Azure Storage keeps up with your growing data needs, with up to 500 TB of total storage per account. A single subscription supports up to 50 storage accounts, delivering petabytes of storage for the largest scenarios. Whether you're building a consumer site or a terabyte-scale big data application, Azure is designed to handle it. Durable & highly available Azure storage automatically replicates your data to help guard against unexpected hardware failures and make sure it's available when you need it. We keep 3 copies within a single region. A georedundancy option creates 3 additional copies hundreds of miles away for higher availability and disaster recovery. Designed for developers Build apps with first class client library support for.net, Java, Android, C++, and Node.js. Data in Azure Storage is also accessible via REST API, which can be called by any language that makes HTTP/HTTPS requests. Azure Storage includes strong consistency guarantees, simplifying cloud application development and enabling predictable experiences for applications built on Azure. Cost effective Pay only for what you use, at a cost lower than onpremises storage options. Our competitive price matching policy also means you're always getting a great deal on cloud storage. 1. Manage petabytes of storage 2. Geo-redundant storage across hundreds of miles 3. Fastest performance in the industry 4. Industry standard SMB file sharing across VMs 5. Pay for what you use with competitive pricing 6. REST,.NET, Java, C++, node.js, PowerShell and more
7 Blobs, Tables, Queues, and Files Azure Storage provides the flexibility to store and retrieve large amounts of unstructured data, such as documents and media files with Azure Blobs; structured nosql based data with Azure Tables; reliable messages with Azure Queues, and use SMB based Azure Files for migrating on-premises applications to the cloud. Table Storage A NoSQL key-value store for rapid development using massive semi-structured datasets Key scenarios: 1.Store semi-structured data that s highly available 2.Create massively scalable apps 3.Create apps that require a flexible data schema 4.Use JSON to serialize data 5.Perform OData-based queries Queue Storage Blob Storage Reliable, cost-effective cloud storage for large amounts of unstructured data Key scenarios: 1.Store petabytes of highly available data 2.Power big data analytics 3.Serve content to web or mobile applications 4.Perform secure backup and disaster recovery 5.Stream video and audio Reliable messaging for scenarios including workflow processing or communication between application components Key scenarios: 1.Effectively scale apps according to traffic 2.Create apps that can handle traffic bursts 3.Build more flexible applications Built for big data When you do big data analysis, you ve got to put all that data somewhere. The Azure Blob service offers several benefits over the traditional Hadoop Distributed File System (HDFS) for big data scenarios including geo-replication, lower storage costs, easier scalability, and data sharing outside of the Hadoop cluster. Use your own on-premises service to analyze blob-stored data, or Azure services including Batch, Machine Learning, and HDInsight (Hadoop). File Storage Shared storage for applications using the standard SMB protocol Key scenarios: 1.Move existing applications to the cloud 2.Use cloud storage with no code changes 3.Share tools and logs between VMs
8 Big Data Reveal new insights and drive better decision making with Azure HDInsight, a Big Data solution powered by Apache Hadoop. Surface those insights from all types of data to business users through Microsoft Excel. HDInsight Our 100% Apache Hadoop-based service in the cloud 1. Scale to petabytes on demand 2. Process unstructured and semi-structured data 3. Develop in Java,.NET, and more 4. No hardware to buy or maintain 5. Deploy in Windows or Linux 6. Spin up a Hadoop cluster in minutes 7. Visualize your Hadoop data in Excel 8. Easily integrate on-premises Hadoop clusters Machine Learning Powerful cloud-based predictive analytics 1. Model your way 2. Deploy in minutes 3. Expand your reach Azure Search Search-as-a-service for web and mobile app development 1. Get your search indices up and running quickly 2. Easily scale up and down as needed 3. Connect search results to business goals with great control over search ranking 4. Leverage Microsoft s deep knowledge of natural language processing 5. Load and update automatically with integrated indexers 6. Easily add geo-spatial search to your app Azure Backup Simple and reliable cloud integrated backup 1. Unified solution for protecting data on premise and in the cloud % availability guaranteed 3. Reliable Offsite backup target 4. Efficient incremental backups 5. Secure, encrypted in-transit and at-rest 6. Geo-replicated backup store Azure CDN A fast and modern global delivery network for highbandwidth content 1. Lower latencies 2. Massively scalable 3. Designed for today s web 4. Improved availability and performance 5. Capacity on demand 6. Robust security
9 Machine learning today Complexities in a nascent market Huge set-up costs create unnecessary barriers to entry Cumbersome data management restricts access to data Complex and fragmented tools limit participation in exploring data and building predictive models Break away from industry limitations Many models never achieve business value due to difficulties with deploying to production
10 Applications Reports Dashboards Natural language query Mobile The Microsoft Orchestration Information management Complex event processing Modeling Machine learning data platform Relational Non-relational NoSQL Streaming Internal & external
11 Business Intelligence in Corporate BI enterprise datawarehouse and standard reporting Self-Service BI provide your business analysts and end-users more capabilities and freedom Advanced Analytics predictive analytics to recognize patterns in data
12 Combine various types of data... Organizational Data Research Data Students Data Individual's Data Community Data World Data
13 What can Big Data do for Education and Research? Social Analytics Associative Analytics Assessment and Prediction Improved Research Collating and Analyzing Data Social Network Awareness Employment Analysis Cross Institutional Learning
14 CITY OF BARCELONA
15 Citizen Dashboard: City by the Numbers https://dashboard.edmonton.ca/
17 LEARN MORE Learn more about we Microsoft Big Data solution and how to get started at https://www.microsoft.com/en-us/server-cloud/solutions/big-data.aspx Learn more about Microsoft Azure at Learn more about Microsoft CityNext at https://www.microsoft.com/en-us/citynext/default.aspx Learn more about Power BI at
David Chappell October 2012 WINDOWS AZURE DATA MANAGEMENT CHOOSING THE RIGHT TECHNOLOGY Sponsored by Microsoft Corporation Copyright 2012 Chappell & Associates Contents Windows Azure Data Management: A
Solution Brief Big Data in the Cloud: Converging Technologies How to Create Competitive Advantage Using Cloud-Based Big Data Analytics Why You Should Read This Document This paper describes how cloud and
April 2013 Operational Intelligence: What It Is and Why You Need It Now Sponsored by Splunk Contents Introduction 1 What Is Operational Intelligence? 1 Trends Driving the Need for Operational Intelligence
An Oracle White Paper June 2013 Oracle: Big Data for the Enterprise Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure
MASARYK UNIVERSITY FACULTY OF INFORMATICS Best Practices in Scalable Web Development MASTER THESIS Martin Novák May, 2014 Brno, Czech Republic Declaration Hereby I declare that this paper is my original
Cloud optimize your business Windows Server 2012 R2 Published: October 7, 2013 Contents 1 Trends 3 Windows Server: cloud optimize your business 5 Windows Server 2012 R2 capability overview 5 Server virtualization
For Big Data Analytics There s No Such Thing as Too Big The Compelling Economics and Technology of Big Data Computing March 2012 By: 4syth.com Emerging big data thought leaders Forsyth Communications 2012.
E-PAPER March 2014 Big Data & the Cloud: The Sum Is Greater Than the Parts Learn how to accelerate your move to the cloud and use big data to discover new hidden value for your business and your users.
SAP BusinessObjects Business Intelligence SAP BusinessObjects Business Intelligence 4.0 Solutions Empowering the Real-Time, Mobile, Social, and Global Enterprise SAP BusinessObjects Business Intelligence
A Forrester Consulting Thought Leadership Paper Commissioned By SAP Real-Time Data Management Delivers Faster Insights, Extreme Transaction Processing, And Competitive Advantage June 2013 Table Of Contents
Plug Into The Cloud with Oracle Database 12c ORACLE WHITE PAPER DECEMBER 2014 Disclaimer The following is intended to outline our general product direction. It is intended for information purposes only,
An Oracle White Paper March 2013 Big Data Analytics Advanced Analytics in Oracle Database Advanced Analytics in Oracle Database Disclaimer The following is intended to outline our general product direction.
A Fresh Graduate s Guide to Software Development Tools and Technologies Chapter 1 Cloud Computing CHAPTER AUTHORS Wong Tsz Lai Hoang Trancong Steven Goh PREVIOUS CONTRIBUTORS: Boa Ho Man; Goh Hao Yu Gerald;
SAP Statement of Direction Business Intelligence Solutions Business Intelligence Solutions from SAP: Statement of Direction Table of Contents 3 Quick Facts 4 Driving Business Innovation Through Radical
IBM Software Thought Leadership White Paper June 2013 The top five ways to get started with big data 2 The top five ways to get started with big data Big data: A high-stakes opportunity Remember what life
WHITE PAPER Embracing Enterprise Hybrid Cloud Storage Sponsored by: Microsoft Ashish Nadkarni Liz Conner November 2014 Laura DuBois IDC OPINION Many businesses are embracing elements that make up the IDC
Big Data at Cloud Scale Pushing the limits of flexible & powerful analytics Copyright 2015 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For
white paper Boosting Retail Revenue and Efficiency with Big Data Analytics A Simplified, Automated Approach to Big Data Applications: StackIQ Enterprise Data Management and Monitoring Abstract Contents
Convergence of Social, Mobile and Cloud: 7 Steps to Ensure Success June, 2013 Contents Executive Overview...4 Business Innovation & Transformation...5 Roadmap for Social, Mobile and Cloud Solutions...7
QLIKVIEW AND BIG DATA A QlikView Technology White Paper July 2012 victa.nl email@example.com +31 74 2915208 Introduction There is an incredible amount of interest in the topic of Big Data at present: for many
IBM Software Big Data Retail Capitalizing on the power of big data for retail Adopt new approaches to keep customers engaged, maintain a competitive edge and maximize profitability 2 Capitalizing on the
Brochure Maximize storage efficiency across the enterprise HP Data Protector software and HP StoreOnce backup systems for federated deduplication and flexible deployment Maximize storage efficiency across