Hybrid Cloud Architectures for Operational Performance Management Delbert Murphy Solution Architect / Data Scientist Microsoft Corporation GPDIS_2014.ppt 1
Delbert Murphy and Microsoft s Data Insights CoE Technologies Areas of Expertise Microsoft Azure SQL Server HDI Insight Azure Machine Learning Customer Analytics Operational Intelligence Analytics for Finance Power BI GPDIS_2014.ppt 2
Quick census? Engineering Information Technology Other Design Manufacturing Architect Data Scientist DBA Developer Manager GPDIS_2014.ppt 3
The NIST cloud definition framework Deployment Models Private Cloud Hybrid Clouds Hosted Cloud Public Cloud SaaS Service Models PaaS Essential Characteristics IaaS On-Demand Self-Service Broad Network Access Resource Pooling Rapid Elasticity Measured Service Common Characteristics Massive Scale Homogeneity Virtualization Low-Cost Software Resilient Computing Geographic Distribution Service Orientation Advanced Security GPDIS_2014.ppt 4
Event volume GPDIS_2014.ppt 5
Event velocity GPDIS_2014.ppt 6
Event source variety 1B 2.5B >5B >10B Connected/ Smart TVs Personal computers Smartphones and tablets Connected internet of things Revenue engines Content Applications Apps and services Services by verticals Source: Gartner, IDC, Strategy Analytics, Machina Research, Company filings, BI Intelligence, Accenture analysis Copyright 2013 Microsoft and Accenture GPDIS_2014.ppt 7
Event veracity ve rac i ty 1. Devotion to the truth: truthfulness 2. Power of conveying or perceiving the truth 3. Conformity with truth or fact: accuracy Record the confidence a source system has in the event value Use a large enough N to confident in the value GPDIS_2014.ppt 8
Event impact Cars Traffic Trains Vessels Letters Packages Smart Logistics Containers Tanks Aircraft Smart Mobility Trucks Buses Bikes Bulkware Remote Servicing Predictive and Reactive Maintenance Events Sports Smart Factory Manufacturing Integration and Automation Games Streaming Smart Entertain -ment Television Pollution Control Water Waste Fire Smart Cities Security Patients Smart Healthcare Clinics Hospitals Emergency Public Safety Law Enforcement Nursing Homes Safety Grid Automation Smart Building Lighting Comfort / Home Mobile Care Smart Energy Renewables Hotels Restaurants Oil/Gas/Coal Recovery and Distribution Smart Retail Fuel Stations Points of Sale GPDIS_2014.ppt 9
Scenarios for operational performance management Predictive Maintenance Avoid costly asset downtime and reduce maintenance costs Reduce warranty claims due to unexpected failures Improve inventory management of spare parts by predicting failures Performance Management Improve Production Quality Assurance and Yield Event and Incident Management and Monitoring Better protect Health, Safety and Environment Improve Situational Awareness, Security and Tracking Route and Capacity optimization Optimize Power grids and networks Optimize Traffic and Goods movement GPDIS_2014.ppt 10
Process Design Emit Transport Ingest Consume Insights Business Objectives KPI s Information requirements Exhaust data Communication Store-and-Forward? Fast and scalable Exception alerting Query predictive analytics models Business Intelligence Train predictive analytics models Visual Studio Visual Studio (device dependent) ISS Event hug ISS Reykjavik Azure NRT ISS Reykjavik Azure NRT Azure ML PowerBI HDInsight Azure ML GPDIS_2014.ppt 11
GPDIS_2014.ppt 12
Device communication patterns GPDIS_2014.ppt 13
Device communication styles GPDIS_2014.ppt 14
Device communication scope GPDIS_2014.ppt 15
Selecting the right components Buy Focus on Business Result and Time To Value Buy Some, Build Some Mix and Match Custom protocols & security models Custom analytics and data processing Build Core Competency in building software & Services Focus on flexibility & control Strategic investment in service platform GPDIS_2014.ppt 16
Azure ISS, no custom code ISS Service Device Registry ISS Device Operator Portal Device Mgmt Services Data Services Limited experience in software and service development Service Bus (Event Hub) Event Processing Azure Storage Very sensitive to time to market and budget ISS Agent ISS Agent ISS Agent Device Custom Code Reference Component Connect remote kiosks to ISS via Windows agents for basic device management Kiosk... Kiosk ISS Component Azure Platform Service Scenarios and Workloads: Entire System Management Station Management Mobile Maintenance Worker Tasking and Support GPDIS_2014.ppt 17
Custom devices & portal on Azure ISS ISS Agent Windows Gateway PLC Maintenance UX on Mobile Device ISS Service Service Bus (Event Hub) Escalator Elevator Powerlines Custom Portal Device Registry Event Processing Azure Storage Device Mgmt API Device Mgmt Services Data API (OData) Data Services Azure Active Directory Device Custom Code Reference Component ISS Component Azure Platform Service Depth and experience in software and service development Very sensitive to time to market (more buy than build) Connect sensors to ISS agents, push data to ISS Command and control from ISS Other Azure services used - AAD Scenarios and Workloads: Entire System Management Station Management Mobile Maintenance Worker Tasking and Support GPDIS_2014.ppt 18
Custom protocol, to cloud gateway to ISS Device Mgmt API Data API (OData) Custom Code ISS Component ISS Agent Service Assisted Connectivity Protocols Schemas Security Custom Device (no agent RTOS, etc) Reference Component Azure Platform Service ISS Service Service Bus (Event Hub) Protocols Schemas Security Custom Protocol Gateway (Reykvavik) Device Registry Event Processing Service Bus (Event Hub) Message Pump Device Mgmt Services Azure Storage Storm Data Services Azure Storage HDInsight SQL DB Limited depth and experience in software development Good fit for turn-key service platform Front end connectivity and data flow with Reykjavik Flow data into ISS for processing, device management ISS publishes data to customer storage account for batch processing ISS publishes copy of live data stream to customer Event Hub for custom processing GPDIS_2014.ppt 19
Custom protocols and Azure building blocks Device Mgmt API Data API (OData) Custom Code ISS Component ISS Agent Service Assisted Connectivity Protocols Schemas Security Custom Device (no agent RTOS, etc) Reference Component Azure Platform Service ISS Service Service Bus (Event Hub) Protocols Schemas Security Custom Protocol Gateway (Reykvavik) Device Registry Event Processing Azure Storage Service Bus (Event Hub) Device Mgmt Services AMQP AMQP Message Pump NRT Data Services Custom Code Azure Storage Worker Role SQL DB HDInsight Depth and experience in software and service development High demand for control and flexibility (more build than buy) Needs custom protocol support for extant devices Build out on Azure services using Reykvavik components to streamline delivery GPDIS_2014.ppt 20
The right strategy Start by connecting the devices you already have Utilize services and the cloud to jump-start your efforts Combine the data you already collect Generate new insights to create new business value Expand by adding new devices, new services, new data GPDIS_2014.ppt 21
Intelligent (cognitive) systems actions data Environment Text Here Intelligent Agents Super Processing Russell and Norvig: AI An Agent-Based Approach percepts algorithms GPDIS_2014.ppt 22
Lidberg s law Data born in the cloud, stays in the cloud. Data born on premised stays on premises. Simon Lidberg, Microsoft GPDIS_2014.ppt 23
Homan s law Think carefully about where you place your data: it forms a gravity well. A data storage mechanism will attract other data in direct proportion to the amount of data it currently holds. Uli Homan, Microsoft GPDIS_2014.ppt 24
Murphy s law of distributed data Don t distribute your data. Compute locally, to join globally. Delbert Murphy, Microsoft GPDIS_2014.ppt 25
Information hierarchy Wisdom Understanding Knowledge Information Data Individual Group GPDIS_2014.ppt 26
Heuristics Synthesis Heuristics Analytical GPDIS_2014.ppt 27
Questions and answers GPDIS_2014.ppt 28
Appendix GPDIS_2014.ppt 29
Microsoft s data platform stack GPDIS_2014.ppt 30
Microsoft s data warehouse stack GPDIS_2014.ppt 31
Microsoft s modern data platform GPDIS_2014.ppt 32
Microsoft data platform and types of data GPDIS_2014.ppt 33