IoT Edge Processing. Evolution of edge computing analytics and long-term data retention. JEFF KIBLER VP Tech Services, Infobright

Transcription

1 IoT Edge Processing Evolution of edge computing analytics and long-term data retention JEFF KIBLER VP Tech Services, Infobright 13 October 2015 AllSeen Alliance 1

2 Agenda 1. IoT Premise and Challenges 2. Exposing Opportunities 3. Directions to Consider 4. Moving from Possible to Practical 5. Wrap-up 13 October 2015 AllSeen Alliance 2

3 IoT Foundations Premise and Challenges 3

4 The Life of the End User Athletics Multi-billion dollar industries where 1% competitive edge decides careers. Infrastructure Increasing reliance on alternative energy, permeable surfaces, and environmental metering. Telemetry Predicting and improving health outcomes. 13 October 2015 AllSeen Alliance 4

5 Premise IoT Presents a Large Market Opportunity Leading Verticals Industrial Equipment Oil/Gas/Energy Automotive Retail / Restaurants Hospitals mhealth/telehealth Infrastructure Leading Challenges Data Security Infrastructure Privacy Governance 13 October 2015 AllSeen Alliance 5

6 Premise IoT Solutions Today are Sexy, Self-Contained Data: NoSQL: Hadoop(Cloudera, Hortoworks, MapR), Cassanndra, MongoDB Analytic: Sybase/IQ, HP Vertica, Amazon Redshift, Infobright, Pivitol Standard Relational: Postgres, MySQL, Oracle, Sybase, Microsoft : Apache Storm, TibcoStreambase, Software AG Apama, Sybase Aleri, Various Coded in (Java, Python, Ruby on Rails), TempoIQ Cloud: Amazon, Rackspace, Dimension Data, Joyent, Cisco, EMC, IBM, Microsoft Alerts, Triggers, Actions Closed Loop Message-Response System Al Cloud Based Central Repository 13 October 2015 AllSeen Alliance 6

7 Key Challenge Evolve with Simplicity Added Complexity Data Exploitation Demands Edge Processing Demands Governance, ownership Centralized Volumes Gigabytes to Terabytes Terabytes to Petabytes Petabytes to Exabytes Privacy 13 October 2015 AllSeen Alliance 7

8 Deliver an IoTPlatform that contemplates enormous sophistication and complexity in a delivery model that is intuitive, accessible, and affordable. 8

9 IoT Foundations Exposing Opportunities 9

10 Gaps Exposing the Opportunity Major Considerations Data How/where to leverage utility value of data Edge Processing Drivers behind and rationale of edge processing both physical and/or virtual Architecture Meeting market requirements over time; getting it right today IoT World Forum Reference Architecture 13 October 2015 AllSeen Alliance 10

11 Many will overkill to address the gaps. The result will be sophisticated yet hardly elegant solutions. 11

12 Viewpoints: Now and Future Vendors and Users Sample Industry Viewpoints Current Equipment Vendor Drivers Current IoTUser Drivers VendorAssumptions about the Future User Assumptions about the Future Industrial Equipment (LutronLighting / Glidden Paints) Better product, higher margins, differentiation, stickiness Higher uptime; easier servicing when needed; better results Control of the silo,data, and devices. Customers will want the value add of accessing the data Devicessupplied by multiple vendors will work together and the data can be leveraged Oil & Gas (FMC / Chevron) Better products; safer products; proactive servicing Safety; efficiency; visibility; uptime; compliance readiness Gain product insight and control devices; value added services Integrated IoT devices; holistic view from rig level up Automotive (Ford / you) Better product info; maintainability; increased margins; more competitive Ease of use; comfort, safety; entertainment Increasingly autonomous; changing models; compliance Fully integrated experience, car as device including data; insurance; ownership Retail& Restaurants (Viking Commercial / McDonald s) Tracking (Beacons); better equipment maintenance; higher uptime; customer stickiness Higher yields per customer; better operational information;better uptime; greater sales Greater level of integration required; anonymization requirements Leverage various IoT silos to create operational efficiencies and greater profits Hospitals (LutronLighting / Mercy Health St. Louis) Higher uptime;greater efficiencies; enhanced supply chain Less shrinkage; better compliance; greater visibility Silocontrol with regulatory oversight; Integrated product suites Exceptional level of integration of patient data and resources; operational efficiency mhealth/ Telehealth (New England BioLabs/ McDonald s) musthave devices for consumers; highly cost effective monitoring solutions Health maintenance; physician accessibility; reduced costs; better outcomes Lower costdelivery; shrinking footprint becoming invisible; Lower energy; multi-point; integration Greater exposure to data; integration with home systems; non-intrusive; lifestyle insights Smart City Infrastructure (Siemens / City of Chicago) Specific silos (lighting/rubbish/streets. Increased efficiency and reduced cost of servicedelivery for various silos Increasing footprint and product suite offerings; Mega vendor based service led engagements Coordination and orchestration of holistic data; lower cost and better service delivery through analytics 13 October 2015 AllSeen Alliance 12

13 Viewpoints: Now and Future Vendors and Users Sample Industry Viewpoints Current Equipment Vendor Drivers Current IoTUser Drivers VendorAssumptions about the Future User Assumptions about the Future Industrial Equipment (LutronLighting / Glidden Paints) Better product, higher margins, differentiation, stickiness Higher uptime; easier servicing when needed; better results Control of the silo,data, and devices. Customers will want the value add of accessing the data Devicessupplied by multiple vendors will work together and the data can be leveraged Oil & Gas (FMC / Chevron) Better products; safer products; proactive servicing Safety; efficiency; visibility; uptime; compliance readiness Gain product insight and control devices; value added services Integrated IoT devices; holistic view from rig level up Automotive (Ford / you) Retail& Restaurants (Viking Commercial / McDonald s) Hospitals (LutronLighting / Mercy Health St. Louis) Better product info; maintainability; increased margins; more competitive Connected Products Tracking (Beacons); better equipment maintenance; higher uptime; customer stickiness Higher uptime;greater efficiencies; enhanced supply chain Ease of use; comfort, safety; entertainment Higher yields per customer; better operational information;better uptime; greater sales Less shrinkage; better compliance; greater visibility Increasingly autonomous; changing models; compliance System of Systems Greater level of integration required; anonymization requirements Silocontrol with regulatory oversight; Integrated product suites Fully integrated experience, car as device including data; insurance; ownership Leverage various IoT silos to create operational efficiencies and greater profits Exceptional level of integration of patient data and resources; operational efficiency mhealth/ Telehealth (New England BioLabs/ McDonald s) musthave devices for consumers; highly cost effective monitoring solutions Health maintenance; physician accessibility; reduced costs; better outcomes Lower costdelivery; shrinking footprint becoming invisible; Lower energy; multi-point; integration Greater exposure to data; integration with home systems; non-intrusive; lifestyle insights Smart City Infrastructure (Siemens / City of Chicago) Specific silos (lighting/rubbish/streets. Increased efficiency and reduced cost of servicedelivery for various silos Increasing footprint and product suite offerings; Mega vendor based service led engagements Coordination and orchestration of holistic data; lower cost and better service delivery through analytics 13 October 2015 AllSeen Alliance 13

14 Deliver an IoTPlatform that accommodates evolving user needs with minimal user requirements. 14

15 Lens of a Vendor Considerations Need Product that performs and adaptable Data Characterization Framing view of product by data Data Use Predict and Evolve Product Constituencies Understand User Segmentation Outlook Integration into larger system of systems Ownership Retain rights to Data Stewardship Data Access by users Management Controlling devices in the field Drivers Decreased downtime, increased utilization and visibility, Upsell 13 October 2015 AllSeen Alliance 15

16 Lens of a User Considerations Need Operate efficiently to gain better insight / make better decisions Data Characterization Products and Services Data Use Holistic understanding on data breadth / avoid silos Constituencies Organization or consumer including various silo systems Ownership Own the data Stewardship Determine user permission Management Product companies manage assets Drivers Cost savings, enhanced outcomes, increased revenue Outlook Move from Silo to greater system 13 October 2015 AllSeen Alliance 16

17 IoT Foundations Directions to Consider 17

18 IoTDirection Pushing Suppliers for more Robust Analytic Stack Alerts, Triggers, Actions Closed Loop Message-Response System Al Cloud Based Central Repository Analytic Workbench: Operational, Investigative, Predictive Analytics and Machine Learning Enterprise Apps: ERP, CRM, and other enterprise apps Possible Specialized Store 13 October 2015 AllSeen Alliance 18

19 IoTDirection to the Edge Increase in Edge Processing for filtering and increased capabilities Analytic Workbench: Operational, Investigative, Predictive Analytics and Machine Learning Alerts, Triggers, Actions Closed Loop Message-Response System Cloud Based Central Repository Edge Processor Apply rules and workflow against that data Take action as needed Filter and cleanse the data exhaust (increasing payload) Store local data for local use Enhance security Provide governance admin controls Possible Specialized Store Enterprise Apps: ERP, CRM, and other enterprise apps 13 October 2015 AllSeen Alliance 19

20 Edge Processing Assumptions Limited or no human resources for maintaining the database or other system capabilities at the edge must be a hands off operation, with remote monitoring or control only Hardware footprint will be limited Not all use cases apply but many do Factories Retail/Restaurants Homes (but with far less data) Buildings Many aspects of smart cities Hospitals (but only marginally for personal health) Cars (Edge on board) Other transportation modalities (especially planes,, trains and ships) Oil and Gas 13 October 2015 AllSeen Alliance 20

21 Architectural Considerations Closed Loop Message-Response System (& Filtering) Analytic Workbench: Operational, Investigative, Predictive Edge Processor Persisted Store ERP, CRM, etc. Publish Subscribe Analytic Workbench Enterprise Apps Cloud-Based Central Cloud-Based Repository Central Cloud-Based Repository Central Repository Vendor Corporate ( Lutron Lighting ) One of multiple vendor silos Analytic Workbench Cloud-Based Central Repository Enterprise Apps User Corporate ( McDonald s Head Office ) Analytic Workbench Various Sensor Devices External Data User Remote Site ( McDonald s/south Boston ) First Receiver Cloud-Based Central Repository Government ( USDA ) Enterprise Apps 13 October 2015 AllSeen Alliance 21

22 Local Back End Data Provisioning Closed Loop Message-Response System Analytic Workbench Enterprise Apps Cloud-Based Central Repository Analytic Workbench: Operational Investigative Predictive ERP, CRM, etc. Various Sensor Devices & Silos Vendor Corporate ( Lutron Lighting ) Analytic Workbench Enterprise Apps Cloud-Based Central Repository Vendor Corporate ( Bosch Appliances ) Analytic Workbench Enterprise Apps Cloud-Based Central Repository Vendor Corporate ( Honeywell HVAC ) (& Filtering) McDonald s Persisted Store External Data First Receiver 13 October 2015 AllSeen Alliance 22

23 Data beyond a certain scale becomes impossible to accommodate and use without vast infrastructure and excessive administration. 23

24 Small Edge Node Volumes Today Alerts, Triggers, Actions Closed Loop Message-Response System Al Cloud Based Central Repository Analytic Workbench: Operational, Investigative, Predictive Analytics and Machine Learning Possible Specialized Store Enterprise Apps: ERP, CRM, and other enterprise apps Most data volumes today and in the near future are exceptionally low by some standards (like Telco and Networking) The key will be to provide the underpinnings to service the full analytic stack and feed enterprise applications Hotel Example Deployed: 100,000 Avg. Message Interval: 5 seconds Exhaust Rate: 100 Avg. Message size: 3kb Data Retention Period: 30 days Required Message Flow Capacity: 2.16M Messages/Hr Required Storage: 2.59 TB 13 October 2015 AllSeen Alliance 24

25 Future Unknown Edge Node Volumes Closed Loop Message-Response System ERP, CRM, etc. Analytic Workbench: Operational, Investigative, Predictive Analytic Workbench Cloud-Based Central Repository Enterprise Apps (& Filtering) Edge Processor Persisted Store Publish Subscribe Vendor Corporate User Corporate Analytic Workbench Cloud-Based Central Repository Analytic Workbench Enterprise Apps External Data Cloud-Based Central Repository Enterprise Apps Various Sensor Devices First Receiver Third Party as needed The combination of many silos with greater reach along with the augmentation with external data will create much higher volumes over time, especially in certain user cases The ability to practically accommodate massive amounts of data in the future will be a critical consideration of IoT architectures 13 October 2015 AllSeen Alliance 25

26 Opportunity for Providers and Users Increased Data Focus & Analytic Capabilities Leveraging the Utility Value of IoT Data Edge & Tier Processing wherever appropriate Publish & Subscribe Architecture 13 October 2015 AllSeen Alliance 26

27 IoT Foundations Moving from Possible to Practical 27

28 Metadata Leveraged Architecture Endpoint Devices 1 st Receiver Edge Processors 1 st Receiver Edge Processors 1 st Receiver Edge Processors Mid-Tier Edge Processors Mid-Tier Edge Processors Leverages Metadata throughout the architecture Includes Infobright Store integrated with Hadoop for enhancing analysis of machine data Establish Metadata at the point of ingestion Provide comprehensive query tools contemplating a variety of needs 13 October 2015 AllSeen Alliance 28

29 Metadata Leveraged Architecture Common Tool Sets, Minimal Administration Affordable and Accessible Endpoint Devices 1 st Receiver Edge Processors 1 st Receiver Edge Processors 1 st Receiver Edge Processors Mid-Tier Edge Processors Mid-Tier Edge Processors Leverages Metadata throughout the architecture Includes Infobright/Metadata Store integrated with Hadoop for enhancing analysis of machine data 13 October 2015 AllSeen Alliance 29

30 Gaps and Opportunities Gap: Leveraging Data and Analytics Lack of data leverage and more robust analytic stack will become an increasing impediment Gap: Edge Processing Edge processing is cloud based filtering and workflow for exhaust Gap: Publish/Subscribe Model Basic monolithic cloud architectures Opportunity: Data Cleansed, Enriched, and Published Analytic stack can be established to provide operational, investigative, predictive, and machine learning. Opportunity: Edge Processing / First Receiver Extensible version of workflow and data cleansing for edge deployments Opportunity: Event-driven Architecture Flexible pub/sub architecture for adaptability in demands with constituencies in a simple, secure, and accessible fashion 13 October 2015 AllSeen Alliance 30

31 Thank you Follow us on For more information on AllSeenAlliance, visit us at: allseenalliance.org & allseenalliance.org/news/blogs 13 October 2015 AllSeen Alliance 31