IoT is a King, Big data is a Queen and Cloud is a Palace

Transcription

1 IoT is a King, Big data is a Queen and Cloud is a Palace Abdur Rahim Innotec21 GmbH, Germany Create-Net, Italy Acknowledgements- ikaas Partners (KDDI and other partnes) Intelligent Knowledge-as-a-Service 1

2 Outline Motivation Convergence/opportunities/applications Challenges and requirements Convergence approach ikaas EU-Japan project Conclusion 2

3 Convergence of Technologies Source-IDC 3

4 Where is the value of IoT? In the past, connectivity and number of the devices were the main driver of IoT Data is nothing without big business value insight IoT without BIG DATA is first generation of IoT 4

5 The real value is not just sheer number of connected devices and data The real opportunity is improved business value-new revenue models, lower cost, improved client experience, better insight improve outcomes Source-IDC 5

6 Big data-how we understand it 6

7 IoT in BIG data IoT presents challenges in combination of all BIG data characteristics (3Vs/4Vs) Most challenging IoT applications match with either or both Velocity & Volume and sometimes also Variety (situation and context) Velocity driven-application A wearable sensor produces about 55 million data points pro day (challenge for storage), whereas some medical wearable's (like ECG) produce up to 1000 events per second (challenge for realtime processing) Volume driven-applications GE each day gathers 50 million pieces of data from 10 million sensors, off equipment worth $1 trillion 7

8 Typical IoT applications 8

9 IoT BIG data applications Massive monitoring/deep understanding (observe of behavior of many thing, gain important insight Health example (understanding the cause of diseases/comorbidities/indicators) Real-time actionable insight (Real-time analytic, detect and react in real-time) Health example (real-time fall detection and potential reaction for aging population) Performance optimization (configuration, energy, health-care) Health example (Improve overall healthcare efficiency) Proactive and predictive functional applications Health example (proactive and prediction identification of diagnostic in healthcare applications (before thing occur) 9

10 Philosophical differences of Big data analytic Traditional methods Centralize More power Summarize data Transform and store Pre-define schema Move data toward compute Less data/more complex algorithms Big data Distributed More machines Keep all data Transform on demand Flexible/no-schema Move compute towards data More data/simple algorithms 10

11 IoT Big data platform requirements Security and privacy Scalable Intelligent and dynamic Real-time Distributed Unified view 11

12 What cloud offers? Dynamic and flexible resources sharing platform Offers scalable, elasticity resources and data management Location independent can be access from any where Reliable and easy access of the services Large amount of computing and storage resources It is also more homogeneous (unified) 12

13 Convergence of IoT-Big data and Cloud "Cloud computing a new business model and management (e.g. data and device) paradigm of Internet of thing and Big data" IoT Big data is to enlarge the opportunities of cloud service provisioning Convergence Approaches Centralize approach (Bring IoT functionalities in Cloud) Distribute approach (Bring Cloud functionalities in IoT) 13

14 IoT-Big data-cloud: Centralize approach Bring IoT data in the cloud Processing and computing the data and deploy management tools in cloud This approach this good if service are provided among objects located in multiple location hosting databases partners SI applications Cogni ve capability IoT Cloud Pla orm Our managed devices All devices your devices 14

15 IoT-Big data- Cloud: Distributed approach Edge/fog computing-stream Processing and storage of data close to users/near to devices To distribute data to move it closer to the end-users to eliminate latency, numerous hop, and support mobile computing and data streaming Usability High-latency and real-time actionable insight (the data flow to fast to be processed) Data/intelligence context are geographically distributed The datasets have strict privacy, security and regulation constraints that prohibit their transfer outside of the paten domain Domain specific service and applications 15

16 ikaas (H2020 EU-Japan) IoT-Big and Cloud Project 16

17 Project objective The goal of ikaas project is to combine ubiquitous and heterogeneous sensing, semantic, big data and cloud computing technologies in a platform enabling the Internet of Things distributed process consisting of continuous iterations on data ingestion, data storage, analytics, knowledge generation and knowledge sharing phases, as foundation for cross-border information service provision. 17

18 Architecture framework (Distributed) App. App. Query Storage KaaS Global Cloud Security GW Knowledge Data Data Knowledge Security GW Query Query Storage Local Cloud Data Storage Local Cloud Data Sensors /IoT Devices Sensors /IoT Devices 18

19 Service and processing migration ikaas Programable Service logic Publish sensor needs, Privacy needs, RT needs, Reliability needs (constraints) Allocation optimizer Allocation decision Cloud, data center Move to the Global Cloud B Cloud Controler or stay in local Cloud Cloud, data center or stay in local Cloud Move to the local Cloud A 19

20 Service deployment and orchestration Smart service logic Autonomously analyse application requirements, user preferences Register the services/deployment of services Allocation manager The most appropriate deployment of service must achieve the best balance among cloud resources, system performance, quality of service and cost. Appropriate service execution Service/task Manager (Query, control, and reconfiguration) Analysis of the application request(s) using ikaas service model/templates; flexible/autonomic selection of more appropriate cloud resources Reconfigure the service logic on run-time (e.g, dynamically changes the services/business logic) Synchronization of the service logic deployment, service migration, decision between local and global cloud

21 Distributed execution environment Service query (Query control) Local Cloud Global Cloud ervice/task Manager Independent Migration Service/task Manager Dependent Programmable application logic Service catalogue Configuration and allocation Manager Smart logic Synchronization Service Query Configuration manager service logic Service Catalogue 21

22 Service orchestration Multi-scale service migration Migration of relationship logic to local cloud Service Logic description Cognitive Engine Analysis Monitoring Decision Making ucore Framework Learning Service request Service and associated meta-data Global Cloud Service results Computing in the Global Cloud Smart Virtual Objects Service component migration Service component results Local Cloud Service execution Local Cloud Service execution Local Cloud Service execution Service orchestration 22

23 Multi-scale application migration Application s logic can be migrated near the data sources multi-scale (recursive) process: the application s logic can be broken down again and further migrated My laptop Service migration Gateway1 Local Proc. 1ms readings Temp. sensor application ikaas Component Service migration Final result Server ikaas Component Gateway2 Local Proc. 1ms readings Temp. sensor In red: application logic deployment In blue: data gathering and consolidating Daily computation results Gateway3 Local Proc. 1ms readings Temp. sensor

24 Security Gateway Global Cloud Security Policy Privacy Policy Local Cloud

25 Security Gateway (2) Security and Privacy by Design Concept Main Functions: Policy Management & Negotiation (Cross-Border) Authentication and Access Control (Service Level) Transformation of Data (Privacy Preserving Way) Application to Cross-border Scenario Local Cloud Cross-Border Use Data Transfer Security GW Global Cloud External App. Internal Use Security GW Local Cloud Local App. Policy Negotiation 25

26 Security Gateway (3) Design of the Security Gateway Privacy CA Application Privacy Certificate DB Global Cloud Global Platform Data Processing Functions Query Control Functions Cache Manager Cache DB Local Cloud Security Gateway Token DB Key DB Owner DB Local Cloud DBs Access Control Functions Privacy Control Functions Local Query Controller Policy DBs Security Policy DB Privacy Policy DB Cache Policy DB

27 Security Gateway (4) Procedure Token Issuance I. An application requests the privacy CA to issue the privacy certificate. II. The application searches the security gateway of the domain where there are the local cloud DBs suited for the objective with using the query control functions on the global platform. III. The application calls function Issuance of Token that the security gateway provides. The application then specifies the DB IDs of the local cloud DBs that it wants access to, and sends the privacy certificate. IV. The security gateway confirms the values of parameters CA Domain Name and Expires listed on the privacy certificate to verify the correctness of the certificate. V. The security gateway checks the values of Application IP, LC Domain Names and LC DB IDs listed on the privacy certificate to validate the application and the request. VI. The security gateway creates a token and returns it to the application. Data Request I. An application generates the MAC of the SGW-query with using the token, which is a common key. II. The application calls function Getting Data that the security gateway provides and transmits SGW-query and the MAC to the security gateway. III. The security gateway extracts the corresponding token from the token DB with the values of the Application ID and Application IP headers and checks the expired date of the token. IV. The security gateway generates the MAC from the token and the SGW-query to verify the authenticity of the query. The value of the Time Stamp header is also confirmed. V. The security gateway transmits the LCD-query to the local query controller. VI. When the data are returned from the local cloud DBs, the security gateway confirms the privacy type of the DBs while searching the token DB. VII. If the data stored in the non-privacy DB are returned, the security gateway returns the data to the application without doing anything. Otherwise, Steps are carried out. VIII. The security gateway extracts the corresponding owner IDs from the owner DB with using the value of the Owner Attributes header. IX. The security gateway searches the privacy policy with using the extracted owner IDs and the values of the Application ID and LC DB IDs headers and confirms the status of the consent of the corresponding data owners. X. The security gateway extracts the data such that the data owner agrees on the transfer and returns the extracted data to the application. 27

28 Application A Security Gateway (5) Example of Security Policy Token Configuration (such as period and accessible information) should be defined for each application category and country of the domain that application is executed. Level DB 1 DB 2 DB N Administrat or 1 1 UK 0 / JP 2mo Non-Privacy UK 3h / JP 3h Non-privacy UK 0 / JP 0 Privacy Administrat or 2 2 UK 1h / JP 2h Privacy UK 5h / JP 0 Non-privacy UK 0 / JP 0 Privacy Administrat or M M UK 0 / JP 0 Non-privacy UK 1h / JP 0 Non-privacy UK 0 / JP 0 Privacy

29 Security Gateway(6) Performance Evaluation Results Transaction time of data collection is practical. Cache function is effective for reducing the transaction time. # of Data Non-Private Private Using Cache Func

30 Take away message Convergence is everywhere If you start innovation think on the how your business will convergence and scale When we talk IoT, it is actually the largescale NEED of large-scale IoT is to exploit Big data for smart IoT services that processed and executed on the cloud to derive business value insight 30