Big Data Driven Knowledge Discovery for Autonomic Future Internet Professor Geyong Min Chair in High Performance Computing and Networking Department of Mathematics and Computer Science College of Engineering, Mathematics and Physical Sciences University of Exeter, U.K. Email: g.min@exeter.ac.uk
Outline Importance and Challenges of AFI for Emerging 5G Network Big Data Analysis: To strengthen AFI standardization Platform for Network Big Data Processing Conclusions
Trends of Future Internet Explosive growth of data traffic Cisco VNI 2014-2019 white paper: The total mobile data traffic is expected to rise at a compound annual growth rate (CAGR) of around 57 percent. Ten-fold increase of the mobile traffic growth from 2014 to 2019. Massive increase in the number of interconnected devices Internet-of-Everything: Everything will be connectted through mobile networks. Cisco Internet of Things White Paper: The number of connected objectives around the world will reach 50 billions by 2020. Continuous emergence of new services and application scenarios Ultra-high-definition Videos, 3D Video, Augmented Reality, Virtual Reality, Tactile Internet, Media-Rich Social Network services, IoV and Smart Home.
Performance Requirements of 5G 5G will bring together the evolved versions of the existing radioaccess, cloud and core technologies with some new complementary ones to build a seamless and high-performance network architecture. Key Performance Indicators of 5G Providing 1000 times higher wireless area capacity compared to 2010 Saving up to 90% of energy per service provided Reducing the average service creation time cycle from 90 hours to 90 minutes Creating a secure, reliable and dependable Internet with a zero perceived downtime for services provision. Facilitating very dense deployments of wireless communication links to connect over 7 trillion wireless devices serving over 7 billion people. Enabling advanced User controlled privacy Timely and challenging issue: how to effectively tackle the increasing complexity of network management and operation, introduced by the heterogeneity of design, implementation and running scenarios of various promising network technologies.
Autonomic Future Internet (AFI) Autonomic networks are regarded as a viable solution for addressing the complexity challenge of 5G. The ambition of AFI is to exploit an autonomic, intelligent and self-managing Future Internet with consequent improvement in network efficiency and performance, increased profitability and reduced OPEX and CAPEX. In order to implement successful transformation and integration of AFI and the existing and future network architecture, especially 5G specification, self-management and cognitive learning in GANA model are the key outputs from AFI to other network architectures. Self-management in AFI is essential for complexity reduction and fast adaptation to changing situations. Cognitive learning can increase the intelligence through flexible knowledge utilization.
AFI: Knowledge Plane GANA Knowledge Plane A pervasive system responsible for building and maintaining high-level models of what the network is supposed to do. A distributed and decentralized construct within the Internet to gather, aggregate and act upon information about network behaviour and operation. Powerful Knowledge Plane To strengthen the AFI standardization Reference: ETSI GANA Model
Network Big Data (1) With the overwhelming amount of data pouring into the future Internet, network domains are embracing an unprecedented wave of traffic flows and are stepping into the era of network big data. Network Big Data: holistic view of network Machine generated data: logs, netflows, SNMP, servers, virtual machines, controllers, events, metrics, data centers. User generated data: social networks, mobile video traffic, end-user information, call detail records (CDRs). Internet-of-Things generated data: sensing networks, smart cities, digital health, RFIDs, machine to machine communication. Big Data brings Big Value: strengthen the knowledge plane of AFI Management: resource utilization, load balance. Service assurance: user behaviour, customer experience. Reliability: cyber-security threats, fault management.
Network Big Data (2) With the overwhelming amount of data pouring into the future Internet, network domains are embracing an unprecedented wave of traffic flows and are stepping into the era of network big data. Network Big Data: holistic view of network Network Big Data In-depth analysis Knowledge Plane Operation, Management, Optimization 5G Communication, transmission Big Data brings Big Value: strengthen the knowledge plane of AFI Management: resource utilization, load balance. Service assurance: user behaviour, customer experience. Reliability: cyber-security threats, fault management.
Features and Challenges of Network Big Data Volume Variety Velocity Veracity Massive scale of data; Streaming data High demand for computational and storage resource Heterogeneous datasets; Diversified format Requirement for efficient and unified representation model Fast generated; Real-time analysis Requirement for online processing of incrementally arriving new data Inconsistent, incomplete, redundant Data Requirement for dimensionality reduction for filtering out undesirable data
Platform for Network Big Data Processing Data Source Stream Processing System Data Storage User Interface Data Collection Scalable Buffer Streaming Processing Aggregated Data Storage Data Index & Search (Flume) (Kafka) (Storm) (HDFS) (ElasticSearch)
Conclusions Autonomic Future Internet (AFI) plays a key role in supporting the emerging 5G networks owing to its self-management and learning features. Effective Network Big Data Analysis is important to strengthen AFI standardization Unified Representation of Network Big Data Online Incremental Dimensionality Reduction Network Big Data Analytics for Network Management Platform design for Network Big Data Analysis
Thank You All!