Measuring Quality of Experience in IPTV Related Services

Transcription

1 1

2 Measuring Quality of Experience in IPTV Related Services 2

3 Agenda Motivation Service Architecture and Related Services IPTV OTT Perceived Quality in TV Related Services Quality of Service (QoS) Quality of Experience (QoE) Related Standards Measuring the Quality of Experience Proposed Techniques and Architectures Case Study Conclusion 3

4 History of Current Presentation Project Title: A Conceptual Model for Monitoring Quality of Experience in IPTV, VoD and OTT Services Start: March 2013 Finish: October 2013 Presentations: CSRI 2014 IST

5 5

6 Data Traffic on Mobile Devices Exabytes per Month Smartphone Non-smartphone 6

7 Connected Devices 12.0 Billion of Devices non-smart phones smartphones M2M laptops tablets other portable devices 7

8 Top Global Mobile Networking Trends Transitioning to Smarter Mobile Devices Measuring Internet of Everything Adoption Analyzing Mobile Applications (Video Dominance) Profiling Bandwidth Consumption by Device Assessing Mobile Traffic/Offload by Access Type (2G, 3G, and 4G) Comparing Mobile Network Speeds Reviewing Tiered Pricing Adopting IPv6 Defining Mobile Prime Time (Peak vs. Average Usage) Cisco VNI Mobile,

9 Smartphones vs. Non-Smartphones Smart Devices and Connections Non-Smart Devices 9

10 Regional Share of Smart Devices Percent North America Western Europe Central and Eastern Europe Latin America Asia Pacific MENA

11 New Access Networks 80% 70% 60% 68% Cisco VNI Mobile, % 50% 40% 30% 20% 10% 0% 48% 44% 29% 26% 15% 8% 3% G 3G 4G 11

12 M2M Connections 2.5 Billion of Devices Cisco VNI Mobile, G 3G 4G 12

13 Mobile Multimedia Traffic 18 Cisco VNI Mobile, 2014 Exabytes per Month Mobile File Sharing Mobile M2M Mobile Audio Mobile Web/Data Mobile Video 13

14 Mobile Traffic Types 120% 100% 80% 60% 40% 20% 0% Mobile Application Traffic 82% 83% 85% 86% 88% 90% Cloud Traffic -Video & Audio Streaming -Online Games -Social Networking - Web Browsing -Online Storage Non Cloud Traffic - IP Voice - File Sharing - File/App Downloading 14

15 Generated Traffic based on Device Types Device Type Non-smartphone M2M Module Wearable Device Smartphone 529 2,672 4G Smartphone 1,984 5,371 Tablet 1,374 5,609 4G Tablet 2,410 9,183 Laptop 2,455 5,095 Megabytes per Month 15

16 Average Mobile Connection Speed Global speed: All handsets Global speed: Smartphones Global speed: Tablets Bandwidth in kbps 16

17 Average Mobile Connection Speed Middle East & Africa Central & Eastern Europe Latin America Western Europe Asia-Pacific North America Bandwidth in kbps 17

18 Other Related Forecasts Just in MENA: More than 321 Million Tablets in MENA upto 2016 (INFORMA) 300% growth in connected smartphones upto 2018 (MASON) 29% yearly growth of news and sport broadcasters in OTT upto 2018 (MASON) 37 billion USD investment on OTT services upto 2017 (INFORMA) 18

19 IPTV Architecture 19

20 Introduction ANY services ANY device ANY time ANY location 20

21 IPTV Related Services. Live TV Linear/Broadcast TV (audio, video and data) Linear Broadcast TV with Trick Modes Multi-angle service Time-shift TV PVR service (network or clientbased) Linear Broadcast Audio 21

22 IPTV Related Services.. Content on Demand Pay Per View (PPV) Video/TV on Demand (VOD) Near VoD (Video on Demand) broadcasting Real VoD Download Based Video Content Distribution Services (Push VOD) Content download service MoD (Music on Demand) including Audio book Pictures Service Information (EPG: Electronic Program Guide, ECG: Electronic Content Guide, etc.) Portal services Hybrid services 22

23 IPTV Related Services... User Generated Services Consumer Originated content (Video, etc. and applications) Consumer Originated broadcast TV (e.g. C2C hosting) Games Regulatory Information services Interactive TV (itv) 23

24 IPTV Related Services... T-Learning Education for children, elementary, middle and high school student, languages and estate, etc. IT Services T-information (news, weather, traffic and advertisement etc.) T-commerce (security, banking, stock, shopping, auction and ordered delivery, etc.) T-communication ( , instant messaging, SMS, channel chatting, VoIP, Web, multiple video conference and video phone, etc.) T-entertainment (photo album, games, karaoke and blog, etc.) 24

25 IPTV Related Services. Value Added Services Presence service Advertising Communications Messaging 3rd party content services 25

26 Subscriber Oriented Services Live TV Timeshift, Catchup and Start over TV VoD nvod and npvr Content Recommendation System Multiscreen and Connected Devices Online Game Location Based Services Advertisement TV Learning TV Banking And etc, 26

27 Television Service History 27

28 Television Services 28

29 Broadcast vs. Narrowcast Broadcast Time Limitation Device Limitation Content Limitation Specific Network High CAPEX and OPEX Narrowcast (IPTV/OTT) Any Time Any Device On Demand Content Any Network Low CAPEX and OPEX 29

30 IPTV vs. OTT IPTV OTT IP managed network Best effort over existing telecom networks Guaranteed Quality of Service NO Guarantee for Quality of Service Main Challenges Infrastructure preparation CAPEX Some Technology Providers: Cisco Huawei Anevia Beenius SAMIM Main Challenges Content delivery network Content compression techniques Some worldwide companies: Apple Google Amazon Sony BBC SAMIM 30

31 IPTV/OTT 31

32 IPTV/OTT Value Chain Subscriber Service Provider Network Provider Content Provider 32

33 Challenges of IPTV Deployment NOT Common standard Brand specific solutions Value added services Business plan Charging and billing systems 33

34 Challenges of OTT Deployment Adaptive streaming Variable bit rate video compression Multimedia delivery network Unicast management 34

35 OTT vs. IPTV Metric OTT IPTV Video Sharing QoS NO YES NO Infrastructure YES YES NO STB YES YES NO Access network YES YES YES Interactivity YES YES YES Content Type Continues Continues Discrete Service Diversity MIDDLE HIGH LOW Main Actor Service Provider Content Provider Service Provider Network Provider Content Provider Content Provider 35

36 IPTV Conceptual Architecture 36

37 OTT Conceptual Architecture 37

38 IPTV Value Chain 38

39 Practical IPTV Architecture 39

40 Standard Committees 40

41 Related Standards DVB ITU-T (ITU-T Focus Group IPTV) ETSI TISPAN IPTV ATIS IIF 3GPP (3GPP MBMS) OMA (OMA BCAST) IETF OIPF CCSA (China Communications Standards Association) 41

42 ETSI Standard Family 42

43 Open IPTV Forum (OIPF) 43

44 ATIS IIF IPTV DRM Interoperability Requirement IPTV Architecture Requirements A Framework for QoS Metrics and Measurements supporting IPTV Services IPTV Packet Loss Issue Report QoS Metrics for Linear (Broadcast) IPTV QoS Metrics for Public Services Standard PKI Certificate Format Interoperability Specification Test Plan for Evaluation of Quality Models for IPTV Services IPTV QoE Measurement Recommendations & Framework IPTV MPEG Transport Stream Monitoring Implementer's Guide to QoS Metrics IPTV QoS Metrics Metadata Technical Report on Mobile IPTV Quality of Service QoS Metrics for Content on Demand Methodology for Subjective or Objective Video Quality Assessment in Multiple Bit Rate Adaptive Streaming 44

45 Quality of Experience 45

46 Quality of Service Definition QoS is one or more measure of desired performance and priorities through the IPTV communication system. Key Measures: Service availability Maximum bit error rate Minimum committed bit rate Packet loss and latency performance Transmission quality 46

47 Quality of Experience Definition QoE is one or more measure of the total communication and entertainment experience from the perspective of end users. Key Measures: End user quality Ability to use the system easily Application driven Content impairments: Blockiness, Jerkiness 47

48 Quality of Experience Overall performance of system from the point of view of the users Measure of End to End performance at the services level An indication of how well the system meets user s need SUBJECTIVE METRIC related to Service pricing Viewing environment Stress level and so on 48

49 Reference Points of Quality Measurement 49

50 Quality of Service Metrics QoS Assessment Control Metrics Network Metrics Content Metrics 50

51 Content Quality Compression Resolution Frame rate Codec Streaming Bit rate Protocol 51

52 Network Metrics Network Metrics Bandwidth Delay Jitter Packet Loss 52

53 Network Assessment Core Distribution Access Home Technology provider Telecom operator Service provider Subscriber 53

54 Control and Signaling Assessment Zapping Time Multicast Join/Leave Delay Assessment of Control System (Middleware) STB Boot Delay Connect Delay Trick Play Delay 54

55 QoE Estimation End user requirements analysis Definition of application layer QoE requirements Translation from subjective QoE requirements to objective service performance requirements end-to-end at the network and application layers Allocation of performance impairments to protocol layers, network segments or node 55

56 Video Quality Measurement Video Quality Subjectively Objectively Indirectly MOS PSNR Using QoS Metrics 56

57 Mapping QoS Metrics to QoE Mappings Linear Logarithmic Exponential Polynominal 57

58 Sample Mapping of QoS to QoE Content Video Stream Transport Transaction Video Quality Error Indicator Continuity Error PCR Jitter PSI Table Error IP Packet Loss, Jitter RTP Packet Loss, Jitter TCP Retransmission Service Provisioning IGMP Latency (Zapping) RTSP Latency (Trick Play) Picture: blurring, edge distortion, visual noise Pixelization, tiling, frame freezes, blue screen Service accessibility, channel change delay, trick play latency 58

59 Reported Problems

60 Reported Problem Groups Other 15% Headend 16% Core Network 12% Home Network 39% Access Network 18% Headend Core Network Access Network Home Network Other 60

61 IPTV QoE in E2E Model Live and VoD asset distribution Audio/Video Encoding EPG/IPG info quality Network Elements (Core, Distribution, Access and Home) Delay Jitter Packet loss and reordering Fast Channel Change Zapping Time STB Audio/Video decoders Lip sync Output interfaces VoD Server Load Distribution 61

62 Recommendation for QoE Estimation 62

63 TM Forum Service Control and Media Delivery Factors STB Factors Transmission Factors IGMP delay RTSP delay EPG delay Command and Control delay GoP delay Buffer delay Decoding delay Playout delay OS Boot delay Service boot delay Processing delay One-way packet delay Key Performance Indicator STB Service Start Delay Channel Change Delay EPG Navigation Delay TV Control Delay VoD Control Delay DVR Control Delay Key Quality Indicator % of sessions that experience delays of X and above % of user actions with delay of X and above 63

64 ITU-T Transport One-way, round-trip Latency Packet loss Jitter Initial buffer time Re-buffer Control and Signaling Join/leave times Middleware latency Video/Audio MOS Start time Picture loss Frame loss First picture time Freeze and skip frames Service Channel availability Channel start time Channel change time DRM response time 64

65 IPTV/OTT QoE Assessment 65

66 IPTV Test Equipment Test Devices Video Network Control Video Analyzer MPEG Generator Network Impairment Monitor Protocol Analyzer Network Monitoring Tools Test Clients (STB) 66

67 Measurement Devices Agilent Tektronix IXIA 67

68 Monitoring Systems WatchiTV AGAMA Bluetop Shenick 68

69 WatchiTV 69

70 AGAMA 70

71 Shenick 71

72 Conclusion Fourth Reference Point Subscriber and Access Network Provider Related Metrics: Network delays (ITU-T Y.1540), TS related metrics and video quality metrics Fifth Reference Point HOME GATEWAY Related Metrics: related delays to Third Reference control and signaling Point system, STB quality, video and audio quality and Access Network Provider and ITU-T G.1082 Distribution Network Provider Related Metrics: Proposed metrics of ITU-T Y.154x e.g. delay, jitter, packet loss, multicast delays First Reference Point Content Provider and Service Operator Related Metrics: media formats and codecs, streaming standards, TS monitoring Second Reference Point Service Operator and Network Provider Related Metrics: network metrics, video quality, control and signaling, middleware quality 72

73 73

74 74