QoE for Cloud Services. Martín Varela VTT Technical Research Centre of Finland September

Transcription

1 . QoE for Cloud Services Martín Varela VTT Technical Research Centre of Finland September

2 M. Varela QoE for Cloud Services Outline Introduction Motivation QoE for Cloud Services Possible Applications Conclusions

3 Introduction QoE the What Several definitions available (ITU-T, Dagstuhl, ) From COST IC - alinet the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user s personality and current state. Patrick Le Callet, Sebastian Möller, and Andrew Perkis (eds). alinet White Paper on Definitions of ality of Experience ( ). Lausanne, Switzerland, June. M. Varela QoE for Cloud Services Slide of

4 Introduction QoE Today An (almost) established research field A buzzword (Emerging) Business potential But still mostly about media M. Varela QoE for Cloud Services Slide of

5 Introduction The Cloud Today An established field, both in research and practice A (somewhat nebulous) buzzword (Proven) Business potential But still mostly about being cheaper M. Varela QoE for Cloud Services Slide of

6 Introduction QoE in the Cloud Today This frame intentionally le blank M. Varela QoE for Cloud Services Slide of

7 M. Varela QoE for Cloud Services Slide of Motivation Plan Introduction Motivation QoE for Cloud Services Possible Applications Conclusions

8 Motivation QoE in the Cloud Motivation ( ) For Cloud-based media: study on willingness to pay in VoD A. Sackl, P. Zwickl, and P. Reichl. The trouble with choice: An empirical study to investigate the influence of charging strategies and content selection on QoE.. In: Network and Service Management (CNSM), th International Conference on. Oct., pp.. : /CNSM M. Varela QoE for Cloud Services Slide of

9 Motivation QoE in the Cloud Motivation ( ) More generally, quality and money go hand in hand Reichl s fixed-point model for QoE-based pricing ality and price are (almost intrinsically) linked Price drives demand Price sets expectations Demand drives quality Peter Reichl et al. A Fixed-point Model for QoE-based Charging. In: Proceedings of the ACM SIGCOMM workshop on Future human-centric multimedia networking. FhMN. Hong Kong, China: ACM,, pp.. M. Varela QoE for Cloud Services Slide of

10 M. Varela QoE for Cloud Services Slide of Motivation QoE in the Cloud Motivation ( ) So, why do we care about quality? Money, mostly Not quite just money, but almost.

11 M. Varela QoE for Cloud Services Slide of Motivation QoE in the Cloud Motivation ( ) So, why do we care about quality? Money, mostly Who is we? Network Operators Cloud / Service / Content Providers Users Not quite just money, but almost.

12 Motivation QoE for the Users Back to the QoE definition: the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user s personality and current state. M. Varela QoE for Cloud Services Slide of

13 M. Varela QoE for Cloud Services Slide of Motivation QoE for the Users The notion of utility, and the users expectations with respect to it, are key here bang for the buck Pleasure, enjoyment Efficiency, fitness for purpose, monetary gain QoE is not utility, per se, but it can be a useful proxy of it The perceived ability of something to satisfy needs or wants.

14 M. Varela QoE for Cloud Services Slide of Motivation QoE for the Service Providers Back to the QoE definition: the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user s personality and current state.

15 Motivation QoE for the Service Providers Decreased QoE = more dissatisfied users Dissatisfied users = churn Churn = decreased revenues You can follow this logic, right? M. Varela QoE for Cloud Services Slide of

16 M. Varela QoE for Cloud Services Slide of Motivation QoE for the Service Providers It actually goes beyond simply avoiding churn: What if we could lower OpEx without decreasing customer satisfaction? What if we could increase margins by charging more for be er quality? What if we could capture a larger market share by providing more quality options? What if we could differentiate from the competition by providing be er quality, or even be er, quality guarantees? Note that this also provides more choice to users.

17 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services Plan Introduction Motivation QoE for Cloud Services Possible Applications Conclusions

18 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services Towards QoE in the Cloud How do we actually use most Cloud services?

19 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services Towards QoE in the Cloud How do we actually use most Cloud services? Web Mobile apps

20 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services Towards QoE in the Cloud How do we actually use most Cloud services? Web Mobile apps Not just those two, of course (e.g. gaming)

21 QoE for Cloud Services Web QoE A first step towards understanding QoE in the Cloud Much more complex than QoE of media services Requires much more information than media QoE Impossible to have a single, universal model for it M. Varela QoE for Cloud Services Slide of

22 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services Web QoE A first step towards understanding QoE in the Cloud Much more complex than QoE of media services Requires much more information than media QoE Impossible to have a single, universal model for it The web can be anything!

23 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services QoE the How ( ) How do we go about measuring QoE, anyway? We can t actually quite measure it Too many unknowns, mostly related to the user Application-specificity Multiple modalities and their interactions Contextual factors But we can, to some degree, estimate it (or parts of it) Considering QoE as a multi-dimensional construct, we can produce good estimates in several perceptually-related dimensions, for many services.

24 QoE for Cloud Services QoE the How ( ) ay Application space: A Dimensions representing application configuration related factors a1 Resource space: R rm Dimensions representing network/system factors related to resource characteristics and performance a2 r2 Estimation of Integral QoE (weight factors assigned to individual QoE dimensions) qn r1 Mapping function Context space: C Dimensions representing factors indicating the situation in which a service or application is being used c1 ck c2 q2 q1 QoE Space: Q Dimensions representing quantitative and qualitative quality metrics User space: U Dimensions representing parameters related to the specific user of a given service or application u1 uz u2 L. Skorin-Kapov and M. Varela. A Multidimensional View of QoE - the ARCU Model. In: Proceedings of th MIPRO. Opatija, Croatia, May. M. Varela QoE for Cloud Services Slide of

25 QoE for Cloud Services QoE the How ( ) In practice: Several means to obtain (perceptual) quality estimates Subjective assessment Objective estimation Many variations of the themes above, including hybrid approaches Varying degrees of accuracy Varying suitability to different contexts (e.g. real-time vs. offline) M. Varela QoE for Cloud Services Slide of

26 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services QoE the How (Cloud edition) We need models for web (and app) QoE: Mapping from ARCU to QoE Using measurable parameters from all system components Including application, context and user information

27 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services QoE the How (Cloud edition) We need models for web (and app) QoE: Mapping from ARCU to QoE Using measurable parameters from all system components Including application, context and user information This is just as complicated as it sounds

28 M. Varela QoE for Cloud Services Slide of QoE for Cloud Services QoE the How (Cloud edition) We need models for web (and app) QoE: Mapping from ARCU to QoE Using measurable parameters from all system components Including application, context and user information This is just as complicated as it sounds But we can start simple

29 QoE for Cloud Services Some Steps Towards Web QoE Performance is critical D. Strohmeier, S. Jumisko-Pyykkö, and A. Raake. Towards Task-dependent Evaluation of Web-QoE: Free Exploration vs. Who Ate What? In: IEEE Globecom. Anaheim, CA, Dec.. Martıń Varela et al. QoE in the Web A Dance of Design and Performance. In: QoMEX. Costa Navarino, Greece, May. M. Varela QoE for Cloud Services Slide of

30 QoE for Cloud Services Some Steps Towards Web QoE Performance is critical Not just the network, but everything D. Strohmeier, S. Jumisko-Pyykkö, and A. Raake. Towards Task-dependent Evaluation of Web-QoE: Free Exploration vs. Who Ate What? In: IEEE Globecom. Anaheim, CA, Dec.. Martıń Varela et al. QoE in the Web A Dance of Design and Performance. In: QoMEX. Costa Navarino, Greece, May. M. Varela QoE for Cloud Services Slide of

31 QoE for Cloud Services Some Steps Towards Web QoE Performance is critical Not just the network, but everything Looks and usability, too D. Strohmeier, S. Jumisko-Pyykkö, and A. Raake. Towards Task-dependent Evaluation of Web-QoE: Free Exploration vs. Who Ate What? In: IEEE Globecom. Anaheim, CA, Dec.. Martıń Varela et al. QoE in the Web A Dance of Design and Performance. In: QoMEX. Costa Navarino, Greece, May. M. Varela QoE for Cloud Services Slide of

32 M. Varela QoE for Cloud Services Slide of Possible Applications Plan Introduction Motivation QoE for Cloud Services Possible Applications Conclusions

33 .. M. Varela QoE for Cloud Services Slide of Possible Applications Monitoring QoE The u Architecture Assuming we can model QoE for these services, we need a means for estimating and exploiting it. OSS. SLAs. X-layer Control.. QoE-Aware Applications. SNMP / Web Services QoE Agent.. Network. Infrastructure. Platform. Service. Context.. Probes. SNMP

34 Possible Applications Monitoring QoE ( ) Recent ETSI Technical Specification, output of the u project. ETSI. TS ality of Experience A Monitoring Architecture. Dec.. M. Varela QoE for Cloud Services Slide of

35 Possible Applications Monitoring QoE ( ) A generic way of building quality models Martín Varela et al. Meta-Modeling QoE - Towards a Generic Methodology for Building QoE Models. In: PIK - Praxis der Informationverarbeitung und -kommunikation ( ), pp.. M. Varela QoE for Cloud Services Slide of

36 M. Varela QoE for Cloud Services Slide of Possible Applications Monitoring QoE ( ) A flexible, agent-based solution with a portable implementation and standard interfaces.

37 Possible Applications QoE and Resource Management Lowest hanging fruit We know how to do this for some services Tractable in practice (for many service types) Direct business value M. Varela QoE for Cloud Services Slide of

38 M. Varela QoE for Cloud Services Slide of Possible Applications QoE and Resource Management Lowest hanging fruit We know how to do this for some services Tractable in practice (for many service types) Direct business value But Non-technical issues exist Standardization is needed

39 M. Varela QoE for Cloud Services Slide of Possible Applications Cloud Brokers Allow SaaS-type providers to (dynamically) pick the best IaaS/PaaS for their service For some definition of best Cheaper? More Reliable? Best QoE for their service?

40 Possible Applications QoE & SLAs Use QoE models to derive, monitor and enforce SLAs Why? For content and SaaS-type providers Being able to properly satisfy their customer demands in terms of quality Product differentiation Tiered subscriptions Etc. For end users Ge ing their money s worth Having recourse when things don t work as expected M. Varela QoE for Cloud Services Slide of

41 M. Varela QoE for Cloud Services Slide of Possible Applications Issues with SLAs How to sell them? ality is not tangible Noticed mostly when missing Conveying quality levels to users is hard Missing in upstream providers E.g. AWS guarantees some. % uptime, and that s it Not clear when or if this will change sufficiently

42 M. Varela QoE for Cloud Services Slide of Possible Applications QoE & SLAs How? Good question! Use QoE models as guides for se ing SLOs Use QoE models for choosing / negotiating proper SLAs Formalize SLAs and their relation to QoE

43 M. Varela QoE for Cloud Services Slide of Conclusions Plan Introduction Motivation QoE for Cloud Services Possible Applications Conclusions

44 M. Varela QoE for Cloud Services Slide of Conclusions Conclusions QoE in the Cloud It can inform business and technical aspects of services It can enable new ways of commercializing services It can keep users happy, decreasing churn Many challenges remain A need for useful models, but also, A need to go beyond modeling, and into exploitation

45 M. Varela QoE for Cloud Services Slide of Conclusions Any estions? Thanks for your a ention!

46 Conclusions Cost/ ality Optimal SLA selection ( ) A simple example Consider a SaaS provider, with customers and upstream providers (network and infrastructure) Set target quality levels to provide (qmin ) Set budget (B) How to choose proper SLAs from the upstream providers? P.A. Frangoudis et al. ality-driven Optimal SLA Selection for Enterprise Cloud Communications. In: Communications Workshops (ICC), IEEE International Conference on. June, pp.. M. Varela QoE for Cloud Services Slide of

47 M. Varela QoE for Cloud Services Slide of Conclusions Cost/ ality Optimal SLA selection ( ) Formulation: S (H) : Set of available hosting (H) SLAs. S (N) : Set of available network (N) SLAs. (H) p i : VM instance specifications for S (H) i (CPU, memory, storage) (N) p k = d k, j k, l k : Worst-case QoS, as per the proposed network SLA (H) c i : Price per VM instance for S (H) i. (N) c k : Price for the overall network service for S (N) k. [ ] Q(i, k, n) Maximize F(i, k, n) = i,k,n C(i, k, n) subject to Q(i, k, n) q min 0, B C(i, k, n) 0

48 M. Varela QoE for Cloud Services Slide of Conclusions Cost/ ality Optimal SLA selection ( ) Solve via scalarization Minimize the distance from an utopic reference point: L (F,w,zu ) = max(w Q Q(i, k, n) Q u, w C C(i, k, n) C u ) Where z u = (Q u, C u ) is the utopic reference point (free, max quality) [ ] wq w = represent the relative importance of each w C criterion in a set of Pareto-optimal solution vectors

49 M. Varela QoE for Cloud Services Slide of Conclusions Cost/ ality Optimal SLA selection ( ) The solution space (for a simplistic cloud-based teleconferencing app) 5 Solution space 4 MOS Solutions Reference point Budget constraint Min quality constraint Cost x 10 4