Arnaud Becart ip- label 11/9/11
RUM Synthe2c Tests
You should measure HTML and RIA (Flash ) Page Rendering Onload + Full Page Load InteracBons in your page Third Party content How Synthe2c / Real browsers (Renderstart + onload + Full page) RUM (onload only) Advanced diagnos2cs (video, HTTP logs, headers, code ) Code OpBmizaBon Build SLA with 3 rd Party Web is complex But not so much from a User Perspec2ve
Before AGer Client GET /psp/crprd/?cmd=login&languagecd=eng HTTP/1.1 Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/xshockwave-flash, application/x-ms-application, application/x-ms-xbap, application/ vnd.ms-xpsdocument, application/xaml+xml, */* Accept-Language: fr User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0;.NET Client CLR 2.0.50727;.NET CLR 3.0.4506.2152;.NET CLR 3.5.30729) Accept-Encoding: gzip, deflate Host: dsxeco.3ds.com Connection: Keep-Alive GET /psp/crprd/?cmd=login&languagecd=eng HTTP/1.1 Host: dsxeco.3ds.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16 (.NET CLR 3.5.30729) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: EN Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Server HTTP/1.0 302 Moved Temporarily Date: Wed, 20 Apr 2011 13:04:24 GMT Location: https://xxxxxxx.com/psp/crprd/?cmd=login&languagecd=eng& Content-Type: text/html Expires: Thu, 01 Dec 1994 16:00:00 GMT Set-Cookie: CRPRDWEB1-PORTAL- PSJSESSIONID=2l97NnhDJhNnllrwQLsM1LJHq9QXqY58!786913731; path=/ Connection: close Server HTTP/1.1 302 Moved Temporarily Date: Wed, 20 Apr 2011 13:11:13 GMT Location: https://xxxxxxxx.com/psp/crprd/?cmd=login&languagecd=eng& Content-Type: text/html Cache-Control: no-cache Set-Cookie: CRPRDWEB1-PORTAL- PSJSESSIONID=y4RYNnbRTV44lR6gzSYQv902TRXfW1kq!786913731; path=/ Transfer-Encoding: chunked + Keepalive
Before AGer
Before AGer +30% Web Server Conf Change
Page + CDN op2miza2on 18% to 41% perf gain
They re everywhere they re moving...
they expect different response 2mes
Measuring Loca2on Good when you know the User CONTEXT Last Mile Backbone 20 Mb 100 Mb actual Mb 2GHz, 4GB RAM actual resource Telefonica Vodafone DTAG Fixed user Mobile user MAC PC Tablet Smartphone IE Firefox Chrome safari How SyntheBc : where your users are RUM, qualifying : Actual user s bandwidth Actual processing capaci2es Isolate Mobiles/Tablets Isolate Synthe2c tests
34 browsers 6.5 M monthly visitors Driving 75% of its revenue on the web Exhaus2ve Browser monitoring
You should measure Compa2bility across releases First Bme / repeated view Cross- browsers tesbng Real browsers observing (in a known context) Mobile browsers How SyntheBc : up- to- date (real) browsers, benchmark RUM : Truly exhaus2ve Fast changing market shares Real- Time (release valida2on) Correla2on with analy2cs
A True Mobile Monitoring story Mobile App + Data App + Voice App
ios5 vs ios 3.1 + 60% performance gain measured with ip- label Cloud Observer Many devices Many OS
Why is mobile monitoring complex? TesBng a site or an app Many devices & OS 3G / 4G / WIFI Emulator Vs Real Device monitoring? Yet it s cribcal Users expect an equal response <me than from desktop How to make it simple? SyntheBc : Real- device & technology independent monitoring RUM : Quite exhaus2ve For a limited cost Automa2c update of devices (js compliant) & new OS
More & More Web- Based internal Apps Web Services based communica2ons What about my employees?
You should measure Employees produc2vity (can they use their applicabons?) Measuring real usages : Browser / Same se`ngs WorkstaBon / Virtual session Environment (internal network) Web + External App usages How? Internal Synthe2c DIY (license) or SaaS Internal RUM : Exhaus2ve Very limited cost Internal SLA
Website : 1 million visitors/day + 82 physical shops 6000 worksta2ons Results : Hardware upgrade Browser updates Release valida2on 13s 6s per page Internal RUM
Origin Hos2ng New Hos2ng + 40% CDN +66% 3/ Benchmarking to choose the good CDN at the good place 2/ How to jus2fy the cost and calculate ROI? 1/ Do I need a CDN?
You should measure CDN effecbveness on a locabon/area ROI CalculaBon SLA based on actual users How Synthe2c : worldwide, mulb- carrier Tracking only CDN performance Benchmarking RUM where Robots are too expensive or complex to deploy don t forget Video Content!
40 sec before playing a live content Robot Streaming Servers Stream Provider Robot Real- 2me or Catch- Up TV Video on mobile Is video cri2cal to your business? Mobile Network
You should measure Buffering Bme Frame- rate & image quality Breakdowns during playing Real- 2me / Live Channels VOD / Progressive Download Mobile video / adapbve streaming How Synthe2c : video/audio streams, live TV, VOD Portals Image quality (MOS) with or w/o reference RUM : stream quality delivered to anyone
VOIP PSTN ISDN Mobile MOS PESQ ITU P.862.1 Speech Quality Is voice cri2cal to your business? Call Center App & IVR availability
You should measure Voice of the Customer (VOC) : can they reach you? Click- 2- call Inbound/oubound calls Speech quality Call center produc2vity if phone system or CRM are down How TesBng real phone calls Monitoring IP & PSTN lines Universal & objecbve quality evaluabon (MOS PESQ) Monitoring InteracBve Voice Responder (IVR) and call centers Mixing Data & Voice monitoring
You should measure A user accessing to a virtual or cloud plagorm External/Internal depending on usages Physical vs. Virtual performance & load impact SLA achievement? < 99.9 uptime for end-users How SyntheBc transacbon in a virtual environment External SyntheBc transacbon on a cloud plalorm Load tesbng : 2VCPU = 1.2 CPU RUM script based CloudObserver Third Party SLA calculabon Cloud Monitoring
Too many KPIs! A minimum to make decisions Sa2sfac2on (EUEM) / Apdex Sa2sfac2on (actual) Conversion Revenue and be customizable An simple Dashboard
IP- LABEL Synthe2c RUM Load Tes2ng BAM
No 1 In Europe OFFICES & PARTNERS in 15 countries Join us at booth 16 TRUSTED THIRD- PARTY