Network Virtualisation in Testing

Transcription

1 Network Virtualisation in Testing Richard Bishop 27 th January

2 Introduction Trust IV Trust IV is an independent specialist provider of software QA solutions and professional IT services across the software development lifecycle. Established 2005 HPE Gold Partner Based in Manchester, UK Richard Bishop Head of Service Delivery at Trust IV Co-leader of Vivit in the UK 17 years experience in IT consultancy, primarily in software risk management Key experience in Finance, Telco and Retail sectors 2

3 2020 vision and challenges Accurately predicting the impact of what if scenarios Unknown or uncontrollable influences Variety of client access methods and channels Source: Cap Gemini / Sogeti WQR

4 Live vs Test environments Can Development you accurately / Unit-Test mimic production Environment usage in your test environment? Tester s PC Application Server Database / Back-end Server 4

5 Live vs Test environments Integration Testing environment Tester s PC Other apps Application Servers Multiple Database / Back-end Servers Remote / third-party sites 5

6 Live vs Test environments Acceptance Testing environment Tester s PC / devices Shared Networks and Cloud Applications Other apps Multiple Database / Back-end Servers Remote / third-party sites 6

7 Live vs Test environments Possible problems Tester s PC / devices Shared Networks and Cloud Applications Other apps Multiple Database / Back-end Servers Remote / third-party sites 7

8 Live vs Test environments Problems resolved Shared Networks and Cloud Applications NV Other apps SV SV NV SV SV Multiple Database / Back-end Servers NV SV Remote / third-party sites 8

9 Real World example Trust IV Blog A day in the life of a forgetful performance engineer 9

10 Real World example Trust IV Blog A day in the life of a forgetful performance engineer 10

11 Third-Party links FaceBook 28 th September 2015 Source: venturebeat.com & catchpoint.com 11

12 Network Virtualisation Concept 12

13 NV use cases Uncontrolled / unreliable connectivity 13

14 3G Mobile Quality at Old Trafford 14

15 3G Mobile Quality at Old Trafford Thursday afternoon During the derby game Before game During game During game Average Latency: 42ms Min Latency: 37ms Max Latency: 62ms Packet loss 0.00% Download 3.4Mbps Source coordinates: (53.46,-2.29) Destination: m.bbc.co.uk Duration: 2min Date and time: 10/22/ :44 Connection: Cellular data network Carrier: vodafone UK OS: ios Device model: iphone 4 Average Latency: 45ms Min Latency: 39ms Max Latency: 124ms Packet loss 0.00% Download 4.0Mbps Source coordinates: (53.46,-2.29) Destination: m.bbc.co.uk Duration: 2min Date and time: 10/25/ :37 Connection: Cellular data network Carrier: vodafone UK OS: ios Device model: iphone 4 Average Latency: 43ms Min Latency: 39ms Max Latency: 153ms Packet loss 0.00% Download 5.4Mbps Source coordinates: (53.46,-2.29) Destination: m.bbc.co.uk Duration: 2min Date and time: 10/25/ :41 Connection: Cellular data network Carrier: vodafone UK OS: ios Device model: iphone 4 15

16 3G Mobile Quality at Old Trafford Test Case Conditions 3G phone with HSPA support iphone 4 specs Expected Results High latency ( >200 ms ) High packet loss ( > 1-2 % ) Limited bandwidth ( 1 Mbps ) Poor mobile data performance Actual Results Variable latency ( ms ) No packet loss ( 0% ) 3-5Mbps throughput ( 4.7Mbps avg ) Good but variable network performance 16

17 3G Mobile Quality at Old Trafford Analysis / Observations 5 Masts with 11 transmitters 4.7 Mbps throughput No packet loss Variable latency Unpredictable network 17

18 Mobile website performance Monitored key betting sites during Manchester derby at Old Trafford on 25 th October

19 Observations BBC website for comparison Occasional spikes vs BetFred and Bet365 similar tech BetFred slow before 10am Early kick-off football games F1 Grand Prix Austin, Texas RWC Semi-final Twickenham PaddyPower and SkyBet similar PaddyPower fastest overall Showing all charts confusing Midday to 4pm shows busy periods Trendline of all betting sites shows two pre-game peaks 19

20 Is 4G always better? 20

21 Is 4G always better? Average Latency: 70ms Min Latency: 31ms Max Latency: 294ms Packet loss: 0.79% Download: 3.32Mbps Average Latency: 67ms Min Latency: 46ms Max Latency: 747ms Packet loss: 0.43% Download: 13.64Mbps 21

22 HPE Network Virtualization* * Network Virtualisation 22

23 Impact of mobile users Mobile Client Network & Security Resources Web Server DB Server Request Connection Pool Resources Response Bandwidth Packet Loss Jitter Latency Session Resources Database Resources 23

24 NV in performance testing WITHOUT accurate network conditions VS WITH accurate network conditions Capacity > 500+ Users Capacity < 200 Users 24

25 HPE Network Virtualization Products NV Analytics Analytics recommends performance improvements Grades applications / websites for mobile / desktop performance Simple results analysis, results shared easily within project teams. Now built into LoadRunner NV Test Manager Test Manager manages network profiles during Manual functional tests Automated functional tests e.g. LeanFT, UFT HPE Mobile Center tests NV Server NV Server can (optionally) be used to store test results User management and pooling of NV reports. Options to use servers to gather network statistics. Network Catcher Express ios and Android Client software Real world performance measurements. Contributes to world-wide database as well as providing personalised network performance statistics for testing. 25

26 NV in standalone mode Client and NV server AUT Servers 26

27 NV in proxy mode NV server configured as MITM proxy server. Default port Multiple MITM instances supported. Clients configured to use NV server as MITM proxy server. Individual flows can be configured based on IP address of client device AUT Servers 27

28 NV as a router NV Test Manager 2 NICs configured as router 28

29 NV in performance tests 3G 4G Wifi Controller Load Generator (s) Application Under Test Results analysis Filtered by location (network profile) Performance improvement recommendations Mobile or Desktop recommendations Drill down to resource usage etc. 29

30 Video Demo Mobile Center & NV integration (functional tests) NV & LoadRunner integration (performance tests) 30

31 Closing statement NV within the application development lifecycle Test composite and cloud-based apps more effectively Improve performance for all users Optimise the mobile user experience 31

32 Closing statement Retail sites Increasingly catering for mobile users Vulnerable to slow users consuming resources StormRunner Quality Center Performance Center Betting sites Customers impatient / less loyal - impact on customer retention NV SV Vulnerable to extreme variations in network Performance, require detailed testing Network Virtualization Service Virtualization Mobile Center ALM Sprinter BPT LoadRunner UFT What if testing only possible with SV / NV together 32

33 Questions Richard Bishop Trust IV 33