A Versatile and Scalable Testing Solution

Transcription

1 A Versatile and Scalable Testing Solution to Meet the Challenges of Testing WebRTC Services By: Chao Zhao Software Engineer at Valid8.com 1

2 The Nature of WebRTC How WebRTC integrates into IMS Four Stages in WebRTC Product Lifecycle, Test Challenges and Case Studies Stage 1 Prototyping Stage 2 Implementation Stage 3 Pre-deployment Stage 4 Post-deployment Conclusion 2

3 WebRTC (Web Real-Time Communication) is an API definition drafted by the World Wide Web Consortium (W3C) that supports browser-tobrowser applications for voice calling, video chat, and P2P file sharing without the need of either internal or external plugins. WebRTC is located in the junction between VoIP and the Web. 3

4 Traditional VoIP testing tools and paradigms would be great assets to WebRTC testing. The lessons learned in traditional POTS and VoIP domains can and must be translated into the WebRTC world. e.g. conformance testing and interoperability testing However, testing from the web perspective seems also reasonable. e.g. use a browser to test WebRTC applications 4

5 SIP ify WebRTC Browser WebRTC ify SIP Network 5

6 Prototyping Implementation Pre-deployment Post-deployment 6

7 Challenges WebRTC is still a work in progress. How to test new ideas? How to rapid prototype for stakeholder demo? 7

8 Wanted to prototype a WebRTC gateway to integrate with their SBC Purely telecom company, no former experience with web Needed to do it quick for stakeholder demo to get the budget 8

9 Solution A test tool continuously testing against latest WebRTC browsers and finding out what is changed from version to version A test tool with multi-domain\multi-protocol capability that can easily integrate into other networks Test Tool Chrome/other browsers Multi-domain\Multi-protocol Powerful testing oriented scripting language 9

10 Challenges How to make sure your implementation is correct and compliant with standard? How to develop test cases efficiently to cover all scenarios? How to do tests in parallel among different development teams? 10

11 Wanted to launch a IMS-based WebRTC service Chose the SIP ify WebRTC browser approach and organized a team to develop a JavaScript SIP library Another team worked on upgrading their SBCs to support WebSocket 11

12 Solution A test tool with a complete conformance test suite A test tool that can easily develop new test cases A test tool that can simulate non-standard behavior for negative testing Test Tool Chrome/other browsers Non-standard Behavior Powerful testing oriented scripting language 12

13 Challenges How to make sure the server can handle the load volume in field? How to scale up to the expected volume when testing? 13

14 Wanted to load test their WebRTC gateway for IMS REST for authentication socket.io for signaling OPUS & VP8 on WebRTC side, G.711 & H.264 on IMS side Scale from 1 to 1000 and two-way test 14

15 Can browsers scale up by themselves? How about building a load test center with 1000 PCs and hiring 1000 people to click the Call button at the same time? Or do it in a VoIP testing fashion generate load directly on the network level. 15

16 Solution A test tool that can scale efficiently A test tool that can generate load in multi-domain\multi-protocol A test tool that can 100% simulate end-user behavior Test Tool Chrome/other browsers Multi-domain\Multi-protocol Efficient scaling Real end-user script execution 16

17 Challenges How to get notified when service is unavailable? How to ensure QoS is always on a high level? How to reproduce the problem discovered in field? 17

18 7x24 Monitoring In the case of any abnormal events, the operator will get an Historical records will provide valuable guidelines in service improvement, e.g. frequent failure point, weak point Proactive Testing Update the test script accordingly when browsers have a newer version Reproduce Problems discovered in Field Deploy the test tool on the cloud closest to failure point 18

19 Solution A test tool that can run test case effectively 24/7 and report any failures A test tool that can be easily updated in accordance with any WebRTC update A test tool that can be deployed to field and send back diagnostics and metrics Test Tool Chrome/other browsers Easy deployment to cloud Event-driven state machine architecture Metrics and diagnostics 19

20 A testing solution evolved from the VoIP testing in the past decades will thrive in WebRTC world with its full skill set, contributing its versatility and scalability. Test Tool Chrome/other browsers Multi-domain\Multi-protocol Efficient scaling Non-standard behavior Powerful testing oriented scripting language Event-driven state machine architecture Metrics and diagnostics Easy deployment to cloud Real end-user script execution 20

21 Thank you! Chao Zhao t: x

22 Q&A session Test Tool Architecture (upon asked by audience) 22