OF 1.3 Testing and Challenges

Transcription

1 OF 1.3 Testing and Challenges May 1 st, 2014 Ash Bhalgat (Senior Director, Products), Luxoft Santa Clara, CA USA April-May

2 Agenda OpenFlow : What and Why? State of OpenFlow Conformance Challenges with OF 1.3 Testing Mitigation ideas Luxoft Twister open source framework References

3 UNINTERRUPTED GROWTH PERSONNEL CLIENTS 20 OFFICES 1 HIGH RETENTION REVENUE CAGR 26% FY 2011 TO FY YEARS TEST EXPERIENCE ONF MEMBER SINCE 2012 ONF CONTRIBUTIONS (OF TEST SPECS, PLUGFEST, OF DRIVER COMPETITION) Data LUXOFT as of November AND/OR 18, ITS 2013 AFFILIATES. ALL RIGHTS RESERVED LUXOFT AND/OR ITS AFFILIATES. ALL RIGHTS RESERVED. 3 3

4 OpenFlow : What and Why? Single control plane controls several forwarding devices Routing TE Mobility Specialized Features Specialized Control Plane Network OS Control plane physically separate from data plane Packet Forwarding Packet Forwarding Specialized Packet Forwarding Hardware Packet Forwarding Packet Forwarding Packet Forwarding

5 Open SDN Ecosystem Application Layer SDN apps Northbound API SDN Controller OpenFlow Agent Linux OS Linux OS Commodity Server Open Compute Switch Control plane Data plane Site (DC, Campus, POP)

6 Twister Test Execution Framework ONF OpenFlow Roadmap OpenFlow 1.1 Multiple tables VLAN & MPLS support OpenFlow 1.3 Stable long-term release ONF approved Extensions Meters, PBB, event filters Dec Feb Dec Apr Aug OpenFlow 1.0 First version Single table OpenFlow 1.2 Flexible match IPV6 support OpenFlow 1.4 Bundles, eviction Optical ports, flow monitoring

7 Twister Test Execution Framework State of ONF OpenFlow Conformance ONF released OpenFlow v1.3 as the stable release ONF s Current Conformance Certification Program Currently only available for OpenFlow v1.0 OpenFlow v1.3 conformance certification program slated for 2H 14

8 OF 1.3 Testing Challenges: Functional Validation OpenFlow 1.3 specification : mandatory and optional functionality OF 1.0 experience shows that not all vendors supports the entire OF spec and we expecting the same in OF1.3 Vendors can support support OF 1.3 conformance at various levels Only mandatory functionality is supported There are a lot of optional functionality that is important and can make a difference L2 only support L3 only support Full conformance ( mandatory + optional is supported at all levels ) Mitigation plan need good tools to verify at all levels and all functionality

9 OF 1.3 Testing Challenges: Inter-operability Conformance (enforced through standards) does not assure interoperability. However, non-conformance definitely means non-interoperability. - Dan Pitt, Executive Director Open Networking Foundation(ONF) What we observed at ONF Plugfest in Fall 2013 Different vendor implementations limit inter-operability despite conformant products Spec misinterpretation between vendors Hardware limitations differs between vendors; even if a switch is OF compliant, the HW limitations can make a difference in stress conditions Error handling is implemented differently by every vendor Mitigation plan need good tools to verify positive and negative testing

10 Interoperability Test Methodology OF Controller Test tool OF Switch 1 OF Switch 2 OF Switch 3 OF Switch 4 Agent Agent OF Controller configures the multi vendor switches to use the red data path Agents can inject traffic and verify that the traffic pass through OF controller can break the red data path Agents can inject traffic and verify that the traffic DOESN T pass through OF Controller configures the multi vendor switches to use the green data path Agents can inject traffic and verify that the traffic pass through Test tool collects statistics from OF switches for both routes Any other inter-operability scenario can be supported

11 OF 1.3 Testing Challenges: Hardware Limitations Not an issue for SW switches Multi-table capabilities dependent on hardware support Hardware switch vendors support full or partial implementations One table only or Multi-table implementations Every table specialized on specific matches so the pipelining is not possible Supports all tables but the number of OF flows is limited Mitigation plan tool to test various above scenarios

12 OF 1.3 Testing Challenges: Scalability and Performance OpenFlow is designed to be highly scalable protocol How to test switch scalability for millions of flows? How fast a switch can generate a packet-in to the controller? How this metric is impacted by a large number of flows already added in switch tables? How fast a switch can add new flows? Simple flows Complex flows Add in multiple tables How does the pipelining impacts the switch performance? What is the performance impact due to traversing multiple tables? Mitigation plan Tools that can benchmark switch flow table capacity

13 OF 1.3 Testing Challenges: Controller Testing Functionality testing Challenge: how to ensure that OpenFlow or OpenDayLight controller conforms to OF 1.3 standard? Issues How many packet-in messages it supports and how fast it replies to these messages? How many flows it can handle and how the large number of flows impacts the packet-in reply message? How the controller behaves when a large number of switches try to register to it? Mitigation plan need test tool that can simulate above scenarios

14 Controller OF Conformance Test Methodology OF Application Test Tool NBI OF controller OF messages Agent Test tool includes an OF application Based on NBI, the OF application could configure OF controller to generate OF messages Agent emulates an OF switch Agent can verify if OF controller comply with OF spec.

15 OF 1.3 Testing Challenges: Protocol simulation (MPLS using OpenFlow) Source: Saurav Das, PhD Thesis, Stanford University Simplification of tradition MPLS networks using Openflow What are testing needs to simulate such environment? Configure the switches Generate traffic and verify the data path Change labels and verify the behavior Mitigation plan: need tool to simulate MPLS traffic using openflow switches

16 OF 1.3 Testing Challenges: Debugging Issues Commonly issues that need further debugging Switch doesn t parse correctly the OF messages Packets not forwarded correctly Flows not present in switch tables Mitigation plan: Common test tools needed OF enabled Packet Sniffers to verify the requests/replies to/from switch Tools to compare the flows present in switch tables and the flows injected from controller side Traffic generators

17 Twister Introduction Twister is a open source, test execution framework that can automate, integrate and manage quality assurance tests and processes. Contributed by Luxoft, source code on Github Freely available under Apache 2.0 license terms Generic core framework designed as QA Lab foundation Extensible w/ commercial modules (e.g. OF 1.3 Conformance) Can address variety of QA needs Functional Testing Integration Testing System Verification Testing Benchmark Testing Conformance Testing Interoperability Testing Regression Testing Application GUI Testing Santa Clara, CA USA April-May

18 Twister Architecture and Key Features Test cases Results DB Modules OF Controller Exec Engine #1 WEB server HTTP XML- RPC Central engine Exec Engine #2 Exec Engine #n DUT Twister Framework Spirent & IXIA Traffic Generator interface Web based user friendly GUI Easy of Use: Drag and Drop test cases Multi-tenancy & resource pool sharing Role Based Access Control (RBAC) Highly scalable due to Parallel and Distributed execution architecture Real time monitoring of test execution Flexible reporting, open schema for custom reports Support for major scripting languages (Python, TCL, Perl) Extensible via plugins for value added functionality Santa Clara, CA USA April-May

19 Twister Open, Extensible and Modular Integrated libraries: telnet, ssh ftp Spirent Test Center & IXIA interface Multithreaded execution Step by step execution Automatic setup/teardown test cases Product Plug-ins: GIT/SVN/Clearcase: Revision control Jenkins/Bamboo: Continuous Integration JIRA: Defect tracking Packet Sniffer: Traffic Capture Integrated execution scheduler Santa Clara, CA USA April-May

20 References Twister open source project : web page, demo video, documentation: Twister Github page: For additional information: [email protected]

21 Thank You!