Matěj Grégr RESILIENT NETWORK DESIGN 1/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Campus Best Practices - Resilient network design Campus Best Practices documents share knowledge within several technical areas (physical infrastructure, campus networking, wireless, security, etc.) Campus Best Practice Documents are available at: http://www.terena.org/activities/campus-bp/bpd.html Resilient network design is described mainly in: Recommended Resilient Campus Network Design, March 2010 (CBPD114, the Czech Republic) Recommended configuration of switches in campus networks, May 2010 (UFS105, Norway) 2/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Resilient network design Enterprise campus requires highly available and secure network infrastructure, to support business solutions such as voice, video, wireless, and mission-critical data applications. Resiliency Ability to provide non-stop business communication with rapid sub-second network recovery during abnormal network failures or even network upgrades. The goal of resilient topology is to eliminate downtime and convergence time during crashes and device upgrades 3/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Network design 1 Single broadcast domain Single security domain No backup Central switch performance 4/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Network design 2 Routers can separate network Smaller broadcast domains Possibility to control traffic path Routers are pretty expensive Number of ports in an ordinary router is limited 5/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Resilient Campus design Servers and users access ports Aggregation traffic from access layer High speed switching/routing 6/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Access layer Entry point for clients into the network Provides Layer 2 (VLAN) connectivity between users High port density PoE Security mechanism 802.1x QoS classification 7/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Access layer Problems: Access switches are single points of failure in a network Redundant connection for end users is very expensive Resiliency has to be integrated into the device Redundant supervisor, power outlet Recommendations: Disable Etherchannel and trunk negotiation for end users Prevents VLAN hopping attacks Enable edge ports (PortFast) for access ports 8/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Server access layer redundancy Servers need redundant network connection Possible solutions: Link aggregation protocol (LACP, PAgP) Ethernet card bonding Server virtualization Service load balancing 9/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Link aggregation (Etherchannel) Allows combination of several physical links to one logical channel Physical view Logical view Load balancing MAC, IP, IP+TCP/UDP Simplify configuration only logical port configuration is necessary Simplify other protocol operation STP sees only Etherchannel link 10/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Link aggregation protocol Two signaling protocols exist (PAgP, LACP) LACP IEEE 802.3ad is recommended, PAgP is Cisco proprietary Modes: Active/Passive: request/response channel establishment On/Off: static configuration Recommendation: Use static configuration (mode on): dynamic configuration delays channel establishment 11/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Etherchannel configuration Prerequisites to established channel Same speed/duplex Same mode (trunk same vlans enabled, access same access vlan) Same STP cost and mode (edge/non-edge) Cisco Switch1(config)# interface range gi 0/1-2 Switch1(config-if-range)# channel-group 1 mode active HP Switch1(config)# trunk g1-g2 trk1 lacp 12/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Ethernet card bonding Useful, when server does not support aggregation protocol Linux - supported in kernel Windows supported in Ethernet card drivers Several modes: backup transmit load balancing: load balance in transmit direction adaptive load balancing: rewrite MAC addresses, different peers use different MAC address, no switch support is necessary 13/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Server virtualization Virtualization brings simplification in network resilient design Virtual servers are connected through Virtual Switch Virtual switch redundant connection to resilient network All virtual servers have resilient connection to Internet 14/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Server Load Balancing SLB provides a virtual server IP address to which clients can connect, representing a group of real physical servers in a server farm Load balancing: according to: L4 - L7 information Software implementation Hardware implementation Cisco Application Control Engine modul needed Advantages: Reduced server load Higher security real IP address is not visible Downtime elimination if more servers are used 15/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
SLB modes Dispatched mode Every server in server farm has own real IP together with virtual IP address (secondary IP or loopback IP) of whole server farm Traffic redirection: packet with virtual IP is put in Ethernet frame with MAC address of real server All servers in SLB farm have to be in same IP subnet Directed mode Every server has only own real IP address Servers do not know virtual IP address of whole server farm NAT is used virtual IP address of the farm is translated to real IP address of a server 16/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Distribution layer Purpose is to provide L2 distribution through switched network Topology contains loops (needed for redundancy) L2 loop protocol (STP) is needed Gateway redundancy, high availability Packet filtering, 17/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Spanning tree protocol Necessary for loop elimination 802.1D Original version has very long convergence time > 30s Did not support VLAN support added in 802.1t, extend BID, now integrated into 802.1D Recommendation is to use RSTP (802.1w), RPVSTP+ or MSTP (802.1s) 18/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Rapid Spanning Tree (RSTP) IEEE 802.1w Convergence time < 1s Backward compatibility with 802.1D Several Cisco 802.1D improvements were integrated into 802.1w standard (UplinkFast, BackboneFast ) Configuration: Cisco: Switch(config)# spanning-tree mode rapid-pvst HP: Switch(config)# spanning-tree force-version rstp-operation 19/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
STP load balancing Scenario: Port-priority or port cost can be used Example: Left(config)# interface gi1/6 Left(config)# spanning-tree vlan 200 port priority 112 Recommendation: Useful is to set higher priority on undesired port instead of setting lower priority on desired port 20/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
MSTP STP or RSTP support only one STP tree for all VLANs RPVSTP+ (Cisco proprietary): STP tree per every VLAN Main idea of MSTP: Administrator can configure several STP instances VLANS are mapped to instances MSTP internally use RSTP 21/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
MSTP configuration Switch(config)# spanning-tree mode mst Switch(config)# spanning-tree mst configuration Switch(config-mst)# name region1 Switch(config-mst)# revision 1 Switch(config-mst)# instance 1 vlan 100 Switch(config-mst)# instance 2 vlan 200 Configuration needed for every switch Increase complexity Proprietary solution: Use VTPv3 22/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
VTPv3 More flexible protocol can distribute any database Better authentication VTP can be turned on/off per port Client can not rewrite database as it was common in previous versions Server/client/transparent switch for databases VLAN, MST, Unknown (another database in future) Primary/secondary server Primary server (only) can modify a database Only one server in a domain Secondary server: backup server, could be primary 23/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
VTPv3 MSTP configuration Configure VTPv3 Switch(config)# vtp version 3 Switch(config)# vtp domain NAME Switch(config)# vtp mode server mst Switch(config)# end Switch# vtp primary mst MSTP configuration similar to previous slide MSTP config is distributed within VTPv3 domain 24/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Gateway redundancy Historic attempts Proxy ARP, ICMP Router Discovery Protocol, routing support in end station Does not scale well, software support is needed Solution: Redundancy using virtual router Virtual IP, virtual MAC No host configuration needed Proprietary solutions HSRP, GLBP Standard solution VRRP Virtual Router Internet, Backbone, etc. 25/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz Forwarder Backup in Standby
VRRP Open standard IETF RFC 3768 version 2 IETF RFC 5798 version 3 (IPv4 + IPv6) VRRP Group virtual router with virtual IP address Virtual MAC address - 0000.5e00.01xx - last byte is group number Master router Highest priority IP address same as virtual IP (IP address owner) always win master role Backup router Other routers in VRRP group 26/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
VRRP configuration Cisco configuration SwitchA(config)# interface vlan10 SwitchA(config-if)# ip address 10.1.10.5 255.255.255.0! Virtual IP for vrrp group 10 SwitchA(config-if)# vrrp 10 ip 10.1.10.1! Priority for router in group 10 (standard priority is 100) SwitchA(config-if)# vrrp 10 priority 150! Preempt delay SwitchA(config-if)# vrrp 10 preempt delay minimum 380 HP configuration hp (config)# vlan 223 hp (vlan-224)# vrrp vrid 1 hp (vlan-224-vrid-1)# owner hp (vlan-224-vrid-1)# virtual-ip-address 10.1.10.5 255.255.255.0 hp (vlan-224-vrid-1)# enable Recommendation Use the first IP address from subnet for Master router Set preempt-delay-time to let routing protocol converge 27/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Gateway Load Balancing Protocol HSRP, VRRP may have inactive routers in a group Standby and Other routers: cannot be used by end station (do not have virtual IP/MAC) Possible solution: several HSRP/VRRP groups with end stations distributed among them Static configuration! GLBP goal is to utilize all routers equally Several members of GLBP group should participate in packets switching/routing GLBP solution Virtual IP per group max. 4 virtual MAC addresses per group 28/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
GLBP fundamental concept GLBP group contains two types of members Active Virtual Gateway Active Virtual Forwarder Active virtual gateway (AVG) Router with highest priority (highest IP address) There is only one AVG per group Assign virtual MAC addresses of other members of the GLBP group Reply to virtual IP ARP requests GLBP group controller: end device requesting virtual IP address obtains some of the assigned virtual MAC addresses Active virtual forwarder (AVF) Max. 4 AVF per group, other routers are in backups AVF are responsible for assigned virtual MAC/IP address AVG is also AVF Communication is done via Hello messages every 3 sec, multicast address 224.0.0.102 UDP 3222 29/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
GLBP operation 30/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
GLBP load balancing techniques Weighted load-balancing algorithm Based on weighting parameter Per-host (Host-dependent load-balancing algorithm) End station has always the same AVF Round-robin load-balancing algorithm (default) Virtual MAC addresses are rotated per ARP request 31/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Core layer High speed routing Fast convergence is necessary Aggregate links from distribution layer Try to avoid any packet manipulation, (access lists and filtering), which would slow down the switching of packets. Smaller campus can combine core and distribution layer functions 32/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Is the core layer necessary? 33/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Is the core layer necessary? 34/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
Summary Resilient network design eliminates downtime and convergence time in a network If is it properly deployed Always depends of How much money do You have 35/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz
36/36 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, igregr@fit.vutbr.cz