Multi-site Datacenter Network Infrastructures Petr Grygárek rek 1
Why Multisite Datacenters? Resiliency against large-scale site failures (geodiversity) Disaster recovery Easier handling of planned outages Workload migration to unaffected site Traffic optimization choose ingress point closer to deliver service to requesting client 2
Interconnection of DC Sites L3 IP or MPLS Optionally L2 traditional design 3
Multi-site PoDs 12
L3 extension between DC sites Dedicated core (IP-only or MPLS) MPLS/VPN DMVPN... 4
Why to extend L2 between sites? Server admins like transparent VM mobility Distributed clusters FWs, Lbs, NASs,... Server clusters e.g. using NLB Application of originally local technologies into multi-site environments needs careful consideration Built-in timers etc 5
L2 Extensions between DC Sites Dual-site: P2P virtual links: QinQ, EoMPLS, AToM, Virtual chassis P2P L2 technologies (e.g. Cisco VPC/VSS) General topology: Redundant switched network with STP non-recommended! Distributed virtual chassis if latency between sites fits into solution's limits VPLS OTV THRILL/FabricPath 6
Transparent Interconnection of Lots of Links (TRILL) IEEE 802.1aq L2 multipath solution eliminates Spanning Tree, no stability issues no blocked ports reduced latency shortest path always used alternative active paths (equal-cost) path selection based on data packet header hash ensures ordered delivery ISIS-like routing L2 frame encapsulation new header carries egreess switch identity 2005 Petr Grygarek, VSB-TU Ostrava, Routed and Switched Networks 8
TRILL Principles Rbridge THRILL-capable bridge Ingress, egress, THRILL cloud internal Switches have identities, ISIS calculates shortest paths between switches ISIS chosen as it runs directly on L2 2-level switching hierarchy Only Rbridge addresses have to be known in core Smaller MAC address tables, better scalability ISIS TLVs also used to propagate current MAC address locations over TRILL cloud Backward learining still used to learn addresses from outtside of THRILL cloud Ingress Rbridge maintain <MAC,egress Rbridge> or <MAC,local port> records VLANs supported 2005 Petr Grygarek, VSB-TU Ostrava, Routed and Switched Networks 9
TRILL Frame Forwarding Rbridges may be interconnected via legacy Ethernet clouds Not meaningful in DC environment Cisco FabricPath (THRILL predecestor) does not support this Next-hop header allows passing THRILL frame over leagacy Ethenet cloud (even VLAN-based) DST MAC addreses next-hop Rbridge For each VLAN, single Rbridge is elected to avoid looping/frame duplication Ingress/egress header allows routing of TRILL frame to egress Rbridge Loop protection: Hop count (TTL) in TRILL header 2005 Petr Grygarek, VSB-TU Ostrava, Routed and Switched Networks 10
TRILL Multidestination Frame Forwarding For unknown unicasts, broadcasts/multicasts One or more distribution tree calculated covering all egress Rbridges Distribution tree (destination root switch) selection based on destination Rbridge address of encapsulated frame 2005 Petr Grygarek, VSB-TU Ostrava, Routed and Switched Networks 11
Multisite DC without L2 extension VM migration across different subnets (keeping original IP address) Keeping VM identity and established sessions Potential solutions Load-balancer frontend Mobile IP LISP 11