Network Virtualization and Data Center Networks 263-3825-00 Data Center Virtualization - Basics Qin Yin Fall Semester 2013 1
Walmart s Data Center 2
Amadeus Data Center 3
Google s Data Center 4
Data Center Evolution 1.0 Computer rooms Large and special installations Supported mainframe systems since 1950 Accommodated central processing units (CPU) and peripherals (storage devices, terminals, printers ) Features Based on a monolithic software architecture A tighter control from an IP perspective A high resource utilization 5
Data Center Evolution 2.0 Background Client-server application model in the 1980s Personal computers Mainframes -> low-end servers Features Low cost of computer hardware Servers accommodated into distributed and improvised DC (closer to the clients) Scarce bandwidth of WAN links 6
Data Center Evolution 3.0 Background Internet boom in the 1990s -> Internet data centers Internetworking and web-based applications -> Increased server utilization Client-server model -> Application tiers (dedicated servers) Problems Space and power saturation Low resource utilization 20% of the capacity on average Root cause: application silos 7
Data Center Evolution 3.0 Infrastructure consolidation Improve resource utilization Increase operational simplicity Virtualization technologies Isolation of environments in a consolidate structure Aggregation of discrete resources into a shared pool Simplification of operational procedures 8
Origins: Virtual Memory CPU directly access a virtual memory address Virtualization Address mapping Data transfer Page replacement 9
Origins: mainframe virtualization IBM System/370 in 1972 Hypervisor VM creation Resource sharing Device management Virtual storage mgmt Etc. Virtual machine Prehistoric virtualization Time sharing Same OS No unequal resource allocation Same failure domain 10
Origins: Hot Standby Router Protocol One default gateway Inherent single point of failure Steps: Router send HSRP Hello Discover and decide which to emulate the virtual IP Active interface fails, a new router will emulate 11
Summary: Virtualization Emulation Main memory, mainframe, default gateway IP address Transparency Consumers cannot make a distinction Benefit Memory expansion, resource optimization, high availability Virtualization is the transparent emulation of an IP resource producing to its consumers benefits that were unavailable in its physical form. 12
Virtualization Type and Subtype Pooling Several physical elements work simultaneously to form a single logical entity that shares characteristics with the original entities Homogeneous & Heterogeneous Abstraction Emulation doesn t increase or decrease the number of physical elements to create a logical entity Address remapping & Structural Partitioning Independent logical partitions that emulate the characteristics of the physical resource Resource allocation & No resource allocation 13
Virtualization Scalability Pooling Max number of devices that can be pooled together Abstraction Max number of address remappings per device Partitioning Max number of partitions 14
Virtualization Areas and Subareas 15
Classification Examples 16
Data Center Network Evolution Ethernet Protocol Frame Broadcast Carrier sense multiple access / collision detection Ethernet devices Hub, repeater, bridge, switch, router 17
Ethernet Data Rate Timeline 18
Data Center Network Topologies Core-aggregationaccess layered data center architecture Network modularity Flexibility Resilience 19
Data Center Network Layers Hierarchical network 1+1 redundancy Equipment higher in the hierarchy handles more traffic, is more expensive (scale-up design) 20
Core Layer Provides the high-speed packet switching backplane for all flows going in and out of the data center Provides connectivity to multiple aggregation modules provides a resilient Layer 3 routed fabric with no single point of failure. Runs an interior routing protocol, such as OSPF or EIGRP Load balances traffic between the campus core and aggregation layers 21
Aggregation Layer Aggregation layer modules provide important functions, such as Service module integration Layer 2 domain definitions Spanning tree processing Default gateway redundancy Server-to-server multi-tier traffic flows through the aggregation layer and can use services, such as firewall and server load balancing, to optimize and secure applications. The smaller icons within the aggregation layer switch represent the integrated service modules which provide services, such as Content switching Firewall SSL offload Intrusion detection Network analysis, and more. 22
Access Layer Where the servers physically attach to the network The server components consist of 1RU servers Blade servers with integral switches Blade servers with pass-through cabling Clustered servers Mainframes with OSA adapters The access layer network infrastructure consists of Modular switches Fixed configuration 1 or 2RU switches Integral blade server switches Switches provide both Layer 2 and Layer 3 topologies, fulfilling the various server broadcast domain or administrative requirements. 23
Design Factors of Data Center Networks Application bandwidth demand Oversubscription Failure domain sizing Application resilience All the factors should be prioritized 24
Access-aggregation Connection Options: Looped Triangle Topology Arguably the most widely deployed in data centers Deterministic characteristics and flexibility Access-to-aggregation oversubscription remains constant in the case of Uplink failure Aggregation switch failure STP does not allow all deployed uplinks to be used 25
Access-aggregation Connection Options: Looped Square Topology Increases the access layer switch density Each access switch demands only one connection to the aggregation layer Traffic oversubscription to the aggregation layer doubles if An aggregation switch fails Uplink fails 26
Access-aggregation Connection Options: Loop-free U Topology No blocked paths because A loop cannot be formed STP is still recommended Like looped square Allows a higher number of access switches per aggregation pair An optimized use of uplinks But Allow one pair of access switches per L2 domain Any switch connection failure will stop all L2 communication 27
Access-aggregation Connection Options: Loop-Free Inverted U Shares all advantages from U topologies Allows more than one pair of access switches on a single L2 domain Uplink or aggregation failures are tricky "black-hole" the server traffic 28
Physical Network Layout - TOR Intra-rack cabling between Servers Small switches Pro Con Reduce cabling Optimize the space A high number of devices 29
Physical Network Layout - EOR Inter-rack cabling between Servers High-density switches Pro Reduce network devices Optimize port utilization Cons A lot of horizontal cabling 30
Network Logical Partition Consolidation is a definitive trend Network partitioning to address Traffic isolation for groups of hosts Distinct security areas Different path behavior Shared failure domains Virtualization 31
Virtual LAN 32
Defining VLANs A VLAN can be defined as a broadcast domain in a single Ethernet switch or shared among connected switches. Within each VLAN A switch emulates an Ethernet bridge Forward Ethernet frames based on their destination MAC address Each port of a VLAN defines a collision domain 33
VLAN trunks Use access ports to connect VLANs Need as many connections as the number of VLANs VLAN trunks Transport multiple VLANs over a single Ethernet interface Each frame has a tag that contains a VLAN identifier 34
IEEE 802.1Q VLAN Tagging 35
Two IP Subnets Sharing a VLAN Direct IP communication occurs among hosts that belong to each IP subnet Every host receives all broadcast and flooded frames from both subnets 36
Two VLANs Sharing a Subnet Layer-2 device Used to bridge both VLANs in a single broadcast domain Traffic analysis Acceleration Content security Load balancing Advantage Traffic manipulation without additional switch deployment or recabling 37
Case study Internet Data Center Layer 3 Internet CR CR AR AR AR AR Layer 2 LB S S LB S S S S Key: CR = L3 Core Router AR = L3 Aggregate Router S = L2 Switch LB = Load Balancer A = Rack of 20 servers with Top of Rack switch 38
Internal Fragmentation VIP: the IP to which requests are sent DIP: the IP of the server over which the request is spread Popular load balancing techniques (destination NAT) require all DIPs in a VIP s pool be in the same L2 domain Fragmentation and under-utilization of resources 39
No Performance Isolation VLANs used for: security, service isolation, traffic management, etc. One service sending/receiving too much traffic hurts all services sharing its subtree Reconfiguration of VLAN trunks painful, error-prone, slow, often manual 40
Limited Server-to-Server Capacity Data center run two kinds of application: Outward facing (serving web pages to users) Internal computation (computing search index like HPC) Comm. between servers in different L2 domains must go through L3 network bw. bottleneck 41