Data Center Evolution and Network Convergence

Transcription

1 Joseph L White, Juniper Networks Author: Joseph L White, Juniper Networks

2 SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced in their entirety without modification The SNIA must be acknowledged as the source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the author nor the presenter is an attorney and nothing in this presentation is intended to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney. The information presented herein represents the author's personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. 2

3 Abstract FC, FCoE, NAS, iscsi, DCB, traditional LAN, internet/wan, HPC, clusters, clouds, server virtualization, storage virtualization, network virtualization, and more are all colliding in your data center. Redundancy, resiliency, security, I/O consolidation, network convergence, dynamic application distribution, and thin provisioning with high levels of service are desired at all layers and all data center sizes across a broad spectrum of use cases. You worry about operational separation, buying decisions, investment protection, cost and energy savings, and evolving standards while maintaining very high levels of service and security. Is the technology evolving to a dream come true or a nightmare? If that doesn t keep you up at night nothing will. This tutorial will untangle, define, and illustrate the main ideas and concepts behind to give context and a solid foundation for discussions with your vendors as well as for your further reading and investigation. The point of view taken for this presentation is that of the network and transport characteristics in the face of the changes taking place. 3

4 Agenda What is Network Convergence? Definitions Why would Data Centers Evolve in this direction? Trends & Pressures Advantages How do Deployments and Infrastructure change Deployment models Supporting Protocols 4

5 Data Center LAN (today) built from Multi-Tier Trees Aggregation Core Routers SSL VPN Firewall IPSec VPN L2/L3 Switch L2/L3 Switch L2/L3 Switch L2/L3 Switch Servers Servers, NAS, Campus/MAN/WAN across switched network Multi-Tier 100 s to many1000 s of ports multi-link redundancy 100s of meters max diameter oversubscribed East-West Latency can be a problem Ethernet carrying predominantly IP traffic Access Firewalls and security in aggregation layer have to be distribute in the data path due to efficiency forced by oversubscription 5

6 Data Center FC SAN (today) Servers Storage across switched network Core Edge or Edge Core Edge in effect one level of tier collapse is done 10 s to 1000 s of ports Full Dual Rail Redundancy 100s of meters max diameter High Bandwidth, Low Latency Lossless Links Fabric Services provide Discovery, Access Control, and Change Notification Gateways and specialized extension devices provide remote access for BC/DR Attached Appliances provide data services Encryption, Block Virtualization 6

7 Network Convergence disk disk SAN disk LAN Running disparate network traffic types across common physical infrastructure Servers Network Convergence Disparate network traffic Block Storage (FC, FCoE, iscsi) Networked file systems (NFS, CIFS/SMB, CAS) Server LAN access High speed clustering and transactions Common Physical Infrastructure disk disk disk Ethernet with multi-protocol switches Fabric based distributed switches Servers Infiniband 7

8 Agenda What is Network Convergence? Definitions Why would Data Centers Evolve in this direction? Trends & Pressures Advantages How do Deployments and Infrastructure change Deployment models Supporting Protocols 8

9 How do the pressures balance? Convergence is in progress, but there is a balance between various factors influencing the adoption rate Existing Operational Models and Infrastructure INHIBITORS Cost of 10G/40G Infrastructure Politics / Team Issues Technology Maturity Data Center Network Convergence Development of best practices for phased deployment of convergence 10GE Performance Cost Reductions Convergence Solves Actual problems of scale, flexibility, and complexity ACCELERATORS On the whole the Accelerators are outpacing the Inhibitors Server Virtualization Network Evolution New Data Center Models 9

10 Overall Trends First Model isolated systems direct attach storage Second Model limited networking direct attach storage Third Model Networking Explosion direct attach storage + local cluster server-server storage Third and a half Model Flexible storage via NAS and network file systems Fourth Model SANs for block storage attach plus fully entrenched NAS Fifth Model ( current one ) Server Virtualization drives first hop I/O consolidation, increased SAN attach Cloud (pooled resources of all kinds with uniform distributed access) Evolving Model Network Convergence Protocols for SAN and LAN on same infrastructure Network Scaling via virtualization and simplification (tier collapsing, distributed control planes) Increasing Scale Increasing Features 10

11 reflected in the Data Center Consolidation Server Trends Mega DCs; 400K sq ft 4K racks, 200K servers Multi-core (8->16 >32,.128, ) Virtualization and VMs Apps Apps Guest OS Guest OS... Hypervisor (VMM) Mgt Hardware DC Scale Want Low Oversubscription Application Trends Interconnect Trends SOA, Web 2.0 MapReduce, Hadoop, Grids Banking System Web Service Request - Pay Balance (SOAP over HTTP) Make_Payment - Web Service Credit Card Systems Convergence to 10 GE Enhancements to Ethernet IP Stack Application Middleware HPC Stack FC Stack iscsi Stack Web Service Response Transaction Confirmed (SOAP over HTTP) DCB Link 10 GigE Physical Increased East-West traffic Large speed increases 10/40/100 GE 11

12 Servers and Storage Attach High End Servers with high IOPS & performance MF / Unix Mid range servers Unix / Wintel Storage is High End High IOPS Storage with Reasonable balance of capability & performance DB tier application tier SAN Attach started at the high end SAN Attach Penetrating lower Server virtualization driving towards 100% attach Rack Dense & blade servers Wintel / Linux Storage with minimal load, no persistent data, just VM images Presentation tier Server virtualization started at the low end Steadily penetrating higher Needs shared storage for VMs 12

13 Leading to Infrastructure Explosion Function of new services, digital media, application design and deployment topologies Fueled by Moore s law and ever increasing price performance Driving issues of managing massive performance and scale 11.8M 15.8M 522 EB 44 EB/yr Servers in US 1 New Storage Shipped 1 (per year) Global IP Traffic (per year) 2.6M 2.6M 54EB 1 EB/yr 146 EB 5 EB/yr 1 Source: IDC Remember: Complexity increases exponentially with scale 13

14 Virtualization OF EVERYTHING Aggregate up and Virtualize down many examples such as storage arrays, servers,... avoid Accidental partitioning embrace Deliberate partitioning Aggregation Physical and Software Bring together and pool capacity with flexible connectivity Virtualization logical partitions of the aggregated systems to match actual need flexibility fungible resources everywhere Utility Infrastructure with just in time & thin provisioning THIS IS HAPPENING TO NETWORKS AS WELL 14

15 Virtualization Drives Storage Connectivity... because Data Centers are always in flux Application life cycle services introduced, updated, retired Load on servers and networks constantly changing can be unpredictable Resource management challenge Minimize the need for excess capacity Reconfigure Reclaim/Reuse Adding resources is last resort Dynamic shared resource pools address these issues Enabled by Virtualization + Full Connectivity Networks Any servers potentially needs access to any storage Drives SAN attach from 20% to near 100% If you don t converge you will end up connecting everything to everything anyway but across additional parallel networks. 15

16 Virtual to Physical Server Trend Physical Server Installed Base (Millions) Logical Server Installed Base (Millions) Complexity and Operating Costs are still present Capital Savings Millions Installed Servers Source: IDC 16

17 Agenda What is Network Convergence? Definitions Why would Data Centers Evolve in this direction? Trends & Pressures Advantages How do Deployments and Infrastructure change Deployment models Supporting Protocols 17

18 Hasn t Convergence already happened? For some aspects of Convergence: YES NAS Allows access to file based storage across the network iscsi Allows access to block based storage across the network SANs have been bridged across metro and wide area networks for 10 years (FCIP, ifcp, & proprietary) FCoE provides an accepted protocol for FC across an Ethernet These are good but not sufficient Data center LANs have issues at scale WAN IP SAN connections do not solve the Local Data Center problem Operational Characteristics of FC based SANs desirable and entrenched for many applications 18

19 Network Convergence Benefits results in logical overlays for forwarding on single, shared HW infrastructure Benefits stocking of spare FRUs combined operations fewer stranded resources better utilization lower latency better flexibility 19

20 Agenda What is Network Convergence? Definitions Why would Data Centers Evolve in this direction? Trends & Pressures Advantages How do Deployments and Infrastructure change Deployment models Supporting Protocols 20

21 Network Convergence Deployments What does the network look like when you overlay the SAN picture... onto the LAN picture You can have converged infrastructure at several places in the network. 21

22 Convergence Considerations Protocols for storage access Block Storage (FC, FCoE, iscsi) Networked file systems (NFS, CIFS/SMB, CAS) Physical Infrastructure Ethernet with multi-protocol switches Fabric based distributed switches Infiniband End to End performance bandwidth & latency & frame processing rate congestion handling logical overlays onto physical infrastructure MAN and WAN extension for storage Under convergence the local network is Ethernet. The devices that make up the local Ethernet are not typically suited to directly support lossless distance extension. 22

23 Using FCoE for Network Convergence Let s now focus on the case of FCoE and convergence in a Data Center as an example of the deployments and considerations Scaling FCF at the TOR hits Domain Scaling problems FC-BB-6 addressing this with FCF-FDF virtual domains Configuration of L2/L3 separation VLAN Virtual Fabric mapping L2 network as access between server and FCF Full FC topology overlay onto data center network Multi-hop FCoE across multiple L2 networks and several FCFs Multiple VLAN/Virtual Fabric Configuration and Management 23

24 l I/O Consolidation CNA CNA N I C N I N C I C N I N C I H C B A H B H A B A N I C N I N C I C N I N C I H C B A H B H A B A N I H C B A H B H A B A I/O Consolidation disk nas vtl tape firewal vpn Unified storage Multiple network connections Separate subnets Separate VLANs May be separate networks May be separate switches in the same network Each configured with different QoS settings For Example Something like this is possible: Campus Access (1+) LAN Backup NIC (1) Application Cluster NIC (1) Vmotion NIC (1) Presentation to application to database private NICs (1+) Storage Access Cards (2) CNA CNA CNA CNA Run multiple independent virtual networks planes across a single physical infrastructure not JUST at the Edge not JUST at the SAN I/O Consolidation reduces this to 2 NICs per server (more only if you need the bandwidth) Unified services 24

25 Converged Deployments for FC/FCoE Server to FC SAN via NPV Gateway with integrated L2 bridge 25

26 Converged Deployments for FC/FCoE server to FC SAN via external NPV Gateway

27 Converged Deployments for FC/FCoE server direct to FCF with integrated L2 bridge

28 Converged Deployments for FC/FCoE Server to FCF via L2 bridges 28

29 Full Convergence goes beyond I/O Consolidation SAN A SAN B Separate Networks No overlap of operations & management separate network characteristics separate teams run them (typically) SAN SAN A B I/O Consolidation overlap confined to server and 1 st hop team consensus needed several detailed ways to accomplish transit switch L2 to access SAN gateway SAN-LAN Convergence unified management and operational model run by single team common redundancy shared bandwidth and connectivity 29

30 Converged Deployments for FC/FCoE Converged SAN/LAN Using FCoE and FCFs with integrated L2 Bridges With FC-BB-6, some of these FCFs can be FDFs 30

31 Converged Deployments for FC/FCoE Converged SAN/LAN Using FCoE and FCFs 31

32 For this to be a reality What does convergence need? Transport Convergence I/O Consolidation out of server Switch forwarding plane aggregation Tier Collapsing Ethernet Fabrics, not to be confused with FC Fabrics Protocol Evolution FCoE (FC-BB-5/FC-BB-6) iscsi (especially over DCB networks) DCB for Ethernet is complete Operational Collections of physical switches acting as single logical switch Scaling of all management tools and infrastructure 32

33 Constructing the Ethernet DC LAN The DC LAN can be built out of L2 Bridges using multi-tier trees The DC LAN can be built out of a fabric based distributed switch The DC LAN can be built out of hybrid of one of the above and FCF/FDF integrated L2 Bridges Would typically have the multi-tier tree characteristics 33

34 Constructing the DC LAN: Protocols In all DC LAN cases the IEEE DCB (Data Center Bridging) and some IETF protocols are required for a properly functioning infrastructure PFC (Priority Flow Control) provides lossless operation Required for FCoE and any high speed SAN traffic where congestion is possible ETS (Enhanced Transmission Selection) Allows configuration of endpoint bandwidth DCBX Allows capabilities information to be exchanged CN (Congestion Notification) Allows the DC LAN to react to congestion by notifying endpoints to slow down L2 multi-path Allows all physical paths to be utilized Not as important with fabric based infrastructure Non-DCB protocols still important to the DC LAN VLAN Allows logical organization and overlay onto a physical infrastructure Link Aggregation - Allows multiple physical links between bridges to act as though they were a single link. QoS/CoS Allows network to manage and prioritize traffic STP (Spanning Tree Protocol) 34

35 Data Center LAN: Multi-Tier Trees Local Area Client Server applications And so we wired the data center the same way Data Center Client Server Applications Need more redundancy: STP and disabled links OR lots of VLANs 20 years ago the Ethernet switch was introduced to solve the LAN problem And it became the basic building block of the network SOA Applications etc Up to 75% of traffic Adding in the complexity of storage networking as well, these pressures force the Ethernet network to change. 35

36 Why not Multi-Tier Trees? BECAUSE Location matters in a tree architecture Bubbles of Optimal Performance Appliances and VLANs create Shadows of Accessibility VM One Hop VM 36

37 Instead use Fabric Infrastructure Simplify the Data Center LAN by aggregation and consolidation Aggregate Switches Multiple physical switches that operate as a single logical device for both management and traffic forwarding Collapse Tiers Use an aggregated switch to do the work of multiple tiers of switches Allows aggregated access to services This gives another way to scale for Network Convergence Create A Fabric Use both techniques at the same time to build a fabric based infrastructure 37

38 Last Words Convergence happening along multiple lines I/O Consolidation Well established now and of direct benefit to server SAN-LAN Fabric Convergence Common Equipment, Shared Infrastructure Multiple Protocols Possible Tier Collapse Local networks replaced by high capacity distributed fabrics Flatter, fewer Tiers, higher utilization Network Virtualization underway Large benefits derived from scaled and converged components Cloud Deployments benefit from Convergence Cloud does not change the fact that there are very large centralized data centers that need scale, one of the levers to achieve scale is network convergence 38

39 Q&A / Feedback Please send any questions or comments on this presentation to SNIA: tracktutorials@snia.org Joseph L White Simon Gordon Gunes Aybay Charles Waters Andy Ingram Many thanks to the following individuals for their contributions to this tutorial. - SNIA Education Committee 39