Expert Reference Series of White Papers. Planning for the Redeployment of Technical Personnel in the Modern Data Center

Transcription

1 Expert Reference Series of White Papers Planning for the Redeployment of Technical Personnel in the Modern Data Center

2 Planning for the Redeployment of Technical Personnel in the Modern Data Center Raymond B. Dooley, CCSI, Global Knowledge Course Director Introduction The modern data center is a highly technical combination of servers, memory devices, management workstations, security devices, software, network-related services, equipment racks, Ethernet LAN switches, multilayer switches, cabling, power systems, cooling systems, fabric switches, and a few people. From the standpoint of providing for the exchange of information to and from a client and one of the servers in the data center, the process is divided into two clearly defined halves. The Ethernet LAN side consists of all the switching, routing, and security technology for the client PC to establish and maintain a TCP/IP connection to the server. So, the LAN side consists of Ethernet technology and TCP/IP technology. The data center is composed of many complex components and protocols, including: The three-layer model Server attachment Choosing a platform and general issues One rack unit switching example Layer 2 or Layer 3 access High availability Services Layer and left vs. right traffic flow Copyright 2014 Global Knowledge Training LLC. All rights reserved. 2 Planning for the Redeployment of Technical Personnel in the Modern Data Center Raymond B. Dooley, CCSI, Global Knowledge Course Director Introduction The modern data center is a highly technical combination of servers, memory devices, management workstations, security devices, software, network-related services, equipment racks, Ethernet LAN switches, multilayer switches, cabling, power systems, cooling systems, fabric switches, and a few people. From the standpoint of providing for the exchange of information to and from a client and one of the servers in the data center, the process is divided into two clearly defined halves. The Ethernet LAN side consists of all the switching, routing, and security technology for the client PC to establish and maintain a TCP/IP connection to the server. So, the LAN side consists of Ethernet technology and TCP/IP technology. The data center is composed of many complex components and protocols, including: The three-layer model Server attachment Choosing a platform and general issues One rack unit switching example Layer 2 or Layer 3 access High availability Services Layer and left vs. right traffic flow Copyright 2014 Global Knowledge Training LLC. All rights reserved. 2

3 Data Center Architecture Overview Web, application, and database multitier environments Layer 2 and Layer 3 access topologies Layer 2 adjacency requirements Dual and single attached 1RU and blade servers Mix of oversubscription requirements Multiple aggregation modules Stateful services for security and load balancing Campus Core Data Center Core Service Modules Data Center Aggregation Layer 2 Clustering and NIC Teaming Mainframe Blade Chassis Blade Chassis with OSA with Integrated with Pass-Through Switch Data Center Access Layer 3 Access The three-layer model consists of access layer, aggregation layer, and core layer and is used to describe functionality as well as hardware and software components that support TCP/IP and LAN traffic into and out of the data center. These functions are summarized in the graphics. It is the best way to simplify and explain data center functions and design as well as help isolate management and troubleshooting issues. Benefits of the Three-Layer Model Layer 2 domain sizing Service module support Supports a mix of access layer models Supports NIC teaming and high availability clustering Campus Core Data Center Core Service Modules Data Center Aggregation Layer 2 Clustering and NIC Teaming Mainframe Blade Chassis Blade Chassis with OSA with Integrated with Pass-Through Switch Data Center Access Layer 3 Access Copyright 2014 Global Knowledge Training LLC. All rights reserved. 3

4 Server Attachment Methods Layer 3 Layer 2 Only One Link Active: Fault Tolerant Mode All Links Active: Load Balancing EtherChannels Server attachment at the access level is shown in the graphic. To support the four (99.99) and five (99.999) high availability objective, most servers are dual-homed to two separate access Ethernet switches for redundancy. This link can be in active/standby or active/active (load balancing) mode. There are also server network interface cards( NICs) that support Cisco EtherChannel for bundling multiple Ethernet links. Considerations for Choosing an Access Layer Platform Cabling design requirements Cooling requirements Power requirements Density 10-Gigabit Ethernet uplink support Resiliency features Intended use The choice of server platform has a big impact on many components in the design of a data center as shown in the graphic. A constant trade-off of benefits vs. disadvantages must be weighed to properly design a modern data center. This paper covers only the main points. Copyright 2014 Global Knowledge Training LLC. All rights reserved. 4

5 Choosing Between Layer 2 and Layer 3 Cabinet Design with 1RU Switching Access Designs 1RU designs Aggregation minimize cabling from the cabinet. 1RU designs have several issues: Cooling requirements limit the number Access of servers. Additional Gigabit EtherChannel or 10-Gigabit Ethernet uplinks are used. Higher STP processing is Layer 3 Layer 2 required. Management complexity is adjacencies increased. Multiple 1RU switches may be needed for port density. NIC teaming and adjacencies High-availability clustering and VLAN extension Custom application requirements Rapid PVST+ or MST Layer 3 Layer 2 Layer 2 Layer 3 Other considerations: Aggregation Access Cabling Remains in Cabinets Ease in managing loops Faster convergence Link utilization on uplinks Broadcast domain sizing Staff skill set Oversubscription Single Rack 2 requirements Switches Dual Rack 2 Switches Service modules support and placement Since It would Layer seem 2 Ethernet that implementing switches are multi-layer used in the switches data center, at the Spanning access level Tree would Protocol provide (STP) a simple is required solution to avoid to many problems, Ethernet loops thereby and eliminating broadcast storms. the need When for STP. VMware However, is used, applications it is necessary such to as span VMware VLANs that between require access Layer 2 layer adjacency switches, and will so not converting work. So, to a modern Layer 3 data the center access design layer is includes not possible both Layer to support 2 and all Layer servers 3 access in the layer data switches. center. The graphic shows a typical 1 rack unit (RU) server/switch design using either standalone or blade servers. This design can be improved with Cisco Fabric Extenders and Nexus switch solutions to reduce cabling requirements. High Availability in the Data Center STP issues can be alleviated by deploying Rapid Spanning Tree Protocol (RSTP) or Multiple Spanning Tree Protocol (MSTP), or using Cisco features such as VSS (Virtual Switching System) with the 6500 switch, Virtual Port Channel (VPC), and Common Virtual Device failures Contexts in the (VDC) with Nexus switches. path from server to Layer 3 aggregation switch: Layer 2 Server network adapter Network links Access switch OSPF, EIGRP No Redundancy Implemented High Availability Recommendations Implemented The first step in implementing high availability in the data center is to ensure that a link or device failure will not prohibit access to the server. A proper design is shown on the right side of the graphic. Copyright 2014 Global Knowledge Training LLC. All rights reserved. 56

6 Integrated Service Modules + Layer 4 through Layer 7 services can be integrated in Cisco Catalyst 6500 Series Switches. Includes server load balancing, firewall, and SSL services Two deployment scenarios: Active/standby pairs (Cisco CSM, and Catalyst 6500 Series FWSM 2.x) Active/active pairs (Catalyst 6500 Series FWSM 3.1) Blades provide flexibility and economies of scale. Many additional services are required in a modern data center, such as: Server load balancers Firewalls Wireless support SSL (Secure Socket Layer) VPN termination Content services If Cisco 6500 switches are used at the aggregation layer, service modules can be utilized to provide these services (as opposed to separate hardware appliances). When Nexus switches are used at the aggregation level, service modules are not an option. Copyright 2014 Global Knowledge Training LLC. All rights reserved. 7

7 Scaling with Service Layer Switches Service layer switches can support service modules: Ideal for CSM and SSL modules Opens slots in aggregation layer for 10-Gigabit Ethernet ports Service Switch1 Data Center Core Service Switch2 Aggregation Access In this case, a separate set of 6500 switches can be attached at the aggregation level to provide these services or separate appliances such as the Cisco Advanced Security Appliance (ASA) may be used. Active STP, HSRP, and Service Context Alignment Align server access to primary components in the aggregation layer: STP root is configured with the spanning-tree vlan vlan_id root primary command. Primary HSRP instance is configured with the standby priority command. Active service context is configured by the placement of service modules. Provides more predictable design: More efficient traffic flow Simplified troubleshooting Root Primary HSRP Primary Active Context Core Root Secondary HSRP Secondary Standby Context To explain all the required functions of the service modules and other components while maintaining a four or five-nines high availability objective, it is necessary to describe traffic flows into and out of the data center. Based on the graphic, traffic going to or from the data center must traverse the aggregation switch on the left side because that is where all the services (firewall, load balancer) are active. The same services are standby (in grey) on the right side of the aggregation switch. It is the purpose of the services and switches on the right side Copyright 2014 Global Knowledge Training LLC. All rights reserved. 8

8 to back-up the left side. This arrangement can be implemented on a VLAN-by-VLAN basis, so roughly one half of the traffic goes and comes on the left and one half goes and comes on the right. This is called the active/standby and sometimes called the active/active model. The left vs. right idea is the same either way. For this sophisticated solution to work correctly, several items must fit together properly. For VLANs assigned to the left, the following should be configured on the left aggregation switch: Spanning Tree Root for VLAN(s) HSRP, VRRP, GLBP (First Hop Redundancy Protocols) active (same VLANs) Firewall active (same VLANs) Load balance active (same VLANs) All other services active (same VLANs) The right aggregation switch would be the STP root and active on all services for the other roughly one half of the VLANs. The Storage Area Networking (SAN) side includes all the switching, routing, and security technology for the server to establish and maintain a Fiber Channel connection to a data storage device. SAN Overview Benefits: Separates storage from the server (historically). High-performance interconnection can provide high I/O throughput. Storage is accessed at a block level. Storage can be shared among servers. The SAN fabric is interconnection hardware. Fibre Channel SAN LAN Servers Block Storage Devices The SAN environment includes a layer of fabric switches to switch file requests over Fiber Channel links. As the network expands, multiple SAN environments can be separated for scalability with Virtual Storage Area Networks (VSANs) and Inter-VSAN routing (IVR) is possible with Fiber Channel Shortest Path First (FSPF), which is similar to Open Shortest Path First (OSPF) for IP routing (which is a different world, however). Copyright 2014 Global Knowledge Training LLC. All rights reserved. 9

9 Major SAN Design Factors Port density and topology requirements Device oversubscription Traffic management Fault isolation Convergence and stability QoS and Congestion Control High Performance Crossbar Large Port Count Directors Host Host Host Host Host Host Failure of One Device Has No Impact on Others Reduce FSPF Routes There are several protocols for delivering the SAN traffic payload (Small Computer System Interface [SCSI]), which has been the standard protocol for disk access since PCs were invented and still is. Note that in the graphic, IP is being used as the transport protocol when SANs are extended over distance. Storage Protocols: FCIP vs. iscsi FCIP FC is tunneled over IP. Connects SAN to SAN. Supports demanding QoS requirements. iscsi SCSI commands are carried using IP. Connects host to storage. Supports demanding QoS requirements. SCSI Applications (File Systems and Datsbases) SCSI Block Commands SCSI Stream Commands SCSI Commands, Data, and Status Parallel SCSI Fibre Channel Fibre Channel Interfaces FCIP Other SCSI Commands TCP IP iscsi Parallel SCSI Interfaces Fibre Channel Ethernet The SAN deployment described leads to the design of two parallel networks in the data center. Copyright 2014 Global Knowledge Training LLC. All rights reserved. 10

10 Parallel Data Center Infrastructure Data Center SAN Traditional data centers often deploy multiple parallel infrastructures: LAN for network connectivity SAN for storage connectivity LAN SAN A SAN B Access Layer Ethernet Fibre Channel HPC Data Center LAN Technical advances in data center networking have been made that allow unification of LAN/SAN. A Converged Network Adapter (CNA) allows the server to use the same adapter to send both Ethernet frames and Fiber Chanel frames bundled inside Ethernet frames Fiber Channel over Ethernet (FCoE). The Cisco Unified Computing System solution includes an interface for the blade server rack that includes FCoE capability. I/O Consolidation in the Data Center I/O consolidation integrates multiple data center infrastructures on a single unified fabric. Benefits: Reduced cabling Fewer access layer switches Fewer network adapters per server Power and cooling savings Management integration Wire once cabling model LAN SAN A LAN SAN B SAN A SAN B Copyright 2014 Global Knowledge Training LLC. All rights reserved. 11

11 The business case for this idea is solid. Removing one layer of switching along with the cabling lowers the cost of equipment, power, and cooling. The resistance to this idea by SAN professionals has been consistent. They see a future where the SAN network no longer exists and everything they have built and operated is collapsed into the LAN side even though the skill sets of the personnel on either side are completely different and they are basically left out, so their resistance is understandable. The SAN professional s point-of-view could include the following: A system that has proven reliable and performed well is being changed. SAN professionals do not know LAN and vice versa. Arbitrary decisions are being made that affect my life and career. I do not think the new technology will work. Will I have job? The following ideas would be beneficial to the transition process: Go over the business case in detail with both sides. Explain that Fiber Channel technology is not being eliminated, it is being encapsulated and SAN expertise will continue to be needed. Explain that interexchange of SAN and LAN expertise benefits the careers of professionals on both sides. Provide a clear career path and career counseling for anyone whose job is affected. Whatever solution is selected for this transition, TRAINING will play a key role. The SAN engineers experience will fall into two categories: 1. The engineer is experienced with Fiber Channel network and has limited experience with the Ethernet/IP/switching and routing environment. 2. The engineer has some experience with the Ethernet/IP switching and routing environment from working with that side of the house in the past. To complete an ideal transition for the SAN personnel, the following approach could be used to have the SAN folks achieve Cisco CCNP certification, which would make them full-fledged routing and switching engineers while retaining all of their SAN skills: Category 1: ICND 1 ICND 2 ROUTE SWITCH TSHOOT DCUFI NEXUS DCNX1K v2 - Configuring the Cisco Nexus 1000V DCNX5K - Implementing the Cisco Nexus 5000 and 2000 v2.0 DCNX7K - Configuring Cisco Nexus 7000 Switches v2.1 Copyright 2014 Global Knowledge Training LLC. All rights reserved. 12

12 Category 2: ROUTE SWITCH TSHOOT DCUFI NEXUS DCNX1K v2 - Configuring the Cisco Nexus 1000V DCNX5K - Implementing the Cisco Nexus 5000 and 2000 v2.0 DCNX7K - Configuring Cisco Nexus 7000 Switches v2.1 Cisco and Global Knowledge are close to offering the equivalent of a CCNA and CCNP certification specializing in the data center. It is basically the same courses with a Data Center focus. The benefits of this training strategy are: The SAN staff is assimilated into the network group. The network group now has more skills than before. A truly integrated network can be supported. About the Author Ray Dooley, BS, MBA, CCSI, CCNA, CCNP, CCDA, CCDP, SE, FE, has been a network professional in several capacities for over 30 years. He is the Global Knowledge Course Director for CCDA, ARCH, SWITCH, ROUTE, TSHOOT, and ICMI. He has done course development for Global Knowledge, Cisco Systems, and GE. Copyright 2014 Global Knowledge Training LLC. All rights reserved. 13