Gaining Control of Virtualized Server Environments

Transcription

1 Gaining Control of Virtualized Server Environments By Jim Metzler, Ashton Metzler & Associates Distinguished Research Fellow and Co-Founder, Webtorials Editorial/Analyst Division Introduction The traditional data center network was based on an architecture and design tailored to satisfy the connectivity requirements of a relatively simple server infrastructure based on the following set of assumptions: Each server was dedicated to a single application and had a single IP address and a single MAC address. Nearly all of the traffic within the data center was client/server related. This type of traffic is often referred to as north-south traffic. The relatively small amount of traffic that went between servers was readily visible on physical switch ports. This type of traffic is often referred to as east-west traffic. The physical separation of servers satisfied most of the requirements for security and for the delineation of departmental resources. The static nature of the application environment lent itself to manual processes for server deployment or redeployment. VLANs tended to be used primarily for traffic isolation and to provide high availability between the access and distribution tiers. As a result, VLANs tended to be localized within application pods. There were clear lines of demarcation between the server and application management domain and the network management domain. Interestingly enough, the implementation of server virtualization has negated the validity of each of the assumptions listed above. As a result, the traditional approach to designing and managing data center networks is no longer valid. The goal of this white paper is to discuss what IT organizations must do to successfully meet the networking challenges associated with a highly dynamic, virtualized data center. In order to achieve this goal, three IT professionals were interviewed. One of the interviewees was the senior manager of global network design for a major health care organization. His organization doesn t allow him to be explicitly named in a document such as this, so he will be referred to in this white paper as The Senior Gaining Control of Virtualized Server Environments June 2011 Page 1

2 Manager. The other two interviewees were Aaron Larkins, the CEO of Profitability.net and Charles Hollingsworth, the manager of IT Architectural Planning at Georgia State University. The Adoption of Server Virtualization There are numerous benefits that can be derived from server virtualization, including: Server Consolidation Once it has been virtualized, a single physical server can support multiple virtual machines (VMs) and as a result, applications that would normally require a dedicated server can now share a single physical server. This enables IT organizations to reduce the number of servers in their data centers which yields a significant savings in both CAPEX (i.e., costs of server hardware, SAN Host bus adapters, and Ethernet NICs) and OPEX; i.e., server management labor expense, plus facility costs such as power, cooling, and floor space. Flexible Server and Application Provisioning A production VM can be transferred without service interruption from a given physical server to a different physical server, either within the same data center or between different data centers. This capability enables workload management and optimization across an IT organization s virtualized data center(s). This capability also helps IT organizations meet business objectives by: o o o o Streamlining the provisioning of new applications. Improving backup and restoration operations. Enabling zero-downtime maintenance. Enable Other Initiatives In order to more effectively manage and control user desktops, many IT organizations are implementing virtual desktops. Virtualized servers are a prerequisite for virtualized desktops because a virtual desktop implementation requires that a VM in a data center server hosts a complete user desktop including all its applications, configurations, and privileges. A recent market research report 1 contained the results of a survey that asked IT professionals about their organization s adoption of server virtualization and the impact of that adoption on networking and management. Throughout this white paper that report will be referred to as The Research Report. The Research Report presented the responses to a survey question in which the respondents were asked to indicate the percentage of their company s data center servers that have either already been virtualized or that they expected would be virtualized within the next year. Their responses are shown in Table Gaining Control of Virtualized Server Environments June 2011 Page 2

3 None 1% to 25% 26% to 50% 51% to 75% 76% to 100% Have already been virtualized 21.6% 33.0% 18.9% 15.1% 11.3% Expect to be virtualized within a year 12.4% 25.6% 21.9% 21.9% 18.2% Table 1: Deployment of Virtualized Servers 2 One observation that can be drawn from the data in Table 1 is that over the next year the deployment of server virtualization will continue to increase. For example, over the next year the percentage of IT organizations that will have virtualized the majority of their data center servers is anticipated to almost double. Georgia State University is an example of an organization that has made only a moderate deployment of server virtualization as they have virtualized roughly one third of their servers. The company that The Senior Manager works for is an example of an organization that has been very aggressive with implementing server virtualization. That company has already virtualized roughly eighty percent of their five thousand servers and their goal is to get as close as they can to having one hundred percent of their servers virtualized. The Senior Manager stated that they will not virtualize all of their servers in the near term in part because in some instances porting the code from a physical server to a VM is too expensive and in part because of the compliance concerns they have relative to some of their healthcare applications. As previously noted, once a server has been virtualized it can support multiple VMs. One measure of the efficiency, and hence the cost savings of server virtualization, is the average number of VMs on a single physical server. This number is typically referred to as the consolidation ratio. In the current environment, organizations are typically implementing consolidation ratios between 5:1 and 10:1. The consolidation ratio is expected to grow rapidly as IT departments become more familiar with virtualization technology and as they deploy multi-core server processors that are optimized for virtualization. The company that The Senior Manager works for is an example of a company that is driving cost savings by implementing a high concentration ratio. According to The Senior Manager they currently average fifty VMs per physical server and that number is likely to increase in part due to the fact that they are aggressively implementing 10 Gbps ports. IT organizations, however, will only be able to appreciate these added cost savings if they redesign their data center networks and change their approach to management. The Research Report indicated that most IT organizations understood that and were making progress in those directions. For example, The Research Report presented the responses to a survey question in which the respondents were asked Has your IT organization already redesigned, or within the next year will it redesign, its data center LAN in order to support cloud computing in general, and virtualized servers in particular? Those responses are shown in Table Gaining Control of Virtualized Server Environments June 2011 Page 3

4 Cloud Computing in General Virtualized Servers in Particular Already Have Will Within the Next Year No Plans 28.6% 42.9% 28.6% 50.5% 30.7% 18.8% Table 2: Redesign of the Data Center LAN The data in Table 2 indicates the significant impact that server virtualization is having on the design of data center networks. As discussed in The Research Report, some of the primary design changes that IT organizations are making to their data center networks in order to better support server virtualization include implementing: A 2-Tier network topology to improve traffic flows among servers. Edge Virtual Bridging (EVB) to provide scalable I/O performance and full visibility of VM-VM traffic on the same physical server. The combination of multi-chassis LAGs and virtual LAN switching to improve performance and availability via support for active-active Layer 2 switching and load balancing across redundant source/destination paths. The Research Report also indicated that most IT organizations understand the importance of modifying how they manage in a virtualized environment. In particular, The Research Report presented the responses to a survey question in which the survey respondents were asked to indicate how important it is to their organization to get better over the next year at managing some key tasks related to server virtualization. They were given the following five-point scale: 1. Not at all important 2. Slightly important 3. Moderately important 4. Very important 5. Extremely important Included in Table 3 are the tasks and the percentage of the survey respondents who indicated that the task was either very or extremely important for their organization to get better at over the next year. Gaining Control of Virtualized Server Environments June 2011 Page 4

5 Server Virtualization Management Task Importance: Very or Extremely Perform traditional management tasks 49% such as troubleshooting and performance management on a per VM basis Keep track of VMs as they are moved 38% between physical servers Dynamically move VMs, and all of the 37% supporting management functionality, between physical servers Discover VMs 33% Manage the traffic that goes between 31% VMs on a single physical server Table 3: Importance of Managing Server Virtualization As shown in Table 3, the survey respondents indicated that getting better at each of the individual challenges associated with server virtualization is important to their organization. In addition, it is reasonable to look at the five challenges contained in Table 3 as being a single challenge - managing server virtualization. When looked at that way, getting better at server virtualization is extremely important to IT organizations. Managing at the Virtual Machine Level The Senior Manager commented that server virtualization is great when it works but that they struggle when it doesn t. When there is a problem caused by server virtualization, the approach that they take is to get all of the appropriate personnel into a war room and tear apart the problem. He added that, this can take hours and can affect business. Hollingsworth concurred and said that, When looked at overall, server virtualization is wonderful, but there are some legitimate issues. He added that server virtualization increases complexity and makes it difficult to know what is running where and when. Below are some techniques and functionality that IT organizations can implement to respond to the challenges that are associated with server virtualization. Network Configuration at the Virtual Machine or Virtual Port Level In the traditional data center, physical servers were typically dedicated to a single application. Therefore, network policies could be implemented by configuring the physical port of the network access switch with the Access Control Lists (ACLs), Quality of Service (QoS) classifications, and the rate-limiting characteristics required for specific applications. With virtualization, multiple disparate applications can be running on VMs sharing a single physical server. IT organizations need the ability to manage network traffic using the same granular policies relating to security, QoS, and regulatory compliance for applications running on VMs as it did previously for applications running on dedicated physical servers. This can be accomplished if the network access switch is capable of mapping each VM to a separate, individually configurable virtual switch port. The network access switch management system can then maintain a profile for each VM, including the VM s IP address and virtual MAC address, VLAN memberships, ACLs and QoS classification. Gaining Control of Virtualized Server Environments June 2011 Page 5

6 The benefits of virtual ports can be maximized in those situations in which the physical network access switch s operating system includes support for switching all VM-to-VM traffic even when the source and destination VMs are residing on the same physical server. With this form of virtualized switching at the network edge, the physical access switch processes all VM traffic rather than having VM-to-VM traffic within the same physical server being switched by a hypervisor s software based virtual switch (vswitch). Virtual port profiles and edge virtualized switching are two subjects that are currently being addressed by the IEEE 802.1Qbg working group which is expected to deliver ratified standards within the next two years. The standards will include specifications for a Virtual Ethernet Port Aggregator (VEPA) in external access switches and a Virtual Station Interface (VSI) Protocol to associate and deassociate VM MAC Addresses with a virtual port profile. The way that VEPA will work is that it will offload networking tasks from the server s vswitch, which will in turn maximize the server s available CPU resources to support higher consolidation ratios. Virtual port capability, together with VEPA, also allows existing traffic monitoring and troubleshooting tools to gain visibility into all the traffic flows for each VM. This visibility makes it possible for IT organizations to continue to manage the availability and performance of applications. Additional benefits of VEPA include high performance, low latency, hardware-based switching for all VM traffic and the ability to minimize the number of switching tiers in the data center by potentially eliminating the hypervisor and blade server switching tiers. In heterogeneous hypervisor server environments, VEPA also eliminates the need to support the multiple vswitch architectures and management tools that result from using multiple vendors, further reducing complexity. The use of VEPA can also eliminate the organizational and economic stress that so many companies have experienced as both the network and the server organizations claimed or disclaimed responsibility or were forced to increase staffing to manage the multitude of vswitches. From the point of view of the network, VEPA functionality can be implemented on most switches as a relatively simple firmware upgrade rather than requiring new ASICs or other hardware. This means that data center networks can deploy pre-standard VEPA without running the risk of having to replace their access switches when the standard is finalized. A number of vendors already support pre-standard VEPA in their switches. However, for VEPA to work as intended, hypervisor vendors will need to provide software support for the server side of VEPA. While this also appears to be a relatively easy modification, the hypervisor vendors seem somewhat reluctant to embrace a pre-standard switching solution that can bypass their software switches. The Senior Manager stated that they are undergoing a proof of concept trial relative to using an external switch to switch between VMs. He added that in order to reduce organizational stress that his organization is discussing the creation of a data center group that is comprised of personnel who have a background in networking, servers, applications, storage and load balancing. Hollingsworth discussed some of the organizational challenges associated with server virtualization. One example he gave is the coordination needed between the network engineering group and the platform engineering group to make sure that vlans are properly exposed on network equipment, virtual switches and virtual servers. This requires a degree of coordination and synchronization of group priorities that was not previously required. For example, it is necessary to make sure the synchronization of vlans is not configured just for the initial host the VM is placed on, but also for any host where the VM might be moved. Hollingsworth added that debugging issues are also much more challenging. For example, in a conventional network connection the network engineering group can deploy packet capture and analysis tools between the host and the nearest switch. That is not always possible within a virtualized server environment where communications between VMs may never leave a physical server. In these cases, the network engineering group is dependent on the platform engineering group either capturing and supplying the appropriate management data for analysis, or Gaining Control of Virtualized Server Environments June 2011 Page 6

7 moving the appropriate VM from one server to another so that the traffic flow is forced to route to the physical network where network engineering can capture and analyze the data. The complexity of these issues are leading both the platform engineering group and the network engineering group to realize that the virtual switch provides an unnecessary layer of abstraction that doesn t help either engineering group. Dynamic Policy In addition to enabling server consolidation, another major benefit of server virtualization is derived from the relative ease with which a VM can be provisioned or migrated among physical servers. This enables the creation of a dynamic server environment in which the compute capacity devoted to an application can expand and contract in real time to meet fluctuations in user demand, and virtual servers can be easily relocated to balance load on physical server platforms or to ensure higher application availability. The Senior Manager stated that in order to support patch management and to ensure adequate application performance that they move VMs between servers in the same data center on a weekly basis. He stated that the majority of these moves are automated and that they would like to move VMs between servers in disparate data centers but that they are constrained both by the requirement to not cross a layer 3 network boundary as well as the difficulty associated with moving the associated storage. Larkins commented that they automatically move VMs between servers within their data center in response to a server failure. They also move VMs manually in order to improve performance. He added that he expects to more fully automate the movement of VMs in the near future as they support more applications built for cloud computing which include APIs that enable appropriate resources to be dynamically added or removed. When a VM is provisioned or migrated to a new physical server, changes in the configuration of the network are required to ensure appropriate policies are enforced for the application that is running in the VM. VMs can be created or moved in a matter of seconds or minutes but it can take hours or even days to implement the appropriate network configuration if manual processes are employed. The delays associated with reconfiguring the network can result in performance shortfalls, security vulnerabilities and gaps in regulatory compliance. Taking full advantage of the dynamic server environment made possible by server virtualization requires that the virtual port profile associated with a VM can be automatically created within the network and migrated to the appropriate access switch whenever a VM is migrated among servers. The dynamic creation and movement of port profiles can be achieved in those instances in which the switch vendor provides the right level of integration between the hypervisor management system and the switch element management system via APIs supported by both systems. This integration allows the hypervisor manager to automatically notify the switch management system when a VM is created or moved, so that the virtual port profile in the network is created or moved as required without human intervention. Fully automated configuration of virtual port profiles not only enables a highly dynamic server environment, but it also minimizes operational workloads, reduces the human errors that occur during configuration and preserves the traditional responsibilities of the server and network administrations. Deployment of multiple types of hypervisors within a single data center is becoming increasingly common. This means that switch element management systems need to support the APIs of each hypervisor management system employed within the data center. Gaining Control of Virtualized Server Environments June 2011 Page 7

8 Visibility The dynamic nature of virtualized server environments can make it difficult for IT organizations to have sufficient visibility to keep track of the status of each VM throughout its lifecycle. This lack of visibility can result in VM Sprawl whereby obsolete or low activity VMs proliferate and waste system, storage and network resources. This lack of visibility can also make it difficult for organizations to comply with the large and growing list of regulatory requirements. From a server management perspective, full visibility into a virtualized server environment requires system management tools that can extract detailed life cycle data from the various hypervisors management systems. This capability enables the management system to catalog each VM in terms of characteristics such as its time of deployment, the version of the OS it is running, its location, the ownership of the VM, the allocated resources, the workload history and the patch status. In a similar fashion, network managers also need a network management tool that can both gather information about, and report on the network services provided to VMs. In those instances in which the data center network supports virtual port profiles, much of the data required by network managers can be collected via enhanced capabilities within the switches element management system. VM location and inventory data can be stored in a network management database where it can be correlated with both historical and real-time network performance statistics. This provides a valuable aid in troubleshooting application performance issues and verifying the correctness of network configurations. VM-aware network element management systems allow network managers to have direct access to the information required to troubleshoot virtualized server networking issues without the time delays that are typically involved when network managers must coordinate manually with the appropriate server management personnel. Larkins stated that driven in part by their regulatory requirements that his clients want to know where their data is at all times. As a result, he needs detailed visibility into the creation and movement of VMs in order to provide that insight. He added that his customers expect that he will be monitoring the systems that he provides to them and that to do this, his company Monitors as much as we can, collects as much data as we can, and trends that data as much as we can. Gaining Control of Virtualized Server Environments June 2011 Page 8

9 Summary The traditional data center network was based on an architecture and design that was tailored to satisfy the connectivity requirements of a relatively simple server infrastructure based on a variety of assumptions. This included the assumption that each server was dedicated to a single application and had a single IP address and a single MAC address and that the static nature of the application environment lent itself to manual processes for server deployment or redeployment. While server virtualization delivers efficiency, flexibility and agility to application provisioning and management, it has negated the validity of most, if not all, of the assumptions that were used to design traditional data center networks. As a result, IT organizations must develop a new architecture and design for their data center networks. As part of their new approach to architecting and designing data center networks, IT organizations need to manage network traffic using the same granular policies relating to security, QoS, and regulatory compliance for applications running on VMs as it did previously for applications running on dedicated physical servers. This can be accomplished if the network access switch is capable of mapping each VM to a separate, individually configurable virtual switch port. This topic is the subject of standards currently being developed by the IEEE. In order for IT organizations to take full advantage of the dynamic server environment made possible by server virtualization requires that the virtual port profile associated with a VM can be automatically created within the network and migrated to the appropriate access switch whenever a VM is migrated among servers. The dynamic creation and movement of port profiles can be achieved in those instances where the switch vendor provides the right level of integration between the hypervisor management system and the switch element management system via APIs that are supported by both systems. Given the dynamic nature of virtualized server environments, in order for IT organizations to have full visibility into these environments requires a system management tool that can extract detailed life cycle data from the various hypervisors management systems. In a similar fashion, IT organizations also need a network management tool that can both gather information about, and report on the network services provided to VMs. In those instances in which the data center network supports virtual port profiles, much of the data that is required by network managers can be collected via enhanced capabilities within the switches element management system. Gaining Control of Virtualized Server Environments June 2011 Page 9

10 About the Author Jim Metzler Jim has a broad background in the IT industry. This includes being a software engineer, an engineering manager for high-speed data services for a major network service provider, a product manager for network hardware, a network manager at two Fortune 500 companies, and the principal of a consulting organization. In addition, Jim has created software tools for designing customer networks for a major network service provider and directed and performed market research at a major industry analyst firm. Jim s current interests include cloud networking and application delivery. About the Webtorials Editorial/Analyst Division The Webtorials Editorial/Analyst Division, a joint venture of industry veterans Steven Taylor and Jim Metzler, is devoted to performing in-depth analysis and research in focused areas such as Metro Ethernet and MPLS, as well as in areas that cross the traditional functional boundaries of IT, such as Unified Communications and Application Delivery. The Editorial/Analyst Division s focus is on providing actionable insight through custom research with a forward looking viewpoint. Through reports that examine industry dynamics from both a demand and a supply perspective, the firm educates the marketplace both on emerging trends and the role that IT products, services and processes play in responding to those trends. For more information and for additional Webtorials Editorial/Analyst Division products, please contact Jim Metzler at jim@webtorials.com or Steven Taylor at taylor@webtorials.com. Published by Webtorials Editorial/Analyst Division Division Cofounders: Jim Metzler jim@webtorials.com Steven Taylor taylor@webtorials.com Professional Opinions Disclaimer All information presented and opinions expressed in this publication represent the current opinions of the author(s) based on professional judgment and best available information at the time of the presentation. Consequently, the information is subject to change, and no liability for advice presented is assumed. Ultimate responsibility for choice of appropriate solutions remains with the reader. Copyright 2011 Webtorials For editorial and sponsorship information, contact Jim Metzler or Steven Taylor. The Webtorials Editorial/Analyst Division is an analyst and consulting joint venture of Steven Taylor and Jim Metzler. Gaining Control of Virtualized Server Environments June 2011 Page 10