Expert Reference Series of White Papers. vcloud Director 5.1 Networking Concepts

Expert Reference Series of White Papers vcloud Director 5.1 Networking Concepts 1-800-COURSES www.globalknowledge.com

vcloud Director 5.1 Networking Concepts Rebecca Fitzhugh, VMware Certified Instructor and Consultant Introduction A VMware vcloud is made up of one or more vcloud Director servers that are integrated with underlying vsphere components. The vcloud is a new abstraction layer above vcenter Server consuming the resources that vcenter manages; this allows a user to self-provision virtual environments utilizing memory, compute, storage, and networking resources. Cloud computing has become a vague, arbitrary phrase, but there are six characteristics that define exactly what a cloud should consist of self-service elasticity pay as you go multi-tenancy resource pooling ubiquitous access A private cloud is an infrastructure whose resources are only used internally. A public cloud is an infrastructure made available to external customers for a price. A hybrid cloud combines two or more clouds with some kind of standardized technology, like VMware vcloud Connector, while each cloud maintains its own unique identity. The foundation of the vcloud centers on the networking configuration. Networking occurs over three different layers: external, organization, and vapp; it is imperative to properly configure and manage these networks so that the vcloud can be consumed. Think of vcloud networking as an onion that will be peeled back to reveal each layer, starting with the organization s networks that are created by an administrator with the system administrator role in vcloud Director. A system administrator is the highest role within the vcloud. This white paper covers the different networking layers present in a vcloud environment as well as some other considerations in the configuration process with the assumption that the reader has some technical experience with vcloud Director. What is an Organization? An organization provides four core resources (compute, memory, storage, and networking) to a particular set of policies dictating how those resources can be consumed. In a private cloud, a business may have different cloud organizations indicative of their business structure: perhaps a Human Resources organization, Finance organiza- Copyright 2013 Global Knowledge Training LLC. All rights reserved. 2

tion, etc. A public cloud may specify different organizations for each of their external customers. Organizations receive their compute, memory, and storage resources from an object called a virtual datacenter. A system administrator can create multiple provider virtual datacenters to meet Service Level Agreement (SLA) requirements with each different provider specifying a different level of guaranteed service. Note that the networking resources don t come from virtual datacenters but from network pools. External Networks The first object that is created within vcloud Director is the External Network. An External Network provides the connection from the cloud to the outside world, allowing inter-cloud connections and is port group based. Even though this connection is called the external connection, an Internet connection is not actually required; this can be set up to provide a connection to several different internal entities, like ESXi hosts, without an actual route to the Internet. Since this connection is port group-based, then the port group needs to exist prior to attempting to establish the connection. The port group can be defined on a standard vswitch, a distributed vswitch, or on a Nexus 1000V. Organization virtual datacenters can use the external networks to provide Internet connectivity to the organizations and the virtual machines that reside within a vapp, given that the vapp network is configured for that. By creating an external network, vcloud Director is effectively configured to send all external traffic using the port group(s) selected. Should there be multiple external networks created then be sure to separate them by using VLANs. Only someone with the system administrator role within the vcloud can create and manage external networks. Figure 1. Selection of existing port group when creating external network. Copyright 2013 Global Knowledge Training LLC. All rights reserved. 3

Organization Network An organization network provides network services to one particular organization, whereas an external network is created at the provider level and supplies connectivity to multiple organizations. There are three options when creating organization networks: internal, NAT-connected, and direct-connected. An organization administrator cannot create an organization network due to the configuration of external IPs; only a system administrator can configure this. Internal An organization can be set up so that it does not have a connection to the Internet or a connection to any other external network, just an internal connection. An internal-only network could be set up for groups of test virtual machines; a virtual machine can be configured with multiple network interfaces so that it has a connection to the internal network as well as one of the other two types. With an internal organization network, vapps can connect, but there is no traffic outside the organization. Network Address Translation (NAT)-Connected Network Address Translation (NAT)-connected, sometimes called a routed network, can be connected to the external network through a vshield Edge device. The vshield Edge device provides port-forwarding services, NAT, DNS forwarding, and DHCP services to the network; the vshield Edge device gets provisioned automatically by vcloud Director as needed. A NAT connection allows for virtual machines to communicate with each other while only having one IP seen from the Internet. Another use of NAT is to fence, which includes two sets of IP addresses: external and internal. Fencing allows for several vapps to utilize the same internal IP addresses and extremely useful for test environments. Direct Communication The last option for an organization network is a direct connection. The organization would use an external network to connect to external systems, including the Internet. Using this method, a user can connect directly to a virtual machine using remote desktop or even SSH. If a vapp configured for a direct connection then the vapp s IP addresses must be statically assigned or a DHCP server must be connected to the external providing the vapp with those IP addresses. Copyright 2013 Global Knowledge Training LLC. All rights reserved. 4

Figure 2. Selection of network access type for organization network. Network Pools All cloud entities consume resources that are pooled; there is no exception for network resources. A portion of a network pool is used whenever an organization network or a vapp network is created and connected to the network layer above. Any time an organization network is created that is either NAT-connected or internal, a network pool is used. Also, all vapp networks use network pools. There are four types of network pools that can be created: VLANbacked port group-backed vcloud Network Isolation backed (VCNI) VXLAN Each pool can be used interchangeably, and each has its own set of requirements. The vswitch, Distributed vswitch, or the Nexus 1000V producing the service needs to have physical uplinks to enable communication beyond the host on which a given virtual machine resides. Copyright 2013 Global Knowledge Training LLC. All rights reserved. 5

Whenever an organization virtual datacenter is created, it is associated directly with a network pool. Each organization must have at least one organization network that is built off of network pools. The organization virtual datacenter can utilize multiple network pools, and one organization can be associated with more than one organization virtual datacenter. Multiple organization datacenters can utilize the same network pool. A maximum of 1,016 port groups can be created per vcenter, and VMware s best practice is to make the port groups have a maximum of 4096 ports instead of the default 128. For the port binding option, it is recommended that ephemeral, or no binding, is chosen for all preconfigured port groups. Ephemeral port binding is done automatically by vcloud Director for auto-provisioned port groups. Also to limit network names to 33 characters or shorter because vcloud Director adds a unique identifier, as long as 47 characters, to the end of the network name when a vshield Edge device is provisioned. VLAN-Backed Network Pools The VLAN-backed model is flexible, can be routed, and does not require any special MTU setting. This option requires a distributed vswitch and a set of unused VLANs. For this option, one or more VLAN IDs need to be specified, making sure not to overlap any existing VLANs. Also, for all VLANs specified in the pool, the physical environment needs to be trunked accordingly. Port groups are dynamically created by vcenter as the VLANs are used. Nexus 1000V and Standard vswitches are not currently supported in 5.1. Port Group-Backed Network Pools The port group network pool requires pre-created port groups within the vsphere environment and is, therefore, the least flexible of the different options. Since the port groups have to be pre-created, the VLANs have to be manually configured, and there is no automatic network deployment, so it can be difficult to manage. This option can utilize Standard vswitches, Distributed vswitches, and the Nexus 1000V so it is the only network pool option for those without Enterprise Plus licensing. There is a one-to-one ratio between the manually created port groups and the networks in the pool. vcloud Network Isolation (VCD-NI)-Backed Network Pools A vcloud Network Isolation-backed pool is driven by the VSLAD (vcloud Director) agent that runs on the ESXi hypervisor. A VCD-NI network isolates network traffic at layer 2. This method uses MAC-in-MAC encapsulation to tunnel traffic between ESXi hosts through the VMkernel module, attaching a packet header before the traffic hits the physical layer. Nothing changes on the vsphere layer when first configuring for this method as a network pool; no vshield device is deployed, and no new port groups appear until a vapp that is connected to this network is powered on. After creating a Distributed vswitch, a transport VLAN needs to be designated for carrying the encapsulated traffic. vcloud Director will create an overlay network for the specified VLAN for each isolated network, at which time it will be assigned a Network ID number. The network overlay encapsulates the data and ensures that it is isolated. The encapsulation contains information regarding the source and destination MAC addresses of the Copyright 2013 Global Knowledge Training LLC. All rights reserved. 6

ESXi host(s) where the endpoint is located as well as the Network ID. When the ESXi host receives the packet, the VCD-NI header is stripped off to expose the MAC address information so it can be delivered to the destination virtual machine. Because of this header, the packet is 1524 bytes instead of the normal 1500 bytes so the Maximum Transfer Unit (MTU) will need to be adjusted on the physical layer. Also, since an ESXi host is the only thing able to decode the packet header, this traffic is non-routable. All switches, Distributed vswitches and physical switches, need to have the MTU settings adjusted accordingly if planning to use this network pool type. Keep in mind that when using jumbo frames, the frame size would need to be reduced by 24 bytes to accommodate the encapsulation. Therefore, if jumbo frames is normally set to 9000 then the virtual machines guest operating system would need to be set to -24, with the MTU defined as 8,976 bytes. VXLAN In vsphere 5.1 and vcloud Director 5.1, VXLAN (virtual extensible LAN) support is introduced, providing the multi-tenant broadcast domains across datacenters enabling a logical network to span physical network boundaries. VXLAN allows compute resources to be pooled across non-contiguous clusters or pods and then segment this pool into logical networks attached to applications. This technology uses MAC-in-UDP encapsulation, adding a 24-bit identifier, providing a layer 2 abstraction to virtual machines regardless of physical location. The ESXi hosts have to be prepared through the vshield Manager (vcloud Networking and Security appliance) that requires a Segment ID Pool and a Multicast address assignment. Once the ESXi hosts are prepared, a VXLAN pool is automatically created. vapp Networks There are three types of network connections for a vapp Network: isolated, bridged, and NAT routed. Isolated networks are totally separate, no connection to another network. These are great for back-end communication such as communication between a database and a web server. So a second interface could be added to the web server and the database so that the traffic between the two servers is isolated and then a second interface could be added for a connection to the Organization network. A bridged network simply means that the vapp is directly connected to the Organization network. This method is commonly used for vapp that need to be accessed from anywhere within the Organization. In the vcloud Director User Interface, this connection is called a direct connection. Creating a vapp network that has a NAT connection to the Organization network results in the creation of a vshield Edge appliance that connects the two different networks. The vshield Edge appliance has two interfaces, internal and external, where the external is the Organization network and the internal is the vapp network. vshield Edge provides services like NAT, DHCP, Firewall and static routing to a vapp network. The term fenced refers to the fact that the vapp is somewhat isolated from the rest of the network. The isolation includes the MAC address of the virtual machines within the vapp, no virtual machine outside the vapp Copyright 2013 Global Knowledge Training LLC. All rights reserved. 7

will have visibility of the IP addresses and MAC addresses. In vcloud Director, this means that both the vapp network and the Organization network are on the same subnet. This idea doesn t seem special; however, the difference is that, with a fenced network, there is a vshield Edge device in between the networks. Connectivity There are many layers and types of cloud inter-connectivity and intra-connectivity networking that an administrator must be able to deploy and manage for the VMware vcloud environment. This can include Virtual Private Network (VPN) tunnels and static routes, as well as the use of VMware vcloud Connector (vcc). Multiple external networks can exist on the same physical LAN as long as they are separated by VLANs. An external network can be dedicated to a sole organization or shared across multiple organizations. A virtual machine within a vapp can be multi-homed; however, each virtual machine s vnic can only connect to one network. Virtual machines can be connected to both vapp networks and organization networks. Multiple vapp network and multiple organization network connections are possible for a virtual machine s vnic, and more than one virtual machine vnics can be connected to the same network. Two vapp networks cannot be connected directly to each other; both vapp networks should both be connected to an organization network for connectivity between them. A vapp network cannot be connected to multiple organization networks. Multiple vapps cannot connect to a single vapp network, but multiple vapps can be connected to each other through an organization network. An organization network cannot be directly connected to another organization. A network cannot be deleted from a vapp, whether the network is a vapp or organization, unless there are no virtual machines connected to it. VPNs A Virtual Private Network (VPN) tunnel is an encapsulated or encrypted network path through a hostile network space. A VPN is anchored on both ends by either a VPN device or a firewall; in the case of vcloud Director the VPNs are anchored by vshield Edge appliances. After a VPN tunnel connects the two systems, communication occurs as if the two devices were on the same network except that any system outside the tunnel cannot intercept the traffic. In vcloud Director, there are three types of VPN tunnels that can be created: VPN between two different organization networks within the same organization, VPN between two organization networks in two different organizations, or a VPN between an organization network and a remote external network. An organization administrator and a system administrator can create VPN tunnels. To create a VPN between two different organization networks within the same organization, both networks must be external; a NAT connection cannot be established to an internal organization network. Both networks must be NAT-connected to the same external network with non-overlapping IP subnets and site-to-site VPN enabled. Copyright 2013 Global Knowledge Training LLC. All rights reserved. 8

For a VPN tunnel between two different organizations, the different organizations can be within the same vcloud or part of different vclouds. Both organizations need to have at least a single organization network that is NAT-connected with an external connection. The organization networks cannot have overlapping IP subnets and must have site-to-site VPN enabled. When creating a VPN tunnel to a remote network, the external remote network can an IPSec-enabled system, a firewall, or a router. Also, the external organization must be NAT-connected. No matter what type of VPN connection is being created, vshield Manager 5.0 (vcloud Networking and Security appliance) or newer must be used, since that is when VPN support was established. Also, vshield Manager (vcloud Networking and Security appliance) requires a special license for this support. If a firewall is present between the two endpoints of the tunnel, then the firewall must be configured to pass IP Protocol ID 50 (ESP) and IP Protocol ID 51 (AH), and needs to have the proper UDP ports open (500 and 4500). Static Routes Most routing is done dynamically where the router automatically chooses the best path between two network endpoints; however, a static route can be created. A static route is a permanent path between two networks used when routers are not configured to create dynamic routes, typically because of security reasons. There are two types of static routes that can be defined within vcloud Director: a route from one vapp network to another vapp network within the same organization or a route from one vapp network to another vapp network in a different organization. Either of these options will enable communications between the two vapps, but this is not a VPN; therefore, the communication between the vapps is not encrypted. Static routing services have to be enabled at the organization level before a static route can be created that allows traffic between vapps that are located in different organizations and routing over the organization networks. Only a system administrator can enable static routing for an organization level, but both a system administrator and an organization administrator can create a static route at a vapp level. If a firewall is located between the source and the destination vapp network, the firewall must be configured to pass the traffic, so firewall rules will need to be configured accordingly. Also, many operating systems have firewalls and may be configured to block incoming traffic so this operating system firewall may need to be disabled, or a rule be created to allow traffic from another network. Conclusion VMware vcloud Director contrives the provisioning of the software-defined datacenter layer to allow for a complete virtual datacenter delivery within a short period of time. This software-defined datacenter level provides the vcloud external connection while the organization and vapp networks are created within that vapp. Understanding how to create the different network layers and what is involved with the creation of network pools is integral to the success of a vcloud environment. Copyright 2013 Global Knowledge Training LLC. All rights reserved. 9

References For more information on vcloud Director and the features mentioned in this paper, see the following documents on VMware s website: VMware vcloud: Architecting a vcloud Technical White Paper vcloud Director Administrator s Guide [v5.1] Learn More VMware vcloud Director: Install, Configure, Manage [V5.1] VMware vcloud: Deploy and Manage the VMware Cloud [v1.5] VMware vcloud: Design Best Practices [v1.5] Visit www.globalknowledge.com or call 1-800-COURSES (1-800-268-7737) to speak with a Global Knowledge training advisor. About the Author Rebecca Fitzhugh is a VMware Certified Instructor and Consultant whose primary focus is on VMware virtual infrastructure products and vcloud Director. Prior to becoming an instructor and consultant, Rebecca served five years in the United States Marine Corps where she assisted in the build-out and administrator of multiple enterprise networks residing on virtual infrastructure. Copyright 2013 Global Knowledge Training LLC. All rights reserved. 10