Automated Provisioning with the VMware Software-Defined Data Center

Transcription

1 Automated Provisioning with the VMware Software-Defined Data Center REFERENCE ARCHITECTURE VERSION 1.0

2 Table of Contents Overview Audience VMware Software Components Architectural Overview... 6 Virtual Management Networks... 6 Management Cluster... 8 Edge Cluster Compute Clusters Physical Component Details Compute Storage Network Software-Defined Data Center Component Details...15 vsphere Data Center Design vrealize Orchestrator vsphere Data Protection NSX for vsphere vrealize Automation Load-Balanced vrealize Automation Configuration vrealize Automation Appliances vrealize Automation Infrastructure as a Service Web Servers vrealize Automation Managers Distributed Execution Managers vsphere Agent Monitoring vrealize Operations vrealize Log Insight SDDC Operational Configuration NSX for vsphere Configuration Tenants Endpoints Fabric Groups Business Groups Network Profiles TECHNICAL WHITE PAPER / 2

3 Reservation Policies Reservations Blueprints Conclusion The VMware Validated Design Team About the Author TECHNICAL WHITE PAPER / 3

4 Overview IT operations teams are being driven to be more efficient and more responsive to the needs of their business users. It no longer is acceptable for IT provisioning requests to take days or weeks to fulfill. Demand is always increasing, as is the expectation of immediate on-demand delivery for repeatable commodity-type requests. An automation cloud based on VMware IT technology enables an IT operations team to automate the provisioning of common repeatable requests and to respond to business needs with more agility and predictability. This reference architecture describes the implementation of a software-defined data center (SDDC) that leverages the latest VMware components to create an SDDC with automated provisioning. The reference architecture is built on the VMware Validated Design for a single-site IT automation cloud. The VMware Validated Design process gathers data from customer support, VMware IT, and VMware and partner professional services to create a standardized configuration that meets the majority of customer requirements. Internally, VMware engineering teams test new product capabilities, installations, upgrades, and more, against the standardized configuration. VMware and partner professional services teams build delivery kits based on this design, knowing that they are deploying with the best possible configuration. Customers planning do it yourself deployments also benefit from following this architecture, confident that future product upgrades, patches, and so on, have already been tested against a configuration identical to theirs. Audience This document will assist those who are responsible for infrastructure services, including enterprise architects, solution architects, sales engineers, field consultants, advanced services specialists, and customers. This guide provides an example of a successful deployment of an SDDC. TECHNICAL WHITE PAPER / 4

5 VMware Software Components This architecture uses the following VMware software components. PRODUCT VERSION DESCRIPTION VMware vcloud Suite Enterprise Edition 6.0 VMware vcloud Suite Enterprise Edition is a comprehensive suite of products used to deliver the SDDC. In this architecture, users leverage the following components of the vcloud Suite Enterprise Edition 6.0: VMware vsphere Enterprise Plus Edition 6.0 U1, VMware vrealize Automation 6.2.2, VMware vrealize Orchestrator 6.0.3, VMware vsphere Data Protection 6.1, and VMware vrealize Operations Manager 6.1. VMware vcenter Server 6.0 U1 VMware vcenter Server is a central platform for managing and configuring the VMware ESXi hypervisor. VMware vsphere Web Client is the centralized point of administration for compute clusters and all networking services provided by VMware NSX for vsphere. VMware Virtual SAN 6.1 VMware Virtual SAN is radically simple, hypervisor-converged storage for virtual machines. It delivers enterprise-class, high-performance storage for virtualized applications, including business-critical applications. NSX for vsphere 6.2 NSX for vsphere exposes a complete suite of simplified logical networking elements and services including virtual switches, routers, firewalls, load balancers, virtual private network (VPN), QoS, monitoring, and security. VMware vrealize Log Insight 2.6 VMware vrealize Log Insight is real-time log management and log analysis with machine learning based intelligent grouping, high-performance search, and better troubleshooting across physical, virtual, and cloud environments. Table 1. Components TECHNICAL WHITE PAPER / 5

6 Architectural Overview This design uses three cluster types, each with its own distinct function. It provides a management plane that is separate from the user workload (compute) virtual machines. It also leverages an edge cluster, which provides dedicated resources for network services such as VMware NSX Edge devices, which provide access to the corporate network and the Internet. Virtual Management Networks Table 2 provides a quick introductory overview of VMware NSX and networking terms and acronyms. TERM DEFINITION VLAN NSX for vsphere Virtual switch VPN MPLS A VLAN is used to partition a physical network into multiple distinct broadcast domains. NSX for vsphere is virtual networking and security software that enables the software-defined network. The NSX virtual switch abstracts the physical network and provides access-level switching in the hypervisor. It is central to network virtualization because it enables logical networks that are independent of physical constructs such as VLANs. A VPN can be used to connect networks to each other or a single machine to a network. Multiprotocol Label Switching (MPLS) is a scalable, protocol-independent transport mechanism. In an MPLS network, data packets are assigned labels. Packet-forwarding decisions are made solely on the contents of this label, without the need for examination of the packet itself. This enables users to create end-to-end circuits across any type of transport medium, using any protocol. Table 2. VMware NSX Overview This design utilizes NSX for vsphere virtual switches in the management cluster for automation and monitoring components. Each management solution vrealize Automation, vrealize Operations, and vrealize Log Insight has its own NSX virtual switch and corresponding NSX Edge device. The grouping of these virtual machines onto a virtual switch is referred to as a virtual management network. Virtual management networks are connected to both the vsphere management VLAN and the external VLAN via an NSX Edge device. Routing is configured between the virtual management network and the vsphere management VLAN. This enables the solution and virtual machines on the vsphere management VLAN, such as the vcenter Server instance, to communicate without exposing the management servers directly to end users. External connectivity to the business function of the virtual management network is provided by a load balancer s virtual IP on the external VLAN. External connectivity to the management function of the virtual management network is provided, as needed, by the method routing to the virtual management network using Ethernet, MPLS, VPN, jump hosts, or other means that best fits the customer s environment. Another option is to deploy the VMware NSX distributed logical router and to enable a dynamic routing protocol, such as OSPF, between the router and the top-of-rack switches. This enables access to all virtual machines on the virtual switches by advertising their IP addresses to the rest of the network. Virtual management networks still require NSX Edge devices to provide load balancer functionality. TECHNICAL WHITE PAPER / 6

7 Virtual management networking isolates management solutions from each other and from compute workloads. More important, it enables disaster recovery of the automation and monitoring stacks without having to change the IP addresses of the virtual machines. The virtual machines can be moved to precreated virtual switches in another site that has been configured the same as the primary site, enabling quick recovery of the solutions. Top-of-Rack Switches VLAN 1701 SVI VLAN 1680 SVI Static Routes / / / External VLAN (1701) /24 vsphere Management VLAN (1680) /24 vrealize Automation Virtual Switch /24 vrealize Operations Virtual Switch /24 VM VM VM VM VM VM NSX Edge (Internal) (vsphere) (External) NSX Edge (Internal) (vsphere) (External) vrealize Log Insight Virtual Switch /24 VM VM VM NSX Edge (Internal) (vsphere) (External) Figure 1. Example Using Static Routing TECHNICAL WHITE PAPER / 7

8 Top-of-Rack Switches External VLAN (1701) /24 vsphere Management VLAN (1680) /24 OSPF Distributed Logical Router vrealize Automation Virtual Switch /24 vrealize Operations Virtual Switch /24 VM VM VM VM VM VM NSX Edge (Internal) NSX Edge (Internal) vrealize Log Insight Virtual Switch /24 VM VM VM NSX Edge (Internal) Figure 2. Example Using Dynamic Routing Management Cluster The management cluster contains the management and monitoring solutions for the entire design. A single management cluster can support multiple pods of edge and compute clusters. The minimum number of hosts required in the management cluster is three, although four hosts are recommended for availability and performance. The management cluster can scale out as the number of edge and compute pods increases. With the exception of vsphere Data Protection, which stores all backups on NFS storage, all virtual machines utilize Virtual SAN storage. A single vcenter Server instance manages the resources in the management cluster. Additional vcenter Server instances manage edge and compute clusters. A Platform Services Controller is deployed for each vcenter Server instance. The Platform Services Controllers are joined to the same VMware vcenter Single Sign-On domain, enabling features such as Enhanced Linked Mode and cross vcenter Server VMware vsphere vmotion. For more information on the Platform Services Controller and new enhancements in vsphere 6.0, see the VMware vcenter Server 6.0 Deployment Guide and the What s New in vsphere 6.0 white papers. TECHNICAL WHITE PAPER / 8

9 The management cluster also contains common core infrastructure. This includes vrealize Operations Manager and vrealize Log Insight. VMware NSX Manager instances, one for each vcenter Server instance, are deployed into the management cluster. NSX for vsphere components, such as VMware NSX Controller instances, are also deployed for and in the management cluster. VMware NSX Controller instances for the edge and compute clusters are deployed into the edge cluster. All vrealize Automation components are also deployed in the management cluster. External Access Internal Access Spine Leaf Virtual SAN Management External Storage Figure 3. Management Cluster TECHNICAL WHITE PAPER / 9

10 Edge Cluster The edge cluster simplifies the physical network configuration. It is used to deliver networking services to the compute cluster virtual machines. All external networking, including corporate and Internet, for user-workload virtual machines is accessed via the edge cluster. The minimum edge cluster size is three hosts, but it can scale depending on the volume of services required by the compute cluster virtual machines. External Access Internal Access Spine Leaf Virtual SAN Edge Figure 4. Edge Cluster TECHNICAL WHITE PAPER / 10

11 Compute Clusters The compute clusters are the simplest of the three types; they run user-workload virtual machines. Compute cluster networking is completely virtualized using NSX for vsphere. A single transport zone exists between all compute clusters and the edge cluster. Virtual switches are created for user-workload virtual machines. The minimum compute cluster size is 4 hosts; the maximum is 64 hosts. Additional compute clusters can be created until the maximum number of either hosts (1,000) or virtual machines (10,000) for vcenter Server is reached. Additional vcenter Server instances can be provisioned in the management cluster to facilitate more compute clusters. Spine Leaf Virtual SAN Virtual SAN Compute External Storage Figure 5. Compute Clusters TECHNICAL WHITE PAPER / 11

12 Physical Component Details Compute Table 3 lists the recommended minimum physical server configuration. COMPONENT SPECIFICATION CPU 24GHz Two 2GHz six-core CPUs (12 total cores) Memory 256GB ECC RAM SD 6GB SD card boot device HDD controller Virtual SAN certified controller* Flash 500GB Virtual SAN certified Flash device* HDD Two 1TB Virtual SAN certified HDDs* Network interface cards Two 10Gb network adapters Power supplies Redundant Fans Redundant Table 3. Minimum Physical Server Configuration * Virtual SAN certified devices can be found at VMware recommends the use of Virtual SAN Ready Nodes for maximum compatibility and supportability. For ease of management and to guarantee resource availability as the solution grows, all physical server hardware, regardless of cluster, utilizes the same configuration. TECHNICAL WHITE PAPER / 12

13 Storage The management cluster utilizes Virtual SAN in addition to NFS datastores. Virtual machines reside on Virtual SAN; vsphere Data Protection backups reside on NFS datastores. The edge cluster utilizes Virtual SAN, which serves the VMware NSX Controller instances for the edge and compute clusters as well as NSX Edge devices. The compute clusters utilize Virtual SAN, NFS, and VMware vsphere VMFS datastores. The size and number, if any, of datastores other than Virtual SAN depend on available capacity, redundancy requirements, and application I/O needs. Table 4 presents some guidelines for sizing storage. STORAGE CLASS IOPS (PER 100GB) MB/SEC (PER 1TB) REPLICATION DEDUPLICATION Gold Yes Yes* Silver No Yes* Bronze 25 2 No Yes* Table 4. Storage Sizing Guidelines * Deduplication is enabled only on storage systems that support this feature. Performance values are based on 100 percent random I/O with 70 percent read and 30 percent write rate at 8KB block size. Network Each rack contains a pair of multichassis link aggregation capable 10 Gigabit Ethernet (10GbE) top-of-rack switches. Each host has one 10GbE network adapter connected to each top-of-rack switch. The vsphere hosts utilize the VMware vsphere Distributed Switch configured with an 802.3ad Link Aggregation Control Protocol (LACP) group that services all port groups Q trunks are used for carrying a small number of VLANs for example, NSX for vsphere, management, storage, and vsphere vmotion traffic. The switch terminates and provides default gateway functionality for each respective VLAN; that is, it has a switch virtual interface (SVI) for each VLAN. Uplinks from the top-of-rack switches leaf layer to the spine layer are routed point-to-point links. VLAN trunking on the uplinks even for a single VLAN is not allowed. A dynamic routing protocol (OSPF, ISIS, or BGP) is configured between the top-of-rack and spine layer switches. Each top-of-rack switch advertises a small set of prefixes, typically one per VLAN or subnet that is present. In turn, it calculates equal cost paths to the prefixes received from other top-of-rack switches. TECHNICAL WHITE PAPER / 13

14 Spine OSPF Leaf (Top-of-Rack) LACP VMware ESXi Host Figure 6. ESXi Host Network Connectivity TECHNICAL WHITE PAPER / 14

15 Software-Defined Data Center Component Details In this section, we will define the VMware software components and their configuration in enabling this solution. COMPONENT NUMBER DEPLOYED DEPLOYED LOCATION CONNECTED NETWORK Platform Services Controller 2 Management cluster vsphere management VLAN vcenter Server 2 Management cluster vsphere management VLAN ESXi hosts Minimum of 11 (varies based on compute cluster requirements) 4 Management cluster 3 Edge cluster 4 Compute cluster ESXi management VLAN vrealize Automation 1 (in a redundant distributed configuration comprising 10 virtual machines) Management cluster vra vcenter Orchestrator 2 (clustered) Management cluster vra Microsoft SQL Server Management cluster vra vsphere Data Protection 1 Management cluster vsphere management VLAN NSX Manager 2 Management cluster vsphere management VLAN VMware NSX Controller 6 3 Management cluster 3 Edge cluster vsphere management VLAN vrealize Operations Manager 4 (1 master, 1 master replica, 2 data nodes) Management cluster vrops vrealize Log Insight 3 (1 master, 2 workers) Management cluster vli Table 5. SDDC Component Details TECHNICAL WHITE PAPER / 15

16 vsphere Data Center Design The vsphere Enterprise Plus Edition is the core that enables the SDDC. All vsphere hosts are stateful installs that is, the ESXi hypervisor is installed to a local SD card. ATTRIBUTE SPECIFICATION ESXi version 6.0 U1 Number of hosts 11 Number of CPUs per host 2 Number of cores per CPU 6 Core speed Memory Number of network adapters 2GHz 256GB 2 x 10Gb Table 6. ESXi Host Details The solution uses VMware vsphere High Availability (vsphere HA) and VMware vsphere Distributed Resource Scheduler (vsphere DRS). vsphere HA is set to monitor both hosts and virtual machines. Its admission control policy utilizes a percentage of cluster resources reserved 25 percent in a four-node cluster guaranteeing sustainability with one node failure. To calculate the percentage, divide 100 by the number of hosts in a cluster; then, multiply the result by the number of hosts that can be eliminated while still guaranteeing resources for the virtual machines in the cluster. For example, an eight-host cluster for which sustainability for a two-host failure is the goal would be = 12.5 x 2 = 25 percent. vsphere DRS is set to fully automated mode. VLAN ID FUNCTION 14 VXLAN (management cluster) 24 VXLAN (edge cluster) 34 VXLAN (compute clusters) 970 ESXi management 980 vsphere vmotion 1020 IP storage (NFS) 1680 vsphere management 1701 External 3002 Virtual SAN Table 7. VLAN IDs and Functions TECHNICAL WHITE PAPER / 16

17 PORT GROUP VDS VL AN I D vds-mgmt-esxi vds-mgmt 970 vds-mgmt-ext vds-mgmt 1701 vds-mgmt-nfs vds-mgmt 1020 vds-mgmt-vmotion vds-mgmt 980 vds-mgmt-vsan vds-mgmt 3002 vds-mgmt-vsphere-management vds-mgmt 1680 VXLAN (VMware NSX autocreated) vds-mgmt 14 vds-edge-esxi vds-edge 970 vds-edge-ext vds-edge 1701 vds-edge-vmotion vds-edge 980 vds-edge-vsan vds-edge 3002 vds-edge-vsphere-management vds-edge 1680 VXLAN (VMware NSX autocreated) vds-edge 24 vds-comp-esxi vds-comp 970 vds-comp-nfs vds-comp 1020 vds-comp-vmotion vds-comp 980 vds-comp-vsan vds-comp 3002 VXLAN (VMware NSX autocreated) vds-comp 34 Table 8. Port Groups DATASTORE TYPE FUNCTION DS-VSAN-MGMT01 Virtual SAN Management cluster virtual machine datastore DS-NFS-MGMT01 NFS vsphere Data Protection backups DS-VSAN-EDGE01 Virtual SAN Edge cluster virtual machine datastore DS-VSAN-COMP01 Virtual SAN Gold-tier storage DS-NFS-COMP01 NFS Silver-tier storage DS-NFS-COMP02 NFS Silver-tier storage Table 9. Datastores TECHNICAL WHITE PAPER / 17

18 ATTRIBUTE SPECIFICATION Number of CPUs Processor type Memory Number of network adapters Network adapter type Number of disks SQL server Operating system Four VMware virtual CPU 16GB One VMXNET3 One 100GB (C:\) VMDK Microsoft SQL Server 2012 SP2 Windows Server 2012 R2 Table 10. SQL Server Configuration ATTRIBUTE SPECIFICATION vcenter version Quantity Appliance size 6.0 U1 (appliance) Four (two Platform Services Controllers, two vcenter Server instances) Small for management vcenter Server, large for compute vcenter Server Table 11. VMware vcenter Configuration ATTRIBUTE SPECIFICATION Data center object Enhanced Linked Mode WDC01 Automatic by joining same vcenter Single Sign-On domain Table 12. VMware vcenter Data Center Configuration TECHNICAL WHITE PAPER / 18

19 vrealize Orchestrator vrealize Orchestrator is deployed using the vrealize Orchestrator appliance. For resiliency, it is set up in a cluster in which the database resides on the SQL Server. NSX for vsphere and vrealize Automation plug-ins are installed in both instances. vrealize Orchestrator is a critical component in the SDDC. All vrealize Automation to NSX for vsphere communication is handled via vrealize Orchestrator NSX plug-in and workflows. vsphere Data Protection vsphere Data Protection is deployed in the management cluster and is responsible for backups and restores of the virtual machines residing in the management cluster. A backup policy was created for each management application, such as vcenter Server, vrealize Automation, and so on. In addition to the full virtual machine backups, the SQL Server vsphere Data Protection agent is installed on the SQL Server, and all databases are backed up as well. Backup frequency and retention periods vary depending on organizational requirements. Nightly backup of all virtual machines and databases is recommended. Figure 7. vsphere Data Protection NSX for vsphere NSX for vsphere provides the virtual switches, routing, and load balancer services used to create the SDDC. All virtual machine traffic, excluding that on the vsphere management VLAN, is encapsulated using NSX for vsphere. All virtual machine to virtual machine, or east west, traffic is encapsulated and then routed between the virtual tunnel endpoints (VTEPs) of the host, where it is decapsulated and delivered to the virtual machine. When a request to or from the external network is serviced, it travels through the NSX Edge device in the edge cluster, which provides all north south routing that is, routing to and from external networks. NSX for vsphere has a one-to-one relationship with vcenter Server, so two NSX Manager instances are deployed, one for the management cluster vcenter Server instance and the other for the edge and compute cluster vcenter Server instance. These are both deployed in the management cluster. NSX for vsphere utilizes controller virtual machines to implement the network control plane. The NSX Controller instances must be deployed in odd numbers to avoid a split-brain scenario. As such, three controllers per NSX for vsphere instance are deployed. The NSX Controller instances for the management cluster are deployed into the management cluster itself; the instances for the edge and compute clusters are deployed into the edge cluster. TECHNICAL WHITE PAPER / 19

20 The ESXi hosts must be prepared for NSX for vsphere. The following values are used: SPECIFICATION VALUE MTU 9000 Teaming mode LACP V2 VLAN 14 (management) 24 (edge) 34 (compute) Segment IDs Transport zones Management (management cluster) Compute (edge and compute clusters) Table 13. NSX for vsphere Values To enable external (north south) connectivity for the compute workloads, an NSX Edge router is deployed in HA mode. This NSX Edge instance is referred to as the provider edge. One interface is connected to the external network; another is connected to an NSX virtual switch, which is also connected to an NSX Edge router. The NSX Edge routers are configured with OSPF to facilitate the exchange of routing information. This enables the virtual machines on the NSX virtual switch to communicate with the external network and vice versa as long as firewall rules permit the communication. TECHNICAL WHITE PAPER / 20

21 External Network/Internet Spine Leaf External VLAN Virtual SAN Virtual SAN Compute Transport Zone VXLAN 5001 VM VM Edge Compute Figure 8. NSX for vsphere North South Routing TECHNICAL WHITE PAPER / 21

22 vrealize Automation vrealize Automation empowers IT to accelerate the delivery and ongoing management of personalized, business-relevant infrastructure, application, and custom services while improving overall IT efficiency. Policy-based governance and logical application modeling ensure that multivendor, multicloud services are delivered at the right size and service level for the business need. Full life cycle management ensures that resources are maintained at peak operating efficiency. And release automation enables multiple application deployments to be kept in sync through the development and deployment process. vrealize Automation provides the portal to request services such as virtual machines. This configuration utilizes the distributed architecture of vrealize Automation and uses NSX for vsphere to create a highly available environment by load-balancing multiple instances of the vrealize Automation components. Load-Balanced vrealize Automation Configuration External VLAN vsphere VLAN VM VM VM VM VM VM VM VM VM VM VM VM VM NSX Edge (Internal) (vsphere) (External) vrealize Automation Appliances Load-Balanced IP vrealize Automation laas Web vrealize Automation Managers vrealize Automation vsphere Agents vrealize Automation DEMs vrealize Orchestrator Appliances Microsoft SQL Figure 9. NSX for vsphere Load Balancer Configuration for vrealize Automation To achieve the architecture shown in Figure 9, we deploy a single NSX Edge device in HA mode. It is configured to load-balance vrealize Automation appliance, vrealize Automation Manager, vrealize Orchestrator, and Web traffic. Because we are load-balancing Web servers that make requests that are routed back to themselves, a registry setting that disables Windows loopback checking was created. See VMware Knowledge Base article for more information. vrealize Automation Appliances vrealize Automation is distributed as a prepackaged appliance in OVA format. For increased redundancy, two of these appliances are deployed and configured for clustering; the internal PostgreSQL database is also clustered. The NSX Edge device shown in Figure 9 is configured to load-balance the traffic to the vrealize Automation appliances. vsphere DRS rules were created to ensure that the vrealize Automation appliances run on different hosts. vrealize Automation Infrastructure as a Service Web Servers The vrealize Automation infrastructure as a service (IaaS) Web servers utilize Microsoft Internet Information Services (IIS) on Windows. For redundancy, two IaaS Web servers are deployed. Both are active and are load balanced by the NSX Edge device shown in Figure 9. The model manager data is deployed to the first IaaS Web server only. TECHNICAL WHITE PAPER / 22

23 vrealize Automation Managers The vrealize Automation managers utilize IIS on Windows. For redundancy, two managers are deployed. Only one is set to active; the other is passive. The NSX Edge device shown in Figure 9 is configured to load-balance the traffic, but only the currently active manager is active on the load balancer. During an outage, manual steps are taken to make the passive server active and to update the load balancer configuration to use the now-active server. Distributed Execution Managers The distributed execution manager (DEM) runs on Windows. For redundancy, two virtual machines are deployed, each configured with three execution managers. The DEM doesn t support a load-balanced configuration, but deploying two virtual machines with three DEM instances each provides redundancy of the service. vsphere Agent The vsphere agent runs on Windows. For redundancy, two virtual machines are deployed, each configured with the same vsphere agent name. The vsphere agent doesn t support a load-balanced configuration, but deploying two virtual machines configured with the same vsphere agent name provides redundancy. ATTRIBUTE SPECIFICATION Number of CPUs Two Processor type VMware virtual CPU Memory 4GB, 6GB for DEMs Number of network adapters One Network adapter type VMXNET3 Number of disks One 50GB (C:\) VMDK Operating system Windows Server 2012 R2 Table 14. Recommended Minimum Windows Server Configuration TECHNICAL WHITE PAPER / 23

24 Monitoring Monitoring the performance, capacity, health, and logs in any environment is critical. Using vrealize Operations along with vrealize Log Insight is a unified management solution for performance management, capacity optimization, and real-time log analytics. Predictive analytics leverages both structured and unstructured data to enable proactive issue avoidance and faster problem resolution. The solution extends intelligent operations management beyond vsphere to include operating systems, physical servers, and storage and networking hardware. It is supported by a broad marketplace of extensions for third-party tools. vrealize Operations vrealize Operations provides operations dashboards, performance analytics, and capacity optimization capabilities needed to gain comprehensive visibility, proactively ensure service levels, and manage capacity in dynamic virtual and cloud environments. vrealize Operations is deployed as a virtual appliance and is distributed in the OVA format. In this architecture, vrealize Operations is deployed on an NSX virtual switch. Four vrealize Operations appliances are deployed. The first is configured as the master node, the next as the master replica, and the last two as data nodes. The four appliances access the vsphere management VLAN via the NSX Edge device configured with either static or dynamic routing. The NSX Edge device also load-balances the four virtual appliances on port 443, providing access to the vrealize Operations cluster via a single FQDN. External VLAN vsphere VLAN NSX vswitch /24 VM VM VM VM NSX Edge (Internal) (vsphere) (External) vrealize Operations Appliances Load-Balanced IP Figure 10. vrealize Operations TECHNICAL WHITE PAPER / 24

25 To ensure a complete picture of how the environment is running, vrealize Operations is configured to monitor the management, edge, and compute vcenter Server instances. Additionally, the NSX for vsphere and vrealize Automation content pack is installed and configured to provide insight into the virtualized networking and automated provisioning environments. Figure 11. vrealize Operations Dashboard vrealize Operations requires updates to the default monitoring settings for most organizations. For more information on how to customize vrealize Operations for a specific environment, see the vrealize Operations documentation. vrealize Log Insight vrealize Log Insight provides in-depth log analysis in an easy-to-query Web interface. It collects syslog data from ESXi hosts or any other server or device that supports syslog. There is also an installable agent for Windows and Linux that enables the collection of event logs and custom logs such as vcenter Server and vrealize Automation server log files. vrealize Log Insight is deployed as a virtual appliance and is distributed in the OVA format. In this architecture, vrealize Log Insight is deployed on an NSX virtual switch. Three vrealize Log Insight appliances are deployed. The first is configured as the master node, and the other two are configured as data nodes. The appliances access the vsphere management VLAN via the NSX Edge device configured with either static or dynamic routing. vrealize Log Insight includes its own load balancer, which is configured to provide a single IP address for access. TECHNICAL WHITE PAPER / 25

26 External VLAN vsphere VLAN NSX vswitch /24 VM VM VM VM NSX Edge (Internal) (vsphere) (External) vrealize Log Insight Appliances Load-Balanced IP Figure 12. vrealize Log Insight The syslogs of all appliances and ESXi hosts are configured to send to the vrealize Log Insight load-balanced IP address. Additionally, the vrealize Automation, NSX for vsphere, and vrealize Operations content packs were installed and are configured to create dashboards to easily monitor the entire environment. Figure 13. vrealize Log Insight Dashboard TECHNICAL WHITE PAPER / 26

27 SDDC Operational Configuration When all components have been installed, they must be brought together to enable the creation of blueprints and to provision services by authorized users. To unify the components to operate as a solution, the following configuration steps are required. NSX for vsphere Configuration First we provision the common network resources for use within vrealize Automation. An NSX Edge device in the edge cluster was created for north south routing; the OSPF dynamic routing protocol is configured between the NSX Edge device and the external physical switches. NSX virtual switches were precreated for use in vrealize Automation for single-machine blueprints; they are connected to NSX Edge devices to provide routing and load-balancing services. vrealize Automation can dynamically create NSX virtual switches in multimachine blueprints. Tenants A tenant is an organizational unit in a vrealize Automation deployment. A tenant can represent the entire organization or specific business units. A default vsphere.local tenant is created during the installation. This tenant is used for administration purposes. A tenant for the IT administrators was created and is pointed to the Microsoft Active Directory environment for authentication. Figure 14. vrealize Automation Tenants TECHNICAL WHITE PAPER / 27

28 Endpoints Endpoints are the infrastructure sources that vrealize Automation consumes. In vcloud Suite and in this architecture, the endpoint is vcenter Server more specifically, the vcenter Server instance that manages the edge and compute clusters. Figure 15. vrealize Automation Endpoints Fabric Groups Fabric groups are groups of compute resources that the endpoints discover; they define the organization of virtualized compute resources. In most single-site environments, a single fabric group is created that contains all nonmanagement clusters. Figure 16. vrealize Automation Fabric Groups TECHNICAL WHITE PAPER / 28

29 Business Groups Business groups define the users and machine prefixes and are used later to grant access to a percentage of resources. Users assigned the group manager role can generate blueprints and see all machines created in the group. Support users can work for another user, and users can be entitled to request blueprints in the catalog. In most environments, business groups are established for department or business units in an organization. Figure 17. vrealize Automation Business Groups Network Profiles Network profiles define the type of connection external, private, NAT, or routed that a resource has. NAT and routed profiles require an external profile. External profiles connect resources to an existing network. Reservation Policies Reservation policies enable a user to associate one or more reservations into a policy that can be applied to a blueprint. Multiple reservations can be added to a reservation policy, but a reservation can belong to only one policy. A single reservation policy can be assigned to more than one blueprint. A blueprint can have only one reservation policy. Reservations A virtual reservation is a share of the memory, CPU, networking, and storage resources of one compute resource allocated to a particular business group. To provision virtual machines, a business group must have at least one reservation on a virtual compute resource. Each reservation is for one business group only, but a business group can have multiple reservations on a single compute resource or on compute resources of different types. TECHNICAL WHITE PAPER / 29

30 Figure 18. vrealize Automation Reservations Blueprints A machine blueprint is the complete specification for a virtual, cloud, or physical machine. Blueprints determine a machine s attributes, the manner in which it is provisioned, and its policy and management settings. In this architecture, blueprints can be either vsphere based that is, single machine or multimachine, which requires one or more vsphere blueprints and provisions and manages them together as a single entity. Multimachine blueprints also enable the dynamic provisioning of networks through the use of network profiles. Figure 19. vrealize Automation Blueprints TECHNICAL WHITE PAPER / 30

31 After blueprints have been created and published to the catalog, authorized users can log in to the vrealize Automation portal and request these resources. Figure 20. vrealize Automation Service Catalog Conclusion An automation cloud based on VMware IT technology enables a more agile and predictable IT response to business needs. The reference architecture presented in this paper describes the implementation of a software-defined data center that uses the latest VMware components to create a single-site IT automation cloud. Customers following this architecture can be confident that they will have the best possible supported configuration, one that is fully backed by the VMware Validated Design process. For a guided tutorial that shows step-by-step instructions for deploying this configuration, see TECHNICAL WHITE PAPER / 31

32 The VMware Validated Design Team Blaine Christian, Scott Faulkner, Phil Weiss, Christian Elsen, Nik Gibson, Randy Jones, William Lam, Nick Marshall, Paudie O Riordan, Kamu Wanguhu, Steven Ching, Michelle Gress, Christine Zak, Yu-Shen Ng, Bob Perugini, Justin King, Karthik Narayan, Sunny Bhatia, Mandar Dhamankar, Olga Efremov, David Gress, Kristopher Inglis, Rama Maram, Hari Krishna Meka, Arvind Patil, Venkat Rangarajan, Lakshmanan Shanmugam, Todor Spasov, Georgi Staykov, Antony Stefanov, Kevin Teng, Todor Todorov, Tuan Truong, Randy Tung, Shivaprasad Adampalli Venkateshappa, Lap Vong, Zhuangqian Zhang, and Mike Brown About the Author Mike Brown is a senior technical marketing architect in the Integrated Systems Business Unit. Mike s focus is on reference architectures for VMware vcloud Suite and the software-defined data center. He has multiple industry certifications, including VMware Certified Design Expert (VCDX), VMware Certified Advanced Professional Cloud, and VMware Certified Professional Network Virtualization. Follow Mike on the vsphere Blog and on TECHNICAL WHITE PAPER / 32

33 VMware, Inc Hillview Avenue Palo Alto CA USA Tel Fax Copyright 2015 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Item No: VMW-RA-Sngl-Ste-Auto-Cloud-USLET-101 Docsource: OIC-FP-1378