System 800xA Virtualization with VMware vsphere ESXi

Transcription

1 System 800xA Virtualization with VMware vsphere ESXi System Version 6.0 Power and productivity for a better world TM

2

3 System 800xA Virtualization with VMware vsphere ESXi System Version 6.0

4 NOTICE This document contains information about one or more ABB products and may include a description of or a reference to one or more standards that may be generally relevant to the ABB products. The presence of any such description of a standard or reference to a standard is not a representation that all of the ABB products referenced in this document support all of the features of the described or referenced standard. In order to determine the specific features supported by a particular ABB product, the reader should consult the product specifications for the particular ABB product. TRADEMARKS ABB may have one or more patents or pending patent applications protecting the intellectual property in the ABB products described in this document. The information in this document is subject to change without notice and should not be construed as a commitment by ABB. ABB assumes no responsibility for any errors that may appear in this document. In no event shall ABB be liable for direct, indirect, special, incidental or consequential damages of any nature or kind arising from the use of this document, nor shall ABB be liable for incidental or consequential damages arising from use of any software or hardware described in this document. This document and parts thereof must not be reproduced or copied without written permission from ABB, and the contents thereof must not be imparted to a third party nor used for any unauthorized purpose. The software or hardware described in this document is furnished under a license and may be used, copied, or disclosed only in accordance with the terms of such license. This product meets the requirements specified in EMC Directive 2004/108/EC and in Low Voltage Directive 2006/95/EC. All rights to copyrights, registered trademarks, and trademarks reside with their respective owners. Copyright by ABB. All rights reserved. Release: Document number: August BSE

5 Table of Contents About this User Manual Version Described in this User Manual...12 User Manual Conventions...12 Warning, Caution, Information, and Tip Icons...12 Terminology...13 Released User Manuals and Release Notes...13 Section 1 - Introduction Prerequisites...15 Overview of Virtualization...16 Changes in ESXi Section 2-800xA Server and Client Node Consolidation Non-virtualized 800xA System...19 Consolidated 800xA System...20 Section 3 - System Planning Software Versions...23 System Overview...23 System Planner...23 Server Provisioning...24 Client Virtualization...24 Determine Required VMware ESXi Server Hardware...25 Local or SAN Storage...25 Server and System Dimensioning...26 Determine Additional Hardware BSE

6 Table of Contents 800xA Licensing Dongle Using the ABB USB Dongle Over the Network xA for Advant Master and 800xA for MOD Dimensioning Rules - Virtual 800xA Application Servers Virtual 800xA Application Servers Example Dimensioning Rules - Virtual 800xA Clients Virtual 800xA Client Example Section 4 - ESXi Installation Install ESXi 5.5 Server...43 Section 5 - ESXi Configuration Console ESXi Login...49 Configure Management Network...50 Section 6 - VMware vsphere Client Installation...57 Log into an ESXi Server...59 Log into VMware vcenter Server...62 Section 7 - Basic Configuration Adding Disks...65 Configure Networks...70 Set the Time Server...76 ESXi USB/SD Install: System Logs Location...82 Section 8 - Create Virtual Machines Create a Virtual Machine...87 Install the Guest Operating System...99 Add ISO file to the Virtual Machine Install VMware Tools Virus Scanning Section 9 - ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi BSE

7 Table of Contents Upgrade to ESXi Section 10 - vsphere Command Line Interface Overview Download Install VMware vsphere CLI Section 11 - USB Licensing Dongle Section 12 - Backup and Restore Backup and Restore of an ESXi Server Backup and Restore of Virtual Machines Section 13 - Security Roles Users Permissions Section 14 - Optimization Virtual CPUs Allocate Virtual CPUs to Physical Cores Section 15 - Monitor ESXi servers using SNMP Traps 800xA Components Required Set ESXi Host Name SNMP Traps Communication Topology Server Hardware ESXi Configuration SNMP Receiving Port Trap Target Check Configuration Firewall Test Basic Communication BSE

8 Table of Contents Test Trap Communication SNMP Traps Service Provider Add PNSM Objects Create a Device Object Filter Trap Messages SNMP Traps - Generic xA Components Required Install PNSM Base Object Install PNSM Workstation Object Download Asset MIB Files Create an Object Type Create an Asset Object Section 16 - VMware Command Line Interface, Drivers, Updates and Upgrades VMware Command Line Interface Prepare the ESXi Server Drivers Apply Patches/Updates Section 17 - Storage Area Network Hardware Configuration Networking ESX Server Network Configuration Add Datastores Performance Section 18 - VMware vcenter Licensing Single Sign-On Web Client and Desktop Client BSE

9 Table of Contents Section 19 - VMware vcenter Installation vcenter 5.5 Supported Operating Systems Hardware Pre-configuration Install vcenter Basic Configuration through the vsphere Web Client Section 20 - VMware High Availability Configure VMware High Availability Add a Cluster Object Add ESXi Servers to the Cluster Specify Failover Host Section 21 - VMware vmotion Section 22 - VMware Storage vmotion Section 23 - VMware Fault Tolerance Section xA in the IT Infrastructure Section xA Virtual Client Virtual Clients Benefits of Virtual 800xA Workplaces Operating Area Planning Graphical Presentation and Physical Server Speed Thin Clients Multiple Screen Section 26 - Timesync ESXi with Windows Communication Topology Configure Windows Firewall Configuration Registry BSE

10 Table of Contents Service Configure ESXi SSH Putty ESXi Configurations Timesync Confirmation Appendix A - ESXi Host Setup Check List Appendix B - ESXi Guest Setup Check List Appendix C - Further Technical Information on VMware ESXi Index 10 3BSE

11 About this User Manual Any security measures described in this User Manual, for example, for user access, password security, network security, firewalls, virus protection, and so on, represent possible steps that a user of an 800xA System may want to consider based on a risk assessment for a particular application and installation. This risk assessment, as well as the proper implementation, configuration, installation, operation, administration, and maintenance of all relevant security related equipment, software, and procedures, are the responsibility of the user of the 800xA System. This User Manual provides an overview of virtualization and how it is to be used with 800xA through the following topics: Overview of virtualization. 800xA server node consolidation based on VMware ESXi. Consolidating 800xA server nodes on VMware ESXi servers. System planning, installation, and configuration using VMware ESXi. Creating Virtual Machines. Backup and Restore of a virtualized system. Security. Optimizations. Monitoring VMware vsphere ESXi. Upgrading from ESX 4.0U1/ESXi 4.x/ESXi 5.x to ESXi 5.5. SAN Storage Server. vcenter Server. VMware High Availability. With the progressive support for the latest releases of VMware ESXi, this User Manual has been updated to remove content that was only applicable to ESX BSE

12 Version Described in this User Manual About this User Manual Version Described in this User Manual Unless otherwise noted, the versions of all 800xA Base System and Functional Area software described in this user manual are the latest release of 800xA 6.0. User Manual Conventions Microsoft Windows conventions as defined in the Microsoft Manual of Style are normally used for the standard presentation of material when entering text, key sequences, prompts, messages, menu items, screen elements, and so on. Warning, Caution, Information, and Tip Icons This user manual includes Warning, Caution, and Information where appropriate to point out safety related or other important information. It also includes Tip to point out useful hints to the reader. The corresponding symbols should be interpreted as follows: Electrical warning icon indicates the presence of a hazard that could result in electrical shock. Warning icon indicates the presence of a hazard that could result in personal injury. Caution icon indicates important information or warning related to the concept discussed in the text. It might indicate the presence of a hazard that could result in corruption of software or damage to equipment/property. Information icon alerts the reader to pertinent facts and conditions. Tip icon indicates advice on, for example, how to design your project or how to use a certain function Although Warning hazards are related to personal injury, and Caution hazards are associated with equipment or property damage, it should be understood that operation of damaged equipment could, under certain operational conditions, result in degraded process performance leading to personal injury or death. Therefore, fully comply with all Warning and Caution notices. 12 3BSE

13 About this User Manual Terminology Terminology A complete and comprehensive list of terms is included in System 800xA System Guide Functional Description (3BSE038018*). The listing includes terms and definitions that apply to the 800xA System where the usage is different from commonly accepted industry standard definitions and definitions given in standard dictionaries such as Webster s Dictionary of Computer Terms. Released User Manuals and Release Notes A complete list of all User Manuals and Release Notes applicable to System 800xA is provided in System 800xA Released User Documents (3BUA000263*). System 800xA Released User Documents (3BUA000263*) is updated each time a document is updated or a new document is released. It is in pdf format and is provided in the following ways: Included on the documentation media provided with the system and published to ABB SolutionsBank when released as part of a major or minor release, Service Pack, Feature Pack, or System Revision. Published to ABB SolutionsBank when a User Manual or Release Note is updated in between any of the release cycles listed in the first bullet. A product bulletin is published each time System 800xA Released User Documents (3BUA000263*) is updated and published to ABB SolutionsBank. 3BSE

14 Released User Manuals and Release Notes About this User Manual 14 3BSE

15 Section 1 Introduction Prerequisites As the use of virtualization has expanded dramatically in the business and financial sector amongst others, ABB is now providing the ability to take advantage of virtualization in the process industries. Virtualization can be used in 800xA systems to combine multiple 800xA server nodes onto a single computer. The total number of physical computers required in an installation is reduced significantly. This also reduces the required space for the computers, hardware acquisition cost for computers and cabinets, and the operating costs (such as energy costs). The following are the prerequisites to be considered for virtualization: For the Windows 2012R2 Guest and E1000 Network consider the following: When using Windows Server 2012R2, do not use E1000 network adapters. Use VMxNET 3 network adapters. With the combination of ESXi and Windows 2012R2 Guest, and E1000, even minimal network traffic between Windows 2012R2 Guests on the same host will cause the ESXi to crash causing the PSOD (Purple Screen Of Death). VMware will provide a post ESXi 5.5 release patch to correct this by early ABB 800xA versions prior to 6.0 should use E1000 virtual network adapters and ABB 800xA versions from 6.0 should use VMxNET 3. Changing the network adapter to VMxNET3 while upgrading the virtual machine s guest operating system may cause network connectivity issues. However, the network adapter should be changed to VMxNET3 to prevent the ESXi PSOD. When adding VMXNET3, if you add it as E1000 and remove it from the Edit settings options and again add the adapter as VMXNET3, the system may display a message saying The adapter with same name already 3BSE

16 Overview of Virtualization Section 1 Introduction exists. In this case, you need to delete the hidden adapters and add it again. Restart the virtual machine after adding the network adapter. ESXi 5.5 Guest hardware version If you are not using vcenter Server, DO NOT UPGRADE the guest hardware to version 10 or above. You will no longer be able to edit the guest. You will be expected to use the web client which is only available through the vcenter Server. This will require the addition of a new node which has vcenter Server installed (a license for this is needed) and the ESXi must have non-free license. Licensing It is essential for you to purchase the ESXi license. This ensures that you have direct access to VMware Support. It also provides access to patches and updates through the VMware website. Node Roles To simplify and consolidate system dimensioning the Node Roles are introduced into System 800xA 6.0. Refer to the System 800xA System Guide Technical Data and Configuration (3BSE041434*) for more information. Overview of Virtualization VMware ESX is a virtualization technology that enables running multiple virtual machines (VM) on the same physical computer. Each VM represents a computer and its hardware components (CPU, RAM, disk drives, network adapters, etc.). Each VM requires an operating system and applications to be installed in the same manner as a physical computer. The VM appears to the operating system as a physical computer. Each VM requires a licensed copy of the Windows operating system. 16 3BSE

17 Section 1 Introduction Overview of Virtualization Figure 1. VMware ESX Virtualization Overview While ESXi can provide the environment that represents a computer, there are limitations for the types of resources made available. When planning a virtual environment, only server nodes that communicate over the network should be included. Interaction with the virtual machines is normally performed through a dedicated client to create, modify, start and stop the virtual machines. It also enables the opening of a console to takeover the virtual machine. VMware vsphere encompasses the VMware ESXi server with the advanced functionality of VMotion, High Availability, Fault Tolerance and Storage Motion. vcenter Server provides central administration of the ESXi servers. VMware vsphere Client is a Windows application used to log on to the ESXi server, or the vcenter Server, for administration purposes. 3BSE

18 Changes in ESXi 5.5 Section 1 Introduction Changes in ESXi 5.5 There are many enhancements in ESXi 5.5 that are relevant to larger enterprise installations. Those that are most relevant to ABB 800xA installations are: Free version is no longer restricted to 32GB RAM. Removal of the VMDK maximum size limit of 2TB to 512 bytes. The limit is now 62TB. 18 3BSE

19 Section 2 800xA Server and Client Node Consolidation This topic describes the details of 800xA Server Node Consolidation. Non-virtualized 800xA System The difference between a non-virtualized 800xA system and a VMware ESXi system is that some, or all, of the 800xA server nodes, run inside VMs. For example, a non-virtualized 800xA system is shown in Figure 2. Figure 2. Example of a Non-virtualized 800xA System The components of the non-virtualized 800xA system are as follows: One redundant pair of Domain Controllers. 3BSE

20 Consolidated 800xA System Section 2 800xA Server and Client Node Consolidation One redundant pair of Aspect Servers. Three redundant pairs of Connectivity Servers for AC800M, including two Asset Optimization Servers. One redundant pair of Foundation Fieldbus Connectivity Servers, including one Asset Optimization Server. Two Information Management Servers. Client nodes. Consolidated 800xA System Using VMware ESXi, the same system can be set up (consolidated) as follows: One primary VMware ESXi server containing all primary 800xA server nodes as VMs. One secondary VMware ESXi server containing all secondary 800xA server nodes as VMs. Main operator clients are not virtualized and remain on physical computers. Ancillary and engineering clients can be virtualized. A separate client application (VMware vsphere Client) is used for configuring the VMware ESXi server (for creating VMs or changing their virtual hardware). The vsphere Client is required to be installed on at least one physical computer running Microsoft Windows. This can be an 800xA Client node, preferably an Engineering Client node. 20 3BSE

21 Section 2 800xA Server and Client Node Consolidation Consolidated 800xA System The resulting 800xA system configuration based on VMware ESXi is shown in Figure 3. Figure 3. Example of a Virtualized 800xA System Compared to the non-virtualized system shown in Figure 2, which requires 14 physical servers, the virtualized system requires only two physical servers. 800xA server nodes are installed and configured into the VMs the same way physical computers are by using the 800xA System Installer tool. However, setting up 3BSE

22 Consolidated 800xA System Section 2 800xA Server and Client Node Consolidation virtualized 800xA server nodes requires added steps to initially set up the VMware ESXi servers. This procedure is outlined in Figure 4. Figure 4. Setup Procedure for VMware ESXi 800xA System The steps of this procedure are discussed in detail in the following topics: System Planning. ESXi Installation. 22 3BSE

23 Section 3 System Planning This topic describes how to plan the features, size, and configuration of the VMware ESXi-based 800xA system. The objective is to determine the hardware and software requirements for VMware ESXi servers to make the right purchases. Software Versions System Overview System Planner The first stage of planning the system is to confirm that the required server node types are supported for virtualization. This is documented in the System 800xA 6.0, 5.1, 5.0, 4.1, 3.1 Third Party Software (3BUA000500*). For 800xA 5.1 Rev B, PNSM SNMP Trap support can be used to monitor ESXi servers. Essentially, the configuration and dimensioning rules for 800xA server nodes stated in the System 800xA, System Guide Technical Data and Configuration (3BSE041434*) instruction should be kept the same when the node is installed inside virtual machines. Given that these rules are followed, with additions and limitations given in this document, the performance data stated in the system guide will be fulfilled. Refer Virtual 800xA Application Servers Example. Use the 800xA System Planner tool to determine the types and number of 800xA nodes for a system. 3BSE

24 Server Provisioning Section 3 System Planning Server Provisioning Servers are provisioned based on the summation of the resources required for the virtual machines. For high system reliability, it is required to use two VMware ESXi servers - one primary and one secondary (see Figure 3). Place all primary 800xA server nodes on the primary VMware ESXi server and place the secondary 800xA server nodes on the secondary VMware ESXi server. If one VMware ESXi server fails, all redundant 800xA server nodes on the other VMware ESXi server are available. Do not place primary and secondary 800xA server nodes on the same VMware ESXi server (single point of failure). Client Virtualization Virtual clients are System 800xA nodes that run as virtual machines. Connection to the virtual clients is through standard Microsoft Remote Desktop. Typically, a thin client which is a small PC with embedded Windows is used at the operator desk. To decide how many virtual clients you should restrict to, refer System 800xA System Guide Technical Data and Configuration (3BSE041434*). More graphical content produces higher loading on the virtual client CPU, which may cause graphic response (call up time) issues. It is recommended to use separate ESXi servers for the virtual clients. That is, ESXi servers that do not have System 800xA Server virtual machines running. This minimizes the risk of resource interference of the virtual clients due to their higher CPU usage. Zero clients are small client computers, which communicate over PCoIP to virtual clients setup using VMware View. This is not supported for System 800xA 24 3BSE

25 Section 3 System Planning Determine Required VMware ESXi Server Hardware Determine Required VMware ESXi Server Hardware Local or SAN Storage Refer ABB certified hardware listings for supported hardware. Where there is a requirement for using non ABB listed server hardware, the following issues must be understood clearly: Technical support from ABB will be limited. The hardware chipset must be Intel based. The hardware must be in VMware's approved compatibility listings. There will be no support for monitoring the ESXi server using 800xA PNSM. This must be handled by an alternative method. The 800xA system configuration rules state that some server nodes may not be combined, such as, Information Management with Foundation Fieldbus. However, it is possible to combine these nodes as separate VMs on the same physical VMware ESXi server, because they still run as separate nodes (separate VMs). Combined server node types with heavy resource usage should be split into separate VMs to prevent resource starvation. Refer the System 800xA System Guide Technical Data and Configuration (3BSE041434*) instruction for more information. Virtual machines can be located in local storage (disks mounted in the ESXi servers), or in SAN Storage (connected on a separate network). SAN storage is essential for VMware High Availability. There are many different possible configurations for SAN storage such as redundant SAN storage. These are not supported for System 800xA. It is critical to ensure that any SAN solution can be correctly maintained. Typical concepts for redundant storage are: Is there a master SAN. How to move/add master status to a slave. What is the procedure for replacing a SAN. Was the failed SAN a master. What is the procedure for replacing a disk. 3BSE

26 Server and System Dimensioning Section 3 System Planning Server and System Dimensioning Dimensioning of ABB 800xA Virtual Hosts follows the same role based guidelines that apply to the ABB 800xA Physical Servers. For example, the same maximum number of roles apply to virtual servers as it does for physical servers. The virtual system is required to separate the roles into different virtual machines. ABB 800xA virtualization guidelines has maintained that for every virtual CPU, there should be a physical logical processor (hyperthreaded core). This is reinforced in the role based dimensioning where each role must have a virtual CPU. That is, for each role, the virtual machine should have one virtual CPU, and for each virtual CPU, there should be a physical logical processor. Terminology Each CPU of a physical computer has number of cores which execute programs. Intel also has hyperthreaded cores, there are 2 locations to run programs simultaneously, these are called Logical Processors. Refer Figure 5. Figure 5. Relationship between Processor, Core, Hyper-threaded Core and ESXi Server Properties Guideline for CPU When a virtual machine executes, all the virtual CPUs must have access to CPU resources at the same time. To ensure that every virtual CPU can execute with 26 3BSE

27 Section 3 System Planning Server and System Dimensioning minimal delay, there should be an physical Logical Processor available. Hence, the sum of virtual CPUs in the virtual machines should be less than or equal to the sum of the physical Logical Processors. The formula is represented as follows: (Virtual CPU) (Physical Logical Processors) Max 2 Quad Core CPU per ESXi Server (16 Logical Processors => 16 vcpu) Max 4 vcpu Per VM. 1 vcpu per Physical Logical Processor. For the virtual clients, the virtual CPU (hence the server CPU) produces the graphics. Hence, a faster CPU is recommended (a 2.4GHz or 3.0 GHz CPU is preferable over a 2.0 GHz CPU). Whilst the hyper-threaded cores share core resources and may result in the few percent performance impact, the critical advantage is to increase the number of execution locations for the virtual machines. This minimizes latency in virtual machine execution (being allocated a physical location to execute) and improves performance reliability. Increasing the number of virtual CPUs in a virtual machine also increases the number of available execution locations (physical CPU logical cores) required to run the virtual machine. Since the ABB guideline is not to have more virtual CPUs than physical logical processors, this issue is minimized. The CPU usage of ESXi is negligible and hence does not need to be calculated in as one of the logical core users. Guideline for RAM The RAM required for an ESXi server is based on RAM for the ESXi server (approximately 2GB), and the sum of the RAM required for the virtual machines. There is also overhead RAM (approximately 0.5 GB) for running each virtual machine. In general the following formula should be considered for minimum RAM requirements, for an ESXi server: ESXi RAM GB = 2GB + (VM RAM) + (0.5GB)x(No. VMs) RAM typically comes in 4GB, 8GB, 16GB increments. 3BSE

28 Server and System Dimensioning Section 3 System Planning If there are two processors, then the memory areas for each processor should be loaded with the same amount of RAM. Future expansion of the requirements of the ESXi should also be taken into consideration. When a server has two CPUs, there are two areas for memory. One for each of the CPUs. Each of these areas should be populated with the same amount of memory. For example, if it is decided to have 20GB of memory, this would be 5 x 4GB modules. There would be 2 modules for one CPU and 3 modules for the other. The correct configuration is to have 24GB of memory, in two sets of 3 modules. Guideline for Hard Disk Physical hard disk configurations for ESXi servers must be a redundant RAID. For example, RAID1, RAID5, RAID6 and RAID10. It is preferable to have a hotspare allocated to return the RAID set to full redundancy as quickly as possible if a disk failure occurs. If the ESXi server is installed on the local hard disks, it typically uses less than 4GB of the hard disk space and can be ignored in storage capacity calculations. The actual capacity is dependent on the RAID type, and is mentioned in the Table 1. Table 1. RAID Storage Capacity Minimum No. of RAID Type Capacity Disks RAID1 2 1 x Disk Size RAID5 3 (No. of Disks - 1) x Disk Size RAID6 4 (No. of Disks - 2) x Disk Size RAID 1 or RAID 6 are highly recommended. RAID 1 provides a simple configuration and maintains performance if a disk failure occurs. RAID6 provides a larger capacity and allows for 2 disks to fail. A hotswap spare is highly recommended to provide rapid recovery in the event of disk failure. 28 3BSE

29 Section 3 System Planning Server and System Dimensioning Each running virtual machine has hard disk space allocated as swap space, which equals the amount of RAM allocated to the virtual machine. The amount of hard disk storage required for the ESXi server is the sum required for the virtual machines and the RAM allocated to the virtual machine. Each virtual machine should have a hard disk defined based on the virtual machines requirements. That is, a domain controller would not require as much as a history server. Also, there must be space in the virtual machine hard disk for typical usage (log files, data storage), and additional functionality (adding programs/hotfixes, etc). Hence, if a VM requires 20GB of disk space, 40GB is allocated, else if the VM requires 40GB, then 60GB to 80GB is allocated. Approximately 200GB should be available for anticipated requirements. This may be more based on future plans. In general the following formula provides a guideline for calculating the hard disks requirement: ESXi Hard Disk GB = 200GB + (VM Hard Disk + VM RAM) Based on RAID configurations, more hard disks will be required than the storage required to run the VMs. For Example, 10 VMs, each requiring 60GB of hard disk and allocated 6GB of RAM. If the storage is to be RAID6 with a hotspare, Basic storage = 200GB + 10 x (60GB + 6GB) That is, the basic storage is 860GB RAID6 => (4 x 300GB) + (2 x 300GB) + (1 x 300GB) => 7 x 300GB Hard Disks Table 2. RAID6 with Hotspare Hard Disk Setup RAID6 Base Storage Raid Overhead Hotespar 4 x 300GB = 1200GB 2 x 300GB 1 x 300GB 3BSE

30 Server and System Dimensioning Section 3 System Planning ESXi Hard disk requirement is 7 x 300GB. and If the storage is to be RAID1 with a hotspare, Basic storage = 200GB + 10 x (60GB + 6GB) That is, the basic storage is 860GB RAID1 => (2 x 300GB) + (2 x 300GB)+ (2 x 300GB) + (1 x 300GB) => 7 x 300GB Hard Disks Table 3. RAID1 with Hotspare Hard Disk Setup RAID1 Base Storage Raid Overhead Hotespar 3 x 300GB = 900GB 3 x 300GB 1 x 300GB ESXi Hard disk requirement is 7 x 300GB. A formatted disk has approximately 6% smaller capacity than the stated disk capacity, hence, the storage is scaled up to help ensure adequate capacity after the disks are formatted in the ESXi server. Guideline for Networking One separate gigabit Ethernet port is required for each physical network in which the VMware ESXi server should be connected to. Additionally, one more gigabit Ethernet port is required for the VMware ESXi Service Console. The number of physical network ports is limited by the hardware, and by the number of ports on the network cards used. The actual verification case described in this document used 10 hardware ports. In some cases the configuration may require more physical ports. The theoretical maximum given by the hardware limitation may be overridden by other resource limitations. From ESX 4.0 each virtual machine can have a maximum of ten virtual ports, refer Appendix C, Further Technical Information on VMware ESXi. 30 3BSE

31 Section 3 System Planning Determine Additional Hardware Refer to the VMware Compatibility Guide ( for any specific VMware ESXi hardware configuration constraints that account for the VMware ESX server model. Determine Additional Hardware Apart form the requirements mentioned in the various subtopics previously, there are also additional hardware required. These are explained in the subtopics below. 800xA Licensing Dongle From System 800xA 5.1 revision, VMware vsphere ESXi 4.1 is supported. This version of ESXi introduces support for physical USB devices being recognized in virtual machines. Previously, an AnywhereUSB device was required for this functionality. Note that where the virtual machine is to be moved to another server, the USB dongle must be moved and the configuration must be reviewed in the ESXi server and the virtual machine. In some circumstances, it still may be beneficial to have the USB dongle available over the network. VMware ESXi does not support a physical parallel port connection. The ABB Parallel Port License Dongle cannot be used for ESXi. 3BSE

32 Using the ABB USB Dongle Over the Network Section 3 System Planning Using the ABB USB Dongle Over the Network Where it may be likely to move the license server to another ESXi server, using the ABB dongle over the network is an advantage. With this configuration, it would not be necessary to move the dongle to the new ESXi server. As shown in Figure 6, the AnywhereUSB is a device which has ports for USB dongles and an Ethernet port for network connectivity. Figure 6. Connection of 800xA USB License Dongle to CLS Server Using AnywhereUSB Tying the 800xA License to a network card MAC address is not supported when running in a virtual environment. 32 3BSE

33 Section 3 System Planning 800xA for Advant Master and 800xA for MOD xA for Advant Master and 800xA for MOD 300 In the case of physical Connectivity Server nodes for 800xA for Advant Master and 800xA for MOD 300, an RTA board or RTA unit is used to provide connectivity to respective networks. Hardware requiring separate drivers is usually not supported in virtual environments. This is solved by using the PU410 respectively PU412 RTA Units which is external to the ESXi server and communicates with it via an Ethernet link. Figure 7. Example Connection of 800xA for Advant Master Connectivity Server to MasterBus Dimensioning Rules - Virtual 800xA Application Servers The dimensioning rules can be summarized as 1 Role per hyper-threaded core with a maximum of 16 roles per physical server. A single quad core processor would support 8 roles and a dual quad core processor would support 16 cores. 3BSE

34 Dimensioning Rules - Virtual 800xA Application Servers Section 3 System Planning RAM requirements is the sum of the RAM for each virtual machine and 0.5GB per virtual machine for ESXi execution and 2GB for ESXi. Since each virtual machine can have only 2 application roles, the fixed value of 8GB of RAM per virtual machine supports the ABB 800xA physical dimensioning guidelines. Hard disk requirements is the sum of the virtual machine allocation plus free storage to support ESXi execution of the virtual machines. This should be at least 300GB. This can be summarized as follows: Table 4. Role Based Server Dimensioning Maximum Roles CPU RAM 8 1 Quad Core 2GB + 8.5GB x No. of VMs 16 2 Quad Core 2GB + 8.5GB x No. of VMs Hard Disk Capacity 200GB + Sum of VM RAM + 80GB x No. of VMs 200GB + Sum of VM RAM + 80GB x No. of VMs The RAM and hard disk requirements may be greater based on usage (for example, history storage) and should calculated into the server configuration. 34 3BSE

35 Section 3 System Planning Virtual 800xA Application Servers Example Virtual 800xA Application Servers Example The Table 5 provides an example of a number of system roles that are to be run in a virtual environment. If there are more roles, an additional quad core CPU can be added to the ESXi server. Table 5. Virtual Machine Definition based on 800xA Roles Roles Virtual Machines Number Item Role VM RAM Hard Disk of vcpu 1 Domain Controller 1 1 8GB 80GB 2 Batch Server GB 80GB 3 Aspect Server 3 2 8GB 80GB 4 CLS License Server 5 AC800 Connect 4 2 8GB 80GB 6 PLC Connect 7 PC, Network and Software Monitoring 5 1 8GB 80GB Total 8 44GB 400GB The example of Table 5 can be visualized by the following table: Table 6. Dimensioning Virtual Machine with regard to Physical CPU Virtual VM VM vcpu Physical LP Core CPU 1 3BSE

36 Dimensioning Rules - Virtual 800xA Clients Section 3 System Planning Virtual machine's CPU (vcpu) is not tied to physical CPU. It is important that there are more virtual CPUs than Physical Logical Processors in an ESXi server As indicated in Table 5, the Batch server is in its own virtual machine and is allocated 2 virtual CPUs and 12GB in accordance to the System 800xA System Guide Technical Data and Configuration (3BSE041434*). Server Dimension Following table depicts dimensioning the CPU, RAM and storage capacity in a server: Table 7. Server Dimensioning CPU RAM Storage capacity 1 Quad Core 56GB (2GB + 44GB + 5 x 0.5GB = 48.5GB) 640GB (200GB + 5 x 80GB + 5 x 8GB) Raided Then the total number of hard disks for RAID6 is as follows: RAID6 = (storage capacity) + (RAID overhead) + (hotspare) =(3 x 300GB) + (2 x 300GB) + (1 x 300GB) =6 x 300GB Hard Disks Whilst 3 x 300GB disks would equal 900GB, approximately 6% is consumed by disk formatting. Dimensioning Rules - Virtual 800xA Clients Virtual clients must be run on ESXi hosts which are not running 800xA servers, that is, one or more separate ESXi servers. To decide how many virtual clients you should restrict to, refer System 800xA System Guide Technical Data and Configuration (3BSE041434*). Observing the load can be performed by opening the task manager in the remote desktop session and checking the AfwWorkplaceApplication.exe CPU usage. Be aware that the number of vcpu must be taken into account. E.g. for a 3 vcpu 36 3BSE

37 Section 3 System Planning Dimensioning Rules - Virtual 800xA Clients virtual machine, a 33% CPU usage is the maximum for a single thread in the AfwWorkplace.exe that is rendering the graphics. (2vCPU would have a maximum of 50%, and 4 vcpu would have a maximum of 25%). Note that this is only for the one thread that renders the graphics. There are other activities handled by other threads handle that is included in the CPU usage. In the Remote desktop session of a virtual client, to see the percentage of CPU usage perform the following procedure: 1. Open the Task Manager. 2. Click the Performance tab. 3. Right-click on the CPU chart, point to Change graph to and then select Logical processors. Figure 8. Viewing Number of CPU in a Remote RDP Session In the Figure 8, 3 vcpu have been allocated to the virtual machine, hence, a single thread in a process can only take a maximum of 33.3% of the total CPU capacity of the virtual machine. 3BSE

38 Dimensioning Rules - Virtual 800xA Clients Section 3 System Planning 4. Click Process tab and select AfwWorkplaceApplication process. Read the CPU percentage. Figure 9. Viewing the CPU Usage of the AfwWorkplaceApplication The above example is a 3 vcpu virtual machine (hence maximum 33% of total virtual machine CPU is used for a single thread). Approximate loading percentage of the graphics is: %Max = 100% * (19.9%/33.3%) % Max = 59.8% To reduce this percentage, either less content (graphics, faceplates, etc) or a faster server CPU is required. An 800xA Virtual Client requires 1 virtual CPU, 4GB of RAM and an 80GB Hard Disk. The number of remote users and virtual machine dimensioning for virtual clients can be summarized as follows: Table 8. Number of Remote Users and Virtual Machine Dimensioning Type Number of users Number of vcpu RAM Storage capacity Virtual Client GB Virtual Client GB 38 3BSE

39 Section 3 System Planning Virtual 800xA Client Example Table 8. Number of Remote Users and Virtual Machine Dimensioning Type Number of users Number of vcpu RAM Virtual Client GB Virtual Client GB Virtual Client GB Maximum number of roles and maximum number of users for a Server dimensioning can be summarized as follows: Storage capacity Table 9. Maximum Number of Roles/Users and Server Dimensioning Maximum Roles Maximum Users CPU Quad Core Quad Core RAM 2GB + Sum VM RAM + 0.5GB x No. of VM 2GB + Sum VM RAM + 0.5GB x No. of VM Storage Capacity 200GB + Sum VMs RAM + 80GB x No. of VM 200GB + Sum VMs RAM + 80GB x No. of VM Virtual 800xA Client Example The Table 10 provides an example of a number of system that requires 8 client users. Table 10. Example Server Dimensioning of 8 Virtual Client Users Roles Virtual Machines Number Item Role VM RAM Hard Disk of vcpu 1 Virtual Client 1 1 4GB 80GB 2 Virtual Client 2 1 4GB 80GB 3BSE

40 Virtual 800xA Client Example Section 3 System Planning Table 10. Example Server Dimensioning of 8 Virtual Client Users Roles Item Role VM Virtual Machines Number of vcpu 3 Virtual Client 3 1 4GB 80GB 4 Virtual Client 4 1 4GB 80GB 5 Virtual Client 5 1 4GB 80GB 6 Virtual Client 6 1 4GB 80GB 7 Virtual Client 7 1 4GB 80GB 8 Virtual Client 8 1 4GB 80GB Total 8 32GB 640GB The example of Table 10 can be visualized by the following table: RAM Hard Disk Table 11. Dimensioning virtual machine CPU with regard to physical CPU Virtual Role VM VM vcpu Physical LP Core CPU 1 2 While it appears that one CPU is left unused, the virtual machines execution is spread across the CPUs which reduces the hyper-threading inefficiencies. 40 3BSE

41 Section 3 System Planning Virtual 800xA Client Example Server Dimension The following table depicts dimensioning the CPU, RAM and storage capacity in a server: Table 12. Server Dimensioning CPU RAM Storage capacity 1 Quad Core 24GB (2GB + 16GB + 4 x 0.5GB = 20GB) 600GB (200GB +4 x 80GB + 4 x 4GB) Raided Then the total number of hard disks for RAID6 is as follows: RAID6 = (storage capacity) + (RAID overhead) + (hotspare) = (2 x 300GB) + (2 x 300GB) + (1 x 300GB) = 5 x 300GB Hard Disks In the example, the storage capacity is rounded up to the typical disk size. It is also assumed that 2 virtual machines will share the same storage location 3BSE

42 Virtual 800xA Client Example Section 3 System Planning 42 3BSE

43 Section 4 ESXi Installation This topic describes how to install the VMware ESXi server for virtual 800xA server nodes.when installing a new server, the vendors custom ESXi installation media should be used as this has the drivers required for the hardware. Before using virtualization in any computer, the BIOS should be checked to ensure that virtualization support has been enabled. If not enabled, then enable the options Execute Disable and Virtualization VT. Install ESXi 5.5 Server While servers can come pre-installed in the ESXi, the procedures outlined are for new installations. Many modern servers are designed with internal USB or SD slots. When extracted with a memory device, they can be used for the installation destination of ESXi. Perform the following to install VMware ESXi server software: 1. Boot the computer with the VMware ESXi server installation media to start the VMware ESXi Installer. If the ESXi boot menu does not appear, then the server BIOS settings should be set to boot from CD. 2. Select the Standard Installer from the ESXi Installer (Figure 10). 3. Press Enter to install (Figure 11) on the Welcome to the VMware ESXi Installer dialog box. 4. Read the End User License Agreement and press F11 to accept and continue (Figure 12). 3BSE

44 Install ESXi 5.5 Server Section 4 ESXi Installation Figure 10. ESXi Installer Figure 11. Installation Welcome Screen 44 3BSE

45 Section 4 ESXi Installation Install ESXi 5.5 Server Figure 12. End User License Agreement 5. Select the Local Storage Device to install ESXi. This can be a USB disk. Use the arrow keys to select the device and press Enter (Figure 13). If a warning message that the device contents will be deleted, think again if there is any data on the storage device that will be lost. If there is, cancel the installation and move the data from the server. Figure 13. Selecting the Installation Destination 3BSE

46 Install ESXi 5.5 Server Section 4 ESXi Installation 6. Use the arrow keys to select the keyboard layout and press Enter (Figure 14). Figure 14. Keyboard Layout Selection 7. Type the Root password (administrator) and confirm. Press Enter to continue (Figure 15). Figure 15. Enter Root Password 8. Press F11 at the Confirm install dialog box (Figure 16). 9. Remove the installation media and press Enter to reboot after the installation is complete (Figure 17). 46 3BSE

47 Section 4 ESXi Installation Install ESXi 5.5 Server Figure 16. Confirming the Installation After installation, the console provides information of the current IP address and provides the ability to perform basic configurations, such as, change the IP address, network adapter, or change the ESXi sever host name (Figure 18). After installation, the host name of the ESXi server should be configured. On many systems, a DHCP server will not be present to provide an initial IP address. A static IP address can also be configured through the ESXi console. Refer ESXi Configuration Console for the examples of configuring the ESXi server through the ESXi console. Figure 17. Installation Complete 3BSE

48 Install ESXi 5.5 Server Section 4 ESXi Installation Figure 18. Basic Console Interface of ESXi 48 3BSE

49 Section 5 ESXi Configuration Console ESXi Login This topic describes how to configure the ESXi console. The ESXi console provides the ability to configure the network properties use for the management interface. The interface is cursor based, where selection and changing of properties is performed using the keyboard. Once the network has been established, the VMware vsphere client is used to interact with the ESXi sever. Perform the following to log on to the ESXi server: 1. Press F2 for the ESXi server dialog box. Figure 19. ESXi Login 3BSE

50 Configure Management Network Section 5 ESXi Configuration Console 2. Type the root password and press Enter. Figure 20. Providing the Root Password for Login to the ESXi Server Configure Management Network To configure the management network, perform the following: 50 3BSE

51 Section 5 ESXi Configuration Console Configure Management Network 1. Use the arrow keys to select Configure Management Network and press Enter. Figure 21. Configure Management Network Option 2. Use the arrow keys to select the Network Adapters option and press Enter. The first stage to ensure that the management network is configured correctly is to ensure that the correct network adapter is used. When initially setting up the network, it can be easier to identify the management network if it is the only network connected. 3BSE

52 Configure Management Network Section 5 ESXi Configuration Console Figure 22. Network Adapters Configuration 3. Select the Network Adapters marked with X by pressing the Spacebar, and then press Enter. Figure 23. Management Network Adapter 52 3BSE

53 Section 5 ESXi Configuration Console Configure Management Network 4. Use the arrow keys to select IP Configuration, and press Enter to change the IP address of the ESXi management interface. Figure 24. Selecting the IP Configuration 5. Use the arrow keys to select Set static IP address and network configuration and then the Spacebar to enable the option. Figure 25. Management Network Settings 3BSE

54 Configure Management Network Section 5 ESXi Configuration Console 6. Use the arrow keys to set the IP Address, Subnet Mask, and Default Gateway (see Figure 25). 7. Press Enter to accept the values. 8. Use the arrow keys to select the DNS Configuration, and press the Enter. It is important to set the host name to a value other than local host to ensure that SNMP Traps sent from the ESXi server contains the IP address and not Figure 26. Access the DNS Configuration to Specify the Host Name 54 3BSE

55 Section 5 ESXi Configuration Console Configure Management Network 9. Select Use the following DNS server addresses and hostname by pressing the down arrow and then the Spacebar to enable the option. Figure 27. Specifying the Host Name 10. Use the Down Arrow key to select the Hostname. 11. Type the Hostname (Figure 27), and press Enter to accept the values. 3BSE

56 Configure Management Network Section 5 ESXi Configuration Console 12. Press the Esc at the Configure Management Network dialog box to apply the changes. This will trigger a request to restart the management network. Figure 28. Exiting the Configure Management Network 13. Press Y to apply the changes. Figure 29. Applying Changes 14. Press Esc to logout (see Figure 21). 56 3BSE

57 Section 6 VMware vsphere Client Installation This topic describes how to install VMware vsphere Client and how to log on to VMware vsphere Client. The VMware vsphere Client is the graphical interface used to interact with the ESXi server to configure the server itself and the virtual machines in the server. Perform the following to install VMware vsphere Client software: 1. Download the VMware vsphere Client from Figure 30. Running the VMware vsphere Client Installation Program 2. Run the.exe on the client to be used to configure the ESXi server. 3BSE

58 Installation Section 6 VMware vsphere Client 3. Select the appropriate language from the drop-down list and click OK. Figure 31. Language Settings 4. Click Next on the welcome screen dialog box. Figure 32. VMware Vsphere Client Installation Welcome Screen 5. Select I accept the terms in the license agreement and click Next. 58 3BSE

59 Section 6 VMware vsphere Client Log into an ESXi Server 6. Leave the destination folder to the default path and click Next. Figure 33. Default Location for the Installation 7. Click Install at the Ready to Install the Program dialog box. Figure 34. Ready for the Installation of the VMware vsphere Client Program 8. Click Finish to complete the installation. Log into an ESXi Server Perform the following to log on to the ESXi server: 3BSE

60 Log into an ESXi Server Section 6 VMware vsphere Client 1. Double-click the on the desktop to start the VMware vsphere Client. From ESXi 5.5, the client informs that all new vsphere features are available only through the vsphere Web Client. This has little impact on basic virtual machine creation and operation where the ESXi server is not a member of a vcenter server. 2. In the VMware vsphere Client dialog box, type the IP address, User Name and Password of the ESXi server, and click Login. Figure 35. Login to ESXi Server 60 3BSE

61 Section 6 VMware vsphere Client Log into an ESXi Server 3. Click Ignore on the Security Warning message. It is recommended to install a trusted certificate in the ESXi server. Figure 36. Security Warning Message 3BSE

62 Log into VMware vcenter Server Section 6 VMware vsphere Client vsphere Client displays the host with virtual machines in the left pane, and a set of tabs to provide more interaction with the host, or virtual machine. Figure 37. vsphere Client Example Log into VMware vcenter Server When logging into a VMware vcenter Server, it is essential to use the SSO credentials. During the vcenter Server quick install, a default domain of vsphere.local is created. This is not a Microsoft Active Directory domain. To login, the correct user, domain, and password have to be used. The format for the user is: username@ssodomain. The password is same as the password for the administrator user, it is configured during the installation of the vcenter Server. The same credentials also apply for logging on to an ESXi server that is a member of an VMware vcenter Server. 62 3BSE

63 Section 6 VMware vsphere Client Log into VMware vcenter Server Perform the following to log on to a VMware vcenter Server: 1. In the VMware Vsphere Client dialog box, type the IP Address, User name and Password, and then click Login. Figure 38. VMware SSO Credentials to Log into the vcenter Server 3BSE

64 Log into VMware vcenter Server Section 6 VMware vsphere Client By logging in to the vcenter Server, the vsphere Client shows the vcenter Object, Datacenter, Clusters, and hosts (see Figure 39). Figure 39. vsphere Client Interface When Logged into a vcenter Server 64 3BSE

65 Section 7 Basic Configuration Adding Disks This topic describes the basic configuration that needs to be done after installing ESXi in a server so as to add the virtual machines. Adding the additional storage is done using the vsphere Client. Perform the following to add additional storage: 1. Open the vsphere Client. 2. Click Configuration tab (Figure 40). Figure 40. Configuring Storage to the ESXi Server 3. In the Hardware pane, select Storage and click ADD Storage (Figure 40). 3BSE

66 Adding Disks Section 7 Basic Configuration 4. Select Disk/LUN and click Next. Figure 41. Disk/LUN Storage to Add 5. Select the available storage to add, and click Next. Figure 42. Local Storage to Add 66 3BSE

67 Section 7 Basic Configuration Adding Disks 6. Select the VMFS-5 and click Next. Figure 43. VMFS-5 Option 3BSE

68 Adding Disks Section 7 Basic Configuration 7. Review the configuration change and click Next. Figure 44. Configuration Changes 68 3BSE

69 Section 7 Basic Configuration Adding Disks 8. Enter a datastore name and click Next. Figure 45. Naming Data Storage 9. Select Maximum available space, and click Next. Figure 46. Selecting Maximum Available Space 3BSE

70 Configure Networks Section 7 Basic Configuration 10. Review the changes and click Finish. The Recent Tasks pane shows the status of adding a storage is completed. The new data storage is available in the list of data stores (Figure 47). Figure 47. Data Store Added Configure Networks Prior to setting up VMs, it is good practice to setup the virtual network using virtual switches. A virtual switch is an emulation of a physical switch. In VMware ESXi, it is possible to create virtual switches and to connect VMs to these virtual switches to form a virtual network. 70 3BSE

71 Section 7 Basic Configuration Configure Networks In the Figure 48 two virtual switches plant network 1 (PN1) and a control network 1 for area 20 (CN1 A20) are connected to various VMs. Figure 48. ESXi Virtual Switches and Physical Network Adapters Connection A virtual switch can be bridged to a physical network adapter of the VMware ESXi server. This is shown in Figure 48 by the dotted lines. The VMware ESXi server itself is connected to the physical plant and control networks through these physical adapters. By doing so, virtual switches provide a means for connecting virtual 800xA server nodes to the physical system networks. From VMware ESXi version 4.0, each VM can have a maximum of ten virtual network adapters. Therefore, a VM can be connected to a maximum of ten virtual switches. In the example shown in Figure 48, the virtual CS ID1001CS1A is connected to the plant network PN1 and to the control network CN1 A20. In the same manner, create a separate virtual switch for each physical plant or control network to which virtual 800xA server nodes require a connection. It is suggested to name a virtual switch like the physical network it is bridged to (for example, PN1 for the virtual switch bridged to the Physical PN1 as shown in Figure 48). For network redundancy, create a second virtual switch (for example, named PN2). 3BSE

72 Configure Networks Section 7 Basic Configuration It can be difficult to know how the physical network adapters are related to the VMware ESXi virtual network adapters. To make this easier, connect each additional network adapter to a physical switch, then setup the virtual switch. This will result in an unused (in ESXi) adapter which is connected. Perform the following to add a virtual switch to the ESXi server: 1. Open the vsphere Client. 2. Click Configuration tab. Figure 49. ESXi Networking Configuration 3. In the Hardware pane, select Networking, and click ADD Networking... (see Figure 49). 72 3BSE

73 Section 7 Basic Configuration Configure Networks 4. In Add Network Wizard dialog box, for the Connection type, select Virtual Machine and click Next. Figure 50. New Virtual Switch for Virtual Machine Communication 5. For Network Access, select vmnic1 check box under Create a vsphere standard switch, and click Next. As seen in Figure 51, one Ethernet adapter is 1000 Full, and other is Down. This clearly indicates which adapter is newly connected. 3BSE

74 Configure Networks Section 7 Basic Configuration Figure 51. Selecting the Network Adapter to be Connected to the Virtual Switch 74 3BSE

75 Section 7 Basic Configuration Configure Networks 6. For Connection Settings, enter a name in the Network Label text box and click Next. Figure 52. Naming the Virtual Switch 7. Click Finish. 3BSE

76 Set the Time Server Section 7 Basic Configuration The new network switch is seen in the network configuration tab of the ESXi server. Figure 53. Network Switch Added 8. Repeat Step 2 through Step 7 to add more virtual switches. Set the Time Server When a virtual machine starts, it initially takes the time from the ESXi server. If there is a time synchronization inside the guest, the time will then be updated by its time source. If the ESXi server is not synchronized to a reliable time server, this creates a step change in the time in the guest node which could interfere with the applications running inside the guest. It is important that the VMware tools are not synchronizing time with the ESXi server. This is the default setting and should not be changed. The ESXi server internal clock must be synchronized to a reliable NTP server. This NTP server must have the same time as the 800xA system and be accessible by the ESXi management network. Configuring of the NTP client in the ESXi server is performed through the vsphere Client. 76 3BSE

77 Section 7 Basic Configuration Set the Time Server Perform the following to configure the NTP client in the ESXi server: 1. Open the vsphere Client. 2. Click Configuration tab. Figure 54. Accessing the ESXi Time Synchronization Configuration 3. In the Software pane, select Time Configuration, and click Properties... (see Figure 54). 3BSE

78 Set the Time Server Section 7 Basic Configuration 4. In Time Configuration dialog box, click Options button to access the NTP servers configuration. Figure 55. Accessing the NTP Servers Configuration 5. In NTP Daemon (ntpd) Options dialog box, to add an NTP server a. Select the NTP Settings and click Add. 78 3BSE

79 Section 7 Basic Configuration Set the Time Server b. Enter address in ADD NTP Server dialog box and click OK. Figure 56. Adding an NTP Server to the ESXi Time Configuration 6. In NTP Daemon (ntpd) Options dialog box, a. Select General. b. Click Start and select Start and Stop with host option. c. Click Ok. The NTP Daemon (ESXi service that update the time) is initially stopped. 3BSE

80 Set the Time Server Section 7 Basic Configuration Figure 57. Starting the NTP Daemon and Selecting it to Start/Stop with the Host 80 3BSE

81 Section 7 Basic Configuration Set the Time Server 7. In Time Configuration dialog box, select NTP Client Enabled and click OK. Figure 58. Enabling the NTP client 3BSE

82 ESXi USB/SD Install: System Logs Location Section 7 Basic Configuration The ESXi shows the NTP client running and NTP servers have been added along with the Data and Time. Figure 59. Running ESXi NTP Client In case, where the NTP server is a Windows computer, additional configuration is necessary. Refer Section 26, Timesync ESXi with Windows. ESXi USB/SD Install: System Logs Location When ESXi is installed to a USB or SD memory card, the ESXi log files are setup to be stored on the USB or SD memory card. Since the storage capacity is limited, the size of the log files is also restricted by ESXi to prevent the file system from being filled up. 82 3BSE

83 Section 7 Basic Configuration ESXi USB/SD Install: System Logs Location ESXi warns that the log files are being stored on the USB or SD memory card by the following message: Figure 60. ESXi Warning Perform the following to address the warning about the storage of the log files: 1. In vsphere Client Data Browser, click the folder icon to create a folder on one of the hard disk datastores to store the log files, see Figure 61. Figure 61. Folder created to store ESXi log files 2. Open the vsphere Client. 3. Click Configuration tab 3BSE

84 ESXi USB/SD Install: System Logs Location Section 7 Basic Configuration 4. In the Software pane, select Advanced Settings. Figure 62. Advanced Settings 84 3BSE

85 Section 7 Basic Configuration ESXi USB/SD Install: System Logs Location 5. In the Advance Settings dialog box, select global under Syslog. Figure 63. Advanced Settings Syslog Destination 6. Enter a name in the Syslog.global.logDir text box and click OK. Follow a naming convention while giving a name to the log directory (see Figure 63). This directory is stored in the datastore folder. 3BSE

86 ESXi USB/SD Install: System Logs Location Section 7 Basic Configuration 7. Go to the folder in the vsphere Client Datastore browser to confirm the settings are correct. A number of log files are present, see Figure 64. Figure 64. ESXi Logs are being Written to the Defined Folder in the Datastore 86 3BSE

87 Section 8 Create Virtual Machines This topic describes how to create a virtual machine through the vsphere Client interface. A wizard assists the creation of the virtual machine, where the most important configuration items are specified, such as, the hard disk size. Whilst most of the virtual machine can be customized after it is created, the hard disk size is more troublesome to reconfigure, hence, the size should be planned based on typical usage of the node type. The data that the applications will create must be taken into account. Space should also be available for the operating system for log files, temporary files, etc. For example, Operating System = 20GB, Application = 10GB, Expected Data = 20GB, Free space = 20GB, then the virtual hard disk should be around 70GB to 80GB. Create a Virtual Machine Perform the following to create a virtual machine: 1. Open the vsphere Client. 3BSE

88 Create a Virtual Machine Section 8 Create Virtual Machines 2. Right-click on VMware ESXi server and select New Virtual Machine. Figure 65. Creating a New Virtual Machine 3. In Create New Virtual Machine dialog box, select Custom and click Next. Custom option is selected to provide advanced selection of the virtual machine configuration. 88 3BSE

89 Section 8 Create Virtual Machines Create a Virtual Machine Figure 66. Specifying Custom Virtual Machine Creation 4. Enter a name and click Next. Figure 67. Specifying a Name for the New Virtual Machine 3BSE

90 Create a Virtual Machine Section 8 Create Virtual Machines 5. Select the location of storage for the virtual machine and click Next. There should be enough free space for the size of the virtual machine disk and a swap file which will be the size of the RAM allocated to the virtual machine. Figure 68. Storage Location for the Virtual Machine 6. Select Virtual Machine Version 8 and click Next. Specify the Virtual Machine version that is applicable to ESXi 5.0 and later. 90 3BSE

91 Section 8 Create Virtual Machines Create a Virtual Machine Figure 69. Specifying the Virtual Machine Version 3BSE

92 Create a Virtual Machine Section 8 Create Virtual Machines 7. Select Windows as Guest Operating System and Microsoft Windows Server 2012 (64-bit) from the drop-down list, and then click Next. Figure 70. Specifying Guest Operating System and Version 8. Select the Number of virtual sockets, and Number of cores per virtual socket from the drop-down list, and click Next. The total number of cores indicates the number of vcpu allocated to the virtual machine. The software to be run in the virtual machine is licensed on sockets, it may be a cost advantage to specify only one socket and multiple cores. 92 3BSE

93 Section 8 Create Virtual Machines Create a Virtual Machine Figure 71. Number of vcpu in the Virtual Machine 9. Select or type the value for Memory Size and type from the drop-down list, and click Next. The sum of the RAM allocated to all of the running virtual machines should be less than the physical RAM of the ESXi server. 3BSE

94 Create a Virtual Machine Section 8 Create Virtual Machines Figure 72. Specifying RAM 10. Under Create Network Connections select a suitable value for the following from the drop-down list and click Next: How many NICs do you want to connect? Network Adapter Up to 4 NICs can be added initially to the virtual machine. More can be added/removed/edited once the virtual machine has been created. 94 3BSE

95 Section 8 Create Virtual Machines Create a Virtual Machine Figure 73. Initial Network Adapters The NIC adapter should not be set to E1000 where Windows 2012 R2 Server is used, network traffic between virtual machines running Windows 2012 R2 Server cause the ESXi server to crash and cause a PSOD (Purple Screen Of Death). A patch is expected to be released by VMware. The virtual adapter to be used in 800xA 5.x and 800xA 4.x is E1000. The virtual adapter to be used in 800xA 6.0 is VMxNet3. 3BSE

96 Create a Virtual Machine Section 8 Create Virtual Machines 11. Select the LSI Logic SAS as SCSI Controller and click Next. Figure 74. SCSI Controller 12. Select Create a new virtual disk and click Next. 96 3BSE

97 Section 8 Create Virtual Machines Create a Virtual Machine 13. Select the Disk Size from the drop-down list. Figure 75. Disk Size 14. Select Thick Provision Lazy Zeroed under Disk Provisioning. 15. Select Store with the virtual machine under Location (Figure 75), and click Next. For 800xA production systems, it is recommended to not use thin provisioning and snapshots. 3BSE

98 Create a Virtual Machine Section 8 Create Virtual Machines 16. Leave the Virtual Device Node and Mode to default settings, and click Next. Figure 76. Virtual Device Node and Mode settings 98 3BSE

99 Section 8 Create Virtual Machines Install the Guest Operating System 17. Review the settings and click Finish to complete the setup. Creation of the virtual machine will take a few seconds and then appear in the ESXi inventory (Figure 77). Figure 77. Virtual Machine is Setup The new virtual machine will not have an operating system or applications installed in it. These will have to be installed in a similar manner to that of a physical computer. Install the Guest Operating System Whilst it is possible to connect the virtual machine to a CD/DVD which has the installation media, a more efficient method is to upload the ISO image of the media to one of the datastores and point the virtual machine CD/DVD to the ISO image. Perform the following to upload the ISO file to the datastore: 1. Browse one of the datastores. 2. Create a folder and name it ISO. 3BSE

100 Add ISO file to the Virtual Machine Section 8 Create Virtual Machines 3. Upload the ISO image into the folder. Figure 78. Operating System ISO image to ESXi Datastore Add ISO file to the Virtual Machine After the ISO file has been uploaded to the datastore perform the following to upload the ISO file to the virtual machine: 1. Open vsphere client. 2. Right-click on the new virtual machine (here it is 800xA_AS1) and select Edit Settings To edit the virtual machine settings, a. Click Hardware tab and select CD/DVD drive 1 (edited). b. Select Connect power on under Device Status BSE

101 Section 8 Create Virtual Machines Add ISO file to the Virtual Machine c. Under Device type, select Datastore ISO file and click Browse to upload the suitable ISO file, and then click OK. Figure 79. Adding the ISO File It is recommended to set the Video card memory to at least 64MB. This provides the ability to resize the virtual machine console to larger sizes. Operating System Installation Perform the following to start the operating system installation: 1. Open vsphere Client. 3BSE

102 Add ISO file to the Virtual Machine Section 8 Create Virtual Machines 2. Right-click on the virtual machine and select Open Console. Figure 80. Accessing the Virtual Machine Console 102 3BSE

103 Section 8 Create Virtual Machines Add ISO file to the Virtual Machine 3. Click the Start button to start the virtual machine. Figure 81. Starting the Virtual Machine 3BSE

104 Add ISO file to the Virtual Machine Section 8 Create Virtual Machines 4. Install the Windows operating system in the virtual machine in the same manner as that of a physical computer. Figure 82. Installation of the Windows Operating System VMware tools should be installed after the operating system is installed. Prior to this, it is required to press the Ctrl +Alt key at the same time to release the mouse and keyboard focus from the virtual machine. VMware tools will also assist in making the mouse cursor more responsive BSE

105 Section 8 Create Virtual Machines Install VMware Tools Install VMware Tools VMware tools provides drivers and services to the guest operating system which also enhances the user interaction with the virtual machine. For example, without VMware Tools installed, to move the mouse and keyboard focus from the virtual machine console, it is required to press the Ctrl + Alt keys. With VMware Tools installed, the mouse moves freely between the computer running the vsphere Client and virtual machine console. Graphics drivers installed by VMware tools improve the responsiveness of the mouse in the virtual machine console. Perform the following to install the VMware tools: 1. Perform Step 1and Step 2 of Operating System Installation to open the virtual machine console. 2. Click on VM and point to Guest and select Install/Upgrade VMware Tools. Figure 83. Initialing the Installation of VMware Tools 3BSE

106 Install VMware Tools Section 8 Create Virtual Machines 3. Click Ok on the Install VMware Tools message. Figure 84. Installation Notification 4. In the virtual machine, open a Windows explorer, and double-click on the DVD Drive(D:) VMware Tools to start the installation. Figure 85. Initiating the Installation of VMware Tools 5. Click Next on the VMware Tools Setup dialog box. 6. For Setup type, select Typical and click Next. 7. Click Install at Ready to install VMware Tools BSE

107 Section 8 Create Virtual Machines Install VMware Tools 8. Click Finish to complete the installation. 9. Click Yes on the VMware Tools Setup machine to restart the machine. Once restarted, it is now possible to resize the virtual machine desktop size. To resize perform the following: 1. Perform Step 1and Step 2 Operating System Installation to open the virtual machine console. 2. Click View and select Fit Guest Now. Figure 86. Resizing the Guest Desktop to fit the Virtual Console Size If it there are difficulties in resizing the desktop, check that the virtual machine video adapter has to be allocated at least 64MB. 3BSE

108 Virus Scanning Section 8 Create Virtual Machines The virtual machine desktop should fit the size of the virtual machine console. Virus Scanning Figure 87. Resized Virtual Machine Desktop After Windows and the VMware Tools are installed, install and configure 800xA software into the VM the same way as on physical computers by using the 800xA System Installer tool. For more information, refer to the System 800xA Installation and Upgrade Getting Started (2PAA111708*). Virus scanning of computers is a CPU and disk intensive activity. It is best practice to stage schedule hard disk scanning in virtual machines so that there is not more than one virtual machine scanning it s hard disk in an ESXi server at a time BSE

109 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi 5.5 Upgrade to ESXi 5.5 The upgrade of an ESXi server requires all virtual machines to be shutdown, it is essential to ensure that there is no risk to production, that is, the upgrade is performed during a factory maintenance shutdown. Before upgrading an ESXi server to ESXi 5.5, consider the following: All virtual machines running on the ESXi server must be shut down. This is done inside Windows in the virtual machine in the same manner as shutting down a physical computer. Ensure that the server vendor supports ESXi 5.5 before upgrading. Use the server vendor custom ESXi installer which is available from the vendors support web site. After migration, the vsphere Client 5.5 needs to be installed on the client computer. Previous versions can remain on the client. An ESXi 5.5 license will need to be installed. The process for upgrading an ESXi to ESXi 5.5 is the same for the different versions of ESXi. It is important to ensure that an upgrade of the existing installation is made which preserves existing VMFS (VMware File System) partitions. Perform the following to upgrade to ESXi 5.5: 1. Open vsphere Client. 3BSE

110 Upgrade to ESXi 5.5 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi Check that the virtual machines on ESXi have been shut down. Figure 88. Checking that all Virtual Machines have been Shut down 3. Right-click on the ESXi server and select Enter Maintenance Mode. You must opt for the ESXi server into maintenance mode to ensure that the virtual machines will be started in a controlled manner, This will prevent the virtual machines from starting automatically when the upgraded ESXi server starts BSE

111 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi 5.5 Upgrade to ESXi 5.5 Figure 89. Opt the ESXi Server into Maintenance Mode 4. Click Yes on the Confirm Maintenance Mode message. 5. Insert ESXi 5.5 installation media into the server. 3BSE

112 Upgrade to ESXi 5.5 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi Right-click on the host in vsphere Client and select Reboot. Figure 90. Rebooting the ESXi Server 112 3BSE

113 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi 5.5 Upgrade to ESXi 5.5 The ESXi boot menu appears if the server has been correctly setup in the BIOS to boot from the CD, see Figure 91. Figure 91. Booting the Server from the Installation Media 3BSE

114 Upgrade to ESXi 5.5 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi Press Enter to continue. Figure 92. ESXi Installation Welcome Screen 8. Press F11 to accept the License and continue. Figure 93. Accepting the VMware Enduser License Agreement 114 3BSE

115 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi 5.5 Upgrade to ESXi If the correct disk has been selected, the disk details will show that an ESXi installation has been found. 10. Use arrow keys to select the disk, and press the F1 to display the disk details. Typically, the ESXi installation is on the first disk, however, it is important to confirm this before commencing the installation. Figure 94. Selecting ESXi Server Hard Disks 11. Press Enter for the Disk Details dialog box.. Figure 95. Disk Details 3BSE

116 Upgrade to ESXi 5.5 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi Press Enter to continue the ESXi upgrade. Figure 96. Continuing with the ESXi Upgrade 116 3BSE

117 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi 5.5 Upgrade to ESXi Select the Migrate ESX, preserver VMFS datastore using the arrow keys, and press Enter to maintain the existing settings and datastores of the ESXi server. Figure 97. Selecting to Upgrade ESX and Preserve Datastores 14. Review the migration settings and press F11. Figure 98. Confirmation of Migration of the ESX server to ESXi 3BSE

118 Upgrade to ESXi 5.5 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi 5.5 When upgrading from 5.x to 5.5, then option used is called upgrade instead of migrate. A non-interruptible progress bar shows the migration status. 15. After completion of the migration, remove the installation media from the drive and press Enter to reboot the server. Ensure that the VMware vsphere Client 5.5 has been installed in the client. Figure 99. Completion of the ESXi to ESXi 5.5 Migration 16. After the server has restarted, open vsphere Client, and review the configuration BSE

119 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi 5.5 Upgrade to ESXi Right-click on host and select Exit Maintenance Mode, and power on the virtual machines. Figure 100. Exiting Maintenance Mode to Enable the Virtual Machines to be Started 3BSE

120 Upgrade to ESXi 5.5 Section 9 ESX 4.0/ESXi 4.x/ESXi 5.x to ESXi BSE

121 Section 10 vsphere Command Line Interface Overview Download This topic describes how to install vsphere Command Line Interface (CLI). The VMware ESXi interface and the vsphere Client does not have the ability to execute commands in the ESXi Server. This functionality is provided by vsphere CLI, allowing the execution of administration commands on the ESXi Server. Where additional files are required, these can be uploaded to a datastore prior to executing the commands. Typical usage for the vsphere CLI is to install patches and setting SNMP trap configurations. The VMware CLI is downloaded from VMware website and installed on a computer that has network connectivity to the management interface on the ESXi server. The version should be same version as the ESXi servers that are to be administrated. Download VMware vsphere CLI 5.5 from Install VMware vsphere CLI The VMware vsphere CLI is installed on computer that is also connected to the ESXi management interface. While this could be a virtual machine, the management interface should have limited restricted access over the network to minimize security threats. To install the VMware vsphere CLI 5.5: 3BSE

122 Install VMware vsphere CLI Section 10 vsphere Command Line Interface 1. Double-click on the VMware vsphere CLI installation program. Figure 101. VMware vsphere CLI Installation Program 2. Check that the publisher is VMware and click Run. Figure 102. Confirmation to Run the VMware vsphere CLI 122 3BSE

123 Section 10 vsphere Command Line Interface Install VMware vsphere CLI 3. Click Next at the welcome screen dialog box. Figure 103. Welcome Screen 4. Select I accept the terms in the license agreement and click Next. 5. Leave the destination folder for the installation to the default location and click Next. Figure 104. Default Location of the VMware vsphere CLI Installation 3BSE

124 Install VMware vsphere CLI Section 10 vsphere Command Line Interface 6. Click Install at the ready to install dialog box. 7. Click Finish to complete the VMware vsphere CLI 5.5 installation. Figure 105. Installation Complete 124 3BSE

125 Section 11 USB Licensing Dongle This topic describes the details of adding a USB License Dongle in the ESXi server and in the virtual machines. In physical 800xA nodes, ABB 800xA USB licensing dongle is attached to the computer where the 800xA CLS service is running. Connecting the 800xA USB Licensing Dongle to the CLS Server To enable the virtual machine to access the USB dongle perform the following: 1. Edit the Settings of the virtual machine. 2. Add Hardware of Device Type USB Controller. The configuration must be allowed to complete before adding USB devices. 3. Insert the ABB USB Dongle into the ESXi Server. 4. Add Hardware of Device type USB Device. 5. Add relevant ABB Licence Dongle USB device type from the listed devices. 6. Click Next, and click Finish 7. Click OK. 8. Start the virtual machine. For more information on Central Licensing Server, refer System 800xA Licensing Information (2PAA111691*). 3BSE

126 Section 11 USB Licensing Dongle 126 3BSE

127 Section 12 Backup and Restore This topic describes the details of backup and restore activities performed in the ESXi server and in the virtual machines. Whilst there are many virtual machine backup solutions, it is important to understand that the backups are usually made when the virtual machine operating system has open files. This results in a backup where the integrity of the files cannot be guaranteed. During the backup operation, there is a high level of disk access. This may impact the performance of the node. If multiple backups are made at the same time, there is a risk that there is an impact on other nodes in the system. These solutions are not supported for System 800xA. Backup and Restore of an ESXi Server Backing up an ESXi server is a different concept to backing up a Windows computer. Virtual machines running in an ESXi server are backed up individually as described in Backup and restore of virtual machines. To install a replacement ESXi server as an assistance, the following configuration details should be noted: Virtual switch configuration. For example, switch names, parameters, and connected VMs. Storage adapter configuration. For example, iscsi addresses. NTP server details. DNS and routing configuration. User and Groups configuration. 3BSE

128 Backup and Restore of Virtual Machines Section 12 Backup and Restore Virtual machine configuration such as resource allocation and user security details. After the virtual machines have been moved to the new server, it is important to check that the network has been correctly set for each virtual machine. When powering on a virtual machine which has been moved to the new server, a question may be asked if it has been copied or moved. Answering moved will keep the unique identity (for example NIC MAC addresses) the same. Answering copied will change these unique identities. Backup and Restore of Virtual Machines Using snapshots is not a supported method for 800xA server nodes. There is a reduction in performance when running a virtual machine that has snapshots. If a snapshot has been made of a running virtual machine it can take longer to start the virtual machine. It is also possible to accidentally snapshot back to a previous state of the virtual machine which could interfere with production and potentially loose a great deal of data. Virtual machines running in an ESXi server are backed up as if they are physical computers BSE

129 Section 13 Security When an ESXi server is installed, the root user is setup as the administrative user on all objects in the ESXi server. Items such as virtual machines, that are added, have the root user added to their permissions by default. A problem with this is that, mistakes or abuse of the system is possible by those who have knowledge of the root password. To limit this, additional users who have limited roles in the ESXi server should be added. These limited roles would only allow the user to start the virtual machine, but not stop, delete, or modify it. The roles should be applied to those virtual machines which are relevant to the user. Roles Roles are a group of privileges that can be allocated to a user and applied as permissions to the objects in the ESXi server. These privileges are grouped into categories such as Datastore and Virtual Machine. By default, there are three roles in an ESXi server. These cannot be edited and new roles need to be created to define the required set of privileges. Creating a role with a limited set of permissions through the following actions: 1. Login to the ESXi server using the vsphere Client. 2. Select Home > Administration > Roles in the taskbar. 3. Click Add Role. 4. Provide a name for the new role. By default, the new role has no privileges. 5. Add the minimal level of privileges required. For example, expand the tree structure of: All Privileges >Virtual machine >Interaction 3BSE

130 Users Section 13 Security Select the item: Power On. 6. Click OK. Users Users are either those defined in the ESXi server, or via vcenter which uses Windows users. Once the users are defined, they are added to objects in the ESXi server and assigned permissions. When the user logs into the ESXi Server, only those virtual machines which have the user assigned to are visible. Users are added through the following procedure: 1. Login to the ESXi server using the vsphere Client. 2. Select the ESXi server icon in the tree structure. 3. Select the Users & Groups tab group. 4. Right-click in the blank area of the Users & Groups tab. 5. Select Add... from the context menu. 6. Provide a Login, User Name and Password. The login name should have alphanumeric characters with no spaces. The password should meet the minimum complexity which is a minimum of 6-8 characters which have upper case, lower case, numeric values and do not contain phrases in the login name. If the password does not meet the minimum complexity, an invalid format message is shown BSE

131 Section 13 Security Permissions Permissions Permissions are added to objects, such as, the virtual machines, to allow the users for limited interaction with the virtual machine. They can be allowed to propagate to child objects. Permissions are added to a virtual machine using the following procedure: 1. Login to the ESXi server using the vsphere Client. 2. Select the virtual machine which requires permission modification. 3. Click Permissions tab. 4. Right-click in the blank area of the Permissions tab. 5. Select Add Permission... from the context menu. 6. Select Add... to select a user. 7. Select the user name in the list. 8. Click Add and then OK. 9. In the Assign Permissions dialog box, select the permission from the Assigned Role drop-down box. If the ESXi server is being managed using vcenter Server, the users of the computer running vcenter Server, or the domain it is a member of, can be used to add permissions to the ESXi server objects. 3BSE

132 Permissions Section 13 Security 132 3BSE

133 Section 14 Optimization Virtual CPUs This topic describes the optimizations required in virtual CPUs. As an initial guideline, it is recommended to have two virtual CPUs in each virtual machine and to have at least one logical processor per virtual CPU. In some cases, where there are a number of processes in the virtual machine which consume high processor load, it may be necessary to add additional virtual CPUs to achieve target performance levels. To add an additional CPU to a virtual machine, it must first be shut down. The hardware configuration is then edited, and the number of CPUs are adjusted to the required count. If a virtual machine has been configured with one processor, Windows will use the uniprocessor kernel. When more than one processor is found, Windows will then install the multiprocessor kernel and associated drivers. After this installation, the virtual machine needs to be restarted. To provide CPU resources to the virtual machines, there should be one logical processor for each virtual CPU. For example, if the computer has 2 Quad Core CPUs (8 cores) and each virtual machine has 2 virtual CPUs, then there should be only 4 virtual machines running on the ESXi server (8 virtual CPUs). With hyper threading enabled, there are twice as many logical cores, hence, there can be twice as many virtual CPUs (16 virtual CPUs). Whilst hyper-threaded cores have a small performance reduction caused by sharing core resources, they provide more execution locations for virtual CPUs, which minimizes delays in virtual machine execution. 3BSE

134 Allocate Virtual CPUs to Physical Cores Section 14 Optimization Allocate Virtual CPUs to Physical Cores It is highly recommended to allow the ESXi Server to allocate resources. However, if it is required to allocate virtual machine to physical cores, the identifier convention in the ESXi server must be understood. In VMware ESXi server, the first hyper-thread of the first core is logical processor 0, the next hyper-thread of the first core is logical processor 1. This is summarized in Figure 106. Figure 106. Core Numbering in ESXi 134 3BSE

135 Section 15 Monitor ESXi servers using SNMP Traps This topic gives an idea of setting up the SNMP trap monitoring in ABB 800xA system. For 800xA 5.1 FP4 system, to receive generic traps, for example, test traps, the following temporary correction is required: 800xA PC, Network and Software Monitoring 5.1 RU1 TC1( ). Prior to 800xA 5.1 Rev. D, an issue with the System Installer may cause tlb files, not to be registered. Run the following command on the AO Server, which creates a new file called PnsmEngineHandler.tlb: C:\Program Files (x86)\abb Industrial IT\Optimize IT\PC, Network and Software Monitoring\bin\RegisterTLB.bat The AO Server node must be restarted for the change to take effect. 800xA Components Required The following is required in an ABB 800xA system for SNMP Trap support: 3BSE

136 Set ESXi Host Name Section 15 Monitor ESXi servers using SNMP Traps ABB 800xA PNSM Server for the trap receiving service. ABB 800xA AO server for alarm/event management. ABB System 800xA PNSM Library Base. ABB System 800xA PNSM Library object types of devices which have SNMP Trap aspects (A generic object is described later in this section). The ABB 800xA PNSM and ABB 800xA AO servers can be combined in the same node. Set ESXi Host Name SNMP Traps For SNMP Traps to be sent, the ESXi server must not have its DNS name set to the default localhost. This will result in trap messages being sent from the server with a non-useful name or to the 800xA system. This prevents the 800xA Trap Service from identifying the source of the trap messages. Use the ESXi Server interface to set the ESXi server name to a unique value. See Section 5, ESXi Configuration Console, Step 8 through Step 12. SNMP Traps are notification messages from a device to a trap receiver. Usually, the trap receiver is part of a network management system. From 800xA 5.1 Rev B, ABB has a trap receiver functionality that enables hardware to send notifications to the 800xA alarm and events. For example, to produce System Alarms or System Events depending on how the receiver is setup. Communication Topology SNMP Traps are received by a ABB Trap service running in an ABB 800xA PNSM Server. Events are forwarded by the ABB 800xA PNSM Server to an ABB 800xA AO Server. Traps messages are received on port UDP 162, hence, this port needs to be open in the ABB 800xA PNSM Server. To determine if a device has network connectivity, 136 3BSE

137 Section 15 Monitor ESXi servers using SNMP Traps Communication Topology the ABB 800xA PNSM server periodically performs SNMP reads of the device. These requests are sent to port UDP 161 of the device, see Figure 107. Figure 107. PNSM SNMP Requests for Status and Reception of SNMP Traps VMware ESXi servers send SNMP Traps through the management network. For monitoring of an ESXi server, the ABB 800xA PNSM Server must have network connectivity with the management network in the ESXi Server. The target IP address for the SNMP Traps in the ESXi server must be set to the ABB 800xA PNSM Server, see Figure 108.The target IP address is the IP address of the 3BSE

138 Server Hardware Section 15 Monitor ESXi servers using SNMP Traps ABB 800xA PNSM server network adapter that is connected to the same switch as the ESXi Management Network. Server Hardware Figure 108. Communication of SNMP Trap Messages Prior to setting up SNMP Traps, hardware vendor offline bundles need to be installed in the ESXi Server. The vendors may supply custom ESXi images which 138 3BSE

139 Section 15 Monitor ESXi servers using SNMP Traps Server Hardware have the required SNMP support installed. The following table shows typical requirements for SNMP support from various vendors: Table 13. Vendor Support for SNMP in ESXi Vendor HP DELL IBM SNMP Support Additional Offline Bundle Dell OpenManage Offline Bundle IBM Offline Bundle 3BSE

140 Server Hardware Section 15 Monitor ESXi servers using SNMP Traps The Figure 109 is an example of the HP DL380p G8 health status without offline bundle. There is no status of the storage shown. This requires the offline bundle to be installed. Figure 109. Server Health Status without Offline Bundle Perform the following to download the custom bundle: Before installing the offline bundle, it is highly recommended to shut down the virtual machines and put the ESXi server into maintenance mode. 1. Download HP Agentless Management Service Offline Bundle for VMware vsphere 5.5 from HP website BSE

141 Section 15 Monitor ESXi servers using SNMP Traps Server Hardware 2. Open vsphere Client. 3. Select a datastore and Upload the file. Figure 110. Vendor Offline Bundle Uploaded to the ESXi Server 4. Start the VMware vsphere CLI with the Command Prompt option. Figure 111. Starting the VMware CLI to Initiate Installation of the Offline Bundle 5. Enter the following in command prompt and execute it to install the bundle: 3BSE

142 ESXi Configuration Section 15 Monitor ESXi servers using SNMP Traps C:\Program Files (x86)\vmware\vmware vsphere CLI>esxcli -s u root -p MyPassword software vib install -d /vmfs/volumes/datastore1/hp-ams-esxi5.5-bundle zip The esxcli command must be entered as a single line. The storage status is present in the vsphere Client Health Status. Figure 112. Server Health Status Showing Hard Disk Health Status ESXi Configuration By default, ESXi servers do not have SNMP enabled, or SNMP Trap destinations configured. This is required to be performed before configuring the 800xA system 142 3BSE

143 Section 15 Monitor ESXi servers using SNMP Traps SNMP Receiving Port to receive the traps. Configuration of SNMP Traps in the ESXi server requires the usage of the VMware vsphere CLI interface. A separate network is used for the ESXi management network which keeps the 800xA network separate from the ESXi management network. In the example, the ESXi management network is in to range. SNMP Receiving Port The receiving port for SNMP should be set to UDP 161 by using the following command: C:\Program Files (x86)\vmware\vmware vsphere CLI>vicfgsnmp.pl --server username root --password rootpass -p 161 Changing udp port to Complete. Trap Target Perform the following to prepare the Trap Target: 1. Trap destination (ABB 800xA PNSM Server) is set in the ESXi 5.5 server using the flowing format: IPADDRESS@PORT/COMMUNITY 2. In the following example, the trap destination (ABB 800xA PNSM Server) is set to : C:\Program Files (x86)\vmware\vmware vsphere CLI>vicfgsnmp.pl --server username root --password rootpass -t @162/public Changing notification(trap) targets list to: @162/public... Complete. 3. SNMP is enabled through the following command: C:\Program Files (x86)\vmware\vmware vsphere CLI>vicfgsnmp.pl --server username root --password rootpass --enable 3BSE

144 Check Configuration Section 15 Monitor ESXi servers using SNMP Traps Enabling agent... Complete. Check Configuration SNMP configuration of an ESXi server can be checked using the show command as shown in the following example: C:\Program Files (x86)\vmware\vmware vsphere CLI>vicfgsnmp.pl --server username root --password rootpass -show Current SNMP agent settings: Enabled: 1 UDP port: 161 Communities: public Notification targets: @162/public Options: EnvEventSource=indications engineid= a loglevel=info 144 3BSE

145 Section 15 Monitor ESXi servers using SNMP Traps Firewall Firewall In the ABB 800xA PNSM Server where the SNMP Traps are to be received, the fire wall must be open for port UDP 162, see Figure 113. Figure 113. Firewall port UDP 162 Open for Receiving SNMP Traps Test Basic Communication To confirm that the network configuration is correct, some simple tests can be performed. These are performed on the ABB 800xA PNSM server where the SNMP traps will be received. The first test is a basic ping of the ESXi server: 1. Open the Command Prompt. 3BSE

146 Test Trap Communication Section 15 Monitor ESXi servers using SNMP Traps 2. Ping the ESXi server. Figure 114. Basic Ping from the AO Server to the ESXi Server If there is no response, the basic network configuration must be checked. Test Trap Communication Next step is to check that the ESXi server is able to send traps to the ABB 800xA PNSM server where the SNMP traps will be received. This can be performed with the assistance of a simple PowerShell script BSE

147 Section 15 Monitor ESXi servers using SNMP Traps Test Trap Communication 1. On the ABB 800xA PNSM server, ensure that the ABB PNSM Trap Receiver server is stopped (Figure 115). Figure 115. Stopped ABB PNSM Trap Receiver to Test Traps 2. In the ABB 800xA PNSM Server, create a text file locating it on the desktop and name it traptest.ps1. Windows hides known file types. Hence, the file may look like it is traptest.ps1, but is really traptest1.ps1.txt. Figure 116. Hiding File Type Information 3. In the Windows Explorer, select Tools and click Folder Options. 4. Choose View menu, and clear Hide extensions for known file types under Files and Folders (Figure 117). 3BSE

148 Test Trap Communication Section 15 Monitor ESXi servers using SNMP Traps 5. Click Apply and close the Windows Explorer. Figure 117. Unhiding Known File Types in Windows After changes have been applied, the file is shown with a.txt extension. Figure 118. File Name with Extension 6. Rename the file to traptest.ps1, the icon changes, see Figure Right-click on traptest.ps1 and select Open to edit the file BSE

149 Section 15 Monitor ESXi servers using SNMP Traps Test Trap Communication Figure 119. File with Correct Extension to be a PowerShell Script 8. Copy and paste the following text into it: $UdpPort=162 $endpoint = New-Object System.Net.IPEndPoint ([IPAddress]::Any, $UdpPort) $udpclient= New-Object System.Net.Sockets.UdpClient $UdpPort while($content=$udpclient.receive([ref]$endpoint)) {Write- Host -nonewline $content} 9. Save and close the notepad editor. 3BSE

150 Test Trap Communication Section 15 Monitor ESXi servers using SNMP Traps 10. Select Start > All Programs > Accessories > Windows PowerShell > Windows PowerShell. Figure 120. Starting Windows PowerShell 150 3BSE

151 Section 15 Monitor ESXi servers using SNMP Traps Test Trap Communication 11. Drag the traptest.ps1 to the PowerShell, and press Enter twice. Figure 121. Running the PowerShell Trap Test Script 12. If a request to execute the script is shown, type Y to allow execution of the script. If the PowerShell is seen as shown in Figure 122, there is a service that is already using port UDP 162. The trap receiver services must be stopped. Figure 122. Indication that a Program/Service is already Listening on Port UDP 162 3BSE

152 SNMP Traps Service Provider Section 15 Monitor ESXi servers using SNMP Traps 13. Use the following VMware vsphere CLI command to request the ESXi server to send a test trap: C:\Program Files (x86)\vmware\vmware vsphere CLI>vicfgsnmp.pl --server username root --password rootpass --test Sending test nofication(trap) to all configured targets... Complete. Check with each target to see if trap was received. The following shows the trap message as received in the ABB 800xA PNSM server: Figure 123. Trap Message in PNSM Server The IP address of the ESXi server is in the numeric values. 14. Close the Windows PowerShell dialog box to ensure that the port UDP 162 is free. SNMP Traps Service Provider SNMP Traps utilizes an ABB 800xA Trap Service that needs to be defined in the Service Structure. To set up Trap Service Provider: 1. Open Engineering Workplace BSE

153 Section 15 Monitor ESXi servers using SNMP Traps SNMP Traps Service Provider 2. Provide a service group and service provider objects, and set the node where the ABB 800xA Trap Service should be run. i Figure 124. Service Group and Service Provider Configuration for SNMP Traps 3BSE

154 Add PNSM Objects Section 15 Monitor ESXi servers using SNMP Traps 3. Confirm that the ABB PSNM Trap Receiver service is started in the ABB 800xA PNSM Server. Figure 125. ABB PNSM Trap Receiver Running in the ABB 800xA PNSM Server Add PNSM Objects The System 800xA PNSM Library Base is required as a starting point for PNSM. Additional object types specific to supported devices are added when available. The ESXi version of the object types has an IT Trap aspect added, which contains the required MIB files. These are installed in the ABB 800xA PNSM server node. If an object does not exist, it may be possible (dependent on available MIB files) to create the asset specific object. Refer SNMP Traps - Generic for more information. Figure 126. Object Types Required for PNSM 154 3BSE

155 Section 15 Monitor ESXi servers using SNMP Traps Create a Device Object Installation of the object types consists of executing the file which expands its contents. For the Base, the Import folder contains a Base afw file and VB6/PG2 afw files, which are imported dependent on usage of VB6/PG2. After importing the afw files, the install.bat file should be executed. Figure 127. Structure of PNSM Library Base Folder Structure The device specific PNSM requires the importing of the afw file: Create a Device Object Figure 128. PNSM Device Library Folder Structure Perform the following to create a device object: 3BSE

156 Create a Device Object Section 15 Monitor ESXi servers using SNMP Traps 1. In the Engineering Workplace, under Control Structure, right-click on a IT Server object and select New Object. Figure 129. Object Creation 156 3BSE

157 Section 15 Monitor ESXi servers using SNMP Traps Create a Device Object 2. Select the PNSM object type, enter a Name and click Create. Figure 130. Creating PNSM Object Type 3BSE

158 Create a Device Object Section 15 Monitor ESXi servers using SNMP Traps 3. For the IT Faceplate PG2 aspect, select the Configuration tab, and enter IP address and SNMP Community password. Figure 131. Providing IP Address and SNMP Password 158 3BSE

159 Section 15 Monitor ESXi servers using SNMP Traps Create a Device Object 4. Select the IT Device Manager aspect and click Populate. Figure 132. Populating the PNSM Object 3BSE

160 Create a Device Object Section 15 Monitor ESXi servers using SNMP Traps 5. Select the IT Device Traps aspect and click Deploy. Figure 133. Deploying the IT Traps to the AO Server 6. Select the IT Faceplate PG2 aspect to confirm that the device is online BSE

161 Section 15 Monitor ESXi servers using SNMP Traps Create a Device Object The Information tab shows the Name and Description of the device populated. Figure 134. Device Properties 3BSE

162 Create a Device Object Section 15 Monitor ESXi servers using SNMP Traps The Conditions tab shows that the device is connected. It is indicated by a green color box. Figure 135. Device Connection Status Traps Testing Final confirmation of the device configuration is to test that the traps are working correctly. Perform the following to test the traps are working: 1. Start VMware vsphere CLI command prompt. 2. Execute the command to request the ESXi server to send a test trap: C:\Program Files (x86)\vmware\vmware vsphere CLI>vicfgsnmp.pl --server username root --password rootpass --test Sending test nofication(trap) to all configured targets... Complete. Check with each target to see if trap was received BSE

163 Section 15 Monitor ESXi servers using SNMP Traps Create a Device Object 3. Click the Event List in the device object to view the resultant events. Figure 136. Successful Reception of an SNMP Trap in 800xA System 4. Final test is to perform a non-destructive hardware failure such as removing one of the redundant power supplies. Figure 137. Testing SNMP Trap Generation of Physical Hardware Failure 3BSE

164 Filter Trap Messages Section 15 Monitor ESXi servers using SNMP Traps Filter Trap Messages Not all trap messages are notifications of failures. One example is uptime notifications as shown in Figure 138, where vmwcimomheartbeat events occur. Figure 138. Cyclic vmwcimomheartbeat Events which may Require Filtering By default, all the trap types are enabled in the IT Device Traps aspect. These can be deselected to remove traps that are not critical to the servers operations. To ignore the event, perform the following: 164 3BSE

165 Section 15 Monitor ESXi servers using SNMP Traps Filter Trap Messages 1. Open the IT Device Traps aspect and clear the vmwcimomheartbeat notification trap and click Apply. Figure 139. Deselecting Trap Messages 3BSE

166 SNMP Traps - Generic Section 15 Monitor ESXi servers using SNMP Traps 2. Click Deploy on the Main View as the changes need to be deployed here. Figure 140. Deploying Changes made to the IT Device Traps Configuration SNMP Traps - Generic Before providing any system/statement on monitoring hardware not supported by ABB, the project must test and verify each hardware item. There might be situations where an asset might not have an ABB PNSM Object type, but there is still a need to monitor the asset using SNMP Traps. This can be done using the PNSM Workstation object type as a base. Success of this depends on the asset being monitored and the availability of MIB files. The asset must be able to handle basic property requests. This will enable PNSM to determine if the asset is connected and the name of the asset BSE

167 Section 15 Monitor ESXi servers using SNMP Traps 800xA Components Required MIB files are required by the IT Traps aspect to enable the interpretation of traps received. These can be difficult to come by and to apply due to the MIB file dependencies. The walkthrough is based on making a object to monitor a DL380G8server. 800xA Components Required The following is required to implement a trap receiving object: ABB 800xA PNSM Server for the trap receiving service. ABB 800xA AO server for alarm/event management. ABB System 800xA PNSM Library Base. ABB System 800xA PNSM Library Asset for Workstation. Asset MIB Files (downloaded from vendor's website). Before attempting any setup, ensure that the MIB files are available from the vendor. Figure 141 shows the files required to implement an object. Figure 141. Files Required to Create a Generic Object which can be Configured to Receive SNMP Traps from an Asset Based on MIB files udp940mib.zip file is downloaded from the vendors website. 3BSE

168 Install PNSM Base Object Section 15 Monitor ESXi servers using SNMP Traps Install PNSM Base Object To install a PNSM Object refer Add PNSM Objects. Install PNSM Workstation Object Perform the following: 1. Download the System 800xA PNSM Library Asset for Workstation program, and execute it. Figure 142. PNSM Workstation Device Library Folder Structure 2. Import the 10_Workstation afw file and VB6 or PG2 afw file based on system requirements. Download Asset MIB Files The MIB files can be downloaded from different websites. As an example in this sub-section it is downloaded from HP website. To download, perform the following: 168 3BSE

169 Section 15 Monitor ESXi servers using SNMP Traps Download Asset MIB Files 1. Search for MIB file in HP website, and click Download. Figure 143. Download MIB File 3BSE

170 Download Asset MIB Files Section 15 Monitor ESXi servers using SNMP Traps 2. Choose the two MIB files that are required for SMNP set, that is, snmpv2- mib.mib and SNMPv2-SMI.mib. Figure 144. Adding MIB Files to the IT Device Traps Aspect 3. Select the VMware files as shown in Figure 145. Figure 145. Vendor Specific MIB Files 170 3BSE

171 Section 15 Monitor ESXi servers using SNMP Traps Create an Object Type The resultant set of files is as shown in Figure 146. Create an Object Type Figure 146. MIB Files Extracted To provide efficient engineering, it is recommended to create a new object type for the device to be monitored. This is performed by making a copy of the Workstation object type and customizing it to suit the device to be monitored. Perform the following to create an object type: 1. Open Engineering Workplace. 3BSE

172 Create an Object Type Section 15 Monitor ESXi servers using SNMP Traps 2. Under Object Type Structure, right-click on the Workstation-v1_3 Object Type and select Copy. Figure 147. Copying the Workstation Object Type 3. Select or create a parent object type group BSE

173 Section 15 Monitor ESXi servers using SNMP Traps Create an Object Type 4. Right-click on the object, and select Paste special > Paste Object. Figure 148. Creating a New Object Type 3BSE

174 Create an Object Type Section 15 Monitor ESXi servers using SNMP Traps 5. The new object type needs to be customized before being used. For example, the name and icon should be modified. Figure 149. Initial Copy of the New Object Type Following is a list of modifications that are required before the object type is used: Change name of the object type. Change icon of object type. Add IT Device Traps aspect. Add MIB files Add Event list. Set Aspect Control to Copy to all instances for the IT Device Traps aspect. 6. Select the new object type, then the Name aspect (Figure 150) BSE

175 Section 15 Monitor ESXi servers using SNMP Traps Create an Object Type 7. Change the Name relevant to the device to be monitored and click Apply. Figure 150. Modifying the Object Type Name 3BSE

176 Create an Object Type Section 15 Monitor ESXi servers using SNMP Traps 8. To change the object icon, delete the existing icon and copy a new icon and click Apply. Figure 151. Updating the Icon In the Figure 151, the icon from the HP_ProLiant_DL380_G6_ESXi_v3_0 is copied to the new object type 176 3BSE

177 Section 15 Monitor ESXi servers using SNMP Traps Create an Object Type 9. Right-click on the object type and select New Aspect..., to add an IT Device Trap aspect. Figure 152. Adding New Aspect 10. In the New Aspect dialog, under Common tab, select IT Device Traps > IT Device Traps > IT Device Traps aspect (Figure 153). 3BSE

178 Create an Object Type Section 15 Monitor ESXi servers using SNMP Traps 11. Let the Name be the default name, and click Create. Figure 153. New IT Device Trap Aspect MIB files have dependencies. When changes are made to the to the MIB files, All the required files must be added again, even if they have been added previously. All the dependencies must be met. You must always have the SNMPV2-mib.mib file and the SNMPv2-SMI.mib files. These provide base definitions that are used by vendor MIB files BSE

179 Section 15 Monitor ESXi servers using SNMP Traps Create an Object Type 12. Add the MIB files to the IT Device Traps aspect, and click Load. Figure 154. Adding MIB Files to the IT Device Traps Aspect If successful, the log will show MIB parsing success. If unsuccessful, the MIB files will be missing. 13. Click Apply after successfully adding the MIB files. 3BSE

180 Create an Object Type Section 15 Monitor ESXi servers using SNMP Traps 14. Add an Event list aspect to the new object type, and in the Configuration view, set the Configuration to IT Asset Event Configuration. Figure 155. Adding an Event List Aspect to the New Object Type 15. Click Open in the Configuration view(figure 155) BSE

181 Section 15 Monitor ESXi servers using SNMP Traps Create an Object Type 16. Select Filter menu, ensure that IT Asset alarms option is selected and click Apply. Figure 156. Enabling IT Asset Alarms 3BSE

182 Create an Object Type Section 15 Monitor ESXi servers using SNMP Traps 17. Select Columns menu and change the column order as shown in Figure 157, so that the Message tab comes after the ObjectName tab. Figure 157. Changing the Event List Column Order By default, the IT Device Traps aspect will not be inherited. This behavior is changed in the Type Definition aspect BSE

183 Section 15 Monitor ESXi servers using SNMP Traps Create an Object Type 18. Select the Type Definition aspect, then the Aspect Control tab. Figure 158. Changing the IT Device Traps Aspect Behavior 19. Select the IT Device Traps aspect, and select Copy to all instances, and then click Apply (Figure 158) to make the changes. 3BSE

184 Create an Asset Object Section 15 Monitor ESXi servers using SNMP Traps Create an Asset Object 1. In the Control Structure, right-click on IT Server, IT OPC Server Network object and select New Object... Figure 159. New Object Creation 184 3BSE

185 Section 15 Monitor ESXi servers using SNMP Traps Create an Asset Object 2. Select New Object type and type a name, and then click Create. Figure 160. Creation of a Workstation Object 3BSE

186 Create an Asset Object Section 15 Monitor ESXi servers using SNMP Traps 3. In the IT Faceplate PG2 aspect, select the Configuration tab, and set the IP Address and the SNMP Community password of the device to be monitored. Figure 161. Setting the IP Address and SNMP Password 186 3BSE

187 Section 15 Monitor ESXi servers using SNMP Traps Create an Asset Object 4. Select the IT Device Traps aspect and click Deploy, so that each new instance will have the MIB files deployed to the AO Server. Figure 162. Deploying Trap Configuration 3BSE

188 Create an Asset Object Section 15 Monitor ESXi servers using SNMP Traps 5. Select IT Device Manager and click Populate. Figure 163. Populating the Device 6. Send a test trap message from the device. In the following example, the ESXi server was requested to send a test trap: C:\Program Files (x86)\vmware\vmware vsphere CLI>vicfgsnmp.pl --server username root --password rootpass --test Sending test nofication(trap) to all configured targets... Complete. Check with each target to see if trap was received BSE

189 Section 15 Monitor ESXi servers using SNMP Traps Create an Asset Object The following is the resultant event message: Figure 164. Trap Message Received from ESXi Server The test trap only checks the base trap functionality. To test that the MIB files are correct, a non-destructive test such as removing a redundant power supply is performed. The following is an example of the expected event message: Figure 165. Event Message of a Power Supply Failure in an ESXi Server 3BSE

190 Create an Asset Object Section 15 Monitor ESXi servers using SNMP Traps 190 3BSE

191 Section 16 VMware Command Line Interface, Drivers, Updates and Upgrades When setting up an ESX server, additional drivers may be required. Typically, this would be for network adapters. Additional updates may also be required. These tasks can be achieved from the Windows computer which is used as a client to the ESX server. Due to the difficulty in identifying hardware which ESXi does not automatically detect, it is mandatory to have an inventory of the computers configuration. This should contain details on, for example, additional NIC cards including the chipset. This information is used to identify the correct driver in the VMware website. VMware has produced a software utility, which provides the functionality to have patches saved on the Windows client be applied to the ESX server through simple commands. To make it easier to identify the updated files when typing the commands, they should be placed in a simple folder, for example, C:\ESXUpdates. Most vendors provide ISO images which are used to create bootable CDs/DVDs to enable the firmware to be upgraded. The relevant offline bundle is extracted for the installation in the ESXi server. VMware Command Line Interface The VMware Command Line Interface (CLI) provides the ability to perform maintenance operations, such as, applying patches, drivers and updates from a Windows client. This removes the necessity to understand the Unix based command required when logging in through the text based terminal console for these tasks. It also provides the ability to apply updates to ESXi when there is no text based console to log on to. Installation of the VMware Command Line Interface (CLI) is described in Install VMware vsphere CLI. 3BSE

192 Prepare the ESXi Server Section 16 VMware Command Line Interface, Drivers, Updates and Prepare the ESXi Server Installing drivers or ESXi updates typically will require the ESXi server to be restarted. Hence, it is highly recommended to shut down the virtual machines and put the ESXi server into maintenance mode prior to upgrading the ESXi server. Drivers Perform the following to download the drivers: 1. Go to and download VMware ESX/ESXi 4.x Driver CD for Intel and Gigabit Ethernet Controller. 2. Save the ISO file to disk. 3. Open the ISO file using Winzip and extract the offline bundle. Figure 166. Extracting the offline bundle from an ESXi driver installation media 4. ESXi patches and updates can be downloaded from VMware website BSE

193 Section 16 VMware Command Line Interface, Drivers, Updates and Upgrades Apply 5. Click the drop-down and select a suitable version of ESXi. Figure 167. Specifying ESXi Version 6. Select the update and click Download. Figure 168. Downloading Updates Apply Patches/Updates Perform the following to apply the patches/updates: 1. Upload the patch/update to a datastore in the ESXi server so that it is used for the patch/updates installation commands. 3BSE

194 Apply Patches/Updates Section 16 VMware Command Line Interface, Drivers, Updates and As seen in the Figure 169, a folder named Updates is created in the datastore to assist in keeping the datastore organized. Figure 169. Update Uploaded to Datastore in the ESXi Server 2. Install the update typing the following in the command prompt: C:\Program Files (x86)\vmware\vmware vsphere CLI>esxcli -s u root -p MyPassword software vib update -d /vmfs/volumes/datastore1/updates/update-from-esxi _update01.zip 3. After completion of the installation of the update, the ESXi server will need to restart. Since this means also shutting down the virtual machines first, care must be taken to ensure there is no interference to production BSE

195 Section 17 Storage Area Network Virtual machines can either be stored on hard disks, local in the ESXi server, or on a network storage device such as a Storage Area Network (SAN) server. In both cases, the storage is added to the ESXi server and seen as datastores. One advantage of having the virtual machines stored on a SAN server is that, the virtual machines can be moved from one ESXi server to another. Since both ESXi servers have access to the same shared storage, it is only the execution context that needs to be moved rather than the much larger virtual machine hard disk. This topic provides an overview of SAN and how it is used with VMware vsphere. While SAN servers support either Fiber Channel, iscsi, or both, this topic provides information only on iscsi. Most major server vendors also supply SAN servers. The specific configuration of each SAN type is not documented. Refer to the vendors specific guides for individual device configuration details. It is essential that the configuration used is confirmed to function as expected before used for production. This includes using the performance monitors in the ESXi servers to ensure that load balancing is operational, and to ensure that network redundancy is working by temporarily disconnecting the network connections. 3BSE

196 Hardware Section 17 Storage Area Network Hardware SAN servers are highly redundant disk arrays that provide storage over a network. Each SAN has redundant power supplies, redundant controller modules, redundant network connections per controller module and redundant disks. Figure 170. Rear View of an SAN Device The following parts as numbered in the Figure 170 are: 1. Power Supplies. 2. Fiber Channel host ports. 3. iscsi ports (host or replication). 4. CLI port (mini-usb). 5. Reserved for future use. 6. Management Ethernet port. 7. SAS expansion port BSE

197 Section 17 Storage Area Network Configuration Configuration SAN servers typically consists of 12 to 24 hard disks. These are usually configured as one or more RAID 5 (block-level striping with distributed parity) disk groups with a number of disks set as hot spares or hot standby, used as a failover mechanism to provide reliability in System Configurations. The SAN server vendor may have best practices for allocation of the storage. When combining disks to create RAID arrays, a storage item is created which is used to create virtual disks. These are the entities that become available as datastores in ESXi. With introduction of ESXi 5.5, the restriction on the datastore size ranging from 2TB to 512 Bytes is no more valid. When formatting datastores, the file block size needs to be changed to accommodate the largest file that is created in the datastore. Figure 171. SAN Server Storage Configuration 3BSE

198 Networking Section 17 Storage Area Network Networking Figure 172 and Figure 173 provides an overview of the networking required for a SAN server implantation. Figure 172. SAN Storage iscsi Network Figure 173. SAN Storage Management Network 198 3BSE

199 Section 17 Storage Area Network Networking One or more ESXi servers are connected to multiple network switches through multiple network connections. The SAN server is also connected to the switches through multiple network connections. There are two types of network traffics in this configuration. One is to access the shared storage through iscsi and the other is for configuration and management of the servers. To provide security and reliability, the two types of traffics are separated using a VLAN. To reduce single point of failures, separate network adapters should be used in the ESXi servers for the storage network as shown in the Figure 174. Figure 174. Multiple Network Adapter Connection for Higher Reliability The multiple network connections provide redundancy, it also provides load balancing. Figure 175 is an example of IP addresses for the storage network. When configuring the IP addresses in the SAN server, the VLAN settings are in the advanced settings, as shown in the Figure 176. In the Figure 177, it is observed that the SAN server does not support VLAN. In this case, separate network switches for iscsi and management traffic should be used. 3BSE

200 Networking Section 17 Storage Area Network Figure 175. IP Address for the Storage Network 200 3BSE

201 Section 17 Storage Area Network Networking Figure 176. Setting the VLAN ID for the iscsi Ports 3BSE

202 Networking Section 17 Storage Area Network Figure 177. SAN Server iscsi IP Configuration 202 3BSE

203 Section 17 Storage Area Network ESX Server Network Configuration ESX Server Network Configuration Implementation of networking for the virtual solution involves separating functionality and implementing redundancy. There should be separate networks for the management and SAN iscsi storage networks. This prevents management network traffic from interfering with the storage network traffic. Figure 178 shows the network configuration. Hardware vendor guidelines may recommend different configurations, but the configuration should always verify that network utilization is optimized. Figure 178. ESXi Server Network Configuration Add Datastores This description is based on using the iscsi Software Adapter in the ESX Server. Perform the following steps in the ESXi Server: 1. Select Configuration > Storage Adapters Configuration. 3BSE

204 Performance Section 17 Storage Area Network I 2. Select iscsi Software Adapter. 3. On the Details dialog box, select Properties. 4. In the iscsi Initiator Properties dialog box, select Configure. 5. Select Enabled and click OK. 6. On the Dynamic Discovery tab, for each SAN IP Addresses, perform the following steps. a. Select Add. b. Enter the IP Address of the iscsi Server and click OK. c. Click Close once all the IP Addresses are added. d. Click Yes to re-scan, on the message stating the rescan of the Host Bus Adapter. If the iscsi storage server is not used before, the datastores must be added and formatted. The procedure is same as adding a local storage disk. When formatting the datastores it is important to consider the largest file size and set an appropriate block size. If the iscsi storage server is already been used in an ESX server, then the datastores appears in the storage configuration without the intervention of the user. Load Balancing Perform the following steps to manage the paths: 1. Select the iscsi Software Adapter. 2. Select a device from the list in the Details dialog box (see Figure 179). 3. Right-click on the selected device and select Manage Paths. 4. On the Manage dialog box, select Round Robin from the Path Selection dropdown list (see Figure 180). Performance It is essential that the configuration provides the best performance before using in a production environment. This involves monitoring the throughput to the datastores and ensuring that all networks are being used BSE

205 Section 17 Storage Area Network Performance Figure 179. iscsi Device Details Figure 180. iscsi Path Selection Policy 3BSE

206 Performance Section 17 Storage Area Network In the network performance chart, 4 network adapters that are supporting the SAN server 1,3,4,7 (Figure 181) are all providing throughput to the SAN sever. Figure 181. Load balancing of SAN Servers If the configuration is not correctly setup, not all the network connections are used for communication with the SAN server. It displays the network traffic. It is important to check the average/maximum values as there may be instances where the network traffic is sampled when there is no traffic. The following are the areas that must be checked: Are the network switches in the ESXi server configured correctly. Are the VLANs setup correctly in the ESXi server and the SAN server. Can the SAN server be pinged on each of the IP addresses from the ESXi server. The vmkping is used from the console of the ESXi server. Is it iscsi Path Selection Policy set to Round Robin BSE

207 Section 17 Storage Area Network Performance Does the SAN server has a preferred path that needs to be reset or reconfigured. Contact the SAN supplier for additional information regarding configuration. 3BSE

208 Performance Section 17 Storage Area Network 208 3BSE

209 Section 18 VMware vcenter VMware vcenter provides a centralized platform for administrating ESXi servers. Administrative access to the ESXi servers is provided without having to separately log on to the ESXi servers. User access rights in the ESXi servers can be applied from a common user access definition. VMware vcenter is installed in to a computer which does not run ABB 800xA software. VMware vcenter is required for accessing functionalities such as VMware High Availability, vmotion, Storage vmotion, and Fault Tolerance. Only VMware High Availability is approved for ABB 800xA production systems. These functionalities typically require common storage of the virtual machines, such as, an external SAN storage. There are two methods to login to the vcenter Server; through the vsphere Web Client, or the vsphere Desktop client. The web client is only available with vcenter as it provides the services for the web client access. With the introduction of ESXi 5.5, some functions are only available through the vsphere Web Client that comes with vcenter. The main differences are outlined in the Web Client and Desktop Client. When ESXi servers in a computing environment increases, the time and effort to maintain the individual servers also increases. Changes in user access are to be applied to each ESXi server individually. To make this easier, VMware developed the vcenter Server. Additional capabilities are also unlocked once the ESXi servers have been added to the vcenter server. These include VMware High Availability, vmotion, Storage vmotion, and Fault Tolerance. Inclusion of the capability depends on the license applied to the ESXi server. 3BSE

210 Section 18 VMware vcenter To facilitate these advanced functionalities, the virtual machines need to be stored on a Storage Area Network (SAN) device. This is a highly redundant disk storage unit which uses either Fiber Channel, or Internet SCSI (iscsi) to communicate to the ESXi servers. The folder that represents the virtual machine (including the virtual hard disk, and configuration files) is stored in the SAN device. Typically, the SAN device has RAIDed storage, redundant controller cards, redundant power supplies, and multiple network connections per controller card (Figure 182). Figure 182. Enterprise Virtualization Components vcenter Server is not included in VMware vsphere ESXi and must be purchased separately to implement VMware vsphere advanced functionality, such as, VMware High Availability. It is important to remember that the free version of ESXi cannot be added to a vcenter Server. License must be purchased and used. vcenter Server is a product that runs on Windows Server (64-bit) and should be installed on to a computer dedicated to run the vcenter Server. Hence, vcenter Server should not be installed on a domain controller, or 800xA Server or 800xA Client Node. vcenter Server stores information into Microsoft SQL Server database. For installations of small number of ESXi servers, it is better to install 210 3BSE

211 Section 18 VMware vcenter Licensing Licensing Single Sign-On Microsoft SQL Server Express that is included in the vcenter Server installation media. While it is not a pre-requisite to have external storage for vcenter Server, advanced functionality such as VMware High Availability requires virtual machines stored in an external storage unit accessible over the network. The same VMware vsphere Client that is used to interact with the individual ESXi servers is also used to interact with the vcenter Server. When logging into the vcenter Server, Windows accounts for the computer where vcenter Server is running. Each vcenter server requires a vcenter license. Each ESXi server to be added requires a full license. that is. the free ESXi license cannot be used. vcenter Single Sign-On (SSO) is introduced in vcenter Server 5.1. It is a mandatory component of the VMware Cloud Suite which deals with identity management for administrators and applications that interact with the vsphere platform. This is separate from Microsoft Active Directory (AD). Hence, it can be installed in a computer that is a member of a workgroup. SSO can have multiple AD domains added to it. When logging into the vcenter Server to perform administrative tasks, the SSO administrative account is used which is defined during the installation. This includes access from either the VMware Web Client, or the VMware Desktop Client. It is recommended to use the simple install option which includes vcenter Single Sign-On. 3BSE

212 Web Client and Desktop Client Section 18 VMware vcenter Web Client and Desktop Client The VMware Web Client is only available when vcenter is installed. There are some functionalities that are only available through the web client and some that are only available through the desktop client. ABB 800xA system installations do not require the advanced functionality that is restricted to the web client. It is important to understand that the VMware Desktop Client will not allow the editing of virtual machines which have version 1.0 or higher. Hence, do not upgrade the virtual machines to version 1.0 or higher unless the ESXi server is connected to a vcenter server. Figure 183. vsphere Client Showing VM Version BSE

213 Section 19 VMware vcenter Installation This topic describes how to install VMware vcenter 5.5. vcenter 5.5 Supported Operating Systems The following operating systems are supported from VMware (only 64-bit versions): Windows Server 2008 Service Pack 2. Windows Server 2008 R2. Windows Server 2008 R2 Service Pack 1. Windows Server The installation operation fails if a user tries to install on other operating systems. Hardware The VMware recommended minimum hardware configuration for Simple Install deployment of vcenter Single Sign-On, the vsphere Web Client, vcenter Inventory Service, and vcenter Server. Following table lists the hardware configuration: Table 14. Hardware Configurations CPU RAM Components Capacity 2GHz Dual Core 12GB 3BSE

214 Pre-configuration Section 19 VMware vcenter Installation Table 14. Hardware Configurations Components Hard Disk Network 40GB - 60GB free space (that is 80GB Hard Disk) 1Gbps Capacity Pre-configuration Ensure that the computer where vcenter is to be installed has the following preconfigurations made to minimize installation difficulties: 1. Ensure that the network adapters have static IP addresses. 2. It is critical that the Fully Qualified Domain Name (FQDN) is pinged (Figure 184). a. To find FQDN, use the following command: ipconfig /all b. If there is no DNS Suffix, then use only the Host Name. Figure 184. Lists all the IP Addresses 214 3BSE

215 Section 19 VMware vcenter Installation Pre-configuration 3. In the Host Name, DNS Suffix combination is: vcenter.ld.local. 4. If there is no DNS Suffix, just use the following Host Name to ping: ping - 4vcenter.Id.local. If there is no response as in Figure 185, then an entry needs to be added into hosts file. Figure 185. No Response in Command Prompt 5. Open the C:\Windows\System32\devices\etc\hosts file and add the line as shown in Figure 186. Figure 186. Hosts File Edited Use the values obtained in the ipconfig /all command. Do not use the lmhosts file as this does not work in Windows 2012 server. 3BSE

216 Pre-configuration Section 19 VMware vcenter Installation 6. In the Command Prompt enter ping -4 vcenter.ld.local. The ping statistics is displayed. Figure 187. Ping Statistics 7. Install DotNet Framework on Windows 2012 Server using the following command (assuming Windows installation media is inserted and is the D drive): dism /online /enable-feature /featurename:netfx3 /all /Source:D:\sources\sxs. Figure 188. Installing DotNet Framework 216 3BSE

217 Section 19 VMware vcenter Installation Install vcenter Install vcenter Perform the following procedure to install vcenter Single Sign-On: During the vcenter installation if the following pop-up window appears, always click No: 1. Insert the vcenter installation media into the computer. 2. If the installation program does not automatically start, click the autorun to start the process. Figure 189. Starting the Installation 3BSE

218 Install vcenter Section 19 VMware vcenter Installation 3. At the VMware vcenter Installer dialog box, select Simple Install and click Install. Figure 190. Welcome Screen 4. Click Next at the Welcome to the vcenter Single Sign-On Setup dialog box. 5. Select I accept terms in the License Agreement and click Next. 6. Click Next at the Simple Install Prerequisites dialog box BSE

219 Section 19 VMware vcenter Installation Install vcenter 7. Enter a Password and Confirm Password, and then click Next. For the vcenter Single Sign-On password setting, there is a bug in the installation procedure whereby the Web Client fails to install if certain special characters are used. The following summarizes the password requirements: - The password must have at least eight characters. - The password must have at least one special character. - The password must have at least one lowercase character. - The password must have at least one uppercase character. - The password must have at least one number. - The password cannot have any of the following special characters: ^ * $ ; " ' ) < > & \ _. - The password cannot have a trailing " " space. - The password cannot have non-ascii characters. Figure 191. Entering the vcenter Single Sign-On Password 3BSE

220 Install vcenter Section 19 VMware vcenter Installation 8. Enter a Site name and click Next. Figure 192. Specifying Site Name 9. Let the HTTPS port be default, click Next. Figure 193. HTTPS Port Number 10. Let the installation be in the default folder, click Next BSE

221 Section 19 VMware vcenter Installation Install vcenter 11. Review the installation information and click Install. Installation of the vcenter Single Sign-On should complete without any interruption (see Figure 194, Figure 195 and Figure 196). Figure 194. vcenter Single Sign-On Installation Figure 195. Installation of the VMware vsphere Web Client 3BSE

222 Install vcenter Section 19 VMware vcenter Installation Figure 196. Installation of the VMware vcenter Inventory Service Finally, the vcenter Single Sign-On simple installation starts. Figure 197. Installation of the vcenter Server 222 3BSE

223 Section 19 VMware vcenter Installation Install vcenter 12. Enter the License Key and click Next. Figure 198. Entering License of the Product 3BSE

224 Install vcenter Section 19 VMware vcenter Installation 13. Let the vcenter Server database be default, click Next. Figure 199. Using the Default Database Option for Small Installations For installations which have maximum 5 Hosts/50 VMs, Microsoft SQL 2008 Express can be used. For larger installations, full version of Microsoft SQL Server 2008 should be used BSE

225 Section 19 VMware vcenter Installation Install vcenter 14. Use the default service account for the vcenter Server Service and click Next. Figure 200. Using the Windows Local System Account 3BSE

226 Install vcenter Section 19 VMware vcenter Installation 15. On the warning message for workgroup installation of the vcenter Server, click OK. Figure 201. Warning Message 226 3BSE

227 Section 19 VMware vcenter Installation Install vcenter 16. Leave the network ports to default values and click Next. Figure 202. Network Ports Specifications 3BSE

228 Install vcenter Section 19 VMware vcenter Installation 17. Let the Inventory Size be small and click Next. Figure 203. Inventory Size 18. Click Install to begin the installation. 19. Click Finish on the installation complete dialog box BSE

229 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client 20. Click Ok on the summary message that appears. Figure 204. Installation Summary Message Basic Configuration through the vsphere Web Client This sub-topic consists an example of some of the basic configuration of the vcenter Server through the vsphere Web Client. This is only to provide familiarity with the interface, and not an in-depth process. Perform the following procedure for configuration: Attempting to access the vsphere Web Client will request Flash Player to be installed. This is found not to work on Windows Server Using Windows client on the same network which has successfully installed Flash, open vsphere Web Client address using the following example: 3BSE

230 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 2. Click Continue to this website. Figure 205. Accessing the vsphere Web Client from a Windows Client 230 3BSE

231 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client 3. Use Single Sign-On credentials and click Login to log on to the vsphere Web Client. Figure 206. Login Credentials 3BSE

232 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 4. Click vcenter in the left pane of the vsphere Web Client. Figure 207. Selecting vcenter Configuration 232 3BSE

233 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client 5. Click Datacenters under Inventories. A Datacenter is a top level container required to enable the addition of hosts and clusters. Figure 208. Selecting Datacenters 3BSE

234 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 6. Click the add datacenter icon (see Figure 209) to add a new datacenter. Figure 209. Adding Datacenter 7. Enter a datacenter name and select the vcenter server to contain the datacenter, and click OK. Figure 210. Naming the Datacenter 234 3BSE

235 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client 8. Add a host to data center by clicking the icon, see Figure 211. Whilst there are a number of items that can be added to the datacenter such as hosts and clusters, the example in this context will only work through adding a host to the datacenter. Figure 211. Adding a New Host 9. Enter IP address and click Next. Figure 212. Specifying the IP Address 3BSE

236 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 10. Enter the administrative User name and Password credentials of the host to be added, and click Next. For an ESXi server, this is typically the root user. Figure 213. Specifying the Administrative Credentials 236 3BSE

237 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client 11. Click Yes on the Security Alert message. Figure 214. Accepting the Host SHA1 Thumbprint 3BSE

238 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 12. In the summary, review the host details to ensure that the correct host is being added, and click Next. Figure 215. Reviewing the Host Details 238 3BSE

239 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client 13. Select the License and click Next. Each host to be added requires a full license to be added in the host through the vsphere Client before it can be added to the vcenter Server. Figure 216. License Selection 14. For the Lockdown mode let the Enable lockdown mode check box be clear and click Next. The Lockdown mode prevents logging directly into the ESXi server. Figure 217. Lockdown Mode 3BSE

240 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 15. Select VM Location > Datacenter and click Next. Figure 218. VM location Selection 240 3BSE

241 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client 16. Review the configuration and click Finish. Figure 219. Summary of the Configurations 17. After a short refreshing period, the Datacenter will be updated with the number of hosts and virtual machines, see Figure BSE

242 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 18. Click vcenter navigation on the left-hand side to return to the main menu. Figure 220. Hosts and VMs Available 242 3BSE

243 Section 19 VMware vcenter Installation Basic Configuration through the vsphere Web Client The vcenter Home shows the updated inventory in the vcenter. Figure 221. Updated vcenter Inventory 3BSE

244 Basic Configuration through the vsphere Web Client Section 19 VMware vcenter Installation 244 3BSE

245 Section 20 VMware High Availability This topic describes how to improve the VMware high availability. VMware High Availability is not a substitute for 800xA redundancy. The VMware High Availability (HA) functionality is designed to recover ESXi servers automatically from a hardware failure. Virtual machines that are running on the failed server are automatically restarted on another ESXi server. The virtual machines are offline for few minutes to the restarted virtual machines. The configuration of VMware HA allows the specification of the ESXi server that hosts the restarted virtual machines. VMware HA requires the virtual machines to be stored on an external storage server, such as, a SAN server, and also requires the ESXi Servers to be added to a vcenter Server. The storage network and the management network must be redundant. With the introduction of USB support in ESXi 4.1, it is possible to have the ABB USB dongle connected directly into the ESXi server and available in the guest. However, when a virtual machine is moved from one ESXi server to another, it is necessary to move the dongle as well. Here, it would be beneficial to use a USB over IP device, such as, AnywhereUSB, so as to maintain continuity of functionality when moving virtual machines. Configure VMware High Availability Before configuring VMware HA, the network setup should be reviewed. Refer ESXi Configuration Console. When a virtual machine is moved from one ESXi server to another, the vswitches must have the same set of names. For example, if the plant network is named PN1 on ESX1, then it must also be named PN1 on ESX2 and ESX3. 3BSE

246 Add a Cluster Object Section 20 VMware High Availability The ESXi servers that are part of the VMware HA configuration, must also be part of a cluster in the vcenter Server as shown in the Figure 222. Add a Cluster Object Figure 222. Cluster in vcenter Perform the following steps to add the cluster: 1. Log in to the vcenter Server. 2. If there is no DataCenter, right-click on the vcenter object in the left pane and select New DataCenter. 3. Provide a name for the datacenter. 4. Right-click on the DataCenter object, and select New Cluster to create a cluster. 5. Enter a name for the cluster. 6. Select the Turn On VMware HA option and click Next. 7. Let the Host Monitoring Status and Admission Control properties be default and click Next. These can be adjusted later. 8. Let the Cluster Default settings be default and click Next. 9. Let the VM Monitoring Status and Default Cluster Settings be default and click Next BSE

247 Section 20 VMware High Availability Add ESXi Servers to the Cluster 10. Leave the Enhanced vmotion Compatibility (EVC) disabled. 11. Let the Swapfile Policy for Virtual Machines be default and click Next. 12. Review the summary of settings and click Finish. Add ESXi Servers to the Cluster To add ESXi servers to the cluster, the following procedure is repeated for each ESXi server: 1. Right-click on the cluster object and select Add Host. 2. Enter the Host name or IP address, Username and Password. 3. Click Next. If the name is used, the vcenter and ESXi hosts must have a functioning DNS server. 4. Click Yes on the Security Alert dialog box. 5. Click Next on the Host Summary dialog box. 6. Enter the license for ESXi server if it is not entered previously and click Next. This cannot be the free ESXi host license. Specify Failover Host 7. Leave the Enable Lockdown Mode unchecked on the Lockdown Mode window and click Next. 8. Click Finish on the Ready to Complete dialog box. In the Storage Area Network, there are three ESXi servers in the configuration. The primary server is used for the primary 800xA nodes, while the secondary server is used for the redundant 800xA nodes. The third server in the configuration is the failover host (Figure 223) that is not running virtual machines. If either the primary or secondary server fails, the virtual machines that are running on the failed server are started on the third server. 3BSE

248 Specify Failover Host Section 20 VMware High Availability Figure 223. Virtual Machines Restarting on the Failover Server To ensure that the failed nodes on the primary server are not started on the secondary, or visa-versa, it is important to specify the third server as the failover server: 1. Right-click on the Cluster object and select Edit Settings. 2. Select VMware HA on the left pane. 3. Select Do not power on VMs that violate availability constraints for the Admission Control in the right pane. 4. Select Specify a failover host and select the third ESXi server from the dropdown list on the Admission Control Policy. 5. Click OK to complete BSE

249 Section 21 VMware vmotion VMware vmotion must not be used during production or engineering. The 800xA System must be stopped and no afw processes running. VMware vmotion is the ability to move virtual machines from one ESXi server to another while it is running. It is required that the ESXi servers are members of a vcenter server and the virtual machines are stored on a SAN server. When a vmotion operation is performed, the memory of the virtual machine is progressively transferred to the other ESXi server. Changes in the memory of the original virtual machine are tracked and transferred. When the remaining memory to be transferred is small enough to be performed in one last operation, the virtual machine stops, the remaining memory is transferred, and the virtual machine execution is continued on the other server. This generally results in a loss of one ping on the network. Depending on the network speed and the CPU activity of the virtual machine, the time may be more or less. The loss of network connection may be in the order of a few seconds, but the applications and services on the computer transferred, or communication with the computer transferred may take longer to return to normal operations. To assist in minimizing interference with the storage network communication, a separate network should be used for vmotion operations. 3BSE

250 Section 21 VMware vmotion 250 3BSE

251 Section 22 VMware Storage vmotion VMware Storage vmotion is not supported for System 800xA. 3BSE

252 Section 22 VMware Storage vmotion 252 3BSE

253 Section 23 VMware Fault Tolerance VMware Fault Tolerance is not supported for System 800xA. 3BSE

254 Section 23 VMware Fault Tolerance 254 3BSE

255 Section xA in the IT Infrastructure Many companies have an IT infrastructure and have already taken advantage of virtualization for consolidating server functionality. Moving the 800xA Server environment to the IT infrastructure is a logical progression for optimizing both hardware and IT expertise. Prior to planning the installation of the 800xA System in the IT Infrastructure, the IT infrastructure administrators must be consulted to establish feasibility. The following must be understood: ABB 800xA Systems should have their own domain, and not attached to the corporate domain. ABB 800xA Systems use multicast communication which is typically blocked in IT infrastructure environments. There is a diverse hardware range in the IT infrastructure, yet only VMware vsphere ESXi servers that are running on Intel servers are supported for 800xA. Typically, these types of environments make heavy usage of SAN storage. Generally, monitoring of the server hardware is already in place, but not integrated into 800xA in these installations. The procedure to notify production environments of server issues needs to be reviewed. While the 800xA server nodes can share resources with other servers, it should be considered as a best practice to have the 800xA server nodes residing on ESXi servers. If there is some sharing, the resource usage should be reviewed. This includes the typical loading of scheduled or unscheduled activities, such as, weekend backup operations. The 800xA is a 24/7 system and should have resources accordingly. Those who are responsible for maintaining the infrastructure must be aware that the 800xA System is a 24/7 System, which cannot be treated in the same fashion as 3BSE

256 Section xA in the IT Infrastructure other servers. Maintenance must always be coordinated with production to ensure that no downtime occurs. Due to the diversity of configurations, ABB cannot fully support various configurations. The project and customer must be in agreement regarding support responsibilities before the project is undertaken. This includes the required knowledge and resources to assume the responsibility of support BSE

257 Section xA Virtual Client Virtual Clients 800xA workstations are dedicated physical computer workstations running Windows 8.1 and 800xA as an operator client. Moving the physical clients to the virtual environment allows the benefits of virtualization to be extended to the clients. Virtual clients do not benefit from physical graphic adapter acceleration. Hence, their usage should be restricted as described in Dimensioning Rules - Virtual 800xA Clients. Virtual clients (Figure 224: comparison between physical and virtual clients) are virtual machines which are running Windows 8.1 and 800xA as an operator client. Access to the virtual client desktop is done over the network through a thin client which has 1 to 2 monitors, a keyboard and a mouse attached. In the first release of virtual client support for 800xA, only Microsoft remote desktop protocol is supported. Thin clients are purpose built computers from hardware vendors such as HP and Dell. These devices normally have low power consumption and runs without a fan. The operating system pre-installed in these devices need to be Windows 7 to handle multiple screen remote sessions, and is usually installed on an internal flash device. 3BSE

258 Benefits of Virtual 800xA Workplaces Section xA Virtual Client Figure 224. Physical Client to Virtual Client Comparison Microsoft allows multiple screen remote sessions in the following versions of Windows: Windows 7 Enterprise - 32-bit (x86) and 64-bit (x64) Windows 7 Ultimate - 32-bit (x86) and 64-bit (x64) Windows 2008 Server R2-64-bit Benefits of Virtual 800xA Workplaces The following are the benefits of virtual 800xA workplaces: Reduced operator room space requirements. Reduced operator room power and heating requirements BSE

259 Section xA Virtual Client Operating Area Planning Reduced operator room cooling requirements. Reduced operator room noise. Fast replacement of thin client. Ability to move virtual client to new hardware without reinstall. Standard installation without messing the drivers. Added security by setting up virtual client without USB. Clients now in server room with cost effective remote solutions. When considering relatively small number of clients that are involved in an 800xA solution compared to the enterprise client virtualization, the initial cost of a virtual client solution is at a disadvantage. However, the subsequent maintenance benefits are a continuous benefit that should always be considered when engineering solutions. Operating Area Planning When consolidating hardware into virtual systems, availability must always be considered. If a area has all clients consolidated into one virtual server host and that host becomes unavailable, there will be no clients available. Hence, there must be at least two ESXi servers supporting the control area clients. 3BSE

260 Graphical Presentation and Physical Server Speed Section xA Virtual Client Figure 225. Control Room Support from Multiple Physical Servers A separate network for the remote access must be configured. This enables secured firewall configurations, which improves security by keeping the remote access separate from plant and control networks. Graphical Presentation and Physical Server Speed Graphical rendering that is presented using the thin client is created in the virtual node using virtual and physical CPU resources. The virtual machine does not take advantage of the physical server graphics adapter capabilities. Specifying the physical server CPU should take this into account. If the 800xA workplaces are expected to be heavily loaded, then a faster CPU would provide more capability to handle the load BSE

261 Section xA Virtual Client Thin Clients Thin Clients Multiple Screen These are small computers that normally have an embedded version of Windows 7 pre-installed by the manufacture of the thin client. One or more monitors, keyboard, mouse and networking are connected to the thin client. Thin client establishes communication with the virtual client using client software, such as, Microsoft's Remote Desktop Client. Minimal configuration is required to configure thin client software. This makes it easy to replace the unit. Thin clients may be initially configured for screen and power saving. For production systems, these features must be turned off. As thin client uses a flash disk, configuration of the device requires the disk to be placed in read-write mode. This is described in the device user manual. Once the configuration is complete, the device must be switched to read-only mode. Currently, thin clients are designed with a single interface. Hence, network redundancy using 800xA RNRP is not possible. It is possible to use a standard approved 800xA workstation PC as a thin client. In this case, RNRP can be installed. While the advantages of a true thin client are lost, the workstation needs only minimal software installation and configuration. Hence, replacement becomes a much easier task. Primary considerations for multiple screens are that the operating system in the virtual client supports multiple screens. For example, if the virtual machine is running Windows 8.1 Pro, a thin client can open a multiple monitor remote desktop session. They can provide higher levels of loading as there is more screen real estate to bring up more objects. 3BSE

262 Multiple Screen Section xA Virtual Client 262 3BSE

263 Section 26 Timesync ESXi with Windows ESXi servers need to be time synced to a reliable time server. By default, when an ESXi server is configured to synchronize time from a Windows computer, the time sync will not work. This topic describes how to configure an ESXi server to use a Windows time sync server. Communication Topology The ESXi server must be able to communicate to the Windows time server. This is performed through the ESXi Management Network. Hence, the Windows computer 3BSE

264 Configure Windows Section 26 Timesync ESXi with Windows which is the time sync server and the ESXi management interface must be on the same network. Figure 226. Communication from ESXi Server to the Windows Time Sync Server The Figure 226 shows a Windows virtual machine as the time sync server, but the Windows computer may be a physical computer as well. Configure Windows The Windows computer to be the time server needs to be configured correctly before it works with the ESXi server. This requires the opening of the appropriate firewall port, setting registry entries, and restarting the Windows time server service BSE

265 Section 26 Timesync ESXi with Windows Firewall Configuration Firewall Configuration Open Windows firewall for UDP port 123 on the Windows computer that serves as the time server. Figure 227. Opening Windows Firewall for ntp Traffic Registry Two changes have to be made in the registry: 1. In the Registry Editor, locate HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32 Time\Parameters and set the Type value to NTP. Figure 228. Setting the NTP type 3BSE

266 Service Section 26 Timesync ESXi with Windows 2. In the Registry Editor, locate HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32 Time\Config and set the AnnounceFlags value to 5. Figure 229. Setting the Announce Flag Service Restart the W32time Service for the changes to take effect. Figure 230. Restarting the Windows Time Server Service 266 3BSE

267 Section 26 Timesync ESXi with Windows Configure ESXi Configure ESXi Configuration of the ESXi server requires the modification of two text configuration files in the ESXi. This cannot be done through the VMware vsphere client and requires the use of a free 3rd party software named Putty. This program enables a secure terminal style session to be opened to another computer (typically, not a Windows computer). By default, the ESXi server has it's SSH service stopped, hence, this also needs to be started before the session can be established to modify the ESXi. SSH Enabling SSH is performed through the VMware vsphere client. 1. Open Vmware v Sphere Client and log on to the ESXi Server to be modified. 3BSE

268 SSH Section 26 Timesync ESXi with Windows 2. In the Configuration tab under Software pane, click Security Profile. Figure 231. Accessing the SSH Configuration in the VMware vsphere Client 3. In the Services section click Properties (Figure 231) BSE

269 Section 26 Timesync ESXi with Windows SSH 4. Select SSH service from the list and click Options... Figure 232. Accessing the SSH Configuration 3BSE

270 SSH Section 26 Timesync ESXi with Windows 5. Click Start and then click OK to start the SSH service. Figure 233. Starting the SSH Service The list of all running services shows SSH service is running. Figure 234. List Showing SSH Service Running 270 3BSE

271 Section 26 Timesync ESXi with Windows Putty Putty Putty is a client that is used to connect the SSH service in the ESXi server. Perform the following to install the Putty: 1. Click on putty.exe to download the Putty from the following address: Figure 235. Putty Download 2. Double-click the icon on the desktop to run the Putty client. 3BSE

272 Putty Section 26 Timesync ESXi with Windows 3. In the Putty Configuration dialog box, enter the Host Name (or IP Address) and click Open. Figure 236. Specifying the IP Address 272 3BSE

273 Section 26 Timesync ESXi with Windows Putty 4. Click Yes on the Security Alert Message. Figure 237. Accepting the Server's rsa2 Key Fingerprint 5. Use the root account as the login user, and provide the root password. Figure 238. Logging into ESXi using Putty 3BSE

274 ESXi Configurations Section 26 Timesync ESXi with Windows On successful login, the # prompt appears. ESXi Configurations Figure 239. Successful Login to the ESXi Server When making changes to the ESXi configuration files, the vi editor is used. This is a difficult editor to use. Hence, care should be taken to only perform the actions described in this topic. The vi editor has two main modes, one for entering text, and one for entering commands. The ESC key is used to leave the entering text mode and change to the command mode. 1. Create a backup of the ntp.conf file using the following command: #cp /etc/ntp.conf /etc/ntp.conf.bak. 2. Edit the ntp.conf file with the following command: # vi/etc/ntp.conf. 3. Use the arrow keys on keyboard and move the cursor to the end of the last line, and press the i key to enable insert mode BSE

275 Section 26 Timesync ESXi with Windows ESXi Configurations 4. Press the Enter key to go to a new line, and add the following to the end of the file: tos maxdist 30. Figure 240. Adding tos maxdist Press the ESC key, then the :(colon) key and the X key to save and exit the vi editor. Figure 241. Vi editor 6. Another file that needs changes is in the following path: /etc/likewise/lsassd.conf file. 7. Create a backup of the Isassd.conf file using the following command: # cp /etc/likewise/lsassd.conf /etc/likewise/lsassd.conf.bak. 8. Enable editing the Isassd.conf file using the following command: # chmod +w /etc/likewise/lsassd.conf. 3BSE

276 ESXi Configurations Section 26 Timesync ESXi with Windows 9. Edit the Isassd.conf file using the following command: # vi /etc/likewise/lsassd.conf. 10. Use the arrow keys to move to the line containing # sync-system-time = no, and place the cursor on the #. Figure 242. Editing Isassd File 11. Press the i key to enable text editing mode, then press the Del key to remove the #. Figure 243. Removing the # 276 3BSE

277 Section 26 Timesync ESXi with Windows ESXi Configurations 12. Press the ESC key, then the :(colon) key and the X key to save and exit the vi editor. Figure 244. Saving the File and Exit the vi Editor 13. Enter the following in Putty to ensure that the changes made are persisted on restarting the ESXi server: # /sbin/auto-backup.sh. Figure 245. Execution of Auto Backup Command 14. Restart the Isassd service with the following command: #./etc/init.d/lsassd restart. 3BSE

278 Timesync Confirmation Section 26 Timesync ESXi with Windows The Isassd service restarts. Figure 246. Restarting the Isassd Service 15. Restart the ntpd service with the following command: #./etc/init.d/ntpd restart. The ntpd service restarts. Figure 247. Restarting the ntpd Service Timesync Confirmation The time sync is updated approximately every 15 minutes. The update requires a refresh in the vsphere Client. Perform the following for the time sync confirmation: 1. Enter the command # watch date in Putty to monitor the time in ESXi Server BSE

279 Section 26 Timesync ESXi with Windows Timesync Confirmation The command results in displaying the current time and date every 2 seconds. Press Ctrl+C to stop the watch date command. Figure 248. Watch Date Command Result 2. Check that the ESXi Server has updated the time to the Windows ntp Server. 3. Type #exit to end Putty session. 4. Click Stop in the SSH Options dialog box, to stop the SSH service in the ESXi server and help to prevent unauthorized access. Figure 249. Stopping SSH Service 3BSE