vranger Version Deployment Guide

Transcription

1 vranger Version Deployment Guide

2 2010 Quest Software, Inc. ALL RIGHTS RESERVED. This guide contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser s personal use without the written permission of Quest Software, Inc. The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document. If you have any questions regarding your potential use of this material, contact: Quest Software World Headquarters LEGAL Dept 5 Polaris Way Aliso Viejo, CA [email protected] Refer to our Web site for regional and international office information. Patents This product includes patent pending technology. Trademarks Quest, Quest Software, the Quest Software logo, AccessManager, ActiveRoles, Aelita, Akonix, Benchmark Factory, Big Brother, BridgeAccess, BridgeAutoEscalate, BridgeSearch, BridgeTrak, BusinessInsight, ChangeAuditor, CI Discovery, Defender, DeployDirector, Desktop Authority, Directory Analyzer, Directory Troubleshooter, DS Analyzer, DS Expert, Foglight, GPOADmin, Help Desk Authority, Imceda, IntelliProfile, InTrust, Invirtus, itoken, JClass, JProbe, LeccoTech, LiteSpeed, LiveReorg, LogADmin, MessageStats, Monosphere, NBSpool, NetBase, NetControl, Npulse, NetPro, PassGo, PerformaSure, Point, Click, Done!, Quest vtoolkit, Quest vworkspace, ReportADmin, RestoreADmin, ScriptLogic, SelfServiceADmin, SharePlex, Sitraka, SmartAlarm, Spotlight, SQL Navigator, SQL Watch, SQLab, Stat, StealthCollect, Storage Horizon, Tag and Follow, Toad, T.O.A.D., Toad World, vautomator, vconverter, vecoshell, VESI, vfoglight, vpackager, vranger, vspotlight, vstream, vtoad, Vintela, Virtual DBA, VizionCore, Vizioncore vautomation Suite, Vizioncore vessentials, Vizioncore vworkflow, WebDefender, Webthority, Xaffire, and XRT are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. Other trademarks and registered trademarks are property of their respective owners. Deployment Guide August 2011 Version 5.2.1

3 Table of Contents Planning Your Deployment...5 Getting Started... 6 Before You Begin... 7 Other Information Sources vranger Overview vranger Overview vranger Architecture The vranger Backup Process Available Backup Architectures The vranger Restore Process vranger Repositories The vranger Replication Process Deploying vranger...42 The Deployment Environment The Deployment Environment Choosing a vranger Deployment Architecture Environment Configuration Recommendations VMware Recommendations vranger Repositories Common Configuration Issues Managing vranger Performance...75 Protection Best Practices Workload Protection Considerations Pre-seeding Replication Jobs Space Saving Technologies... 85

4 Table of Contents 4 Filtering Catalog Collections...86 Integrating vranger...88 Integrating vranger...89 vranger and Tape Backup Vendors...89 Application Backup & Recovery: Recovery Manager for Exchange...97 Data Domain and vranger Repositories...98 Using vranger and Microsoft PowerShell...98 Index...101

5 Planning Your Deployment This section provides general information on vranger features and functions, as well as some key questions intended to guide your deployment planning. This portion of the document contains the following sections: Getting Started...6 Other Information Sources...11 vranger Overview...14 vranger Architecture...14 The vranger Backup Process...18 The vranger Backup Process...18 The vranger Restore Process...28 vranger Repositories...29 The vranger Replication Process...34

6 Getting Started This chapter provides information on key questions and information sources used to plan a vranger deployment. This chapter contains the following sections: Before You Begin...7 Learn About vranger...8 Setup Your Environment...8 Install vranger...9 Test Your Installation...9 Tune Your Deployment...10 Get More Out Of vranger...10 Other Information Sources...11

7 Getting Started 7 Before You Begin When planning any deployment, there are several steps that all customers will go through. Each of these steps is listed in the graphic below. The table below the image list the most common questions for each step, and provide links to the relevant sections of the document.

8 Getting Started 8 Learn About vranger What does vranger do? vranger Overview on page 14 How does vranger work? vranger Architecture on page 14 The vranger Backup Process on page 18 The vranger Restore Process on page 28 The vranger Replication Process on page 34 vranger Repositories on page 29 What are my options for using vranger? The vranger Backup Process on page 18 The vranger Replication Process on page 34 The vranger Restore Process on page 28 Where can I get more information? Other Information Sources on page 11 Setup Your Environment What are the requirements for deploying vranger? How should I configure my environment for vranger? What are some common configuration issues I should avoid? Hardware Requirements on page 74 Software Requirements on page 74 Application Databases on page 16 Source and Destination Requirements on page 75 VMware Requirements on page 45 Port Requirements on page 44 Recommendations for Network Backups on page 54 Recommendations for Fibre Backups on page 55 Recommendations for Replication on page 56 VMware Recommendations on page 59 vranger Repositories on page 68 Installing the Databases on page 78 Common Configuration Issues on page 71

9 Getting Started 9 Install vranger I m upgrading from vranger 3.x - what do I need to know? Upgrading from vranger Pro 3.x on page 80 Performance Expectations on page 82 Job Migration on page 84 Where should I install vranger? Where to Install? on page 75 What should I do with the vranger database? Database deployment options on page 16 Installing the Databases on page 78 Test Your Installation I ve got my repositories set up - how can I test them? Testing Repository Connections on page 69

10 Getting Started 10 Tune Your Deployment How should I configure by backup jobs for the best performance? How can I improve my data consistency? How does vranger reduce the storage footprint of archives? Workload Protection Considerations on page 77 Load Balancing on page 84 Pre-seeding Replication Jobs on page 80 Space Saving Technologies on page 85 Data Domain and vranger Repositories on page 98 Get More Out Of vranger How do I get vranger to work with my tape software? How can I extend vranger? I have a Data Domain appliance. Can I use that with vranger? vranger and Tape Backup Vendors on page 89 Application Backup & Recovery: Recovery Manager for Exchange on page 97 Using vranger and Microsoft PowerShell on page 98 Data Domain and vranger Repositories on page 98

11 Getting Started 11 Other Information Sources Before deploying vranger, it is recommended to familiarize yourself with the available information sources. Documentation The vranger documentation set consists of the following: Release Notes (PDF) Getting Started Guide (PDF) What s New Guide (PDF) System Requirements Guide (PDF) Installation and Setup Guide set PDF): User Guide (PDF and online help) The vranger documentation is available online at: Quest Online Resources: The Quest website contains a wealth of information, including product features and functions, release information, product documentation, training, and support forums. General Product Page - Feature Information, Demos, etc: vcommunity - A collaborative environment for Quest customers featuring forums, wikis, and more. Training - Free access to online product training. Contacting Support Quest Support is available to customers who have a trial version of a Quest product or who have purchased a commercial version and have a valid maintenance contract. Quest Support is easily accessed in the following ways: support directly at [email protected] for automatic case creation.

12 Getting Started 12 Contact Quest support directly via our global and local telephone numbers. Log and create/update your case, and check its status via the Quest Support Case Management portal. View the Quest Support Guide for a detailed explanation of support programs, online services, contact information, and policy and procedures. The guide is available at:

13 vranger Overview This chapter details the vranger features and functions and provides an overview of the backup and restore process. This chapter contains the following sections: vranger Overview...14 vranger Architecture...14 Application Architecture...14 Application Databases...16 The vranger Backup Process...18 The vranger Restore Process...28 vranger Repositories...29 The vranger Replication Process...34

14 vranger Overview 14 vranger Overview vranger offers comprehensive data protection and management that supports vsphere. Delivering fast and reliable backup and recovery, vranger can capture the entire image of a VM or an individual file. All jobs can be executed without interrupting service--that is, while the source machine is running. Multiple ESX hosts can be used for simultaneous jobs, reducing the backup window. To save time and disk storage space after a full backup, you can run differential backup job to capture only the data that has changed since the full backup. You can run backup and restore jobs immediately or schedule them. You can track the progress of all jobs and their component tasks. vranger allows you to connect to multiple VirtualCenters (VCs) and back up multiple ESX hosts. vranger Architecture vranger is a multi-tiered application with both the client and server components installed on the same machine. vranger will use a SQL Express database by default, but can easily be configured to use an existing SQL database installed anywhere in the network environment. Application Architecture The diagram below shows the various components, and where they are utilized during the backup process.

15 vranger Overview 15 vranger Client vranger Service vapi The vranger Client is the main user interface for vranger. The Client must be installed and run on the same machine as the vranger Service. The vranger Service is the scheduling and communication engine for vranger. The service runs independently of the vranger Client, which means that task scheduling is not affected by closing the client. vapi is an exposed web service through which third-party vendors can integrate with vranger. A subset of the vranger vapi is the vapi cmdlets for PowerShell. These cmdlets can be used to script vranger features and functions.

16 vranger Overview 16 USDI Service Repository Database VSS Module The USDI Service is responsible for FLR, specifically for the mounting of the.var archive and launching vzwino.exe to allow file extraction from within the GUI. In vranger, this is the configured storage location for backup jobs. A repository can be: CIFS, FTP, SFTP, or NFS v 3. vranger requires a database (SQL or SQL Express) to store configuration information and task details. The database is installed by default on the vranger machine, but can easily be installed on an existing SQL server as well. vranger Application Databases vranger utilizes a SQL database to store application and task configuration data. The database can be either SQL Express 2005, which is the default, or an external SQL database running on your own SQL Server 2005 or SQL Server 2008 database server. Note The cataloging function of vranger requires that the application and catalog databases be installed on the vranger server. For more information, see Installing the Databases on page 78 Database deployment options The database deployment occurs during the initial installation of vranger. Default The Installation Wizard will default with a selection to install vranger with the embedded SQL Express 2005 database. The default database can only be installed on the vranger server. While the embedded SQL Express database is free and simple to install, there is a size limit of 4GB per database. External SQL Server The Installation Wizard will guide you through configuring vranger with an external SQL database. There is also an option in the Install Wizard to configure the database connection manually, but the guided approach is recommended.

17 vranger Overview 17 Most larger organizations chose to use the embedded database to comply with internal policies and best practices. A significant advantage of using an external database, however, is that all settings and configurations (which are stored in the application database) are stored separate from the vranger installation. In the event of a failure on the vranger machine, the application can be quickly installed on another machine and configured to use the existing database. Using an external database also makes it possible to move the vranger installation with minimal effort. The following versions of Microsoft SQL are supported for operation with vranger. SQL 2005 SQL 2005 SP1 SQL 2005 SP2 SQL 2005 SP3 SQL 2008 SQL 2008 R2 Caution If you chose to use Microsoft SQL Server instead of SQL Express, the SQL Server application must be installed on the vranger Machine in order for the cataloging feature to function. For more information, see Installing the Databases on page 78

18 vranger Overview 18 The vranger Backup Process With Inventory Node Selection, you can browse the VirtualCenter (vcenter) inventory and select which VMs or groups (folders) you wish to backup. You can select a VM, ESX host, folder, resource pool, datacenter, or vcenter and backup all of the VMs located under that node in the tree. vranger uses VMware snapshot technology to temporarily store incoming write requests while the VM is being backed up. After the backup completes, the snapshot is deleted, which commits those pending writes to disk. vranger can backup a VM that already has an open snapshot and will backup the open snapshot, but any secondary consolidated helper backups will be closed prior to running the backup. Backup Limitations vranger cannot backup physical RDMs (raw device map) partitions. You should see the warning: incompatible drive detected. Virtual RDMs (vrdms) are supported. For more detailed information on backup features supported by various configurations, please see the tables below. Usage Compatibility Matrix - Compression on page 110 Usage Compatibility Matrix - Space Saving Technologies on page 111 Usage Compatibility Matrix - VM Disk Configurations on page 112 What Changes in the Guest OS By default, vranger does not place any code into the Guest OS. Starting with vranger 4.5, the vzshadow.exe executable will be included with the application download. vzshadow.exe is an optional component that can, if you choose, be installed on Windows VMs to provide an additional level of consistency. For more information, please see Guest Quiescing on page 82. What Changes on the Source While vranger does not permanently install anything on the source ESX Servers, several tools are uploaded when a backup or restore task is run. Once the task is completed, these tools are removed. The tools are uploaded to the /tmp/.vzbin directory. The table below lists the tools that are uploaded:

19 vranger Overview 19 Tool backup.exe cleanup.exe delete.exe deleteall.exe filter-and.exe free.exe metadata.exe restore.exe sizes.exe sprinkle.exe ssh.exe touch.exe vddkd.exe vznwinio.exe verify.exe Description backs up VM disks to a repository as a archive file deletes any orphaned temp files removes one archive file from a repository removes all archive files from a repository combines 2 filter files (for backup) into 1 by doing bitwise-and on each bit. Checks the repository for sufficient free space. writes metadata associated with a VM backup restores an archive file to a host returns the size of a var file re-arranges the original vranger 4.0 repository structure to the current model. Used for SSH connections. verifies access to a repository by writing and deleting a file used to engage the vstorage API for ESXi backups and LAN-Free backups. an intermediate tool used in conjunction with vranger s file-level restore tool. verifies the existence of files in a repository Note For VMware ESX servers, these files are uploaded and run in the COS. For VMware ESXi, nothing is changed on the source server.

20 vranger Overview 20 Available Backup Architectures vranger can be used in either a network based mode (using a Direct-To-Target configuration for ESX backups) or in a LAN Free mode (using Fibre or HotAdd). Each option is described below, along with some common advantages and considerations. Network Mode Network Mode within vranger can be configured one of two ways, depending on whether you are using ESX or ESXi as your hypervisor. When using ESX, Quest refers to the vranger architecture as Direct-To-Target. When using ESXi as the hypervisor of choice, the preferred option is to use the vranger s HotAdd functionality. This will yield performance similar to ESX network backups. Direct-To-Target For network-based backups when using ESX, the backup data flows from the ESX Host to the target repository. This means that the vranger server does not process any of the backup traffic. This Network Mode configuration, also known as "Direct-To-Target", provides the best scalability when using ESX as your hypervisor platform, as the number of concurrent backup jobs can be scaled across multiple hosts to write to multiple data repositories.

21 vranger Overview 21 For smaller environments, the Direct-To-Target configuration is simple to configure and requires no additional hardware. For larger VMware deployments, this configuration allows for a highly scalable backup solution that distributes load across multiple hosts and repositories while minimizing single points of contention. For example, assume that you have 10 ESX hosts, each with 1 GBit/s network connection. Your total backup bandwidth is theoretically 10Gbit/s. As 10 GBit/s networks become more common, this configuration will be able to handle even higher throughputs. Advantages Easiest method to install. Works this way "out-of-the-box" Disadvantages Performs better with a larger number of ESX servers

22 vranger Overview 22 Allows for leveraging Direct-to- Target for optimal scalability Sufficient configuration for small and large environments alike Service Console NIC is a limiter of throughput Works well in conjunction with Data Deduplication appliances like Data Domain, ExaGrid, and Quantum. vranger can be installed in a VM HotAdd When using vranger in Network mode with ESXI, the best performance will be achieved by using HotAdd. In this configuration, as illustrated below, the VMDK(s) of the source VM are attached (via HotAdd) to the vranger machine, yielding direct access to source data. Overall, the performance in this configuration is slightly less than network backups from ESX servers, but significantly faster than previous ESXi backup methods.

23 vranger Overview 23 Advantages Easiest method to install. Works this way "out-of-the-box" vranger is installed in a VM Disadvantages Performance is slightly slower than traditional LAN-Free methods. All traffic must flow through vranger VM. LAN Free Mode vranger can operate in two LAN-Free modes, traditional (using a physical proxy connected to your fibre infrastructure, or HotAdd (with vranger installed in a VM). LAN-Free- Traditional Note The information in this section applies to a traditional LAN-Free configuration using physical proxies. To configure vranger for LAN-Free backups and restores with vranger installed in a virtual machine, see HotAdd vranger provides a method to offload backup traffic from the network and can perform backups directly via iscsi or Fibre connectivity. The LAN Free configuration is identical whether ESX or ESXi is used, providing the best mix of performance and compatibility for protecting your data, especially if your environment has a mix of VMware hypervisors. In order to perform LAN Free backups, vranger must be installed on a physical system attached to your SAN environment. This is a high performance configuration that requires vranger to be installed on a physical proxy server connected to your fibre or iscsi network. In addition, the VMFS volumes containing the VMs to be protected must also be properly zoned/mapped to the vranger proxy server.

24 vranger Overview 24 Advantages Backups are isolated to the fibre channel infrastructure or iscsi network Provides high performance throughput for backup traffic Offloads ESX/ESXi hosts and network Communicates via vstorage APIs Disadvantages Performs best when backing up data between LUNs within the same SAN infrastructure More complicated to setup and configure than network backup If not configured correctly there is risk of VMFS volume corruption. Make sure to follow the instructions below. vranger must be installed on a physical proxy server

25 vranger Overview 25 Requirements for a LAN Free Configuration In order to implement a LAN Free configuration, you must install vranger on a physical proxy server connected to your Fibre or iscsi infrastructure. You will also need to enable LAN Free backups for each backup job. When using the vstorage API, plan on 1 concurrent backup per CPU core. To calculate the maximum number of concurrent backup tasks per proxy server, simply identify the number of CPU cores on that server - that is the maximum number of concurrent backups. For example, a Dual-Socket, Quad-Core system will be able to perform up to 8 concurrent backup jobs. SAN Configuration Requirements There are several SAN configurations that should be addressed before installing vranger on the proxy server. Caution Do NOT initialize or format the unknown or offline disks from the backup server - these represent your VMFS volumes. Any changes could potentially corrupt your VMFS volumes. Disable automount on the vranger machine: From the start menu, select "run" and enter diskpart. orun the automount disable command to disable automatic drive letter assignment. Run the automount scrub command to clean any registry entries pertaining to previously mounted volumes. The multi-pathing software from your SAN vendor should be installed and configured for the best throughput performance LUNs that are not accessible to the proxy server should be masked in the storage array configuration On your storage device, zone your LUNs so that the vranger HBA has Read- Only access (for backups). If you want to restore over fibre, zone your LUNS so that the vranger HBA has Read-Write access Only one proxy should see a set of VMFS LUN's at one time. LAN Free - HotAdd vranger includes support for VMware's HotAdd disk transport functionality. With the proper configuration, HotAdd allows you to perform LAN-Free backups with vranger installed inside a virtual machine. This configuration that requires vranger to be

26 vranger Overview 26 installed on a virtual machine residing on an ESX(i) host connected to your fibre or iscsi network. Advantages Backups are isolated to the fibre channel infrastructure or iscsi network Provides high performance throughput for backup traffic Disadvantages Performs best when backing up data between LUNs within the same SAN infrastructure More complicated to setup and configure than network backup Offloads ESX/ESXi hosts and network Communicates via vstorage APIs

27 vranger Overview 27 Requirements for a LAN Free - HotAdd Configuration In order to use vranger with HotAdd, vranger must be installed in a VM, and that VM must be able to access the target VM's datastore(s). In addition, all hosts that the vranger VM could be vmotioned to must be able to see the storage for all VMs that vranger will be configured to back up. You will also need to enable LAN Free backups in the vranger backup job options. When using the vstorage API, plan on 1 concurrent backup per CPU core. To calculate the maximum number of concurrent backup tasks per proxy server, simply identify the number of CPU cores on that server - that is the maximum number of concurrent backups. For example, a Dual-Socket, Quad-Core system will be able to perform up to 8 concurrent backup jobs. Configuring vranger for HotAdd When using HotAdd, make sure to disable automount on the vranger machine. This will prevent Windows on the vranger VM from assigning a drive letter to the target VMDK. To disable automount: From the start menu, select "run" and enter diskpart. Run the automount disable command to disable automatic drive letter assignment. Run the automount scrub command to clean any registry entries pertaining to previously mounted volumes.

28 vranger Overview 28 The vranger Restore Process Using vranger, recovery of an entire VM, multiple VMs, or individual files are simple processes. Configure a restore job to include particular VMs but exclude certain disks. Not only can you elect to restore only two of a VM s four disks, you can do this while the VM is running and the other disks are not impacted. vranger s inline data validation for restore ensures that the data in the backup matches the data in the restore. Data streaming is used with incremental and differential restores to speed up the restoration process. During the traditional restoration process of a VM, you restore the full backup first and then restore each incremental -one at a time - starting with the oldest one. This is very time consuming and inefficient. vranger Data Streaming allows each savepoint in a set, if it has the most recent instance of a block, to transfer that block from the repository to the VM(s) specified in the restore job. All of the savepoints transfer their blocks in parallel, resulting in an extremely fast restore, almost as fast as restoring a single full backup. With the speed of incremental restores, backup administrators may consider doing incrementally rather than doing full backups every night and saving lots of repository disk space. The only caveat is that if any of the incremental savepoints in the set are corrupted on disk or corrupted during network transfer, the set becomes unusable. For more detailed information on restore features supported by various configurations, please see the table below. Usage Compatibility Matrix - Restore on page 114 FLR Process vranger allows the restore of one or more files from a compressed or uncompressed full, differential, or incremental backup. FLR Limitations Linux - File level restores from Linux VMs requires the use of a virtual appliance Files can only be restored to a Windows server. FLR is not supported with GPT partitions. FLR is not supported from dynamic disks.

29 vranger Overview 29 Catalog Search vranger includes the option to collect and record file and directory information during the backup process. This information is stored in a catalog database, and used to enable a Browse and Search function that speeds and simplifies file level recovery. During the backup transfer process, the source volume is mounted to achieve volume level access using VMware s vstorage API. As a process independent of the backup transfer, data about the volumes files and directory structure is captured and recorded. Once the backup has been verified as successful, the catalog data is activated and available for searching. If there is a backup failure, the catalog data is deleted. Catalog data is also deleted when the corresponding savepoint is purged by retention policies. Catalog collections are disabled by default, as there may be a slight performance impact. Collection must be enabled globally through vranger s Tools>Options menu. Collection must also be enabled on a per-job basis. Once enabled, if catalog collections are disabled in the future, the catalog data will be retained in line with retention policies. In order to get the best performance and reliability from the cataloging function, the following recommendations are made: The catalog database can grow quite large. Please give careful consideration to the location of the database installation. If the default SQL Express installation is used, the catalog database will have a size limit of 4 GB.Please refer to Sizing the Catalog Database on page 78 for sizing information and recommendations for the cataloging database. Take advantage of the catalog filtering option to avoid recording data for system and program files. This will greatly reduce the number of files cataloged per backup, which will improve performance and limit database growth. For more information on filtering, see Filtering Catalog Collections on page 86. vranger Repositories Designed for ease-of-use in recovery operations, repositories eliminate the need for countless backup locations and endless configurations. With vranger, you can configure a repository once, and use it forever. Repositories can be either in the CIFS, FTP, SFTP, or NFS (version 3) format. VMware VMFS does not support SFTP, so the SFTP repository must be located on a separate Linux or Windows server running the SFTP service.

30 vranger Overview 30 Repository Structure A repository is essentially a directory on a supported file system that vranger uses to store savepoints (backup archives). When viewed from outside vranger (through Windows Explorer, for example), a repository consists of a configuration file (GlobalManifest.metadata) and root level directories for each protected object. Any time you add a repository in vranger a GlobalManifest.Metadata XML file is created in the selected directory. It is the presence of that manifest file that tells vranger that a repository exists in that folder. Whenever you add a repository, that repository contains a global manifest XML file that provides details about the repository. As you do backups, an entry is placed in the global manifest file. The global manifest file is required when restoring a deleted repository. Inside the root level directory for each object are sub-directories created for each full backup of the object in question. Differential or incremental savepoints will reside within the full backup sub-directory. Inside the sub-directories are archives for every file protected during that task. Also in the sub-directory are two metadata files. The image below depicts a basic repository structure: The tables below describe the naming conventions for the repository files and folders. Root Level Directories - Naming Structure Name Structure <ObjectName>_<ObjectUUID> ObjectName The name of the protected object

31 vranger Overview 31 Root Level Directories - Naming Structure ObjectUUID A unique identifier for the protected object. In the case of a VMware ESX VM this would be its UUID.BIOS Full Backup Sub-directory - Naming Structure Name Structure <ObjectName>_<Date>_<Time>_<JobTemplate UUID> ObjectName Date Time JobTemplate_UUID The name of the protected object The date the task started in YYYYMMDD format. The time the task started, in 24 format HHMMSS. The UUID of the job template that the backup task was spawned from. Note This is not the same as the object UUID. Archive File - Naming Convention Name Structure <ObjectName>_<Date>_<Time>_<JobType>_<FileIndex>_<FileName> _<Extension> ObjectName Date Time JobType FileIndex FileName The name of the protected object The date the task started in YYYYMMDD format. The time the task started, in 24 format HHMMSS. A single character representing the type of job. F - for full backup, I - for incremental, D - for differential The index of the file in the backup order for the current task. This is not guaranteed to remain constant for the same input file between tasks. This is present to ensure there can be no filename collisions. The name of the input file minus extension

32 vranger Overview 32 Archive File - Naming Convention Extension The extension of the file. If the input file has no extension it should be set to an empty string Metadata File - Naming Convention Name Structure <ObjectName>_<Date>_<Time>_<JobType>.<Key> ObjectName Date Time JobType FileIndex The name of the protected object The date the task started in YYYYMMDD format. The time the task started, in 24 format HHMMSS. A single character representing the type of job. F - for full backup, I - for incremental, D - for differential The key portion for the metadata file. This value is either Manifest or VmConfig. Savepoints When a backup task runs, a savepoint is written to the specified repository. A savepoint consists of three files - the archive itself and two small XML files, the Manifest.metadata and the VmConfig.metadata. The savepoint manifest and the VmConfig.metadata file together are equivalent to the.info file used in vranger 3.x. Whenever you do a backup, the job writes a savepoint to the specified repository along with a corresponding savepoint manifest XML file. The savepoint manifest provides details about the backup job. The savepoint manifest is placed in the repository s folder for that job. The items in the blue box below represent one savepoint.

33 vranger Overview 33 If we assume an incremental backup job with a Threshold Count setting of 2, you will get a full backup savepoint and two incremental savepoints (the first savepoint of an incremental job is always a full backup). Each of the three savepoints will have their own savepoint manifest. These savepoints and their respective manifests will be placed in a single folder in the repository that represents the job. Repository Size There is no limit to the number of savepoints that can be stored in a vranger repository. There are, however, environmental limits on the size of a single directory. The available options, and their limits, are described below. Tip When sizing vranger repositories, allow for maintaining at least 5 GB of free space. Note The volume limitations described in this section are limitations within the Microsoft Windows environment. Default Configuration- A standard volume, with an MBR partition on a basic disk, has a limit of 2 TB. This is the default configuration for Windows Server In this configuration, the vranger repository cannot exceed 2 TB. Dynamic Disks - Dynamic disks contain dynamic volumes, including simple volumes, spanned volumes, striped volumes, mirrored volumes, and RAID-5 volumes. A repository located on dynamic disk volumes can be as large as 64 TB.

34 vranger Overview 34 For more information, see the Microsoft TechNet article: technet.microsoft.com/en-us/library/cc773268(ws.10).aspx GPT Volumes - GPT provides a more flexible mechanism for partitioning disks than the older Master Boot Record (MBR) partitioning scheme that has been common to PCs. GPT partitions are supported on Windows Server 2003, SP1 and later, and can reach up to 256 TB. For more information, see Note FLR will not work from a GPT Partition. The vranger Replication Process vranger includes an integrated replication based on the proven technology of vreplicator. There are two options for replication: standard replication using the service console; or replication via virtual appliance (required for replication to or from VMware ESXi). The replication process is the same for each method, except as noted. A VM is made up of a set of files, which means that replicating a VM is, in essence, replicating the set of files that make up the VM, with changes to these files that reflect user specified settings for the target VM. The set of files replicated by vreplicator is listed below: File Extension.vmx.vmxf.nvram.vmdk -flat.vmdk -delta.vmdk -ctk.vmdk Description The VM config file, one per VM The extended VM config file, one per VM The VM BIO file, one per VM The VM hard disk file, one per hard disk or snapshot The VM hard disk data file, one per hard disk or snapshot The VM snapshot data file, one per snapshot The VM hard disk change tracking file when CB is enabled on the disk

35 vranger Overview 35 During the replication process, the configuration files will be created and modified on the target server via the VMware API. A set of working files is also created and used during the replication process. These files and their purposes are listed below: File Extension.vzmap vzundo-script.vzundo.vmdk-abbt.vztemp Description Records data block offset and hash of files on the target VM. A.vzmap file is created for each of the files replicated at the end of the replication. The.vzmap file is used by the next replication pass to detect any data changes since the previous pass. It stays on the target VM as long as the job is still configured to run. Note These files are only used in Hybrid mode for VM disk -flat and -delta files. A script created by the replication process to roll back changes on the target VM in case of a replication failure. This file created in the target VM folder at the beginning of the replication process and removed at the end. This is a temporary file that records original data of changed blocks since the replication started. One for each file replicated. In case of a replication failure such as network failure, vzundo-script can be executed to restore files to their original state. These files are created by the replication process and removed when the job is completed. Active block filter file. One for each hard disk data file when ABT is enabled. It records active data block offsets for source VM disks. This file is used against the.vzmap file to figure out data blocks that need to be streamed to the target. It is created at the beginning of the replication process and removed when disk replication is completed.

36 vranger Overview 36.vmdk-abfg.vztemp -flat-map.vztemp.vzcid.vzmutex Change block filter file. One for each hard disk when CBT is enabled. It records changed block offsets for source VM disks. This file is only generated when CBT and ABT are both enabled. It will later be combined with the.vmdk-abbt.vztemp file into -flat-map.vztemp and removed. Disk data filter file. One for each hard disk when one of two situations are true: CB is enabled, or both AB and CB are enabled. It contains active/changed data block offsets that need to be compared to the.vzmap file at the target in order to figure out data blocks that need to be streamed to the target. It is created right before file replication starts and removed when file replication is completed Records target VM disk CIDs at the end of the replication pass. Lock file. Indicate ownership of the VM by the current Quest application. When replication is executed through the virtual appliance, handling of these files will change: Working files that only exist during the replication process shall be written to a temporary directory of the VA, since their existence is relevant to the BET on the VA Working files that exists across replication processes shall be written to their original location, since these are not truly temporary files and should be reference-able even when the job is modified to not use a VA The.vzmutex file shall no longer be checked similar to how it is handled in ESXi backup vranger support three modes of replication: Change Block Tracking (CBT), differential and hybrid, all with the option of Active Block Mapping (ABM). Note that there is no option for CBT replication. If CBT is enabled on a VM, differential repliction will query for the changed blocks without a scan.

37 vranger Overview 37 VM replication in general starts with replicating the source VM to the target host. Changes are applied to the target VM at user designated intervals to keep the target in sync with the source. Thus the key areas of difference among the replication modes are: Base target VM creation Detection and application of changes to the base target VM The rest of the section describes each replication mode in more detail. Differential/CBT Replication The flowchart below, and the steps that follow, describe the process for differential replication.

38 vranger Overview 38 Base target VM creation 1 Snapshot source VM 2 Capture source VM config 3 Create target VM based on source config

39 vranger Overview 39 4 Fill disk data and retain disk hash map 5 Persist target CID map 6 Register target VM 7 Fix network configuration on target VM 8 Delete source snapshot Subsequent replication pass 1 Verify CID in sync 2 Snapshot source VM 3 Capture source VM config 4 Update target VM based on source config 5 Compare source disk hash with vzmap on target 6 Transfer changed blocks over to the target disk and retain new target hash 7 Update CID map 8 Refresh target VM 9 Fix network configuration on target VM 10 Delete source snapshot Hybrid Replication The flowchart below, and the steps that follow, describe the process for hybrid replication. Note You should only use Hybrid Replication if CBT is not an option. Replication with CBT is the preferred method. Hybrid replication is not supported when using the vranger virtual appliance.

40 vranger Overview 40 Initial pass Hybrid replication is also known as snapshot rotation replication. It makes use of the differential engine for its initial pass and leaves the source snapshot open after the initial pass for snapshot rotation.

41 vranger Overview 41 Subsequent passes 1 Verify CID in sync 2 Add new rotational snapshot to source VM 3 Snap target VM 4 Capture source VM config 5 Update target VM based on source config 6 Replicate rotational snapshot from source to target 7 Delete target snapshot 8 Delete old rotational snapshot from source 9 Update CID map 10 Refresh target VM 11 Fix network configuration on target VM (this step merges into step 5 in FMO) Switching from Differential to Hybrid Since Hybrid replication depends on an open snapshot at the source VM which is not created by differential replication, when a replication job is changed from Differential to Hybrid, the first replication after the job configuration change should still be differential on the back end. However, the source snapshot is not removed at the end of the replication pass in order to prepare for the next replication pass. Switching from Hybrid to Differential The initial pass after the job configuration change should delete the rotational snapshot created by the previous hybrid pass and reopen a new one to perform a full replication.

42 Deploying vranger This section provides information on configuring your environment and deploying vranger. This portion of the document contains the following sections: The Deployment Environment...44 Environment Configuration Recommendations...54 VMware Recommendations...59 Common Configuration Issues...71

43 The Deployment Environment This chapter outlines the recommendations and best practices for configuring your virtual environment with vranger. This chapter contains the following sections: The Deployment Environment...44 Environmental Requirements...44 Choosing a vranger Deployment Architecture...47 Small Environments (1-30 VMs)...47 Mid-Size Environments ( VMs)...49 Large Environments (100+ VMs)...52 Environment Configuration Recommendations...54 Recommendations for Network Backups...54 Recommendations for Fibre Backups...55 Recommendations for Replication...56 SAN Setup Recommendations...59 VMware Recommendations...59 vranger Repositories...68 Common Configuration Issues...71

44 The Deployment Environment 44 The Deployment Environment The performance and reliability of vranger depends, in large part, on the configuration of the environment in which it is used. Please use these configuration recommendations and guidelines to ensure proper operation of vranger. Environmental Requirements There are some configurations required to be in place before vranger will function properly. Port Requirements The diagram below depicts the ports used by each of the vranger components. Caution Replication with vranger requires an SSH tunnel between the production site and D/R site. Port forwarding is not supported with vranger replication. Note A continuous connection between source and target sites is required when replication is taking place. Excessive network packet loss could result in replication failure. Replication

45 The Deployment Environment 45 will work with links having average packet loss of less than 2%. Replication is not designed to work in replication environments where packet loss can exceed commercially accepted limits. Networks having an of 99% Uptime/Availability will generally provide for good Replication performance. VMware Requirements The table below lists the versions of VMware platform components supported by vranger. vranger supports backup, restore, and replication operation against the following versions of VMware Infrastructure: ESX(i) Server* vcenter Server VCB Update Update Update Update Update Update Update Update Update Update Update Update Update 1 VCB integration is not supported in this release. * Support for ESXi requires VMware s vstorage API. The vstorage API is not available with the free version of ESXi Server. Please be sure to have the appropriate platform licensing level before attempting ESXi backups. * ESXi replication requires the use of the vranger virtual appliance. Caution vranger only supports VMware ESX(i) Servers running in English. No other language packs are supported. Other VMware Requirements Licensing - Every ESX/ESXi host for which vranger is expected to provide protection must be properly licensed, both by VMware and in the vranger Host Licensing tab. While vranger can restore to a host for which you have not

46 The Deployment Environment 46 purchased a vranger license, the application does not support the free versions of VMware ESXi. Storage Requirements Dynamic Disks - While not a requirement, several vranger features (most notably File-Level Restore) are not compatible with dynamic discs.

47 The Deployment Environment 47 Choosing a vranger Deployment Architecture Choosing the correct architecture for your vranger deployment is the single most important decision you can make in order to optimize the speed of your backups and restores. Choose the fastest option available given your network and storage considerations. Small Environments (1-30 VMs) Smaller environments will typically deploy a simple, network-based architecture using local storage or a small array. The image below depicts the logical deployment of vranger in a typical small environment. The key elements of this deployment are identified by numerals in white circles. The numbers in the image correspond to the descriptions below. 1 The vranger machine - in a smaller environment, this can be a VM or perhaps an administrators workstation. In a network-based deployment, no backup traffic flows through the vranger machine.

48 The Deployment Environment 48 2 The vcenter Server - while vranger can connect directly to an ESX host, only editions of VMware Virtual Infrastructure or vsphere that include vcenter are supported. It is not recommended to install vranger on the vcenter Server. 3 VMware ESX hosts - in the network deployment shown above, backup data is sent directly from the ESX host to the repository. Processing functions (compression, block comparisons, etc) occur in the Service Console of the ESX Server hosting the VMs being backed up. 4 vranger Repository - Repositories are a supported storage device located somewhere in your environment, and configured as a repository within vranger. Backup data flows directly from the ESX hosts to the Repository- bandwidth to the repository is a significant factor in determining the maximum number of simultaneous backups jobs. 5 D/R Location - in order to protect business operations in the event of a localized failure, business critical VMs should be replicated to an offsite D/R location. This may be a remote office, but for smaller deployments will most likely be capacity rented from a service provider. Note D/R locations must be connected to the production environment via an IP tunnel for successful replication. For more information, see Recommendations for Replication on page 56. Network Based Architecture This is the default option out of the box, as it will apply to most configurations. In this configuration vranger is typically installed in a VM. The table below lists the key benefits of this configuration, along with some important considerations. Benefits Communications via Service Console NIC Easiest method to install. Works this way out-of-the-box Allows for leveraging Direct-to- Target Sufficient throughput for small environments vranger can be installed in a VM or on a Physical server Considerations Performs better with a larger number of ESX servers that are less VM/data dense SC NIC is a limiter of throughput Works well for small environments or deduplication environments

49 The Deployment Environment 49 Mid-Size Environments ( VMs) Mid-sized environments offer more complexity as the number of ESX hosts and VMs increases and multiple storage configurations are used. In a mid-sized environment, backup traffic is typically isolated to avoid impacting the production network. Figure 1 The key elements of this deployment are identified by numerals in white circles. The numbers in the image correspond to the descriptions below. 1 The vranger machine - in a mid-sized environment, this should be a virtual machine with access to the datastores for any VMs it will be configured to back up. Many environments of this size will include both VMware ESX and ESXi servers. With this configuration, vranger can leverage VMware s HotAdd functionality to speed network backups of ESXi servers. 2 The vcenter Server - while vranger can connect directly to an ESX host, only editions of VMware Virtual Infrastructure or vsphere that include vcenter are supported. It is not recommended to install vranger on the vcenter Server. 3 VMware ESX(i) hosts - in the deployment shown above, backup data is isolated to a separate backup network. If ESXi hosts are to be backed up, this traffic will be routed through the vranger machine to the repository.

50 The Deployment Environment 50 4 vranger Repository - Repositories are a supported storage device located somewhere in your environment, and configured as a repository within vranger. Backup data flows directly from the ESX hosts to the Repository- bandwidth to the repository is a significant factor in determining the maximum number of simultaneous backups jobs. 5 D/R Location - in order to protect business operations in the event of a localized failure, business critical VMs should be replicated to an off site D/R location. This may be a remote office, or capacity rented from a service provider. Note D/R locations must be connected to the production environment via an IP tunnel for successful replication. For more information, see Recommendations for Replication on page 56. Mid-sized environments will typically use a combination of backup methods based on server or workload configuration. For information on network based backups, see Network Based Architecture on page 48. For information in iscsi and VMware ESXi backups, please see the sections below. iscsi Backups This is a high performance configuration that requires that vranger is installed on a physical or virtual server with access to the target datastores. The table below lists the key benefits of this configuration, along with some important considerations. Benefits Communications via vstorage API to the ESX server and the iscsi initiator to the target. Performs very well. Considerations VERY IMPORTANT: Do not initialize the VMFS volume from the backup server. This will wipe out your VM data. ESXi Backups vranger includes support for VMware's HotAdd disk transport functionality which enables faster backup speeds for network backups of ESXi.

51 The Deployment Environment 51 The table below lists the key benefits of this configuration, along with some important considerations. Benefits Target VM disks are mounted directly to the vranger VM using VMware's I/O stack rather than through the network. vranger can be installed in a VM. Backup processing (whitespace removal, compression) done before data is sent over the network. Considerations No direct-to-target. vranger cannot be installed on a physical server.

52 The Deployment Environment 52 Large Environments (100+ VMs) Larger environments are the most complex to configure, with a large number of VMs and storage devices, and often multiple repositories. The preferred architecture for larger environments is most often a LAN-Free configuration over fibre. Figure 2 The key elements of this deployment are identified by numerals in white circles. The numbers in the image correspond to the descriptions below. 1 The vranger machine - in a larger environment, this should be a dedicated physical server. Many environments of this size will include both VMware ESX and ESXi servers. Backup traffic for ESXi backups must flow through the vranger machine as there is no Service Console to process backup operations. 2 The vcenter Server - while vranger can connect directly to an ESX host, only editions of VMware Virtual Infrastructure or vsphere that include vcenter are supported. It is not recommended to install vranger on the vcenter Server. 3 VMware ESX(i) hosts - in the deployment shown above, backup data is isolated to a separate backup network. If ESXi hosts are to be backed up, this traffic will be routed through the vranger machine to the repository. 4 vranger Repository - Repositories are a supported storage device located somewhere in your environment, and configured as a repository within vranger.

53 The Deployment Environment 53 Backup data flows directly from the ESX hosts to the Repository- bandwidth to the repository is a significant factor in determining the maximum number of simultaneous backups jobs. In larger environments, multiple repositories may be desired. 5 D/R location - in order to protect business operations in the event of a localized failure, business critical VMs should be replicated to an off site D/R location. This may be a remote office, or capacity rented from a service provider. Note D/R locations must be connected to the production environment via an IP tunnel for successful replication. For more information, see Recommendations for Replication on page 56. Larger environments will typically use a combination of backup methods based on server or workload configuration. For information on Fibre Channel backups, please see the section below. For information on the other types of backup architecture, please see: Network Based Architecture on page 48 iscsi Backups on page 50 ESXi Backups on page 50 Fiber Channel Backups This is a high performance configuration that requires that vranger is installed on a physical proxy server connected to your fiber network. The table below lists the key benefits of this configuration, along with some important considerations. Benefits Communications via vstorage API to the ESX vranger must be installed in a physical proxy server Backups are isolated to the fiber channel infrastructure Performs very well. Considerations Performs best when backup from LUNs to other LUNs in the same fiber infrastructure VERY IMPORTANT: Do not initialize the VMFS volume from the backup server. This will wipe out your VM data.

54 The Deployment Environment 54 Environment Configuration Recommendations In order to maximize the performance of vranger, and safeguard your production network, the following recommendations should be considered. Recommendations for Network Backups vranger, by virtue of it s Direct-To-Target architecture, pushes a lot of data through the network very quickly. While this performance is good for minimizing your backup window, if not configured properly it can degrade your production network. An important best practice is to separate the backup traffic from the production network by configuring a dedicated backup network. Note This approach requires that each ESX host and the vranger machine have two NICs installed. 1 Using the first (primary) NIC, connect the vranger server, the vcenter Server, and Service Console 1 of each ESX Server host. This becomes the primary production network for VM traffic. 2 In each ESX Server host, configure a second Service Console, connecting it to a dedicated NIC.

55 The Deployment Environment 55 3 Using the second NIC, connect: the vranger server; each Service Console 2; and each repository. This becomes the dedicated backup network. Note For the backup network, fibre NICs are preferred and will provide better throughput than 1Gb/sec Ethernet nics; NIC Teaming NIC Teaming is a feature of VMware Infrastructure that allows you to connect a single virtual switch to multiple physical Ethernet adapters. To utilize NIC teaming, two or more network adapters must be uplinked to a virtual switch. The main advantage of NIC teaming is increased network capacity for the virtual switch hosting the team. When bonding NICs into a team, it is important to use NICs from the same vendor as different NIC vendors achieve bonding differently. When using teamed NICs with vranger, it is critical that the NICs are teamed for performance rather than load balancing. vranger backups are streamed as a continuous file - changing NICs during a data stream will cause backup errors. For more information on NIC teaming, please refer to the VMware Knowledge Base article : Recommendations for Fibre Backups vranger uses VMware s vstorage API for fibre (LAN-Free) backups through your storage infrastructure. You must be using a version of VMware ESX(i) that supports vstorage API in order to use LAN-Free backups. vranger requires the use of physical backup proxy servers to take advantage of LAN- Free backups. Use the requirements and recommendations in the sections below to guide you in the proper configuration of your backup proxies. Proxy Requirements The backup proxy server must be a physical machine running one of the supported operating systems below. Depending on the size of your environment, more than one proxy may be required. Windows Server 2003 SP2 (x86, x64) Windows Server 2008 SP2 (x86, x64) Windows Server 2008 R2 (x64) The backup proxy must use Fibre Channel HBAs to connect to the SAN on which virtual machines reside.

56 The Deployment Environment 56 It is recommended to use two HBAs - one for read operations and one for writing. vranger must be installed on each proxy server. Sizing Recommendations When planning your proxy configuration, you must evaluate a number of factors and identify which are likely to become bottlenecks in your environment. Factors that you should consider when sizing your deployment. The number of virtual machines in your environment is the main factor that determines the size of your backup infrastructure. If there are a great many virtual machines, you must either allow longer backup times or allocate a significant amount of resources to meet time constraints. As a starting point for testing your deployment, Quest recommends using 1 backup proxy for every 10 hosts or 100 VMs. A factor coinciding with the number of VMs is the number of VMs that can be processed at once. vranger includes the Maximum number of tasks running locally configuration option that will limit the number of simultaneous backups per proxy server. As a starting point, plan on 1 concurrent task per CPU core. For example, if your proxy server has 8 cores, you should set the Maximum number of tasks running locally setting to 8. Recommendations for Replication The sections below describe the configurations recommended for best replication performance. Choosing a D/R Site A disaster recovery site can be cold, warm or hot. Each type of site has different recovery times and different corresponding infrastructure requirements and costs. A Cold, or Offline, site consists of powered off servers. A cold site is generally the most affordable DR strategy, and provides the highest RTO of any of the site types. In a DR event the servers are booted up and backup images are restored to them. Replication is not applicable to a cold site, but backups can be used in this situation. A Warm site holds a number of ESX hosts, powered on. Network communication is fully enabled between the production site and the DR site. The ESX hosts hold powered

57 The Deployment Environment 57 off replicas of production VMs, ready to be booted up at a moment s notice. With a warm site, no production services are running. Similar to a warm site, a Hot site consists of powered on ESX hosts. In addition, a hot site includes running production services. for example, Active Directory Domain Controllers, DNS servers and File Servers. A hot site usually results in the fastest time to recovery as many essential infrastructure services are already running, reducing the time required for business applications to come online. When implementing a hot disaster recovery site, you can choose to use a combination of replication tools native to certain infrastructure applications as well as vranger for other applications. For example, Active Directory, DNS, Exchange 2007, SQL Server and many other applications include native replication functionality. When deciding whether to use vranger or native replication functionality, consider the following questions: Are there any extra licensing costs for using vendor native replication rather than using vranger to replicate to a powered off image? Will using multiple replication techniques for different applications complicate the DR plan for limited marginal benefit? It is usually beneficial to have live critical infrastructure components such as Active Directory and DNS available at a hot site to assist with a quick recovery. Network Connections to D/R Site Network architecture is a significant component of any disaster recovery infrastructure. Components to consider include bandwidth, link contention and latency. Correct sizing of a link for replication traffic is part science and part art. As a general rule of thumb, you should ensure you have sufficient bandwidth to handle replicated changes to the source virtual machines, any extra traffic that may utilize the link, and overhead for large data changes such as service packs and application upgrades. When sizing a link, ensure that all network traffic that will traverse the link is taken into account, as this may have an impact on replication times. Replication traffic is transmitted using the TCP protocol, and is susceptible to lower performance if transmitted across a link with a very high latency, such as a satellite link. Low latency links such as Ethernet, DSL and MPLS allow TCP to transmit data at its highest rate possible. If there is a significant amount of production network traffic also using the link, implement network quality of service (QoS). QoS will ensure that network traffic is correctly prioritized, granting all applications a fair percentage of the bandwidth

58 The Deployment Environment 58 available. vranger replication traffic can be identified for QoS as a transmission between ESX hosts on TCP port 22 (SSH). IP Addressing Probably the most complicated and most overlooked component of designing a disaster recovery solution is the IP Addressing. Designing the IP Addressing architecture correctly will allow your production site to communicate with your DR site, and, more importantly, will allow your client computers to communicate with your servers in the event of a disaster. A bridged network connection between sites will ensure the highest probability of a successful failover in the case of a disaster. With a bridged network connection, the Layer 2 network protocol is spanned between sites so a single IP address range can be shared between sites. This simplifies the disaster recovery failover process significantly and aids with any live failover testing. Changing IP address during a high pressure disaster recovery event always complicates the process. This introduces a potential for error and may lengthen the time to recover business operations at the DR site. Bridging the production and DR networks should always be your first choice. An alternative configuration is to use two virtual network cards for each virtual machine that you replicate. The dual virtual NIC approach complicates the architecture of the solution. It can, however, prove more cost effective in some situations. Both of the virtual network cards should be configured with IP addresses; a production virtual NIC with a production IP, and a DR virtual NIC with a DR IP. When running in production, NIC1 will have the production IP address and be connected to the production virtual switch on the production network. NIC2 will have the DR IP address, and will be connected a DR virtual switch disconnected from any networks. On the replicated virtual machine, the configuration will be the same except on the ESX host layer. The DR ESX host s Production virtual switch should be disconnected from the physical network, whereas the DR virtual switch should be connected to the network at the DR site. If choosing to implement the dual network card approach, ensure you comprehensively test a full site failover to ensure that all routing considerations are taken into account. Latency Limitations Excessive network packet loss could result in replication failure. Replication is not designed to work in replication environments where packet loss can exceed commercially accepted limits, summarized by the requirements below:

59 The Deployment Environment 59 Replication will work with links having average packet loss of less than 2%. Networks having an of 99% Uptime/Availability will generally provide for good Replication performance. SAN Setup Recommendations There are several SAN configurations that should be made before installing vranger on the proxy server. The multipathing software from your SAN vendor should be installed and configured for performance. LUNs that are not accessible to the proxy server should be masked. On your storage device, zone your LUNs so that the vranger HBA can see and read them. Only one proxy should see a set of VMFS LUN s at one time. The proxy server should have only read-only access to the LUNs. Disable automount on the vranger machine: From the start menu, select run and enter diskpart. Run the automount disable command to disable automatic drive letter assignment. Run the automount scrub command to clean any registry entries pertaining to previously mounted volumes. VMware Recommendations During standard backup operations, the ESX Service Console is used to run vranger backup tools. The additional performance load placed on the Service Console should be addressed by implementing the suggestions below. Service Console Configurations Quest recommends that the two changes below are made on your ESX hosts to optimize the regular backup of VMs. These ESX resource reservations are not mandatory and recommended only for operation efficiency. Increase the VIM CPU reservation ( MHz): 1 In the VI Client inventory, select the ESX host > Configuration tab > System Resource Allocation. In the System Resource Pools view, select VIM and click Edit Settings.

60 The Deployment Environment 60 2 Adjust the CPU reservation slider up to the equivalent of one core ( MHz). 3 Select Expandable Reservation and Unlimited. 4 Click OK to save. Increase the Service Console CPU Reservation to 1500 MHz 1 In the VI Client inventory, select the ESX host > Configuration tab > System Resource Allocation. In the System Resource Pools view, select Console and click Edit Settings. 2 Adjust the CPU reservation slider up to 1500 MHz.

61 The Deployment Environment 61 3 Select Expandable Reservation and Unlimited. 4 Click OK to save. Increase the RAM allocated to the Service Console to 800 MB. 1 In the VI Client inventory, select the ESX host > Configuration tab > Memory. Click Properties. 2 On the Memory window, enter a value between 256MB and 800MB for the service console parameter. Note: For troubleshooting purposes, VMware recommends that you increase the service console RAM to 800MB. 3 Click OK to save Note Changes do not take effect until the ESX host is rebooted. Enabling Changed Block Tracking In order to take advantage of CBT for backup and replication jobs, it must be enabled for each VM for which you will use it. CBT is only available on vsphere (ESX 4.x) hosts and requires that VMs be at hardware version 7.

62 The Deployment Environment 62 You can enable CBT on a per-vm basis via the vranger interface. If desired, you may also enable CBT on a large scale basis using a PowerShell script. For detailed examples see the vcommunity blog posting Bulk Managing CBT for VMs Creating a vcenter User for vranger vranger requires a vcenter account to function properly. To comply with security best practices, Quest recommends creating a vcenter user account with the minimum required permissions for vranger to use. The procedures differ slightly depending on which version of vcenter you are using. For vcenter 2.5, see the section below. For vcenter 4.0, please see To create a vranger vcenter account - vcenter 4.0 on page 64. To create a vranger vcenter account - vcenter Using the VI Client, navigate to Administrator>Roles. 2 Select Add Role. 3 Enter a name for the Role, such as vranger Non-Admin. 4 In the Privileges section, set the permissions according the table below: Section Global Datastore Host > Local Operations Virtual Machine > Inventory Virtual Machine > Interaction Virtual Machine > Configuration Privileges Log Event Licenses Browse Datastore File Management Create Virtual Machine Create Power On Power Off Device Connection Configure CD Media Configure Floppy Media Select all options in this section.

63 The Deployment Environment 63 Virtual Machine > State Virtual Machine > Provisioning Resource Create Snapshot Remove Snapshot Mark As Template Mark As Virtual Machine Allow Disk Access Allow Read-only Disk Access Allow Virtual Machine Download Allow Virtual Machine Files Upload Assign Virtual Machine To Resource Pool 5 Navigate to the Inventory view, and right-click the desired area to assign user permissions, such as Host & Clusters. Select Add Permission 6 Add and locate the desired user, select the newly created User Role and click OK.

64 The Deployment Environment 64 To create a vranger vcenter account - vcenter Navigate to Administration > Roles. 2 Select Add Role. 3 Enter a name for the Role, such as vranger Non-Admin. 4 In the Privileges section, set the permissions according the table below: Section Datastore Global Host > Local Operations Network Privileges Allocate Space Browse Datastore Licenses Log Event Create Virtual Machine Reconfigure Virtual Machine Assign Network

65 The Deployment Environment 65 Resource Virtual Machine > Configuration Virtual Machine > Interaction Virtual Machine > Inventory Virtual Machine > Provisioning Virtual Machine > State Assign virtual machine to resource pool Select all options in this section. Configure CD Media Configure floppy media Device Connection Power Off Power On Create new Allow disk access Allow read-only disk access Allow virtual machine download Allow virtual machine files upload Mark as template Mark as virtual machine Create Snapshot Remove Snapshot 5 Navigate to the Inventory view 6 Right-click the desired level to grant user permission, such as the main VC level. 7 Add and locate the desired user account, and select the recently created User Role

66 The Deployment Environment 66 VM Configuration for Replication The configuration of the source virtual machines (the VMs that are being replicated) can have an impact on replication performance and the success of the DR plan. vranger supports replicating virtual machines that have VMDK and virtual mode RDM virtual disks. Ensure that any disk volumes that must be replicated are either VMDK or virtual mode RDMs. Note Physical mode RDMs are not supported for replication. As vranger uses VMware snapshots for replication, ensure there is adequate storage for snapshots during the replication process and between replications (if using Hybrid replication). Snapshots are stored with the VMX virtual machine configuration file in the virtual machine s primary data store.

67 The Deployment Environment 67 To ensure best replication performance, ensure that disk intensive activities are not scheduled to occur within virtual machines during a replication. Examples of these activities can include anti-virus cans or traditional, file-level backups. Note As vranger replicates at a block level, disk intensive activities that make a large number of block level changes such as disk defragmentation may cause a large number of changes to be replicated by vranger and as such should be planned appropriately. Data consistency of the replicated virtual machine at the point in time of replication can assist with a quick recovery of transactionally consistent environment if a disaster is encountered. Different replication configurations are available for different platforms. Many database platforms support VSS, such as Microsoft Active Directory, Exchange, SQL and certain versions of Oracle. For more information, see Pre-seeding Replication Jobs on page 80.

68 The Deployment Environment 68 vranger Repositories Repository location, along with the configuration of jobs to those repositories, plays a significant role in the performance of vranger. Use the recommendations below to aid on planning your repository configuration. Tip When sizing vranger repositories, allow for maintaining at least 5 GB of free space. Repository Storage Devices Slow disk performance has been shown to negatively impact the backup performance of vranger. When configuring repositories, special attention should be paid to the type of storage devices used. The use of SAS (Serial Attached SCSI) disk drives are recommended. SAS drives typically offer a 30% performance improvement over SATA drives. The use of external USB drives or low quality NAS devices is not recommended. If these types of storage are used, the vranger configuration settings must be adjusted to accommodate the slow devices. Recommended configuration settings for slower repositories are shown below. These configurations can be made in the vranger Configuration Options dialogue, available on the Tools>Options menu. Maximum number of tasks running off a LUN = 3 Maximum number of tasks running off a host = 1 Maximum number of tasks running per repository = 2 If no errors are received with these settings, increment the tasks per repository value by 1 to find the best fit for your environment. Bandwidth to Repositories Backups and restores go direct-to-target, using a direct communication path between the ESX hosts and repositories. This allows for a high data throughput and a highly scalable backup engine. Without due consideration, however, this can also lead to network saturation and decreased backup performance. While performance varies based on environmental factors, data throughput during a single backup task can reach up to 100 MB/s. If we assume a standard case of a Repository connected via a Gigabit network, then as little as ten concurrent jobs can saturate the link to that repository.

69 The Deployment Environment 69 Although there is no ability to throttle data transmissions from an ESX host, vranger can limit the number of simultaneous backup tasks on a per-repository level. Note This configuration is a global configuration, meaning that it applies to all repositories. If you have need a repository located at a remote site, the WAN link will most likely become the limiting factor in job configuration. Rather than limit all vranger tasks to fit the limited bandwidth, it is recommended to install a second vranger application to be used to manage backups to remote repositories. On this second vranger installation, you may configure the per-repository limit to match the available bandwidth without affecting the local backup tasks. Testing Repository Connections In order to eliminate potential problems during backup jobs, you should test your repository configuration prior to configuring and running backup tasks. Test Connection Speed The connection between the VMware ESX host and the repository is critical for backup success. A simple ping test from the ESX host to the repository should provide an initial verification of connectivity. To Ping the Repository 1 Launch a TTY terminal emulator (PuTTY, for example). 2 Provide the FQDN of the ESX host. This will confirm that DNS is functional. 3 Login with root or other ESX service console credentials. 4 At the command prompt, type ping <repository-host-name> to confirm that DNS is working. 5 If the ESX host has multiple NICs, optionally specify the NIC on the same subnet as the repository: ping -I <interface-address> <repository-hostname>. 6 Record your response times. 7 Logout or press Ctrl+D to end the TTY session. Results The expected result is a response time around 1 ms (or less). Response times in the ms range usually indicate that traffic is crossing one or more routers or firewalls. Due to the excessive CPU/memory load that backup traffic

70 The Deployment Environment 70 will place on a router and/or firewall, it is best to avoid passing that traffic through them. Response times longer than 100ms may cause TCP/IP timeouts during backups. Usually, response times in this range indicate that the backup traffic will be crossing a WAN link. If this is the case, consider a separate vranger installation to manage these remote jobs. Test Repository Permissions To eliminate the possibility of ESX server repository combination problems, do a simple read/write test from the ESX service console to the repository by mounting a share on the ESX host and then using the Linux touch command to create new, empty files. To test permissions: 1 Confirm that the share has Full Control share permissions and at least Modify folder permissions. 2 Using SSH, connect to the ESX host. 3 Run the command mkdir /mnt/backup. 4 Run the command esxcfg-firewall -e smbclient. 5 Run the command mount -t smbfs //<repository-server>/<share> /mnt/ backup -o username=<useraccount>. Note For vsphere 4.0, replace smbfs in the above command with cifs. 6 If the mount fails, set the folder permissions to Full Control. or If the mount is successful, run the command touch /mnt/backup/testfile. If the touch command is successful, the ESX host should be able to perform a backup to the repository. 7 Run the command unmount /mnt/backup to clean up the mount.

71 The Deployment Environment 71 Common Configuration Issues There are configuration issues that can have negative effects on the performance of your VMware environment and your DR operations. It must be understood that the cumulative effect of having many of these small problems means major performance issues in your virtual environment. The good news is that these problems are easily identified with the right knowledge and tools and easily fixed once identified. The sections below list common issues and tips for each of the main configuration areas. Storage The three most common storage configuration issues are described below: Too many LUNS per Array Many types of storage are improperly configured as one large array. While this is the only available option for some iscsi devices, this configuration will not scale well in disk I/O intensive environments. In this configuration, you may find that your SAN becomes unusable from a practical standpoint long before you run out of storage space because a VM disk I/O will affect all other VMs. For more information on LUN configuration in a VMware environment, please refer to VMware s VMFS Technical Overview and Best Practices white paper. Wasted Space It is a common industry practice to over-allocate disk space to allow for data expansion. When planning, virtual administrators have to guess how much space to assign, with the primary consideration being How much is more than enough? The reality is that a few of the VM s will eventually use all of the space assigned, but most will not. All of these VMs contribute to WASTED SPACE and shorten the life of expensive storage devices.

72 The Deployment Environment 72 Mis-aligned Volumes VMFS partitions that are not aligned to 64kb boundaries suffer a performance decrease of approximately 7-12%. In addition, if you have VMs that are being replicated using block level replication techniques - and you properly align those VMs - you can reduce your replication bandwidth requirements by up to 50%. Incremental/differential space requirements can also be reduced by up to 50%. Note If your partitions were created using the vsphere (ESX 4.0) client, then they should have been automatically aligned to the 64kb boundary. For more details on partition alignment, see the VMware document Recommendations for Aligning VMFS Partitions. Network The two most common network configuration issues are described below: Understanding Default Load Balancing The default load-balance configuration in VMware ESX is NOT statistical load balancing. Instead, it is a round-robin approach to load balancing which can lead to one NIC much more heavily used than the others. The default option will not provide aggregate bandwidth to a single virtual machine. This will effectively function as active/passive with only 1 NIC being active. ESX supports alternative load-balancing methods such as 802.3ad link aggregation Broadcasts Broadcasts can crush your ESX server performance even when the total amount of broadcasts is low in terms of bandwidth. This can be resolved by the proper allocation

73 The Deployment Environment 73 of VLANs. When configuring your virtual network, make sure that server VMs are in VLANs that are isolated from user networks. User networks contain a large number of PC s that issue a lot of broadcasts, which slow server performance because of interrupts. Memory The following memory configuration issues are commonly found in VMware environments. Improper VM limits A limit is a restriction on the VM, so it cannot use more memory than this limit. VMs often have limits set that are much lower than their RAM allocation. If the limit is set lower than the allocated memory value, the ballooning driver will start to inflate as soon as the VM demands more memory than this limit. For more information on this topic, please see the blog article below: Low Amounts of Shared RAM ESX servers scan pages of RAM periodically, searching for duplicate pages. When duplicate pages are found, the ESX server uses a single, read-only copy of the page in RAM and shares it among the VMs that have that page in common. If a VM attempts to change a shared memory page, a new read-write copy is created in RAM for that VM. The others continue to use the shared RAM as long as they have it in common. Note that when an ESX server is booted, the common RAM is not instantly shared. The memory scanning process occurs at intervals over time - an overcommitted environment with a lot of shared RAM could still experience performance problems if multiple VMs are booted at the same time. Using the same OS and patch revisions in your VMs can significantly increase the amount of shared RAM in your ESX environment. This, in turn, allows you to use more VMs with the available RAM in your environment. CPU The most common CPU configuration issues involve incorrectly configuring multiple vcpus, which lead to a high % Ready value. This is described in more detail below.

74 The Deployment Environment 74 Multiple vcpu VMs and % Ready VMs don t reside on a processor all of the time- they are scheduled on demand onto a processor. Note that a VM cannot be schedule to a processor until all of the cores required for that VM are available simultaneously. Mixing single, dual and quad vcpu VMs on the same ESX server will lead to an increase in CPU % Ready that will severely affect VM performance. A % Ready measurement should never exceed 4%, and even at that rating you may experience performance degradation. To avoid this, reduce your VMs (where possible) to single vcpu VMs. If you have large amounts of VMs that require multiple vcpus, separating these VMs onto their own clusters improves scheduling. When performing P2V migrations, reduce low utilization servers to a single vcpu. To do this, you will have to: change the HAL (Windows) or kernel configuration (Linux) change this inside of the guest OS. Changing it at the VM setting level is not enough. If there is a mismatch between the number of CPUs the guest OS thinks it has and the setting on the VM in vcenter, the VM will idle at high CPU utilization levels.

75 Managing vranger Performance This section provides information on configuring vranger and your deployment environment for the best performance. This portion of the document contains the following sections: Workload Protection Considerations...77 Space Saving Technologies...85 Filtering Catalog Collections...86 Integrating vranger...89 vranger and Tape Backup Vendors...89 Application Backup & Recovery: Recovery Manager for Exchange...97 Using vranger and Microsoft PowerShell...98

76 Protection Best Practices This chapter outlines the hardware and software requirements for installing vranger. This chapter contains the following sections: Workload Protection Considerations...77 Configuring Backup Jobs...77 Configuring Replication Jobs...78 Load Balancing...84 Space Saving Technologies...85 Filtering Catalog Collections...86

77 Protection Best Practices 77 Workload Protection Considerations The main goal when configuring backup jobs is to minimize the backup window while protecting infrastructure performance. Backup performance may be affected by the available network bandwidth, available processing power on the ESX Servers, and by the I/O capacity of the configured storage devices. Use the available limiters (described below) in vranger to configure the maximum number of backup jobs that run on a given infrastructure component. Configuring Backup Jobs There are a variety of factors to consider when configuring backup jobs. The backup workload needs to be effectively planned to avoid over-taxing storage and network resources. vranger includes configurable limits for the maximum number of concurrent tasks on a per-lun, per-host, and a per-repository basis. You can also configure the overall number of concurrent backup tasks at the application level. vranger Job Limits The number of concurrent jobs that vranger can handle is largely dependant on the hardware configuration of the vranger machine. Per LUN Limits In order to avoid storage I/O contention issues, it is recommended to limit the number of jobs that can be processes based on their storage location. vranger defaults with a limit of 3 backup tasks per LUN. Per Host Limit This setting should be set fairly low as all of the backup processing (for standard backups) occurs on the host. The default per-host limit is set to 1. Due to the amount of CPU and memory used by the host to process backups, it is recommended to use caution when changing this value. Per Repository Limit While performance varies based on environmental factors, data throughput during a single VM backup task will routinely reach MB/s. For repositories on a Gigabit network, Quest recommends a maximum setting of 3 concurrent backups, which can consume 75% percent of the bandwidth to the repository. If you have need a repository located at a remote site, the bandwidth between sites will most likely become the limiting factor in job configuration. Rather than limit all

78 Protection Best Practices 78 vranger tasks to fit the limited bandwidth, it is recommended to install a second vranger application to be used to manage backups to remote repositories. On this second vranger installation, you may configure the per-repository limit to match the available bandwidth without affecting the local backup tasks. Configuring Replication Jobs vranger offers three different replication modes: CBT, Hybrid, and Differential. Each replication mode works in a particular way. Regardless of which replication method is chosen, only the changed data will be replicated on each replication pass. VMware vsphere 4.0 introduces Changed Block Tracking (CBT). CBT will track the disk block changes made by the source VM. CBT, once enabled on the source host, records the blocks that have changed since the last replication pass and transfers them to the target host without scanning the VMDK. CBT removes the need to scan for changed blocks, as vsphere keeps track of the changes. vranger will simply query for a list of changes, and replicate those blocks only. Tip If CBT is supported in your environment, this is the recommended replication method. Note that CBT must be enabled on a per-vm basis. In Differential replication, a differential block scan occurs to check for changes from the last replication pass each time vreplicator hits a replication interval. The differential scan speed varies based on host resources and disk speed, however, a good rule of thumb for the scan speed is approximately 1GB per minute. The entire source VMDK will be scanned. With differential replication, a snapshot is held open on the source VM during the replication pass only. Hybrid replication is based on snapshot rotation, with a fallback to differential scanning in certain circumstances. With hybrid replication, a snapshot is opened against the source VM to capture changes that occur to the VM without a block scan. On each replication interval, the snapshot containing the changed data is sent to the replica site, closed on the source side, and a new snapshot is opened to capture the next set of changes. Block scans are not done during a hybrid replication, except in the case where the destination VM state is altered (e.g. a failover test). Note Hybrid replication is not compatible with CBT. While the choice of Hybrid or Differential replication should be made for each VM based on the RPO, RTO, IO characteristics of the VM and other requirements, there are

79 Protection Best Practices 79 some general rules of thumb that assist with determining which method of replication to use for each VM: VM Size Replication Interval Recommended Replication Settings Small (< 20 GB) Medium- Large ( GB) Medium- Large ( GB) Very Large (500 GB+) Any Frequent (e.g., once per hour) Infrequent (e.g., once per day) Any Use CBT if your virtualization platform supports it. If not, consider using Differential. Small VMs are replicated very efficiently using Differential replication. Use CBT if your virtualization platform supports it. If not, consider using Hybrid. By using Hybrid replication, replication intervals can be very close due to the elimination of scan time. Use CBT if your virtualization platform supports it. If not, use Differential. Using differential replication ensures that snapshots will not grow throughout the day, as they would if using Hybrid. Use CBT if your virtualization platform supports it. If not, use Hybrid, and replicate frequently. Hybrid ensures the elimination of scan time, and replicating frequently ensures that snapshots do not grow to an unmanageable size. While large VMs can be replicated less frequently, ensure that you have an adequate replication interval to replicate the entire VM.

80 Protection Best Practices 80 Pre-seeding Replication Jobs Ranger replication is intended to replicate changes from a source VM to a remote target. It is often not practical to perform the first replication pass (which sends the full VM) to a remote site over a WAN link. You may use vranger to seed a replication job locally in order to reduce the amount of data sent over the WAN. The pre-seeding procedure differs based on the type of host you are using. Pre-seeding with ESX Hosts Pre-seeding to a remote ESX host requires the following actions: 1 Create and run a replication job for the VM that you wish to pre-seed. Instead of configuring the job with the final target host, select a local host as the target. This will generate the.vzmap file which is required by vranger to identify changed blocks. 2 Move the host to the remote location. or Move the entire target VM directory (including the.vzmap file) to the remote target host and register the VM. Do not power on the VM on the remote host - this will invalidate the.vzmap file. 3 Edit the original replication job to point to the new remote host. Copying the.vzmap file and using the original job allows vranger to continue replication without needing to send the entire image over the WAN. Pre-seeding with ESXi Hosts Pre-seeding to a remote ESXi host requires the following actions: 1 Create and run a replication job for the VM that you wish to pre-seed. Instead of configuring the job with the final target host, select a local host as the target. Make sure you replicate locally to a host using a virtual appliance. This will generate the.vzmap file which is required by vranger to identify changed blocks.

81 Protection Best Practices 81 2 Move the host and VA to the remote location. or Move the entire target VM directory to the remote target host and register the VM. Do not power on the VM on the remote host - this will invalidate the.vzmap file. 3 Move the VA to the remote host. or Copy the directory on the VA that contains the.vzmap file to the VA on the remote host. 4 Edit the original replication job to point to the VA on the remote host. Data Consistency vranger, by default, provides no quiescing during backups. Quiescing in vranger is provided by leveraging VMware Tools installed in the VM. This can provide three different levels of backup consistency, as described below: Crash-consistent - A crash consistent backup is analogous to pulling the plug on a server and then backing up the data. The state of the data that is being backed up with respect to the users of the data is indeterminate. Restoring a crash-consistent image is equivalent to rebooting a server after a hard shut-down. File System Consistent - File system consistency is achieved through standard quiescing (via the VMware Sync Driver.) which ensures that no file system writes are pending when the snapshot is taken. For normal VMs, file-system consistency is adequate, although it can cause corruption in database applications. Application consistent - Consistency of VSS compatible applications is achieved by freezing application I/O just prior to creating the VM snapshots. This ensures that all application writes requests in the machines memory are committed to disk before the snapshot is taken. Application consistency is achieved by leveraging the Microsoft VSS driver in VMware Tools (for VMware ESX 3.5 Update 2 and later, make sure it is enabled in the VMs by going to Add/ Remove Programs> VMware Tools >Modify>Check VSS Sync Driver).

82 Protection Best Practices 82 Guest Quiescing The level of consistency provided by the Enable Guest Quiescing option is dependent upon the version of VMware ESX (and the corresponding VMware tools) and the Guest OS. In addition to the standard VSS implementation using VMware Tools, Quest provides an optional method for application quiescing - vzshadow.exe. vzshadow.exe is an optional component that can be installed on Windows VMs to provide an additional level of consistency. The tables below provide more detail on what is needed to achieve various levels of consistency: File-level Quiescing ESX Version Windows Server 2003 Windows Server 2008 (includes R2) ESX(i) 4.0 VMware VSS VMware VSS ESX(i) 4.1 VMware VSS VMware VSS Note for VMs created on ESX(i) 4.1, you must also set the disk.enableuuid flag to "false" Application-level Quiescing ESX Version Windows Server 2003 Windows Server 2008 (includes R2) ESX(i) 4.0 VMware VSS vzshadow.exe ESX(i) 4.1 VMware VSS vzshadow.exe Note for VMs created on ESX(i) 4.1, you must also set the disk.enableuuid flag to "false" Truncation of Exchange Logs ESX Version Windows Server 2003 Windows Server 2008 (includes R2) ESX(i) 4.0 vzshadow.exe vzshadow.exe

83 Protection Best Practices 83 ESX(i) 4.1 vzshadow.exe vzshadow.exe Note for VMs created on ESX(i) 4.1, you must also set the disk.enableuuid flag to "false" Deploying vzshadow.exe: Note vzshadow.exe works in conjunction with VMware Tools VSS function. VMware Tools VSS must be enabled when using vzshadow.exe. 1 Install vranger. Tip Refer to the vranger Installation and Setup Guide. 2 Download and install the appropriate C++ Redistributable package on the same machine that hosts the VSS aware application you want to quiesce. For 64-bit (x64) machines, go to: downloads/details.aspx?familyid=bd2a6171-e2d B809-9A8D7548C1B6&displaylang=en For 32-bit (x86) machines, go to: downloads/details.aspx?familyid=9b2da534-3e a4d-074b9f2bc1bf&displaylang=en#RelatedLinks 3 The installed vranger software (version 4.5 and later) contains two files, one of which must be moved to the root directory of the machine that hosts vranger. The files can be found: C:\Program Files\Quest Software\vRanger\VSS 4 Depending on your system, move the appropriate file to the machine s root directory (typically the C:\ directory). vzshadow_x64.exe or vzshadow_x86.exe 5 Create a directory on the VM: C:\Program Files\VMware\VMware Tools\backupscripts.d\ 6 In this directory, create a batch file called freeze.bat. A sample batch file is shown below; c:\vzshadow.exe x:

84 Protection Best Practices 84 where X equals the drive that hosts the application -- if there are multiple volumes on this VM, list all volumes, separated by colon space (: ), as in: c:\vzshadow.exe c: d: e: When VSS is triggered by VMware tools, the pre-freeze script will execute vzshadow.exe. Changing the disk.enableuuid flag To change the disk.enableuuid flag: 1 Open the VMware vsphere Client, and log in to a vcenter Server. 2 Select Virtual Machines and Templates and click the Virtual Machines tab. 3 Locate the Windows 2008 virtual machine for which you are disabling the disk UUID attribute. Shut down the guest or power off the virtual machine. 4 After power-off, right-click the virtual machine. Choose Edit Settings. 5 Click the Options tab, and select General in the settings column. 6 Select Configuration Parameters. 7 Click Add Row. Note This row will already exist if the 2008 VM ware created on ESX(i) In the Name column, enter disk.enableuuid. 9 In the Value column, enter false. 10 Click OK. Click Save. 11 Power on the virtual machine. Load Balancing vranger will use the configured limiters and job scheduling information to balance the backup load across as many hosts as possible. For example, assume a simple scheduling scenario of 10 ESX hosts running 10 VMs each (for 100 VMs total). If you set limits of 1 backup job per host and 10 backup jobs per repository, vranger will process the backup tasks in parallel, ensuring that each host is operating at capacity. Note that VMs are not vmotioned for load balancing purposes. Assume, like the above scenario, we have a schedule with 10 ESX hosts running 10 VMs each. In this example, however, we have a small backup job of only 10 VMs, all of which happen to be on one

85 Protection Best Practices 85 ESX host. In this case, vranger will process the backup tasks serially because of the per host limit. Space Saving Technologies vranger offers the following options to reduce the storage footprint and network load of VM backups. Full Backups For full backups, vranger performs a full scan and copy of the source VM. Full backup jobs can be modified with the following operations: Active Block Mapping (ABM): ABM will scan the disks on a virtual machine, and detect the blocks being actively used by the disk, as opposed to blocks that have been deleted by the Windows operating system. With ABM enabled, vranger will only backup that part of a virtual disk that has active data on it. Caution Incremental Backups ABM does not backup deleted data. If a VM backed up using ABM is restored, undelete operations will not be possible. For incremental backups, vranger will perform a full scan of the source VM, but copy only blocks that have changed since the last backup. Incremental backups are typically the fastest backup option and consume less storage space per archive. It is important to note, however, that in order to restore from an incremental backup, each incremental archive between the full backup and the desired restore point must be available. When creating incremental backup schedules, use caution to minimize the risk introduced by long incremental chains. Restoring an incremental backup can take longer than restoring a full or differential backup, because each previous incremental archive needs to be restored as well. Incremental backups can be modified with the following operations: Active Block Mapping (ABM): ABM will scan the disks on a virtual machine, and detect the blocks being actively used by the disk, as opposed to blocks that have been deleted by the Windows operating system. With ABM enabled, vranger will only backup that part of a virtual disk that has active data on it. Caution ABM does not backup deleted data. If a VM backed up using ABM is restored, undelete operations will not be possible.

86 Protection Best Practices 86 Changed Block Tracking (CBT): Changed Block Tracking reduces the time needed for incremental and differential backups by only backing up the portions of a disk that have changed since the last backup. By determining which blocks changed within the VMDK file, vranger only backs up the portions of a disk that have changed since the last backup. This will often result in shorter durations for backup operations, and reduce resource consumption on network and storage elements. Note CBT will copy deleted blocks if ABM is not also enabled. Differential Backups For differential backups, vranger will perform a full scan of the source VM, but copy only blocks that have changed since the last Full backup. Differential backups are typically slower than incrementals and can consume more storage space. They are usually faster to restore, however, and require only one differential backup and the parent full backup to restore. Differential backups can be modified with the following operations: Active Block Mapping (ABM): ABM will scan the disks on a virtual machine, and detect the blocks being actively used by the disk, as opposed to blocks that have been deleted by the Windows operating system. With ABM enabled, vranger will only backup that part of a virtual disk that has active data on it. Caution ABM does not backup deleted data. If a VM backed up using ABM is restored, undelete operations will not be possible. Changed Block Tracking (CBT): Changed Block Tracking reduces the time needed for incremental and differential backups by only backing up the portions of a disk that have changed since the last backup. By determining which blocks changed within the VMDK file, vranger only backs up the portions of a disk that have changed since the last backup. This will often result in shorter durations for backup operations, and reduce resource consumption on network and storage elements. Note CBT will copy deleted blocks if ABM is not also enabled. Filtering Catalog Collections While there are thousands (or hundreds of thousands) of files in a typical VM, most are not relevant to file level recovery operations. In order to streamline cataloging operations, and reduce impact to the catalog database, vranger filters files to be indexed in two ways:

87 Protection Best Practices 87 Path - by default, vranger does not catalog any files in the directories listed below. Path filtering is determined by entries in the PathFilterTokens.txt file, located at C:\Program Files\Quest Software\vRanger\Service\Configuration. Program Files Windows $Extend $TxfLog $Txf RECYCLER System Volume Information I386 File - By default, vranger does not catalog files of the type below. File filtering is determined by entries in the FilesFilterTokens.txt file, located at C:\Program Files\Quest Software\CatalogManager\Config\Files..lnk $MFT $Volume $AttrDef $BitMap $Boot $BadClus $Secure $UpCase $Quota $ObjID $Reparse $RmMetadata $Repair $Tops $TxfLog Note File filtering applies to un-filtered paths. If a path is filtered, files in that path do not need to be. For most situations, the default filtering options will be sufficient. If you want to filter out additional paths or files, simply add the path or file to the appropriate text file.

88 Integrating vranger This chapter outlines the hardware and software requirements for installing vranger. This chapter contains the following sections: Integrating vranger...89 vranger and Tape Backup Vendors...89 Application Backup & Recovery: Recovery Manager for Exchange...97 Using vranger and Microsoft PowerShell...98

89 Integrating vranger 89 Integrating vranger Like any component of your virtual infrastructure, vranger can be used in conjunction with other applications to achieve more complex tasks. vranger and Tape Backup Vendors To configure the vranger sweep-to-tape feature, both vranger and the tape vendor s software must point to a common vranger repository folder located locally on the vranger server. The vranger backup administrator creates an on-demand (not scheduled), incremental job with a specific name, appending -Tape as a case-sensitive suffix (e.g., any-jobname- Tape). The -Tape suffix can be changed by editing the PowerShell script if you prefer something else. There should be a dedicated repository for each on-demand job. The tape software administrator schedules a recurring incremental backup job whose PRE field calls a.cmd file containing the PowerShell script that searches for and then starts the vranger on-demand job(s) with the -Tape suffix. The vranger on-demand job performs the incremental backup, placing the savepoint files in the repository and updating the repository's global manifest file. After the vranger job completes (e.g., the job status equals either success or failure), the PowerShell script exits, thus completing the Backup Exec PRE command. With the tape software s PRE command completed, the tape software, which is also pointing to the vranger repository folder, runs its scheduled incremental job, backing up the vranger savepoints and global manifest file to tape or whatever media server type is configured. Required Components The components required differ depending on the installation scenario. Scenario 1 - vranger & 3rd-party tape software on the same server Tape vendor s software on a Windows 2003 server. Note Note: a physical server will provide better performance than a VM. vranger resides on the tape software server. A local repository shared by vranger and the tape software.

90 Integrating vranger 90 PowerShell (PowerShell and vranger must always be installed on the same server). A PowerShell script that finds and launches the vranger on-demand job. A CMD file that contains the PowerShell script. Scenario 2 - vranger & 3rd-party tape software on different servers The tape software is on its own server. All other components, including a remote tape backup agent, reside on the vranger server. Note: Scenario 2 is not discussed in this document. Server 1-Backup Exec: Tape vendor s software on a Windows 2003 server Server 2-vRanger/PowerShell: Tape software s remote tape backup agent that can run a PRE command. The agent runs the PowerShell script contained in the.cmd file called via the PRE command vranger resides on its own server A local repository located on the vranger server and shared by vranger and the tape software PowerShell (PowerShell and vranger must always be installed on the same server). A PowerShell script that finds and launches the vranger on-demand job A CMD file that contains the PowerShell script. Configuration Steps Use the procedures below to configure integration of your tape software and vranger. Note Although the procedure below uses Symantec Backup Exec, the procedures should be similar for any tape vendor s software. These procedures assume that: You have your components installed according to Scenario 1 as described above. The customer already will have a Backup Exec physical server and vranger will be installed on that server.

91 Integrating vranger 91 Installing vranger 1 Install vranger onto your Backup Exec server. Consider placing the local repository on the D: drive of the vranger server. 2 Create the local share D:\vranger_backups (or similar). a b Assign the group Everyone (or similar) Full Control for the Share. The user credentials specified in the next step when adding the repository should have Full Control NTFS security. This folder will become the repository that contains the incremental job savepoints and global manifest file. The folder should have enough disk space to hold a full backup and several incremental backups. 3 Add a local repository on the vranger server that points to the vranger_backups share. Click My Repositories > Add > specify the necessary repository details, pointing to the share. 4 Create an on-demand, daily, incremental backup job that backs up your environment. In this paper, we will backup a single server for simplicity. Click: a b c d e f g h i My Inventory > Add > Backup Job > Advanced > Job Name = vranger- Sweep-to-Tape. Click Next. Select VMs to exclude. Click Next Select hard disks to include. Click Next. Select your options. Click Next. For Retention Policy, select Savepoint Count=7, Space Saving Technology=Incremental, Threshold Count=6. Click Next. If Backup Exec calls this on-demand job daily, there will be a full backup performed the first day and an incremental backup performed each of the remaining days of the week. Select the This will be an on demand job and does not require a recurrence schedule checkbox. Click Next. Click the vranger-sweep-to-tape repository to select it. Click Next. Select the address(es) that should be notified when the job has completed running. The a report after the job has finished running checkbox should remain checked. Click Next. Review your settings and click Finish.

92 Integrating vranger 92 Installing and Configuring PowerShell 1 From details.aspx?displaylang=en&familyid=10ee29af-7c3a c9c1dab6e2bf, download PowerShell 1.0 for Windows Server 2003 to the vranger server. 2 Install PowerShell. PowerShell appears in Add/Remove Programs as a hotfix: 3 Click Start > All Programs > Quest Software> vranger > vranger Console to launch the vranger Console. 4 Set the PowerShell Script Execution Policy to RemoteSigned. a b Set the PowerShell Script Execution Policy to RemoteSigned by running the cmdlet: Set-ExecutionPolicy RemoteSigned. Confirm the policy setting by running the Get-ExecutionPolicy cmdlet. c Exit PowerShell by running the Exit cmdlet. See for an explanation of the available PowerShell Execution Policies. 5 Edit the vranger config files with Notepad (or similar) to use the vranger server's local IP address instead of localhost. a Edit the vranger.api.exe.config file, located at: \Program Files\Quest Software\vRanger\PowerShell \vranger.api.exe.config. Change:<endpoint address=" VAPIHost.svc" binding="wsdualhttpbinding"

93 Integrating vranger 93 b To: <endpoint address=" VAPIHost.svc" binding="wsdualhttpbinding" Edit the vranger.api.powershell.dll.config file, located at \Program Files\Quest Software\vRanger\PowerShell \vranger.api.powershell.dll.config. Change:<endpoint address=" VAPIHost.svc" binding="wsdualhttpbinding" To:<endpoint address=" VAPIHost.svc" binding="wsdualhttpbinding" Using the PowerShell Script 1 Download the PowerShell script that will launch the vranger on-demand job. a b c Go to the URL and save the zip file to any folder (e.g., c:\download) on the vranger server: PowerShellScripts/vRangerTape.ps1.zip. Extract the script from the zip file by right-clicking the file and clicking Extract All. While you do not need to edit the script, you can open it with Notepad to understand why your vranger job name has to have the suffix -Tape (case sensitive). The PowerShell $JobString = "*-Tape" must match the suffix used in your vranger job name. As long as they match, you can change the $JobString and vranger job name suffix to something other than -Tape. d e Close the file. Using Notepad, create a CMD file in the C:\Download\vRangerTape.ps1 folder titled vrangertape.cmd. Its contents off c:\windows\system\system32\windowspowershell\v1.0\powershe ll.exe c:\download\vrangertape.ps1\vrangertape.ps1 Note Make sure that the vrangertape.cmd file has.cmd extension, not a.txt extension.

94 Integrating vranger 94 Configuring Backup Exec 1 Install and configure Backup Exec according to the applications instructions. 2 When configuring the Backup-to-disk folder type, select Backup-to-disk folder. Click Next. 3 At the Enter a name for the backup-to-disk-folder prompt, type vranger Daily and click Next. 4 At the Enter a path for the backup-to-disk-folder prompt, type or browse to D:\Backup_Exec Backups and click Next. 5 At the Enter the maximum size for a backup-to-disk file prompt, type a number of GBs (default=4). Note You can later change this number from within Backup Exec. 6 At the Allocate the Maximum Size for Backup-to-Disk Files prompt, click the Yes radio button and click Next. Pre-allocating the space improves performance. 7 At the Enter the maximum number of concurrent jobs prompt, type 1 (the default) and click Next. 8 At the Enter the size for the low disk space threshold prompt, type 0 (the default) and click Next. 9 Review your Device Configuration Wizard settings and click Next. 10 Click Finish. 11 Create a media set named vranger Daily Backups. Set the Overwrite Protection Period to 4 weeks. 12 Set the Append Period to 4 weeks. Create a Backup Job 1 Click on Create a backup job. This launches a wizard. a b c d At the Welcome to the Backup Wizard page, click Create a backup job with custom settings and click Next. At the Backup Selections page, drill down to and select the vranger repository e.g., D:\vRanger_Backups, and click Next. At the Select Volume Credentials page, click Test All. When the results are successful, click Next. At the Select Volume Order page, click Next.

95 Integrating vranger 95 e f g h i j k l m n o p At the Backup Names page, title the backup job vranger Daily Job, title the backup set vranger Daily Backup, and click Next. At the Backup Device and Media page, at the Which device would you use to backup your data drop-down box, select vranger Daily. At the Which media set would like to use to back your data drop-down box, select vranger Daily Backup. When finished making your selections, click Next. At the Backup Overwrite Method page, accept the default Append to media, overwrite if no appendable media is available radio button. Click Next. At the Backup Options page, click the Incremental - Changed Files - Reset Archive Bit method from the drop-down list. Click Next. At the Completing the Backup Wizard page, click the No, schedule the job to run later radio button and click Finish. At the Schedule Options page, click the Run according to schedule radio button and click the Edit Schedule Details button. At the Edit Calendar schedule by box, click the Recurring Days of the Month option and then click the Select All button. At the Edit Calendar schedule by box, click the Effective Date option and confirm that today's date is placed in the Make the schedule go into effect on drop-down box. At the Edit Calendar schedule by box, click the Time Window option and accept the defaults: Start no earlier than 11:00:00 PM and no later than 10:59:59 PM the following day. Click Submit. At the Completing the Backup Wizard page, click the No, schedule the job to run later radio button and click OK. Click Close to close the Getting Started with Backup Exec page. 2 Specify the vrangertape.cmd file in the Backup Exec job's PRE field. a b Close the Backup Exec Assistant window. Click on the vranger Daily Job and click Properties.

96 Integrating vranger 96 c In the vranger Daily Job Properties column, click Pre/Post Commands and type the PowerShell script path\command (e.g., C:\Download\vRangerTape.ps1\vRangerTape.ps1). d e Click Submit. At the Job Summary page, click OK. - Run Sweep-to-Tape 1 In Backup Exec, highlight the vranger Daily Job and click Run Now. Click Yes to confirm. 2 Click the Job Monitor tab and double-click the Active job. Restore From Backup Exec Restoring from Backup Exec is the reverse procedure of doing backups. 1 From within Backup Exec, restore from tape or disk to the vranger repository.

97 Integrating vranger 97 2 From within vranger, restore from savepoints, using the global manifest file or savepoint manifest files. Troubleshooting Sweep-to-Tape Sweep-to-tape has three main components. Each component should be tested individually. The vranger on-demand, incremental backup job. The PowerShell script contained in the.cmd file. The Backup Exec scheduled, incremental backup job. 1 From with vranger, manually run the vranger on-demand job and confirm that it works. 2 Run the PowerShell script from a CMD prompt. Manually cut and paste the executable statement into the CMD prompt and check for errors to troubleshoot further: c:\windows\system32\windowspowershell\v1.0\powershell.exe c:\download\vrangertape.ps1\vrangertape.ps1 3 Launch PowerShell a b Run the Get-PSSnapin command. vranger.api.powershell should be registered. Run the Get-Executionpolicy command. The execution policy should be set to RemoteSigned. 4 From Backup Exec, confirm that its job runs. If the job fails quickly, confirm that the Backup Exec services are using credentials with local administrator rights. If so, then the PRE command is probably failing due to a PowerShell issue. Application Backup & Recovery: Recovery Manager for Exchange Recovery Manager for Exchange (RME) provides object-level restore of individual messages, folders, and other objects from Microsoft Exchange databases which are protected in vranger backup images. Restoring an individual object is faster and easier than recovering the entire Virtual Machine image. For more information on how vranger can be used with Recovery Manager for Exchange, see the document Application-level Backup & Recovery with vranger Pro and Quest Recovery Manager for Exchange.

98 Integrating vranger 98 Data Domain and vranger Repositories Quest vranger and Data Domain appliances provide companies with a simple and efficient method for backing up and recovering VMware environments. vranger provides significant enhancements with regard to architecture, performance and communications over traditional backup solutions. Data Domain provides superior data de-duplication capabilities. The combination of vranger and Data Domain Technology will dramatically reduce your backup and recovery time for virtual machines on VMware ESX hosts. For more information about integrating vranger and Data Domain, see the document vranger Pro & Data Domain Best Practices. Using vranger and Microsoft PowerShell vranger includes a way to remotely connect to the vranger Service using Microsoft PowerShell. This allows an administrator to use PowerShell (and Virtualization EcoShell) to create a client utility that can interact with vranger from anywhere in the environment. Configuring the vranger Server To get started, you must make a few quick configuration changes on the vranger server. Step 1: Edit the Config File 1 Browse to the PowerShell directory in the vranger installation path. By default this is: C:\Program Files\Quest Software\vRanger\PowerShell 2 Using a text editor, open the file vranger.api.exe.config. 3 In the file, find the following line: <endpoint address=" binding="wsdualhttpbinding" 4 Replace localhost with the IP address of the vranger server. Save the file. Step 2: Edit the Client File 1 In the same directory (C:\Program Files\Quest Software\vRanger\PowerShell) find the file vranger.api.pwershell.dll.config.

99 Integrating vranger 99 2 Using a text editor, open the file and find the following line: <endpoint address= binding= wsdualhttpbinding 3 Replace localhost with the IP address of the vranger server. Save the file. Step 3: Restarting vranger Services 1 Restart the vranger services so that the changes may take effect. The easiest method is to simply restart the machine hosting vranger. If that is not possible, ensure that you restart each of the three vranger services: vranger API Service vranger FLR Service vranger Service Configuring the Remote Machine You may now copy the entire vranger PowerShell directory (C:\Program Files\Quest Software\vRanger\PowerShell) and move it to any system that has Windows PowerShell (Version 1 or Version 2) installed. Once you have the directory copied over, simply open up a standard Windows PowerShell prompt and navigate to the vranger PowerShell directory. Type the following command to launch the vranger console for the first time:./vrangerconsole.ps1 This command will register the proper components and allow you to add the Quest vranger PowerShell library into Virtualization EcoShell. This will also allow PowerShell to interact remotely with your vranger server to create a client-server architecture using the vranger Cmdlets. To import a library into Virtualization EcoShell: This procedure assumes that you have Virtualization EcoShell installed on the machine to which you copied the vranger PowerShell directory.

100 Integrating vranger From the Virtualization EcoShell console, click File>PowerShell Libraries. 2 Select the vranger.api.powershell entry. 3 Click OK.

101 Index 101 Index B backup process 18 dfferentials 86 incrementals 85 limitations 18 C configurations common issues 71 CPU 73 memory 73 network 72 storage 71 Fibre backups 55 network backups 54 SAN setup 56 task limits 76 VMware 59 D Data Domain 98 deployment reference architectures 47 disk.enableuuid 84 F File-level restore 28 limitations 28 K key questions 8 M Microsoft Exchange backups 97 P port requirements 44 PowerShell 98 proxy sizing 56 R reference architectures large 52 mid-sized 49 small 47 repositories 29 bandwidth 68 size 33 storage devices 68 structure 30 testing 69 using a Data Domain appliance 98 requirements licensing 45 platforms 45 port 44 restore process 28 S savepoints 32 T tape backups 89 teaming NICs 55

102 Index 102 V VMware vcenter permissions 62 vranger and Data Domain 98 and PowerShell 98 and Recovery Manager for Exchange 97 and tape backup vendors 89 application architecture 14 backup process 18 configuring limits 76 information sources 11 key questions 8 repositories 29 restore process 28