for Virtual Environments Data Protection for VMware Wolfgang Hitzler Technical Sales IBM Tivoli Storage Management hitzler@de.ibm.com 2011 IBM Corporation
Agenda Introduction for Virtual Environments 6.2 Demo 1
Traditional In-Guest Backup and Recovery Model Install a backup agent in the guest OS and manage like a physical server Disadvantages Deploying, managing, maintaining backup agent sprawl Impacts processor, memory, I/O resources on host Data movement limited to LAN VMware ESX / ESXi Server Hypervisor Virtual Machines Virtual Disk Volumes Backup Server Tiered Storage 2
Backup Archive Client VMware Backup Version 6.2.2 In-guest backup Yes On-host backup (ESX /Linux B-A) No 2 Off-host backup (file) 5.2 Yes Yes technology not available 5.3 Yes Yes VCB (experimental 1 via integration module) 5.4 Yes Yes VCB (via integration module) vstorage API Off-host backup (full-vm) technology not available VCB (experimental via integration module) VCB (via integration module) 5.5 Yes Yes VCB VCB (via integration module) 6.1 Yes Yes VCB VCB 6.2 Yes Yes vstorage API VCB VCB/vStorage API 6.2.3/4VE Yes No 2 vstorage API VCB/vStorage API 1 5.3 was certified with VCB and integration modules by VMware but not by IBM 2 VMware trend from ESX to ESXi means applications will not be installed on hosting machine in future 3
VMware Update Dec 2010 6.2.2 B/A Client - Full VM block level backup / restore (all VMware supported guest platforms) from B/A client running on Windows and integrated with vstorage API for Data Protection (VADP) 1Q2011 VE 6.2 Announced: February 22 http://www-01.ibm.com/common/ssi/rep_ca/8/897/enus211-028/enus211-028.pdf 6.2.3 B/A Client - Adds block level incremental backup (enabled with for Virtual Environments license) for Virtual Environments 6.2 - Data Protection for VMware Recovery Agent provides file restore and instant volume restore capabilities for Windows and Linux guest platforms 4
2011 IBM Corporation for Virtual Environment VMware Integration New TDP New for Virtual Environment component enhancing the Backup-Archive client support Support multiple recovery options from VM image-level backups by exploiting VMware s vstorage APIs for Data Protection including change block tracking Added Value Single source backup Incremental backup File-level recovery from any OS Near-Instant volume restore for VMware Windows centralized backup server* Tivoli Storage Manager Storage Pool vstorage API 1 Backup VM image full or incremental changes 2 New Mount image directly from disk storage pool, expose it locally or using an iscsi target interface New 3 Restore single file directly to guest (or any other target) New *Centralized backup server can be a physical or virtual machine 5
B/A Client: Full-VM Backup with Streaming and Changed Block Tracking C: /a SAN VM1 D: VM2 ESX/ESXi Server 1. client initiates snapshot of virtual guest volumes and backups control information (vsphere API) 2a. Full Backup: agent moves all used data blocks from snapshot (vstorage API for Data Protection) or 2b. Incremental backup (enabled with VE): agent moves changed blocks reported by VMware Changed Block Tracking vstorage Backup Server (physical or virtual machine) B/A Client 6.2.3 VMFS vmdk vmdk vmdk SAN Storage Subsystem Snaps VM1 Server 3.Client sends data via API to server 4. Client removes snapshot 6
for Virtual Environments 6.2 Centrally managed data protection for VMware hypervisors. Remove requirement for deployment of agent within each VM guest. Quick and efficient incremental block-level backups utilizing VMware's vstorage APIs for Data Protection and Changed Block Tracking (incremental and content aware backups) Automatic discovery and backup of newly created virtual machines Capability to offload backup workload to one or more vstorage backup servers Non-disruptive snapshot at the virtual machine image level to help minimize disruptions to virtual guests Multiple recovery options from single-source image-level backup Granular file level recoveries Near-instant volume recoveries Full virtual machine recoveries Integrate with capabilities Data reduction (compression and deduplication) to reduce network and storage capacity requirements server scalability and storage pool management LAN or SAN data paths to server Common / centralized approach for physical and virtual environment VMware ESX / ESXi Server Hypervisor Virtual Machines vstorage API (VADP) B/A Client DP for VMware Agent API Backup Server Virtual Disk Volumes Backup Server vstorage Backup Server 7
for Virtual Environments 6.2 Components vstorage Backup Server Machine where the B/A client is installed VM guest machine or off-host physical machine 6.2.3 B/A Client Installed on 1 to n vstorage backup servers vstorage backup server can host 1 to n B/A client instances. 6.2 Data Protection for VMware Recovery Agent Centralized file-level restore for Windows and Linux Individual VM guest file-level restore and instant volume restore for Windows and Linux 8
B/A Client: Off-host Backup Deployment The vstorage Backup server can be a physical server Offloads backup processing from ESX host High performance configuration that can utilize SAN data path Data path configurations VADP: LAN or SAN API: LAN or SAN VMware ESX / ESXi Server Hypervisor Virtual Machines vstorage API (VADP) API Backup Server Virtual Disk Volumes vstorage Backup Server 9
B/A Client: In-Guest Backup Deployment The vstorage backup server can be a virtual machine No additional HW needed Use ESX host resources for backup Data path configurations VADP: LAN or Hot Add API: LAN VMware ESX / ESXi Server Hypervisor Virtual Machines VADP API Backup Server Virtual Disk Volumes 10
B/A Client Block Level Backup with VADP Supported Environment Backs up all VMware supported guest-os platforms B/A client running on vstorage Backup Server: Windows 2003, Windows 2008, Windows 2008 R2 (32 and 64 bit) Server Levels: 5.5, 6.1 and 6.2 vsphere 4.0 and 4.1 (ESX/ESXi) and VI3 (ESX/ESXi 3.5) Recommend VMware vsphere 4 and 4.1 (ESX / ESXi) and VM s at HW level 7 (required to utilize Changed Block Tracking) VM data accessed via vstorage / vsphere APIs No staging area required on vstorage backup server Changed Block Tracking Incremental backups (with periodic fulls) No scan of guest OS file system Content aware backup only processes used areas of disks VMotion aware Complete full snapshot of live (running) virtual machine Windows utilizes VSS Pre / post snapshot support available (VMware tools) Note - applications can still use a guest level TDP installation 11
B/A Client Full VM Backup vstorage API 1. BA client uses vstorage API Data API contacts VMware (vcenter, ESX, etc) 2. Invokes vmtools on guest 3. ESX creates Snapshot No staging disk on proxy (re. VCB support) 4. B/A client captures configuration information 5. B/A client reads VMDK (full or inc) via VADP 6. B/A client sends backup data to server via API Multiple objects including control info for each VMDK Grouped on server in \VMFULL filespace Different filespace name on server from VCB backups 7. B/A client removes snapshot 12
B/A Client: Installing VADP Support 2011 IBM Corporation 1. Select Custom Install 2. Select VMware Backup Tools 13
B/A Client Configuration 2011 IBM Corporation Recommendation: Use Preference Editor in B/A Client GUI (i.e. dsm) Configure communication with vcenter / ESX VMFULLTYPE = VSTOR VMBACKUPTYPE = FULLVM VMCHOST: specifies where to connect VMCUSER: vcenter / ESX Administrator ID GUI preferences editor / command dsmc set password -type=vcb host userid password Auto discovery and backup of VM guests DOMAIN.VMFULL defines guest to process for full VM backup. Options include VMHOST, VMFOLDER, ALL-WINDOWS, ALL-VM VMLIST can be used to specify list of VM s based on VMware display name Management Class / Copy Group VMMC (optional use of VMCTLMC for tape configurations) Storage pool destination Retention policy based on full backups (i.e. associated incrementals are expired based on policy applied to full) : VEREXISTS and RETAINEXTRA Scheduling Separate schedules for full backup vs. incrementals 14
2011 IBM Corporation Example Node Definitions VMware Datacenter (DC1) = ESX1 and ESX2 Guest Machines VM1 VM2 VM3 vstorage Backup Server Server Disk storage pool Host Machine ESX1 DB2 Database VMFS DataStore VMware Target Node Name = DC1 Guest Machines VM4 VM5 VM6 dsm/dsmc node=dm1 asnode=dc1 vmchost=esx1 dsm/dsmc node=dm2 asnode=dc1 vmchost=esx2 B/A client instances running on vstorage Backup Server Organization of VM s in DB Filespaces for DC1 VM1 VM2 VM3 VM4 VM5 VM6 Host Machine ESX2 VMFS DataStore Recommendation Configure 1 VMware target node to map to 1 VMware Data Center For additional throughput, consider using more than one dsmc instance on a single vstorage Backup server. Use a separate dsmc instance to process each VMware ESX server or VMware folder 15
Example Security Definitions DC1 grant proxynode target=dc1 agent=dm1 DM1 grant proxynode target=dc1 agent=dm2 DM2 dsm / dsmc Full VM Backup / Restore: asnode=dc1 DC2 grant proxynode target=dc2 agent=dm3 DM3 dsm / dsmc Full VM Backup / Restore: asnode=dc2 16
Example Schedule Definitions DC1 DM1 DM2 dsmdm1.opt vmfulltype vstor vmchost=vcenter1 domain.vmfull vmhost=esx1 dsmdm2.opt vmfulltype vstor vmchost=vcenter1 domain.vmfull vmhost=esx2 Schedule Definitions One schedule for full backup and one backup instance / esx server One schedule for incremental backup and one backup instance / esx server Full VM Backup define schedule vmdomain vmschedfullesx TYPE=CLIENT ACTION=BACKUP SUBACTION=VM OPTIONS= -ASNODENAME=dc1 MODE=FULL STARTDATE=mm/dd/yyyy STARTTIME=hh:mm SCHEDSTYLE=ENHANCED DAYOFWEEK=SATURDAY define association vmdomain vmschedfullesx1 dm1, dm2 Incremental VM Backup define schedule vmdomain vmschedincesx TYPE=CLIENT ACTION=BACKUP SUBACTION=VM OPTIONS= -ASNODENAME=dc1 MODE=INC STARTDATE=mm/dd/yyyy STARTTIME=hh:mm SCHEDSTYLE=ENHANCED DAYOFWEEK=SUNDAY, MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY define association vmdomain vmschedincesx dm1, dm2 Client Definitions dsmcutil inst scheduler /name:" Client Scheduler1" /node:dm1 /password:dm1pass /clientdir:"c:\tsm\baclient" /optfile:"c:\tsm\dsmdm1.opt" /autostart:no /startnow:no dsmcutil inst CAD /name:" Client Acceptor1" /optfile:"c:\tsm\dsm.opt1 /cadschedname:" Client Scheduler1" /node:dm1 /password:dm1pass /autostart:yes dsmcutil inst scheduler /name:" Client Scheduler2" /node:dm2 /password:dm2pass /clientdir:"c:\tsm\baclient" /optfile:"c:\tsm\dsmdm2.opt" /autostart:no /startnow:no dsmcutil inst CAD /name:" Client Acceptor2" /optfile:"c:\tsm\dsm.opt2 /cadschedname:" Client Scheduler2" /node:dm2 /password:dm2pass /autostart:yes 17
2011 IBM Corporation Tape Configuration MBLK0000 Disk 0 MBLK0001 MBLK0002 JOB1 is full backup of VMDK JOB2 is incremental (only blocks in MBLK0000 and MBLK0003 changed) MBLKnnnn server Disk storage pool for meta-data B/A Client JOB1 MBLK0000.CTL JOB1 MBLKnnnn.CTL JOB1 BITMAP.DAT VMCTLMC DB2 Database JOB2 JOB2 MBLK0000.CTL MBLK0003.CTL JOB2 BITMAP.DAT Policy based on management class specification Client option (VMMC) or default MC Placement: copy group DESTINATION Retention: copy group VEREXISTS & RETAINEXTRA Best Practice Recommendation when using disk / tape - Set the MIGDELAY (e.g. 10 days) parameter for the disk storage pool to enable the majority of mount requests to be satisfied with snapshots that are on disk. Tape Storage Pool for data \vmname.ovf JOB1. MBLK0000.DAT JOB1 MBLKnnnn.DAT \vmname.ovf JOB2 MBLK0000.DAT JOB2 MBLK0003.DAT VMMC 18
Backup Find Files (Backup) - CBT status 2011 IBM Corporation 19
B/A Client: Full VM Recovery Full Virtual Machine Recovery (any supported Guest OS) 2011 IBM Corporation Server 1. Administrator launches B/A client to restore VM from vstorage backup server and selects recovery point. Note can use new name to clone VM 2. B/A client will coordinate the restore with ESX/ESXi server VMware ESX / ESXi Server Hypervisor Virtual Machines 20
Restore - FULL/INCR - VSTOR/VCB - Display name VSTOR - FULL/INCR display name VCB - FULL hostname 21
Query VM 2011 IBM Corporation dsmc query vm * -inactive 22
Data Protection for VMware Recovery Agent 2011 IBM Corporation Mount Quickly mounts a snapshot, as a partition of ISCSI target device from the server Mounts snapshots without actually restoring the files to guest Interfaces: GUI and CLI Instant Restore Restores volume to guest (non-os volume) Enables on demand access to data while the volume is being restored Interface: GUI only 23
Data Protection for VMware Recovery Agent: Installation Mount and Instant restore will be packaged as part of for Virtual Environments 6.2 Two separate installation packages Windows (32 and 64 bit) Linux (32 and 64 bit) Windows package installed on vstorage backup server (with 6.2.3 B/A client) or on Windows VM guest. Linux package installed on off-host Linux machine or on Linux VM guest. 24
Data Protection for VMware Recovery Agent: Installation 25
Data Protection for VMware: File Recovery File Recovery from Partition Mount (Windows and Linux) 2011 IBM Corporation Server Agent Deployment Linux and Windows install packages 1. In-guest for VM user initiated restore 2. Off-host centralized file restore (windows package typically on same machine as B/A client) 1. User launches Windows or Linux Data Protection for VMware Restore Agent to mount partition a. Enters node name for authentication with server b. Select VM, recovery point and disk c. Select partition mount / partition 2. Volume is presented to local system (if off-host mount, make volume available to VM guest) 3. User copies files (files restored from storage pool) VMware ESX / ESXi Server Hypervisor Virtual Machines 26
Data Protection for VMware: File Recovery File Recovery from iscsi Mount (Windows and Linux) 2011 IBM Corporation 1. User launches Windows DP for VMware Restore Agent a. Enters nodename for server authentication b. Selects VM, recovery point and disk c. Selects iscsi mount d. Enters iscsi target name and VM guest iscsi initiator name 2. From VM guest, user invokes iscsi initiator to discovery and log into iscsi target. 3. For Linux, user issues mount command (Windows can automatically discovery new target) 4. User copies files (files restored from storage pool) Server Agent deployment: Widows install package (typically on same machine as B/A client) VMware ESX / ESXi Server Hypervisor Virtual Machines 27
Data Protection for VMware: Instant Restore Instant Restore of Volume (Windows and Linux) 2011 IBM Corporation 1. User launches DP for VMware Restore Agent Authenticates with server Selects VM, recovery point and non-os volume 2. Volume restore background process initiated 3. Data restored on-demand based on access (while background restore in process) VMware ESX / ESXi Server Server 1. Agent deployed in VM guest Hypervisor Virtual Machines 28
Tape Restore Considerations 2011 IBM Corporation B/A Client Full VM Restore Physical tape or VTL Fully Supported DP for VMware Recovery Agent Mount Physical tape or VTL- Supported with limitations (re. tech doc being created to discuss details) DP for VMware Recovery Agent Instant Restore Not supported for physical tape or VTL 29
VMware Transport 2011 IBM Corporation VMware Transports(SAN, hotadd, NBDSSL, NBD) - VMVSTORTRANSPORT option Use the VMVSTORTRANSPORT option with the backup VM or restore VM command to specify the transport to be used with the vstorage API for Data Protection(VADP). The transport determines how VADP accesses virtual disk data. Valid transports include any order or combination of san, hotadd, nbdssl, and nbd separated by a colon. The first transport in the list that is available in the environment will be used. NBD or network based data transfer is the LAN transport and should be available in all environments. It is not necessary to set this option. The default value is to use the VADP order which is currently defined as "san:hotadd:nbdssl:nbd". This option is passed directly to the VADP API. Place VMVSTORTRANSPORT option in the client options file (dsm.opt). Some common examples: Current default order of transports when none is specified. VMVSTORTRANSPORT san:hotadd:nbdssl:nbd When SAN path is temporary unavailable fail the backup so as not to impact LAN. VMVSTORTRANSPORT san Disable the use of hotadd when running backup server inside VM. VMVSTORTRANSPORT nbdssl:nbd Select NBD even when NBDSSL is available for better performance. VMVSTORTRANSPORT nbd 30
Backup 31
Instant Restore Instant Restore allows users to start using applications on the same disk to which the volume is being restored, while the restore operation is still in process. (Note: The process can t be activated against system volumes) Typical Production Disk 1. Activate Instant Restore 2. Background Process restores blocks gradually 3. Write IOs are performed as usual 4. Read IOs from un-recovered areas create restore on demand 5. All other reads are performed as usual New Production Disk Guest Machine Server New Guest Machine 32
Off-Host File Level Restore (Windows) VMWare vsphere 4.0 Infrastructure Backup/Restore Off-Host Server Server ESX Server ESX Server B/A B/A Client Client Production Storage Storage Pool 33
Off-Host File Level Restore (Linux) VMWare vsphere 4.0 Infrastructure Backup/Restore Off-Host Server Server ESX Server ESX Server B/A B/A Client Client Linux Proxy Production Storage Storage Pool 34
In-Guest File Level Restore (Windows) VMWare vsphere 4.0 Infrastructure Backup/Restore Off-Host Server Server ESX Server ESX Server B/A B/A Client Client Production Storage Storage Pool 35
In-Guest File Level Restore (Linux) VMWare vsphere 4.0 Infrastructure Backup/Restore Off-Host Server Server ESX Server ESX Server B/A B/A Client Client Production Storage Storage Pool 36
In-Guest Instant Restore (Windows) VMWare vsphere 4.0 Infrastructure Backup/Restore Off-Host Server Server ESX Server ESX Server B/A B/A Client Client Production Storage Storage Pool 37
In-Guest Instant Restore (Linux) VMWare vsphere 4.0 Infrastructure Backup/Restore Off-Host Server Server ESX Server ESX Server B/A B/A Client Client Production Storage Storage Pool 38
Off-Host iscsi Target VMWare vsphere 4.0 Infrastructure Backup/Restore Off-Host Server Server ESX Server ESX Server I-SCSI B/A B/A Client Client Production Storage Storage Pool 39
40 2011 IBM Corporation
Trademarks and disclaimers Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries./ Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. The customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Prices are suggested U.S. list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features. Contact your IBM representative or Business Partner for the most current pricing in your geography. Photographs shown may be engineering prototypes. Changes may be incorporated in production models. IBM Corporation 2011. All rights reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. Trademarks of International Business Machines Corporation in the United States, other countries, or both can be found on the World Wide Web at http://www.ibm.com/legal/copytrade.shtml. 41