Online Disk-to-Disk Backup with Veritas NetBackup and VSS

Transcription

1 Online Disk-to-Disk Backup with Veritas NetBackup and VSS This solution guide describes how to configure Intransa Large Scale IP SAN storage System with Veritas NetBackup and integrate with VSS for best performance as destination for backups. It also compares the benefits and performance characteristics of backup-to-disk verses backup-to-tape implementations.

2 Intransa, Incorporated. All rights reserved worldwide. Unauthorized duplication or reproduction of this document by any means, including electronic, reprographic, or any other form without written permission from Intransa, Inc., is strictly forbidden. Use of this manual and the software program is subject to the terms of the License Agreement between you and Intransa, Inc. You should not use the software until you have read the License Agreement. By using the software program, you signify that you have read the License Agreement and accept its terms. For a complete explanation of the terms of this agreement, see the License Agreement booklet. Trademarks The Intransa logo, Intransa Storage System, StorControl, StorControl Management Tool, IntraStor, and the Simply Smarter logo are all registered trademarks of Intransa, Inc. Microsoft is a registered trademark of Microsoft Corporation. Windows is a trademark of Microsoft Corporation. Java is a registered trademark Sun Microsystems, Inc. Linux is a registered trademark of Linus Torvalds. Oracle is a registered trademark of Oracle Corporation. All other brand or product names are trademarks or registered trademarks of their respective companies or organization. All names of people and companies used for examples in this manual are fictional. Any similarity to names of real companies and real people is coincidental. Revisions Intransa reserves the right to revise this document and the information within at any time and as necessary without the obligation to notify any person of revision. Contact: Intransa, Inc Zanker Road Suite 200 San Jose, CA Phone: Fax: Web:

3 Contents 1. Executive Summary... 1 Overview...1 Intransa Disk-to-Disk (D2D) Backup Strategy...2 Conclusion Preface... 5 Introduction...5 Disk Backup Overview...5 Advantages of Backup-to-Disk...6 About Intransa...8 Intransa A brief profile...8 Intransa Approach to Solutions...8 About Veritas...10 Users...10 Other Useful Resources Veritas NetBackup Overview...12 About Advanced Client...14 Snapshots NetBackup Qualification Setup Getting Started...16 Assumptions...17 Configurations used in the Qualification Environment Configuring IP SAN Volumes Intransa Large Scale IP SAN Overview:...22 What is an IP SAN?...23 Solution Specific Configurations...24 Windows File System Tuning Installing Intransa VSS Provider Installing NetBackup Server and Client NetBackup Client Installation...58 Contents i

4 8. Testing Backup and Restore Defining Storage Unit...66 Defining Backup Policy...68 Verifying the Snapshot operations Performance Benchmarking Summary Conclusions...88 Summary Appendix A.VSS Concepts...90 A.1 What is VSS:...90 A.2 VSS Components...91 A.3 How VSS works...91 B. Overview to VSS and VDS Design...93 C. Volume Shadow Copy Service...95 D. Tape Backups: Limitations...99 E. Essentials of Veritas NetBackup Administration E.1 NetBackup Management E.2 NetBackup Configuration Wizards E.3 Disk Staging Storage Unit Considerations E.4 Types of Backup E.5 Configuring an Advanced Client Policy Reference and Acknowledgements References Acknowledgements Contents ii

5 1. Executive Summary In a 24x7x365 world with geographic distribution of users, even shrinking back up windows is becoming unacceptable, especially for mission-critical applications. Enterprise environments need up-to-the-minute data protection without sacrificing high availability. Especially in customer centric and service centric organizations, every minute of backup restore is important. In case of outage or disaster every minute of delay in putting back the operations translates into opportunity loss in terms of money, customers and image. In some cases it can be as serious as closing business. Traditionally backup was taken in tape cartridges. However off late, disk capacities have overtaken single tape cartridge capacities. There seems to be a lot of interest by organizations to go for near line storage by deploying disk to disk backup solutions rather than taking directly into tapes. This makes business and economic sense in the whole run. Overview Backup to disk solutions provide tremendous benefits compare to traditional backup-to-tape. These benefits are as follows: Faster recovery of mission critical data Faster restore from disk Greater reliability of the backup media Multiple backup and restore data streams to disk Backup-to-disk is emerging as powerful solution for faster recovery of missioncritical data and it is complementary to backup-to-tape for long term storage. Most of the applications today require frequent retrievals of recently captured data, as well as the need for backup of business critical data which is required to be preserved and retrieved quickly and efficiently. Today most of the leadingedge customers prefer using disk as the destination for nearline, faster backup as backup windows are shrinking and importance of data is increasing every day. 1. Executive Summary - 1

6 Advanced Technology Attached (ATA) disk technology provides greater benefits over traditional tape with the increasing performance and reducing cost of disk. Disk-based backups will not replace tape, but will definitely shift tape into an archival role. Both of these media will co-exist. Today, Intransa recommends adopting a backup-to-disk solution when performance and data reliability are the primary concerns of the customer. Intransa Disk-to-Disk (D2D) Backup Strategy Intransa has developed Large Scale IP SAN that is built around a set of modular components to deliver exceptional value, low total cost of ownership, high scalability, excellent performance, and built-in redundancy & reliability. These requirements define IntraStor architecture, our foundation for delivering block-level, distributed storage over IP-networks. Intransa IntraStor is a 3-tiered distributed architecture that fully leverages the ubiquity of IP infrastructure and the iscsi protocol to deliver intelligent, networked block-level storage. This offers the flexible scalability options in scaling up the storage capacity or storage throughput I/O performance or both. This solution of Disk-to-Disk backup is using Veritas NetBackup software and Intransa VSS Provider software. Intransa offers the flexibility in performing the backup from the snapshot. Intransa VSS Provider smartly takes care of creating snapshot of the production data volume during the backup operations and deletes the snapshot after completing the backup. This reduces the burden of backup from the Production data volume and keeping it free only for the production purpose. Intransa s scalable architecture offers the unique combination linear scalability in performance and capacity in pay as you grow mode. This paper gives an overview of how to deploy Disk-to-Disk Backup Solution on Intransa Large Scale IP SAN using Veritas NetBackup and Intransa VSS Provider. 1. Executive Summary - 2

7 The paper is lucidly divided into the following logical sections: Section 2 is Preface gives introduction and brief need of the Solution. It also covers brief profiles about Intransa and Veritas. Section 3 provides the overview of Veritas NetBackup Software, its architecture, basic functionality. Section 4 provides the overview of Veritas NetBackup Qualification configurations setup and brief specifications of the servers used in the qualification environment. Section 5 provides the overview of Intransa Large Scale IP SAN and step-by-step guide in configuring the IP SAN Volumes for the Disk-to-Disk Solution. Section 6 provides step-by-step instructions for installing and configuring the Intransa VSS Provider software for the Disk-to-Disk backup Solution. Section 7 provides step-by-step instructions for Installation and configuring NetBackup Server and Clients for the Solution using the snapshot feature. Section 8 provides brief about defining the Disk Storage Unit, Backup Policy for using the snapshot feature for the solution and also provides the steps for conducting the backup and restore operations Sections 9 provides the performance benchmarks for the backup & restore tests conducted. Section 10 provides summary of the solution and the conclusions of the tests. Section 11 is the appendix containing the additional information about the components used in the solution. Brief concepts of Microsoft Virtual Shadow Copy Service. Brief essentials of Veritas NetBackup Software, backup types and some useful guidelines and suggestions in configuring the solution. 1. Executive Summary - 3

8 Section 12 is about the references used for this solution document and acknowledging the teams involved in formulating this solution. Conclusion The performance characteristics are very encouraging. It shows that backup windows can typically shrink by 3 times or reduce back up times by 66 % compared to the fastest LTO based solutions in the market. Veritas NetBackup Software seamlessly gets integrated with the Intransa VSS Provider. This gives immense benefit for configuring the Disk-to-Disk Backup Solutions with Intransa IP SAN in using the automated snapshot feature. A D2D solution of the nature proposed will benefit the IT staff to increase the numbers of backups or simply ease the management and cost of backups. This also insures that the backups are always near online and easily accessible. 1. Executive Summary - 4

9 Introduction 2. Preface This solution guide explains implementing backup-to-disk technology with Veritas NetBackup and the advantages over traditional tape based backup. These implementations have been tested by Intransa Solution Engineering teams. This document contains information relating to configuring Veritas NetBackup and Intransa IP SAN, Intransa VSS provider to form the backup-to-disk solution. It also compares performance with the traditional tape backups. Intransa Solution Engineering teams performed tests using specific system configurations. Due to variations in hardware, software, compressibility of data system usage and other factors, backup rates achieved in some situation may be different than those shown herein. All testing performed by the Solution engineering teams used the most up-to-date backup software available at that time. Other versions of these applications any yield other results. Disk Backup Overview Traditionally, backup software was created to write to a tape device. Today, most backup software products also support write to disk, which means writing to disk file in a file system. The file system may be on a Windows, Netware or UNIX platform, depending on the backup server. Disk-based RAID configurations enhance data protection beyond what tape can provide. Backup and restore time using disk provides customers an advantage over using traditional tape. Disk drives are random access devices and can instantly start to transfer the files, whereas with tape, the tape must be loaded and then accessed, increasing the overall time. File system overhead and tape with on-board compression add more time lags. Under normal conditions, I/O will always go to physical disk. Performances are characterized by the physical disk rotational speed and seek time together with the file system characteristics. Using a larger file system block size or allocation unit size may improve performance of disk backups by using larger contiguous 2. Preface - 5

10 address space for files because the operating system allocates fewer groups of contiguous sectors. Benefits of IP SAN disk-based backups that are discussed in more detail in this solution document include: Random access characteristics of disk versus sequential access for tape simultaneous capability to read/write to disk. Multiple host data streams Disk performance compared to tape performance with compression. Aggregate performance RAW disk performance Advantages of Backup-to-Disk Traditionally, tape has been the backup medium of choice, due to its cost-per-mb advantages compared with disk. However the economics of disk narrowing that gap. The advantages of using disk over with backup solutions can be grouped into four major categories: Backup performance Restore performance Media reliability and data availability Overall IT efficiency. The following sections summarize these benefits. Backup Performance IP SAN storage systems based backup are much faster than tape drives. Some tape technologies respond to a minimal data stream by shoeshining or excessive positioning. Disks do not experience this behavior as they are inherently random access. 2. Preface - 6

11 Restore Performance Data Recovery from disks are much faster than tape. The difference can be seconds or minutes, verses hours with tape. Disk supports random and sequential access. Tape support sequential access only. This enables faster access to data files, improving overall performance. If data is on several tape cartridges, the following steps are required to restore the data: 1. Each tape must be mounted by the library (up to a minute per tape) 2. Tape must load (30 seconds to a few minutes ) 3. Tape must be positioned to the desired data ( an average access time is a few minutes ) 4. Tape must be rewound and unloaded ( 30 seconds to a few minutes ) 5. Load the next tape and repeat cycle. Time to first byte takes milliseconds for disk versus seconds to minutes to tape. Media Reliability and Data Availability Media-specific errors, including faulty tape media, are reduced. Disk system RAID protection prevents data unavailability or data loss in the event of a disk drive failure. Tape handing is reduced or eliminated. Maintaining a set of tapes from a tape library can be problematic and requires properly trained personnel. Overall IT Efficiency Disk does not require the tape handling /positioning and RAID protection makes it inherently more reliable. There is less need to perform frequent full backups. Fewer backups need to be performed, saving network and CPU load. Tape undergoes a technology shift every three years, so a conversion process from old to new media must be undertaken at that interval. Disk technology does not go through these types of transitions since the format of the data is not changed as it is with tape technology. New larger-capacity disk drive reduces floor space requirements compared with equivalent-capacity tape libraries. 2. Preface - 7

12 About Intransa Intransa A brief profile Founded in September 2000, Intransa is an innovator of enterprise-class, intelligent IP storage products and software. Based on technology that leverages the low cost and maturity of Ethernet technology, Intransa s IP SAN solutions dramatically reduce the cost and complexity of deploying scalable network storage. Intransa s products re-define network storage to deliver the first fullfeatured IP SAN at a price attractive to organizations. Intransa believes that network storage should be easy, dynamic and intuitive. By achieving these goals, Intransa is enabling business customers to focus their resources onto missioncritical projects to improve their business. Intransa s new IP SAN Storage System is a second-generation iscsi-enabled, networked storage solution that implements the IntraStor architecture. This solution offers low cost, block-level, networked storage that can be deployed anywhere on the IP network and managed centrally. The enterprise-class features, such as fail-over and load balancing, support the high-availability needs of online data. The policy-based virtual volume management and dynamic volume expansion capabilities, in conjunction with the Intransa s inexpensive disk storage pool, simplify capacity management and improve capacity utilization for online, nearline, and backup/restore staging applications. Intransa s snapshot capability allows many point-in-time virtual copies of data, representing days and weeks of data, thereby obviating the need to incrementally backup to tape on a daily basis. Intransa Approach to Solutions Intransa, with its partners, is focused and committed, to delivering Solutions to market that allows customers to implement end-to-end storage solutions. The Solutions are targeted to meet demanding customer needs. 2. Preface - 8

13 Intransa s solution leverage on the simplicity, scalability and flexibility of the IntraStor Architecture. IntraStor architecture is an enabler for a solutions-based platform. The above figure gives a conceptual overview of the N-way Scaling in performance and capacity. The following approach is helping Intransa to rapidly build a growing and satisfied client base all over the world. Create a storage network as you would create your client/server (data) network Leverage your existing IP infrastructure for solutions such as Disaster Recovery Leverage existing IP expertise and resources in organization Minimize the number of disparate networks that organizations have to manage and support Increase return on your investment via Simplicity of Scaling the IntraStor architecture and solution Intransa has various families of products like IP3000, IP5500 and IP7500 to address storage, price, performance and availability requirements of the customer. These scalable solutions use IP as a storage medium and can be scaled to more than 50 TB. The N-way clustering of Storage Controllers help in scaling 2. Preface - 9

14 up the performance on demand. Intransa will continue to develop value add features that allows our customers to take advantage of the IP network investment that they have already made. For more details about Intransa please visit About Veritas VERITAS Software is a leading provider of software and services to enable utility computing. In a utility computing model, IT resources are aligned with business needs and business applications are delivered with optimal performance and availability on top of a shared computing infrastructure, minimizing hardware and labor costs. The VERITAS NetBackup mission is to provide heterogeneous data protection solutions ranging from the workgroup or departmental level to the enterprise level. The NetBackup Enterprise Server product provides powerful functionality, flexibility, mainframe-caliber robustness, and high performance in large enterprise environments. The NetBackup Server product is a high-performance solution offering ease of use and functionality that is appropriate to smaller work environments. NetBackup contains many significant enhancements in the area of scalability, application protection, clustering, advanced client and off-host backups, and integration with VERITAS Backup Exec. For more details about Veritas please visit Users This guide is intended to be used by Backup, System and network administrators. Throughout this document we assume basic understanding of: Local Area Networks (LAN) Ethernet and Ethernet-switching concepts Storage Area Network (SAN) Backup and System Administration 2. Preface - 10

15 Other Useful Resources Intransa IP SAN Administrator s Guide. Intransa IP SAN Command Line Interface Guide. Intransa IP SAN StorControl Management Tool Guide. Veritas NetBackup Installation Guide Veritas NetBackup System Administration Guide Veritas NetBackup Advanced Client Quick Start Guide 2. Preface - 11

16 3. Veritas NetBackup This section provides brief introduction of Veritas NetBackup software, architecture, functionality and basics of snapshot. Overview Veritas NetBackup is based on client/server architecture. Each NetBackup client and server belongs to a storage domain. A storage domain consists of a single master Server, its associated Media Servers and NetBackup clients. The Master Server controls and directs all NetBackup operations in its storage domain. Each Media Server controls the backup devices it is connected to, including direct-attach and SAN devices. A Media server can have only one Master Server, but a Master Server can control more than one Media Server. The NetBackup clients are any systems containing data to be backed up. A Master Server can act as a Media Server, and are capable of being clients. A NetBackup client is any system with data to be backed up. The client software is tailored to the operating system on which it is installed. Normally, a client operates under the control of the Master Server according to the rules and schedules that an administer establishes. A backup client accesses the storage media through a Media Server. A NetBackup Media Server hosts one or more backup devices. Storage devices on the same SAN can be shared between Media Servers using the Shared Storage Option (SSO). The Master Server directs a client to send its data to a Media Server for backup. SSO does not support disk devices. A NetBackup Master Server is the manager of the storage domain. An administrator can control all NetBackup functions in the storage domain from the Master Server. NetBackup provides high-performance backups and restores for a variety of platforms, including Microsoft Windows, UNIX, and NetWare systems. 3. Veritas NetBackup - 12

17 Administrators can set up schedules for automatic, unattended backups for clients anywhere in the network. These backups can be full or incremental and are managed entirely by the NetBackup master server. Users can start backups and restores from the computer where they are working. A user can also archive files. An archive operation backs up a file and then deletes it from the local disk if the backup is successful. Once started, user operations are managed by the NetBackup server. NetBackup s Media Manager software manages the media and storage devices. Robots require no intervention on the part of the administrator, operator, or the user. Standalone drives (those not in a robot) that contain appropriate media also require no intervention. NetBackup includes both the server and client software: Server software is on the computer that manages the storage devices. Client software is on the computer whose data you want to back up. A server also has client software and can be backed up like other clients. NetBackup supports both master and media servers. The master server manages the backups, archives, and restores. Media servers provide additional storage by allowing NetBackup to use the storage devices that they control. Media servers can also increase performance by distributing the network load. 3. Veritas NetBackup - 13

18 During a backup or archive, the client sends backup data across the network to a NetBackup server that has the type of storage specified for the client. The storage requirement is specified during NetBackup configuration During a restore, users can browse and then select the files and directories that they want to recover. NetBackup finds the selected files and directories and restores them to the disk on the client. About Advanced Client Advanced Client option is required for taking the snapshots and Veritas NetBackup integration with Intransa VSS Provider. Advanced Client combines the features of snapshot backup, FlashBackup, BLI Agent, offhost backup, and Instant Recovery. Snapshots A snapshot is a disk image of the client s data made almost instantaneously. NetBackup backs up the data from the snapshot image, not directly from the client s primary data. This allows client operations and user access to continue without interruption during the backup. A snapshot image is required for all features of Advanced Client. A number of methods are provided for creating snapshots. You can select the snapshot method manually from the Policy dialog of the NetBackup Administration Console, or allow NetBackup to select the method for you. Snapshot Basics Large active databases or file systems that must be available around-the-clock are difficult to back up without incurring a penalty. Often, the penalty takes one of two forms: The entire database is taken offline or the file system is unmounted, to allow time for the backup, resulting in suspension of service and inconvenience to users. The copy is made very quickly but produces an incomplete version of the data, some transactions having failed to complete. A solution to this problem is to create a snapshot of the data. This means capturing the data at a particular instant, without causing significant client downtime. The resulting capture or snapshot can be backed up without affecting the performance or availability of the file system or database. Without a complete, up-to-date snapshot of the data, a correct backup cannot be made. 3. Veritas NetBackup - 14

19 When a backup is managed by a backup agent on a Storage network, the data to backup must be contained in a snapshot. The backup agent can only access the data by means of the raw physical disk. Once the data is captured as a snapshot, the NetBackup client maps the logical representation of the data to its absolute physical disk address. More details about administering the NetBackup Server and Microsoft s Volume Shadow copy Service (VSS) frame work along with related technical details are covered on Appendix. 3. Veritas NetBackup - 15

20 4. NetBackup Qualification Setup This section provides guidelines and in setting up the servers for Configuring the IP SAN volumes and Configuring Veritas NetBackup Server and Client to provide Disk-to-disk backup using Microsoft VSS framework. This section also provides brief specifications of the servers used in the qualification environment. Getting Started Before beginning the setup process, make sure you have gathered the following equipment and necessary information: Intransa IP SAN along with Intransa VSS Provider Software. The Veritas NetBackup Software Enterprise Edition (Version 5) with its Advanced Client license. Windows 2003 Server with ServicePack-1, a Gigabit NIC and Microsoft iscsi initiator. One other NIC for management interface. Network settings specific to your network, such as: IP addresses, Network setting, Netmask setting, and Gateway setting. Network Switches with proper V-LAN configurations. 4. NetBackup Qualification Setup - 16

21 Assumptions It is assumed that the following conditions exist before proceeding with the setup procedure: NetBackup Server (NS) and NetBackup Client (NC) are installed with Windows 2003 Enterprise Edition with Updated Service Packs. NS and NC are configured with Intel Gigabit Ethernet NIC with Microsoft initiators to connect to the Host Access Network (HAN) of IP SAN. The setup machines have two network adapters: One Gigabit adapter to connect to the iscsi Network of IP SAN. Second adapter to connect the NS, NC and other Client machine for backup management traffic on LAN switch. Gigabit Ethernet Switch is used to connect the NS and NC to Host Access Network of IP SAN. Jumbo frames are enabled on the Storage Data Network (SDN) GigE switch. 4. NetBackup Qualification Setup - 17

22 The proposed D2D solution comprises of the following architecture: The NetBackup Server (NS) is installed on an independent server and connected IP SAN for taking the backup on the disk. The NetBackup Clients utilizes IP SAN volumes to host the data. The IP SAN provides the snapshot taking capability through built-in snapshot feature in the StorControl Software and Intransa VSS Provider software installed on the NC of which backup is to be taken. 4. NetBackup Qualification Setup - 18

23 NetBackup Server provides scheduling and management functionality for initiating snapshots, scheduling the backup/restore operations, and deleting snapshots. In the above architecture, the backup data traffic flow from the IP SAN Snapshot Volume mounted on NC to the IP SAN netbackup volume. The data backup occurs from snapshot volumes (provided by IP SAN) mounted at the Client to a backup volume on the IP SAN. Configurations used in the Qualification Environment Following is the qualification environment configured for the evaluation. Servers The configurations used in qualification environment are two Pentium 4 servers connected via Intel GigE Ethernet Adapter on LAN. NetBackup Server (NS) Pentium 4 server named as Rasgulla is configured as NetBackup Server. NetBackup Client (NC) Pentium 4 server named as Jupiter is configured as NetBackup Client. Both machines are connected to Intransa large scale IP SAN storage via Gigabit Ethernet NIC with Microsoft iscsi initiator Networks There are three types of Networks involved in the configurations 1. LAN: Local Area Network Gigabit Ethernet LAN switch: It is used for establishing the connectivity between the Servers and LAN Clients. 2. HAN: Host Access Network Gigabit Ethernet HAN switch: The connectivity between the IP SAN Host Access Network (HAN) and NetBackup Servers is done using the HP Gigabit Switch (HP procurve switch 2824) with no jumbo frame support. 3. SDN: Storage Data Network Gigabit Ethernet SDN switch: Storage Data Network (SDN) is comprises of ETH3 and ETH4 network ports of storage controllers and connections from the each Disk 4. NetBackup Qualification Setup - 19

24 Software Enclosures. This is connected using Foundry EdgeIron 48G switch with jumbo frame enabled on the switch. Windows 2003 Server Enterprise Edition with updated service pack and NTFS file system is installed on NS & NC machines. VERITAS NetBackup Server for Microsoft Windows Version 5.0 Intransa VSS Provider software. Volumes IP SAN Volume namely netbackup is configured using NTFS-Optimized custom policy (which is described latter in the document) for taking the backup. Also IP SAN Volume namely source1 to source 10 are created using simple stripe policy and used as source volumes for backup. Volume netbackup is assigned to Netbackup Server Rasgulla via Storage Controller Module (SCM 1) and Source volumes are assigned to Netbackup Client Jupiter via Storage Controller Module (SCM 2) Data Set of 10 GB streaming datasets are prepared on the NetBackup Client (Jupiter) as source data for backup. 4. NetBackup Qualification Setup - 20

25 Qualification Configuration in brief is in given in Table 1 below. Table 1 : No. Item Configuration NetBackup Server and Client 1 Processor Pentium 4, 2.4 GHz Server with single CPU 2 Memory 1 GB (DDR266) 3 Local Hard disk 40 GB ATA 4 Network card Intel Pro Ethernet NIC 5 Host Based Adapter Intel Pro GigE Ethernet NIC with Microsoft iscsi initiator IP SAN : Intransa IP Storage Controller Module 2 X SC5500 in cluster mode with build Disk Enclosure 1 X DE5200 (16 X 250 GB Maxtor ATA disk drives) 9 Operating System Windows 2003 Server Enterprise Edition and Service Pack 1 10 Host Access Network 12 Storage Data Network HP procurve switch 2824 Foundry EdgeIron 48G 13 Veritas Veritas NetBackup Server Intransa VSS Provider Software 15 Data 10 GB streaming data 4. NetBackup Qualification Setup - 21

26 5. Configuring IP SAN Volumes Intransa Large Scale IP SAN Overview: IP SAN overview 5. Configuring IP SAN Volumes - 22

27 What is an IP SAN? The IP SAN is an iscsi-based storage system. It provides a large, scalable, easily managed pool of storage. The IP SAN is the best of both worlds: the block-level abilities of a Storage Area Network (SAN), and the simplicity and cost benefits of a Network Attached Storage (NAS) device. In other words, it s Simply Smarter. The IP SAN is more flexible than a traditional Fibre Channel SAN. It communicates with host application servers using the iscsi protocol over Ethernet, rather than Fibre Channel, allowing for less costly Ethernet switches and tighter integration with an existing network. The IP SAN connects the components by switched Ethernet connection, allowing drive capacity and Storage Controller Modules to be managed independently. The IP SAN is the perfect solution for decentralized, growing, and dynamic storage needs. It includes a scalable number of IP-addressable Storage Controller Modules and a scalable number of IP-addressable disks, which form a logical unit called a Realm. A Realm is an administrative domain made up of a set of designated Storage Controller Modules, Disk Enclosures, associated software and network interconnects. The Storage Controller Modules within a Realm are managed by the StorControl software as a single pool of controllers, except in cases of single- Storage Controller Module systems. The storage space provided by the Disk Enclosures within a Realm is managed by the StorControl software as a single virtual pool of storage space. The Realm is the entirety of the managed system: each part of the system comes together to become the Realm. You manage the Realm as a single entity by managing its components. Storage traffic moves from your Initiator to the IP SAN via an Ethernet (GigE) switch. By having a centrally-managed system, you can create easily managed volumes for your users. Here s how it works: Small Computer System Interface over IP (iscsi) uses standard IPaddressable Ethernet connections for interaction between components. The Initiator uses a standard network interface card (NIC) with a software iscsi driver OR a hardware-based HBA with driver. SC5500 Storage Controller Modules send requests and responses through a GigE (Ethernet) switch, handling traffic to the Initiator and the Disk Enclosure. Acting together, the Intransa modules shown in Figure above create an intelligently managed, single, unified system of interactive parts. The benefits of this are: 5. Configuring IP SAN Volumes - 23

28 Shared arrays of inexpensive disks A modular, shared processor pool, simplifying shared storage; storage is added and removed as needed A switched GigE Ethernet infrastructure Unified management simplifies IP SAN administration Both graphical and command line-based management interfaces High availability: system-wide failover is handled automatically with minimal need for administrator intervention. Solution Specific Configurations Virtual volumes are configured on the Intransa Large Scale IP SAN one for Backup for NetBackup Server named netbackup and other named source1 to source5 on NetBackup Client as a source for data operations in such a way that they will occupy the separate set of disks. To setup the volumes follow the following steps: Step 1: Create policy for creating the volumes To create policy: Logon to StorControl command line using the realm mgmt IP Create a Policy using the below StorControl> Policy create name = NTFS-Optimized virtualdisktype = RAID10 numberofstripemembers = 3 stripeelementsize = 64 Step 2: Volume Creation Backup Volume (named netbackup ) Volumes can be created using graphical user interface (GUI) or by using the command line interface. Both of these methods are demonstrated one by one. Part I: Volume Creation using CLI. StorControl> volume create blocksize = 512b policy = NTFS- Optimised size = 100GB name=netbackup virtualdiskplacement = new Response: Successful 5. Configuring IP SAN Volumes - 24

29 Part II: Volume Creation using GUI Logon to StorControl GUI using the realm management IP, admin user and appropriate password. Create a Volume using the below Right click on All Volumes for popup menu 5. Configuring IP SAN Volumes - 25

30 Use iscsi compliant naming convention for volumes Set RAID level through policy Select correct unit for desired volume size Placement - performance versus space efficiency 512-byte block size best for most applications on Microsoft Windows platform Initialize zeros-out blocks volume is created on 5. Configuring IP SAN Volumes - 26

31 Assign initiator to the Volume Confirm new volume parameters 5. Configuring IP SAN Volumes - 27

32 Volume is created and given an iscsi Qualified Name (IQN) Now go to the NetBackup Server ( Rasgulla ). Install the Microsoft iscsi Initiator software. Double click on the Microsoft iscsi Initiator icon Available Targets will show the assigned target status as Inactive 5. Configuring IP SAN Volumes - 28

33 Log on to the Target. After log on you will see the target status changed to Connected 5. Configuring IP SAN Volumes - 29

34 Now open the Computer Management window 5. Configuring IP SAN Volumes - 30

35 Windows File System Tuning Windows system create a 63-sector hidden are on disk to house the Master Boot Record (MBR). This cause all I/O in the file system made on the subsequent partition to be misaligned ( not on a 256KB boundary), and therefore I/O performance will suffer. By using a program called diskpar,available on the Windows Resource kit, you can alter the number of hidden block to a larger number so that the disk array I/O are aligned and better optimized. We now have initialized basic raw disks. Query the partition information using the diskpar i command. (In this example, we installed the diskpar utility in a Windows utility directory on the C: drive.) This indicates the disk is accessible. C:\Documents and Settings\Administrator>diskpar -i Drive 1 Geometry Information ---- Cylinders = TracksPerCylinder = 255 SectorsPerTrack = 63 BytesPerSector = 512 DiskSize = (Bytes) = (MB) End of partition information. Total existing partitions: 0 C:\Documents and Settings\Administrator>diskpar -s 1 Set partition can only be done on a raw drive. You can use Disk Manager to delete all existing partitions Are you sure drive 1 is a raw device without any partition? (Y/N) y ---- Drive 1 Geometry Information ---- Cylinders = TracksPerCylinder = 255 SectorsPerTrack = 63 BytesPerSector = 512 DiskSize = (Bytes) = (MB) We are going to set the new disk partition. All data on this drive will be lost. Continue (Y/N)? y Please specify starting offset (in sectors): 64 Please specify partition length (in MB) (Max = ): Done setting partition New Partition information ---- StatringOffset = PartitionLength = HiddenSectors = Configuring IP SAN Volumes - 31

36 PartitionNumber = 1 PartitionType = 7 You now should use Disk Manager to format this partition Verify the partition information. (Notice the starting offset is and the number of hidden sectors is 64.) C:\Documents and Settings\Administrator>diskpar -i Drive 1 Geometry Information ---- Cylinders = TracksPerCylinder = 255 SectorsPerTrack = 63 BytesPerSector = 512 DiskSize = (Bytes) = (MB) ---- Drive Partition 0 Information ---- StatringOffset = PartitionLength = HiddenSectors = 64 PartitionNumber = 1 PartitionType = 7 End of partition information. Total existing partitions: 1 Now right click on the new volume and select the following options one by one 5. Configuring IP SAN Volumes - 32

37 Mark Partition as active. Change Drive letter and Paths and select the appropriate drive letter. Format : Format with o Volume Label : NetBackup Volume o File System : NTFS o Allocation unit size: Default Perform a quick format In the similar fashion create volume Source1 to Source5 on the IP SAN and assign it to NetBackup Client (Jupiter) StorControl> StorControl> volume create blocksize = 512b policy = stripe size = 10GB name=source1 virtualdiskplacement = new Response: Successful 5. Configuring IP SAN Volumes - 33

38 6. Installing Intransa VSS Provider Successful Installation of Windows 2003 Operating systems with Gigabit Ethernet NIC and Microsoft iscsi initiator will lead to installation and configuration of Intransa VSS Provider software on the NetBackup Client machines (Jupiter). The Intransa VSS Provider is instrumental in talking to Intransa IP SAN for creating snapshots of the source production volumes. Backup will be performed from the snapshot and after the backup snapshot will be automatically removed from the IP SAN. Copy the Intransa VSS Provider software. Double click on the executable. Click Next 6. Installing Intransa VSS Provider - 34

39 Enter the User name and Organization Click Next 6. Installing Intransa VSS Provider - 35

40 Click Install 6. Installing Intransa VSS Provider - 36

41 Click Finish. Now You have successfully installed Intransa VSS Provider This will create the icon on the desktop. Now double click on it for Intransa VSS Configuration. Specify the Startup type as Automatic Specify the Realm iscsi target IP address, its User account and password. Click Add realm. 6. Installing Intransa VSS Provider - 37

42 Click OK Congratulations!! You have successfully completed the Installation and configuration of Intransa VSS Provider for the target Realm 6. Installing Intransa VSS Provider - 38

43 7. Installing NetBackup Server and Client The volumes are available for Veritas NetBackup installation and configuration. We presume the VSS installed successfully without any error. Up until now we have configured netbackup volume and source volumes on the Intransa IP SAN. Also assigned them to hosts and formatted them for use. You have also successfully installed and configured the Intransa VSS provider software on NC. The following section explains about how to install Veritas NetBackup Software. Now we are going to install the Veritas NetBackup Software and configure the server and clients. Following are the simple steps to achieve this goal. Installing the Veritas NetBackup Software. Put the Veritas NetBackup Software media in the NS. Click on the NetBackup Installation. 7. Installing NetBackup Server and Client - 39

44 Click on Install Server Software. Click OK 7. Installing NetBackup Server and Client - 40

45 Click on I accept the terms of the license agreement. Select Install to this computer only and click Next. 7. Installing NetBackup Server and Client - 41

46 Enter the License Key and Select the NetBackup Master Server. Select Custom and click Next. 7. Installing NetBackup Server and Client - 42

47 Select the NetBackup, NetBackup Debugger Symbols and select all options under it. Note: Do not select the Veritas Snapshot Provider option since we are going to use the Intransa VSS Provider software for taking the snapshots. Click Next 7. Installing NetBackup Server and Client - 43

48 Select Use the built-in system account, In Startup select Automatic and select Start NetBackup Server Services After Install. Note: Do not modify the Default Ports except under unusual circumstances. Click Next 7. Installing NetBackup Server and Client - 44

49 Now Enter the Master Server Name. In this case it is rasgulla configured as NetBackup Server. Click Next Click Install 7. Installing NetBackup Server and Client - 45

50 Click Finish This completes the installation of the Veritas NetBackup Software. 7. Installing NetBackup Server and Client - 46

51 Now the NetBackup Configuration wizard will start automatically. This will help us in configuring the NetBackup Server. Click Next. Click Next. 7. Installing NetBackup Server and Client - 47

52 Click Next. In Device Hosts select rasgulla 7. Installing NetBackup Server and Client - 48

53 Click Next. Note: Since IP SAN disks are not come in to the category of removable media devices, Device Configuration Wizard will not find it and gives following error Click OK. 7. Installing NetBackup Server and Client - 49

54 Click Next. Click Next. 7. Installing NetBackup Server and Client - 50

55 Click Next. Click Finish. 7. Installing NetBackup Server and Client - 51

56 Click Next. Click Next. 7. Installing NetBackup Server and Client - 52

57 Click Next. Click Next. Please specify the full path for the Catalog backups are to be saved 7. Installing NetBackup Server and Client - 53

58 Click Next. Click Next. 7. Installing NetBackup Server and Client - 54

59 Click Finish. 7. Installing NetBackup Server and Client - 55

60 Now configuration wizard will start with Backup Policy Configuration. Click Next. Click Next. 7. Installing NetBackup Server and Client - 56

61 Click OK. This completes the Veritas NetBackup Server Software installation. Now we have start with the Veritas NetBackup Client Software installation on NetBackup Client machine Jupiter 7. Installing NetBackup Server and Client - 57

62 NetBackup Client Installation The following section explains about how to install Veritas NetBackup Software on NetBackup Client machine.now we are going to install the Veritas NetBackup Software and configure the client. Following are the simple steps to achieve this goal. Installing the Veritas NetBackup Software. Put the Veritas NetBackup Software media in the NS. Click on NetBackup Installation 7. Installing NetBackup Server and Client - 58

63 Click on Install Client Software Click OK 7. Installing NetBackup Server and Client - 59

64 Click Next Click on the I accept the terms of the license agreement. 7. Installing NetBackup Server and Client - 60

65 Click on the Install to this computer only. Click Next Select Custom Click Next Note: Do not select the Veritas Snapshot Provider option since we are going to use the Intransa VSS Provider software for taking the snapshots. 7. Installing NetBackup Server and Client - 61

66 Click Next 7. Installing NetBackup Server and Client - 62

67 Click Next Click Next 7. Installing NetBackup Server and Client - 63

68 Click Next Click Next Click Install 7. Installing NetBackup Server and Client - 64

69 Click Finish This completes the Veritas NetBackup Client Installation. In the next section we will test the backup and restore operations. 7. Installing NetBackup Server and Client - 65

70 8. Testing Backup and Restore Successful Installation of NetBackup Server and Client software will lead to define the storage units and proper policy for taking the backup from the snapshot. While configuring NetBackup Server please take maximum care of defining proper policy, storage units, Retention etc. Defining Storage Unit Right Click on Storage Unit 8. Testing Backup and Restore - 66

71 Select New Storage Unit Fill in the relevant details and click OK Similarly you can create multiple disk storage units to suite your backup needs. 8. Testing Backup and Restore - 67

72 Defining Backup Policy Successful Installation of NetBackup Server and Client software along with defining storage units will lead to define the proper policy for taking the backup from the snapshot. Give appropriate Policy name eg. snapshot1 Click OK 8. Testing Backup and Restore - 68

73 Click Next Click Next 8. Testing Backup and Restore - 69

74 Specify clients for this policy. Click Add and enter the name of client Jupiter and click Next. 8. Testing Backup and Restore - 70

75 Click Next Now specify files and folder to backup. Click Add and enter drive name eg. D: 8. Testing Backup and Restore - 71

76 Define Backup Type, viz. Full Backup, Incremental Backup and User Backup. 8. Testing Backup and Restore - 72

77 Note: Please refer to appendix for more details about the backup types Specify the Rotation. Click Next Specify the time window during which backups are allowed to start Click Next 8. Testing Backup and Restore - 73

78 Click Finish Now double click on the snapshot1 policy and mark the box Perform snapshot backups for performing the backup operations from snapshots. Also mark the Allow multiple data streams for performing multiple backups 8. Testing Backup and Restore - 74

79 This is an important step in defining snapshot based backup method 8. Testing Backup and Restore - 75

80 Click on Clients tab to see the clients 8. Testing Backup and Restore - 76

81 Click on Backup Selections tab to see the volumes to be backup Click OK. Now it will validate the policy. It will give an error is policy is not proper. 8. Testing Backup and Restore - 77

82 Now click on the newly created snapshot1 policy and click on Manual Backup to start the backup immediately. Select Schedules and Clients and click OK Click OK 8. Testing Backup and Restore - 78

83 Now click OK to open the Activity Monitor. Now monitor the progress of the job using the Activity Monitor on NetBackup Server. Following Job Details window will give Job Overview and Detailed Status as shown below 8. Testing Backup and Restore - 79

84 Verifying the Snapshot operations The snapshot operation can be verified from the Intransa StorControl window. Following steps and screenshots demonstrate the same. Open the StorControl window Click on source1 volume. In General tab section shows the camera type icon on the volume. Indication the snapshot is initiated. All snapshots tab shows the snapshot volume for source1 volume connected to Jupiter in the Assigned initiator tab. 8. Testing Backup and Restore - 80

85 Congratulations!! You have successfully taken the backup from the snapshot volume. Following screen shot shows the backup operation is successful. 8. Testing Backup and Restore - 81

86 This will verify the snapshot operation. After the Backup completes the snapshot volume automatically gets deleted. Similarly you can create the multiple policies to suit your backup needs. Note: Please refer to Appendix for guidelines for defining the backup policies. Restore Operation The Backup data can be restored using the Backup, Archive and Restore Window Open the Backup, Archive and Restore Window on NetBackup Client or Server Select the folders or files to be restored and click on icon to Start Restore of Marked files. If the data is backed using the snapshot policy then we need to restore it to the source volume using the option Restore everything to a different location and specifying the appropriate Destination Path 8. Testing Backup and Restore - 82

87 Click on Start Restore button. Click on View Status button to monitor the progress. 8. Testing Backup and Restore - 83

88 Monitor the progress of the job using the Activity Monitor on NetBackup Server. Congratulations!! You have successfully taken restored volume. Following screen shot shows the restore operation is successful. 8. Testing Backup and Restore - 84

89 8. Testing Backup and Restore - 85

90 9. Performance Benchmarking Following performance tests were conducted on the IP SAN. No. Test Cases Average speed (Mb/sec) Peak speed (Mb/sec) A NetBackup Server & Client connected using LAN and NetBackup Server & Client are connected to IPSAN using Intel Gigabit NIC. 1 Backup from Local Disk Backup of NetBackup Servers local disk using Localdisk policy to IP SAN (NetBackup Volume) on NetBackup Server 2 Backup from Snapshot Volume Backup of Clients data (Source Volume) from the snapshot on IP SAN using Snapshot policy to IP SAN (NetBackup Volume) on NetBackup Server 3 Backup from Source Volume Backup of Clients data (Source Volume) on IP SAN using Regular policy to IP SAN (NetBackup Volume) on NetBackup Server Restore to IP SANDisk Restore from NetBackup Volume to IP SAN Source Volume Performance Benchmarking - 86

91 Case Study: Comparison of Backup Performance with Tape Media No. Media Mfg. Xfer Rate Realistic Xfer Rate 25 Gb Backup 1 4mm DDS Mb/s 01.2 Mb/s 5.80 hours 2 4mm DDS Mb/s 02.4 Mb/s 2.90 hours 3 AIT Mb/s 03.0 Mb/s 2.30 hours 4 AIT Mb/s 03.0 Mb/s 2.30 hours 5 DLT Mb/s 01.5 Mb/s 4.60 hours 6 DLT Mb/s 05.0 Mb/s 1.40 hours 7 DLT Mb/s 06.0 Mb/s 1.20 hours 8 SDLT Mb/s 11.0 Mb/s 0.65 hours 9 SDLT Mb/s 16.0 Mb/s 0.45 hours 10 LTO 32.0 Mb/s 16.0 Mb/s 0.45 hours 11 Intransa IP Mb/s 38.0 Mb/s 0.15 hours (Source: The above chart was taken from an online web resource; it is only intended to convey the difference between using tape as a backup medium and using the IP 5500 as a backup medium. Values represented may vary. The chart does give a general comparison in the difference of times to perform backup operations between tape and the IP Performance Benchmarking - 87

92 10. Summary Based on solution results, following are the conclusions drawn: Conclusions Summary NetBackup Server with IP SAN gives Backup peak performance of Mb/sec with average of Mb/sec. NetBackup Media Server with IP SAN gives Restore peak performance of Mb/sec with average of Mb/sec. Disk Storage configuration is flexible and can be easily tuned to the available disk space. Intransa VSS Provider works well with the NetBackup Software makes it possible to automatically start the snapshot for backup. The backup happens from the snapshot volume and Original source volume is available and remains online for the regular use. This helps immensely in shorting the backup window of mission critical volumes without bringing them offline for backup. Restore operation can be controlled using the option Restore everything to a different location to make sure the data is restored at the original source volume or the required destination. IT organizations can increase their productivity and decrease costs by implementing a disk-to-disk backup solution. The Intransa IP 5500 provides a robust platform for easily implementing and managing backups to disk. Veritas s NetBackup software was tested and performance difference was minimal for backup with a varying difference in restore times. Together it offers the extremely flexible and high performing fast backup solution for the enterprise. Veritas NetBackup Software seamlessly gets integrated with the Intransa VSS Provider. This gives immense benefit for configuring the Disk-to-Disk Backup Solutions with Intransa IP SAN in using the automated snapshot feature. 10. Summary - 88

93 The volume configuration tested under Windows 2003 with ATA internal disks for the source and an Intransa stripe of mirrors (RAID10) as the target for the backups. The IP 5500 was configured for optimal throughput based on initial testing by creating a stripe width of 64k per stripe member. The IP 5500 performed much better when compared to conventional tape media. It is important to note that only a single volume was used for the storage pool. If multiple volumes and backup streams were used then this number could potentially double being limited by only the software or source data read rate. From this minimal testing, the IP 5500 decreased backup times by 66% which drastically reduces the backup window for administrators. This has the added benefit of allowing IT staff to increase the numbers of backups or simply ease the management and cost of backups. This also insures that the backups are always online and easy accessible. For more information on specific configurations please contact your local Sales Representative or visit Summary - 89

94 11. Appendix A.VSS Concepts This section provides theory details of the technical concepts about Microsoft Virtual Shadow copy Service (VSS). A.1 What is VSS: Microsoft s Volume Shadow-Copy Service (VSS) provides a framework for creating a point-in time copy of a single volume or multiple volumes. It specifically aims to eliminate the problem of corrupt snapshots or backups due to open files. The Volume Shadow Copy Service (VSS) is an infrastructure that makes possible enhanced data protection though high fidelity backups, rapid data restores, and data transport. VSS is a component of the operating system, and as such, is not directly accessed by users. Instead the Volume Shadow Copy Service coordinates with user applications, backup applications and storage hardware to enable the creation of point-in-time shadow copies of data on single or multiple volumes without significantly impacting performance. Open files have historically been a backup administrator s biggest challenge, as they are often skipped or corrupted during the backup process. The alternative forcing open files closed or asking users to log out so applications can be shut down is no longer viable. Microsoft Windows Server 2003 VSS partially addresses this problem by allowing users to access certain files while a backup is taking place, without compromising the fidelity of the backup. (A drawback to VSS is that the files must be related to a VSS-aware application in order for all volumes to be synchronized to a single point in time,) Another feature of VSS is the ability for users to view and recover previous versions of their files without IT intervention. 11. Appendix - 90

95 By enabling read-only, point-in-time copies of an entire volume, the problem associated with backing up files that are open or in use would seem to be taken care of. That is Microsoft s intent, and in some situations, it is an intent that has the potential to be fulfilled. A.2 VSS Components Requestors o Invokes VSS to a create shadow copy Backup applications Shadow copy management applications Writers Represents Apps and Windows Services (i.e., SQL, Exchange, AD, etc.) o Differentiates VSS from competitors o Participate in shadow copy creation process Providers o Create shadow copies System provider Hardware snapshots A.3 How VSS works Microsoft Windows Server 2003 VSS works with applications that are in use to determine when a volume shadow-copy (snapshot) can be made. A snapshot is a copy of some data set, for example, a disk volume, at a point in time. VSS then communicates with the operating system and applications and freezes computing tasks to perform copy on write tasks on a sector-by sector basis. Without the freeze function, data could become corrupted when the snapshot is taken because the system might be in the middle of a task. Meanwhile, applications continue to run uninterrupted. The original volume continues to change as the process continues, but the shadow copy of the volume remains constant. VSS also gives permission to the backup application to access the volume and back it up in its frozen state. The shadow copy volume is then used for the actual backup. After the shadow copy volume is saved on the backup device, the shadow copy is deleted. The process uses free disk space on any NTFS volume (the file system used by Windows NT for storing and retrieving files) to store a record of the differences between the original volume and the shadow copy volume. If there isn t enough temporary disk space, the volume shadow copy cannot be completed and the backup program skips open files. 11. Appendix - 91

96 In order to ensure absolute data integrity for all volumes, especially those that are interrelated, the application, the backup program and the backup device should be VSS aware. This means that the components participating in the backup must be specifically programmed to work with VSS. Towards this end, Microsoft is providing a software development kit (SDK) to enable independent hardware and software vendors to interface their components, including applications, data stores, snapshot engines, and backup applications, with VSS. These components (the application, backup program, and storage software/hardware) are called writers, requestors, and providers, respectively, as described in more detail in the table below. VSS Component Description Requestor Writer Provider The requestor is backup software that initiates the creation and destruction of the shadow copy. The writer is any application software that participates in the shadow copy process and whose files and data are included in the process. The provider is the storage technology that does the actual shadow copy (snapshotting). Table: VSS Components All components of VSS --writers, requestors, and providers-- must be VSS-aware in order for VSS to be able to orchestrate the communication among them. The following diagram shows the three-way orchestration among shadow copywriters, requestors, and providers, as performed by VSS. 11. Appendix - 92

97 B. Overview to VSS and VDS Design There are two approaches to managing storage networks. One is to manage storage through some mechanism on the server; the other is to manage it through a separate storage product that interfaces with some hardware component of the storage network. Both VSS and VDS are designed to enable host-based and hardware-based solutions, as shown in Figure below. In this figure, Windows host software components are unshaded; third party vendor components are shaded. The additional details of this diagram are explained in later sections. Note that some of these components might not be present. 11. Appendix - 93

98 VSS and VDS Components in Windows Server 2003 Software Providers Software solutions implemented on the host computer are referred to variously as host-based or software solutions, or as software providers. Software providers can be supplied by third party vendors, or by Microsoft (and in some cases, both are present). In the latter case, this software is referred to as the system software provider, and is an in-box solution that ships with the product without requiring any additional installation steps. The advantage of host-based solutions is that they are inexpensive and easy to deploy. Updates and bug fixes are much simpler to manage, since they are part of the operating system service packs. However, host-based solutions do have drawbacks: one is that in high workload settings, the host can be overloaded and performance impaired; another is that advanced functionality might not be provided. Hardware Providers Hardware vendors have also developed their own storage management solutions. These solutions, known as hardware providers, can generally provide higher performance capabilities than solutions built into the system software, since the workload is offloaded to the external disk subsystem. Additionally, these solutions can provide maximum configuration flexibility, as well as advanced capabilities such as remote management. 11. Appendix - 94

99 C. Volume Shadow Copy Service The Volume Shadow Copy service provides an infrastructure for creating high fidelity, point-in-time images known as shadow copies. These copies of a single volume or multiple volumes can be made without affecting production server performance. VSS is able to produce high fidelity shadow copies through its coordination with business applications, backup applications, and storage hardware. Shadow copies can be used for a number of purposes: Rapid backup and restores. Individual file restores (using the Shadow Copy for Shared Folders feature). Transport (using a hardware provider) for tape backups, testing and data mining. VSS Components Involved in the Shadow Copy Creation Process Successful shadow copy creation requires the completion of a number of steps, the details of which are quite complex. This section outlines the components of the Volume Shadow Copy service requestors, writers and providers necessary for creating high fidelity shadow copies. 11. Appendix - 95

100 1) Requestors: Initiating Shadow Copy Creation The Volume Shadow Copy service is invoked by the requestor, software that initiates basic VSS functions such as creating, importing, breaking or deleting a shadow copy. Depending on the problem to be solved, different types of requestor applications are invoked: backup applications for VSS-enabled backups, management applications to manage shadow copy creation and usage, or utility applications designed to help solve other IT problems. (One such example is vrfydsk, which helps to verify whether chkdsk should be run). 2) Writers: Preventing Data Inconsistencies If, during the shadow copy creation process, an application writes to disk, data inconsistencies can occur, compromising the integrity of the point-in-time data image. The Volume Shadow Copy service has been designed to eliminate these drawbacks. Key to this design is the writer, an application-specific software module that acts to ensure that an application s data is consistent when a shadow copy is created. Under this design, the responsibility for data consistency has been shifted from the requestor application to the production application itself. The advantage of this approach is that application developers those most knowledgeable about their applications can ensure, through development of their own writers, maximum effectiveness of the shadow copy creation process. 3) Providers: Creating Shadow Copies In the context of shadow copy creation, a provider is the interface to the point-intime imaging capabilities, either on either the storage array (hardware-based) or in the operating system (software based). Windows Server 2003 includes a system software provider with shadow copy functionality in-box; alternatively, third party hardware and software vendors can develop their own hardware or software providers to provide point-in-time imaging functionality. Windows Server 2003 supports multiple hardware and software providers which can be used in combination to solve many different IT operational scenarios. VSS selects the provider to use during shadow copy creation using the following default hierarchy: hardware provider software provider system software provider. If a specific IT operational problem requires it, the requestor can override this hierarchy. 11. Appendix - 96

101 Coordination of Shadow Copy Creation: the Complete Sequence The various roles of the requestor, writer and provider are put into context in this section, which lists the steps that need to be taken to create a shadow copy. Overall coordination of the requestor, writer and provider is under VSS control. 1) The requestor notifies the VSS coordinator service to prepare for shadow copy creation. 2) The VSS coordinator notifies the application-specific writer to prepare its data for making a shadow copy. 3) The writer prepares the data in whatever way is appropriate for that application, such as completing all open transactions, rolling transaction logs, and flushing caches. 4) When the data is prepared for shadow copy creation, the writer notifies the VSS coordinator. 5) The VSS coordinator relays the message to the requestor, and the requestor initiates the commit snapshot phase. 6) The VSS coordinator temporarily halts (quiesces) application I/O write requests (I/O read requests are still possible) for the several seconds required to create the shadow copy of the volume or volumes. 7) The provider creates the shadow copy (a maximum of 10 seconds). 8) After the shadow copy is created, the VSS coordinator releases the writer from its temporary quiescent phase. 9) All queued write I/Os are completed. 10) VSS queries the writers to confirm that write I/Os were successfully held during shadow copy creation. 11) If the writes were not successfully held (meaning the shadow copy data is potentially inconsistent), the shadow copy is deleted and the requestor is notified. 12) The requestor can retry the process (loop back to 1) or notify the administrator to retry at a later time. As a direct result of steps 6 and 10-11, the VSS coordinator service ensures that the shadow copy is a highly consistent copy of the original data. Moreover, because all the preparation for making the shadow copy is completed prior to halting the application write I/Os, production performance is not impacted. VSS for Fast Backups and Restores The most common implementation of a requestor is a backup application. Backup applications can benefit from the application-aware design of VSS for a number of reasons. Some backup applications are designed to check the application data including metadata and log files for inconsistencies. Other backup applications do not implement consistency checks, and instead require the administrator to ensure that applications are closed before the backup 11. Appendix - 97

102 process. However, even the backup applications that do complete consistency checks have drawbacks: Excessive Time. Depending on the amount of data involved, the consistency check, done at the completion of a backup, can take hours. If, by the end of the check, inconsistencies are found, the backup must be redone. For many businesses, there simply is not the time. Complexity. Because each application SQL, Exchange, etc is designed differently, the location of the data, metadata and log files varies widely. Backup application developers have traditionally borne the responsibility of developing a reliable interface between the backup product and the data to correctly find and use this information. These companies have also been responsible for ensuring that application patches are captured and integrated into the backup application. These fixes are not always captured. Using the VSS infrastructure, backup application developers can take advantage of requestor and writer functionality. Together, these functionalities reduce the time and complexity involved in preparing data for backup and they ensure high fidelity point-in-time shadow copies that can be successfully and quickly restored. VSS for Individual File Restores The system administrator can make shadow copies available to end users through a feature called Shadow Copies for Shared Folders. The administrator uses the Properties menu (see Figure 4) to turn on the Shadow Copies feature, selects the volume(s) to be copied, and determine the frequency with which shadow copies are made. 11. Appendix - 98

103 D. Tape Backups: Limitations Until recently, backups to tape were the dominant technology for protecting data. While tape backups continue to be essential for long-term archiving solutions, they have a number of limitations that the storage manager should consider when planning and implementing effective data protection solutions: 1. Open files cannot be backed up. Open files must either be skipped during the tape backup process meaning the backup data is incomplete or the application must be closed ( cold backups), thereby disrupting user productivity. 2. Tape backups are time intensive. Because of the problems associated with backing up open files, backups are made during evening and weekend hours. However, as businesses amass greater quantities of mission critical data, increasingly long backup windows are required. For many businesses, this time simply does not exist information must be available 24x7. 3. Tape backups are often unsuccessful. Estimates of backup failures put the rate anywhere between 20 and 50% depending on the environment. Worse, it is all too common that backup failures are not discovered until the recovery process after production data has been corrupted or deleted. Poor backup success rates result from a number of factors, including poor media, open files, and incomplete backups (the backup window was too short) any of which the administrator may be unaware. 11. Appendix - 99

104 4. Tape restores are time intensive. Tapes must be found and retrieved from offsite storage. While restoring a full backup is relatively straightforward, differential and incremental backups are far more challenging. For those backup methods to work, all the correct tapes must be located and restored in the correct order. 5. Single file restores are costly. If an individual file is corrupted or accidentally deleted, the user can either rebuild the file or request that an earlier version be restored from tape. However, finding the most recent version of a file is a non-trivial process. Because tape media is read sequentially, the tape drive must first stream through all data physically ahead of the sought-after file. These limitations to tape backups can be circumvented using a relatively new technology, variously referred to as frozen imaging, point-in-time imaging, or snapshotting. Snapshots are fully useable disk-based copies that reflect the state of the original data at an earlier point-in-time. The Volume Shadow Copy service, new to Windows Server 2003 and Windows Storage Server 2003, is the infrastructure that provides built-in snapshotting (shadow copy) capabilities. Shadow copies created by Volume Shadow Copy service augment the storage administrator s tape backup archival solutions, providing high fidelity point-in-time copies that can be created and restored easily and effectively, thereby helping to simplify several aspects of storage and data management, as explained in the following section. High Fidelity Copies for Backup and Rapid Restores 1) Backups Shadow copy creation through the Volume Shadow Copy service is designed to be both an extremely fast means of creating a point-in-time image for backups on the order of seconds to a minute, depending on the amount of data and a process that produces high fidelity images without disrupting production servers. Because shadow copy creation is non-intrusive, point-in-time copies can be made more frequently than tape backups, thereby reducing the amount of time a businesses data is at risk should an outage occur. 2) Restores Scheduling non-disruptive tape backups has always been the system administrator s concern. Successful and rapid data restores, on the other hand, is a concern that the entire organization shares. Unfortunately, it is also the part of the process that most often fails. Recognizing this, organizations are taking steps to ensure that the more complex operation of data restoration from tape, which often must be done manually, is tested and proven effective before a problem arises. 11. Appendix - 100

105 Using shadow copies to restore data from disk provides a simple and extremely fast means of recovering data. The VSS infrastructure can be used to restore both large amounts of data, and individual files or folders: Large Data Restores. Shadow copies can protect single or multiple volumes from the risks of data loss. By default, shadow copies are read-only; in the event of data loss, they can be changed to read/write status and restored for use. Unlike tape restores, the shadow copy restore process is very rapid minutes rather than hours (or more). Individual File or Folder Restores. The Shadow Copies for Shared Folders feature lets administrators determine which volumes to copy and to set the frequency of shadow copy creation. The Previous Versions feature enables end users to quickly recover point-in-time copies of individual files or folders that have been accidentally deleted or overwritten. File recreation or lengthy single file restores from tape are unnecessary. Instead, end users can restore earlier versions of their own data by using a point and click process that takes only a few seconds. High Fidelity Backups The biggest cause of un-restorable backups is the lack of coordination between the backup application and end-user applications, such as databases. If applications are open and being written to during the backup, the saved data is inconsistent. VSS, with its shadow copy technology, solves this problem by coordinating with backup service and the application to ensure that no writes to disk are made during the backup process. Should there be a problem during the shadow copy creation, VSS notifies the requestor that the shadow copy is inconsistent, and the backup can be rescheduled. The net result of these two factors is high data integrity. Transport VSS enables several advanced storage management solutions based on the ability to transport data between servers accessing the same storage array. By design, storage allocated to one server is not accessible by another server; if it were, the stored data could be simultaneously written to by both servers, which would be disastrous. In order to share data between systems, the source data must be copied and the shadow copy made available to the second system. These shadow copies can then be used to: Offload backups to another server. Make a shadow copy of production database for testing purposes. Keep a shadow copy of data nearby for quick restore. The net result is enhanced data protection, and better data management. 11. Appendix - 101

106 E. Essentials of Veritas NetBackup Administration The NetBackup Administration Console provides a graphical-user interface through which the administrator can manage NetBackup. The console is available in two forms: A Windows-based interface to run on a Windows system. A Java-based interface able to run on a Windows system. This interface is called the NetBackup-Java Windows Display Console and is not automatically installed on the system. Installation is available on the main NetBackup for Windows installation screen. Using the NetBackup Administration Console The NetBackup Administration Console provides a Windows-based interface through which the administrator can manage NetBackup. 11. Appendix - 102

107 User Backups, Archives, and Restores To perform backups, restores and archives for this system and other clients, open the client interface by clicking on the Backup, Archive, and Restore button in the toolbar. Users can back up, archive, and restore files, directories, and raw partitions that reside on their own client computer. A user can restore files at any time but can back up and archive only during the time periods that the administrator defines with the schedules. Users can also view the progress and final status of the operations they perform. Activity Monitor Use the Activity Monitor to monitor and control NetBackup jobs, services, processes, and drives. E.1 NetBackup Management Reports Use Reports to compile information for verifying, managing, and troubleshooting NetBackup operations. Policies Use Policies to create and specify the backup policies which define the rules for backing up a specific group of one or more clients. For example, the backup policy specifies when automatic backups will occur for the clients and when users can perform their own backups. The administrator can define any number of backup policies, each of which can apply to one or more clients. A NetBackup client must be covered by at least one backup policy and can be covered by more than one. The properties of a backup policy include the following: General attributes that define the: Priority of backups for this policy relative to backups for other policies. Storage unit to use for backups of clients covered by this policy. Volume pool to use for backups performed according to this policy. A volume pool is a set of volumes that the administrator can assign to specific backup policies or schedules. For example, it is possible to have one volume pool for weekly backups and another for quarterly backups. List of client computers covered by the policy. 11. Appendix - 103

108 List of files to include in automatic backups of the clients. The backup selection list does not affect user backups because the user selects the files. Schedules that control when backups and archives can occur for the clients. As mentioned above, each backup policy has its own set of schedules. These schedules control when automatic backups start and also when users can start a backup or archive. Each schedule is unique with attributes that include: Type of schedule. Specify schedules for automatic full or incremental backups or user backups or archives. There are also schedule types that apply only when separately-priced options are installed (for example, a backup schedule for Microsoft Exchange or Oracle databases). Backup window. For automatic full or incremental backup schedules, this is the time period when NetBackup can start automatic backups of clients covered by this policy. For user schedules, this is the time period when users can start a backup or archive of their own client. Frequency. How often automatic and calendar-based backups should occur and which dates should be excluded from the schedule (dates when backups should not occur). Retention. How long NetBackup keeps the data that is backed up by this schedule. Storage unit. The storage unit for the data that is backed up by this schedule. This setting, if used, overrides the storage unit specified at the backup policy level. Volume pool. The volume pool to use when saving data backed up by this schedule. This setting, if used, overrides the volume pool specified at the backup policy level. The administrator can also manually start a backup schedule for an automatic full or incremental backup. Manual backups are useful if, for example, a client system is down and misses its scheduled backup. Configuring Backup Policies: The easiest way to set up a backup policy is to use the Backup Policy Configuration Wizard. This wizard guides you through the setup process, simplifying the process by automatically choosing default values that are good for most configurations. 11. Appendix - 104

109 Policy Planning Guidelines for Backups Policies allow you to meet the needs of a wide variety of clients in a single NetBackup configuration. However, taking full advantage of policies for use in backups requires careful planning before starting your configuration. The following procedure provides planning guidelines. 1. Divide clients into groups according to the types of work they perform. 2. Gather information about each client. Include information relevant to the backups such as the names, size, and number of files. 3. Create backup policies to accommodate special storage requirements. 4. Create additional backup policies if one set of schedules does not accommodate all clients and files. Best times for backups to occur. How frequently the files change. How long backups have to be retained. 5. Create separate policies for clients that require different general attribute settings than other clients. 6. Create separate policies as necessary to maximize the benefits of multiplexing. 7. Evaluate total backup times for each schedule and further subdivide your policies to reduce backup times to an acceptable level. When to Use Multiple Data Streams Reduce Backup Time Multiple data streams can reduce the backup time for large backups. This is achieved by splitting the backup into multiple streams and then using multiplexing, multiple drives, or a combination of the two for processing the streams concurrently. In addition, configuring the backup so each physical device on the client is backed up by a separate data stream that runs concurrently with streams from other devices can significantly reduce backup times. Note For best performance, use only one data stream to back up each physical device on the client. Multiple concurrent streams from a single physical device can adversely affect backup times because the heads must move back and forth between tracks containing files for the respective streams. 11. Appendix - 105

110 Setting the Number of Streams That Can Run Concurrently The number of streams that can run concurrently for a policy or client is determined by the following: Storage unit and schedule multiplexing limit Number of drives that are available Maximum concurrent jobs settings for the policy and client Each storage unit and each schedule has a maximum multiplex setting. The lower of the two settings is the limit for a specific schedule and storage unit. The maximum number of streams that can be multiplexed is limited to the sum of the multiplexing limits for all drives available in the storage unit and schedule combinations. For example, assume there are two storage units with one drive in each. MPX on storage unit 1 is set to 3 and MPX on storage unit 2 is set to 5. If MPX is set to 5 or greater in the schedules, then 8 streams can run concurrently. The maximum jobs settings also limit the maximum number of streams: Maximum Jobs Per Client (Host Properties > Master Server > Global NetBackup Attributes) Limit jobs per policy (policy attribute) Maximum Data Streams (Set the number in Host Properties > Master Server > Client Attributes) The maximum job settings are interdependent as follows: o If Maximum Data Streams is not set, the lowest value of Maximum Jobs Per Client and Limit Jobs Per Policy is the limiting factor. 11. Appendix - 106

111 o If Maximum Data Streams is set, then NetBackup ignores Maximum Jobs Per Client and uses the lowest value of Maximum Data Streams and Limit Jobs Per Policy as the limiting factor. Storage Units Use Storage Units to display storage unit information and provide commands for managing NetBackup storage units. A storage unit is a group of one or more storage devices of a specific type and density that attach to a NetBackup server. The media can be removable (such as tape) or a directory on a hard disk. Removable media can be in a robot or a standalone drive. The devices in a removable-media storage unit must attach to a NetBackup master or media server and be under control of Media Manager. The administrator first sets up Media Manager to use the drives, robots, and media and then defines the storage units. During a backup, NetBackup sends data to the storage unit specified by the backup policy. Media Manager then picks an available device within the storage unit. When the storage unit is a directory on a hard disk, the administrator specifies the directory during configuration and NetBackup sends the data to that directory during backups. Media Manager is not involved. Storage units simplify administration because once they are defined, the NetBackup configuration points to a storage unit rather than to the individual devices it contains. For example, if a storage unit contains two drives and one is busy, NetBackup can use the other drive without administrator intervention. Disk Storage Unit Considerations A disk type storage unit consists of a directory on a hard disk that stores the backup or archive data. NetBackup permits an unlimited number of disk storage units. A disk type storage unit is useful for testing and is useful during busy periods because it allows quick backups. However, you must be careful that it does not fill up your disk. Before using a disk storage unit, configure the disk as explained in your operating system documentation. To calculate the approximate disk space that NetBackup requires as it creates backups, use the following formula: (Largest backup size x (number of backups + 1)) + Space for the restores that are concurrent with backups 11. Appendix - 107

112 Catalog Use Catalog to create and configure a special type of backup NetBackup requires for its own internal databases a catalog backup. These databases, called catalogs, are on the NetBackup serverʹs disk and have setup information as well as critical information on client backups. The catalog backups are set up and tracked separately from other backups to ensure recovery in case of a server crash. Catalog is also used to search for a backup image in order to verify the contents of media with what is recorded in the NetBackup catalog, to duplicate a backup image, to promote a backup image from a copy to the primary backup copy, to expire backup images, or to import expired backup images or images from another NetBackup server. Host Properties Use Host Properties to customize NetBackup configuration options. In most instances, no setting changes are necessary. However, Host Properties settings allow the administrator to customize NetBackup to meet specific site preferences and requirements for master server, media servers, and clients. Media and Device Management The software that manages the removable media and storage devices for NetBackup is called Media Manager. This software is part of NetBackup and is installed on every NetBackup server. The administrator can configure and manage media through Media and Device Management in the NetBackup Administration Console. Access Management Customers can protect their NetBackup configuration by using Access Management to define who may access NetBackup and what functions a user in a user group can perform. 11. Appendix - 108

113 E.2 NetBackup Configuration Wizards The easiest way to configure NetBackup is to use the configuration wizards. The wizard selection is visible in the Details pane on the right varies depending on what NetBackup node is selected in the left portion of the screen. Getting Started Use the Getting Started Wizard if you are configuring NetBackup for the first time. It leads you through the necessary steps and other wizards to get you up and running with a working NetBackup configuration. Configure Storage Devices Use the Device Configuration Wizard to guide you through the entire process of configuring a device and storage unit. Configure Volumes Use the Volume Configuration Wizard to guide you through the entire process of configuring removable media. Configure the Catalog Backup Use the NetBackup Catalog Backup Wizard to set up your catalog backups, which are essential to recovering your data in case of a server failure or crash. Create a Backup Policy Use the Backup Policy and Configuration Wizard to add a backup policy to your configuration. Import Images Wizard Use the Import Images Wizard to import NetBackup images in a two-part process. Configure a Shared Drive Use the Device Configuration Wizard to configure NetBackup to use shared drives or to reconfigure an existing shared drive. 11. Appendix - 109

114 E.3 Disk Staging Storage Unit Considerations Disk staging provides a method for administrators to create images on disk initially, then later copy the images to another media type (as determined by the disk staging schedule). The later media type would typically be tape, but could be disk or disk staging. This two-stage process allows the NetBackup administrator to leverage the advantages of disk-based backups in the near term, while preserving the advantages of tape-based backups for long term. Disk staging may be appropriate for your NetBackup environment to meet the following objectives: To allow backups when tape drives are scarce. To allow for faster restores from disk. To facilitate streaming to tape without image multiplexing. Disk Staging is conducted in two separate operations: Stage I: A backup creates an image on the disk staging storage unit. Stage II: A relocation schedule determines when the image from the disk staging storage unit should be relocated to the destination storage unit. The image continues to exist on both the disk staging storage unit and the destination storage unit. File restores are done from the disk staging storage unit copy, while the destination storage unit copy can be considered the long term copy. 11. Appendix - 110