EMC DiskXtender Release 6.5 SP7

Transcription

1 EMC DiskXtender Release 6.5 SP7 Microsoft Windows Version Administration Guide P/N REV 01 EMC Corporation Corporate Headquarters: Hopkinton, MA

2 Copyright EMC Corporation. All rights reserved. Published September, 2014 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC 2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to EMC Online Support ( 2 EMC DiskXtender Microsoft Windows Version Administration Guide

3 Figures Title Page 1 File System Manager Administrator window About dialog box EMC Centera C-Clip for a DiskXtender file Embedded BLOB for a file stored on EMC Centera Separate CDF and BLOB for a file stored on EMC Centera Single thread per piece of virtual media Simultaneous moves and fetches Random migration to EMC Centera Media service wizard - Centera Information page Options for automatic creation of EMC Centera media Tiered migration DiskXtender with NAS DiskXtender connection with MediaStor Single DiskXtender server connecting to a single MediaStor server Single DiskXtender server connecting to multiple MediaStor servers Multiple DiskXtender servers sharing a MediaStor server MediaStor Administrator DiskXtender with ACSLS Available Media tree Media group Automation page Compact media task Format media task after Compact media task Label media task after Compact and Format media tasks DiskXtender with TSM DiskXtender with cloud storage Disabling real-time moves Standard migration Multi-target migration Multi-target media group Sample file migration pattern for multi-target migration Tiered migration DiskXtender architecture for indexing Disabling real-time moves Indexing attributes on the DiskXtender File Properties dialog box Disabling index rule qualification Disabling the indexing schedule Promote/demote arrow buttons Media Properties dialog box Add Counters dialog in Microsoft Windows 2008 Windows Performance Monitor Extended Drive Meta-Data Export dialog box EMC DiskXtender Microsoft Windows Version Administration Guide 11

4 Figures 41 Full Meta-Data Export Frequency dialog box Export folders created in Microsoft Windows Explorer EMC DiskXtender Microsoft Windows Version Administration Guide

5 Tables Title Page 1 Supported removable media types Supported DiskXtender environments for License Server Types of retention Supported CenteraStar versions Best practices for EMC Centera media service options Extended drive operations for EMC Centera files NAS media types Extended drive operations for Standard NAS files Extended drive operations for aggregate NAS files Extended drive operations for retained NAS files Minimum and recommended hardware requirements for MediaStor Supported removable media types FIle system support by media type Subsystems for the OTG file system UDF file systems Status of media in the Available Media tree Extended drive operations for files migrated to MO and UDO media Extended drive operations for files migrated to MO WORM or UDO WORM media Extended drive operations for files migrated to tape or tape-worm media Extended drive operations for files migrated to DVD media Media group options for removable media Service options for optical and tape media DiskXtender edits for TSM dsm.opt file Extended drive operations for TSM files Extended drive operations for files migrated to cloud media Product interoperability and international character support Background scan tasks Scan maintenance options Available criteria for selecting files to move Recommended media group options for each media type Types of retention Retention editing options Icons for files on the extended drive Prerequisite steps for purging Purge process overview by purge method Recommended environments for each purge method Available criteria for selecting files to purge File recovery requirements by media type Event Viewer icons for events, warnings, and errors EMC DiskXtender Microsoft Windows Version Administration Guide 13

6 Tables 40 Media status indicated by color Media Properties dialog box tabs Reports Available events for audit logging Access to the audit log for DiskXtender security groups Audit log fields Recommended media protection methods Compatible media types for copy media File recovery requirements by media type Media support for moving files within a media folder DiskXtender edits for TSM dsm.opt file Extended drive options to adjust for a storage transition Media fill methods EMC DiskXtender Microsoft Windows Version Administration Guide

7 Preface As part of an effort to improve and enhance the performance and capabilities of its product lines, EMC periodically releases revisions of its hardware and software. Therefore, some functions described in this document may not be supported by all versions of the software or hardware currently in use. For the most up-to-date information on product features, refer to your product release notes. If a product does not function properly or does not function as described in this document, please contact your EMC representative. Audience Related documentation Conventions used in this document This document is part of the EMC DiskXtender for Windows documentation set, and is intended for use by system administrators responsible for installing software and maintaining the servers and clients on a network. Readers of this document are expected to be familiar with the following topics: Microsoft Windows network administration Storage media and hardware device management Related documents include: EMC DiskXtender Release 6.5 SP 7 Microsoft Windows Version Installation Guide EMC DiskXtender Release 6.5 SP 7 Microsoft Windows Version Release Notes EMC DiskXtender Release 6.5 SP 7 Microsoft Windows Version Utilities Guide EMC uses the following conventions for special notices. Note: A note presents information that is important, but not hazard-related.! CAUTION A caution contains information essential to avoid data loss or damage to the system or equipment.! IMPORTANT An important notice contains information essential to operation of the software. EMC DiskXtender Microsoft Windows Version Administration Guide 15

8 Typographical conventions EMC uses the following type style conventions in this document: Normal Used in running (nonprocedural) text for: Names of interface elements (such as names of windows, dialog boxes, buttons, fields, and menus) Names of resources, attributes, pools, Boolean expressions, buttons, DQL statements, keywords, clauses, environment variables, functions, utilities URLs, pathnames, filenames, directory names, computer names, filenames, links, groups, service keys, file systems, notifications Bold Used in running (nonprocedural) text for: Names of commands, daemons, options, programs, processes, services, applications, utilities, kernels, notifications, system call, man pages Used in procedures for: Names of interface elements (such as names of windows, dialog boxes, buttons, fields, and menus) What user specifically selects, clicks, presses, or types Italic Used in all text (including procedures) for: Full titles of publications referenced in text Emphasis (for example a new term) Variables Courier Courier bold Used for: System output, such as an error message or script URLs, complete paths, filenames, prompts, and syntax when shown outside of running text Used for: Specific user input (such as commands) Courier italic Used in procedures for: Variables on command line User input variables < > Angle brackets enclose parameter or variable values supplied by the user [ ] Square brackets enclose optional values Vertical bar indicates alternate selections - the bar means or { } Braces indicate content that you must specify (that is, x or y or z)... Ellipses indicate nonessential information omitted from the example Where to get help Your comments EMC support, product, and licensing information can be obtained as follows: Product information For documentation, release notes, software updates, or information about EMC products, go to EMC Online Support at: Technical support Go to EMC Online Support and click Service Center. You will see several options for contacting EMC Technical Support. Note that to open a service request, you must have a valid support agreement. Contact your EMC sales representative for details about obtaining a valid support agreement or with questions about your account. Your suggestions will help us continue to improve the accuracy, organization, and overall quality of the user publications. Please send your opinion of this document to: techpubcomments@emc.com 16 EMC DiskXtender Microsoft Windows Version Administration Guide

9 If you have issues, comments, or questions about specific information or procedures, please include the title and, if available, the part number, the revision (for example, A01), the page numbers, and any other details that will help us locate the subject you are addressing. EMC DiskXtender Microsoft Windows Version Administration Guide 17

10 18 EMC DiskXtender Microsoft Windows Version Administration Guide

11 1 The DiskXtender Advantage The following topics provide an introduction to and describe the benefits of EMC DiskXtender for Windows: Seamless extension of disk space Automated file management Flexible support for back end storage The DiskXtender Advantage 19

12 The DiskXtender Advantage Seamless extension of disk space DiskXtender for Windows is so named because it extends the amount of space on a server s local New Technology File System (NTFS) and Resilient File System (ReFS) volumes, or hard drive partition. This can either be a volume physically located on the DiskXtender server or a fibre-connected drive that has been mapped as a local drive. DiskXtender extends the amount of space on a drive by migrating files from the drive to external storage media, while maintaining the appearance that the files are still on the drive. A drive that has been extended through DiskXtender is called an extended drive. The management of files on the extended drive through DiskXtender is actually a two-part process. Those two parts are referred to as a move and a purge and can be performed separately or simultaneously. In DiskXtender, a move is actually a copy. When DiskXtender moves a file to media, it is really copying the file data out to media and adding extended attribute information to the file on the drive. The file is then managed by DiskXtender, and the data resides both on the extended drive and on the media. After file data is moved to storage media, that data can be removed, or purged, from the extended drive, which frees that space for additional files. The purging of a file removes the file data from the drive, and leaves behind a file tag on the drive. The data then resides only on the media. The file tag serves two purposes: It allows the file to appear as if it still resides on the drive so that users can easily find it. It contains the extended file attribute information that DiskXtender needs to manage and retrieve the file from media. To a user who retrieves files from a drive extended by DiskXtender, all files appear to be present, regardless of whether the file data is on the drive or only on storage media. When the user requests the files, DiskXtender accesses the media where the files are stored and retrieves the file data, and displays it for the user. This file retrieval process is called a fetch. Full NTFS compatibility Since DiskXtender uses the NTFS file system as the gateway to its remote storage management functions, all native features of NTFS are supported, including support for: Long filenames (up to 256 characters) Filenames with Unicode characters Folder and file-level security Offline files Regardless of the media type that is being used for remote storage, full security can be applied to any file or folder on an extended drive, even if the storage media itself does not support Microsoft Windows security (for example, magneto-optical WORM or tape). Note: Encryption of files and folders on the extended drive is not supported because the encryption prevents DiskXtender from gaining access to the files to manage them. 20 EMC DiskXtender Microsoft Windows Version Administration Guide

13 The DiskXtender Advantage ReFS compatibility Apart from NTFS, DiskXtender is also compatible with ReFS. While DiskXtender uses extended attributes to store migration information in NTFS, it uses reparse tags in ReFS. You can extend (or archive data) from both NTFS as well as ReFS volumes. All the DiskXtender functionality supported on NTFS (apart from incremental scan) is supported on ReFS. Native Windows network connectivity and security DiskXtender does not interfere with Windows networking. Users that access an extended drive, usually through network shares, can connect by using any installed transports and protocols. DiskXtender also does not interfere with Windows security. Users that access an extended drive are authenticated through native Windows security. The operating system manages security at all levels. Seamless extension of disk space 21

14 The DiskXtender Advantage Automated file management DiskXtender automates the migration of files to media by using a rule-based system. Rather than just migrate all files to media without distinction between the files, you can select which files should be moved to what types and pieces of media, and when those files are moved. After file data is moved to storage media, automatic file purge options can remove the file data from the drive, and leave behind a file tag so that the file can be retrieved when necessary. File purging can occur: Immediately for all files, as soon as they are moved to media When extended drive disk space is needed When the files meet other criteria, such as a certain age If necessary, you can protect migrated files from being edited or deleted during a specified time period. This protection is called file retention, and is required in some environments to comply with government and organizational regulations. Retention is available if you are migrating files from DiskXtender to one of the following devices: EMC Centera EMC Celerra with the File-Level Retention (FLR) file system Network Appliance (NetApp) NAS device with SnapLock software When you apply retention, the files are protected both on the extended drive and on the storage hardware device. In some instances, you may also want to delete files completely when they are no longer of use. You can either delete files manually (for example, through Windows Explorer), or you can delete files automatically through DiskXtender by using rule-based criteria. When you delete a file from the extended drive, the file is also deleted from any associated storage media, if the media allows it. 22 EMC DiskXtender Microsoft Windows Version Administration Guide

15 The DiskXtender Advantage Flexible support for back end storage DiskXtender supports many high-capacity storage media types, including several removable media and disk-based storage options. Table 1 on page 23 provides details on the supported removable media types. Table 1 Supported removable media types Media type Details Tape and tape-worm LTO, SDLT, DLT, SAIT, AIT, 9840, and 9940 DVD Optical and optical WORM DVD-R, DVD-RAM, and DVD-ROM magneto-optical, Ultra Density Optical These media types are supported through a wide variety of hardware devices. For a complete list, refer to the DiskXtender for Windows Supported Device List, available on EMC Online Support. Disk-based storage options include EMC CLARiiON, Celerra, EMC Isilon, EMC VNX/VNXe series, as well as many other Redundant Array of Independent Disks (RAIDs) and network-attached storage (NAS) devices. Essentially, a server with a network share that is available to DiskXtender can be used as storage media. DiskXtender considers all of these shared devices to be NAS media. DiskXtender also supports the use of an EMC Centera content-addressed storage (CAS) system, the first magnetic disk-based WORM device. DiskXtender and EMC Centera are tightly integrated to optimize file storage and retrieval performance. Several DiskXtender file migration and retention features are designed specifically to take advantage of the features available with an EMC Centera device. DiskXtender works effectively with the highly scalable virtual media delivered by the cloud storage system. The cloud media is elastic, enabling you to use as much media as your data demands. DiskXtender allows for file migration to both private as well as public cloud systems. DiskXtender supports the use of EMC Atmos. DiskXtender transparently manages data movement between the extended drive and the storage media. Communication with the hardware devices that contain the storage media is established through the creation of DiskXtender media services. DiskXtender transfers files to the media provided by the media services, and sends requests for pieces of media to the media services as needed. DiskXtender manages all functions related to the transfer of information to and from media, such as the migration and retrieval of files. The only role a media service plays is to provide access to the media so that DiskXtender can work with it. For a media service that provides access to removable media, this means that the service places the requested media into a drive or prompts the administrator of the media service to insert the appropriate piece of media. For a media service that provides access to disk-based devices, this means that the service provides access to a place where the data is written. You can use any or all of these media types with a single DiskXtender installation. Hardware scalability With DiskXtender, hardware can grow with your storage needs, and you do not need to reconfigure or reformat the extended drives. DiskXtender interfaces with the software that manages the storage devices. The devices are completely independent Flexible support for back end storage 23

16 The DiskXtender Advantage of the extended drives they support, even though DiskXtender controls the media in those devices. If you are using removable media, you can change hardware and relocate media without affecting the files on your extended drive or the ability of DiskXtender to find files on the storage media. DiskXtender stores an internal inventory of all online and offline media in the system. This inventory system means that removable media is not dedicated to a specific hardware device. When hardware configuration or removable media locations change, the internal inventory is automatically updated so that DiskXtender always knows where media is located. When file migration or retrieval is required, DiskXtender uses its internal inventory to automatically find the proper media for the requested file. Automated media management DiskXtender can automatically prepare media for file migration when it is needed, as well as reclaim space that is no longer used on some media types. If you are using removable media, DiskXtender can automatically format, label, and assign media for file migration from a group of blank media. You can also reclaim space on removable media by compacting it when wasted space exceeds a certain percentage. When you compact a piece of media, active files on the media (current versions of written files and files not marked for deletion) are written back to the extended drive and are remigrated to another piece of eligible media. You can then reformat the compacted media so that it can be used again. Compaction optimizes media usage even if file deletions are frequent. If you are using an EMC Centera device, DiskXtender can automatically create the "virtual" media that DiskXtender uses to simulate divisions of an EMC Centera cluster. The use of virtual media enables you to take advantage of the flexible file migration features available in DiskXtender. If you are using the cloud media, you can add EMC Atmos as a media service and create virtual media. EMC Atmos provides the big advantage of flexible media, that expands depending on your file migration needs. If you are using DVD-R media, DiskXtender can automatically finalize the media when it is full. Finalization closes the media, which prevents the writing of any more files to the media. Finalization also increases the stability of the media so that data is better protected. Offline media support Offline media support is very important for large systems with limited hardware resources and large storage requirements. DiskXtender fully supports offline media. With DiskXtender, media can be online or offline. When online, users can access any file on the media. If media is taken offline and a file is requested that is not on the extended drive, the user receives an error message and the DiskXtender system administrator is notified that the requested media must be mounted. One important benefit of DiskXtender offline media support is that the files on your extended drive are not affected by the state of the media. Unlike other products, where files disappear when media is taken offline, DiskXtender maintains file and folder information whether the media is online or offline. This enables you to take hardware devices offline for maintenance without concerns about whether DiskXtender will be able to find the files or media when the device is set online again. Users and applications can always see their files on an extended drive. In addition, if you know in advance what media will be offline for a period of time, you can copy certain files from the media back to the extended drive. This prefetch feature ensures that files are available for user requests even when media is offline. 24 EMC DiskXtender Microsoft Windows Version Administration Guide

17 2 Getting Started The following topics provide an overview of DiskXtender for Windows and instructions for configuring the system after installation: Understanding DiskXtender for Windows Starting the File System Manager Administrator Understanding the Administrator interface Verifying the DiskXtender version How to configure DiskXtender Licensing Getting Started 25

18 Getting Started Understanding DiskXtender for Windows EMC DiskXtender for Windows is an automated, policy-based file system archiving solution for long-term data retention. DiskXtender extends primary volumes by making them seem to have an unlimited amount of space. User data is automatically migrated from the local Windows NT File System (NTFS) or Resilient File System (ReFS) volume to secondary storage based on system policies. However, the files still appear to reside on the local volume. Users and applications can seamlessly recall the data from secondary storage when necessary for viewing or editing. For example, users on the network may typically save data to a drive on a Microsoft Windows file server. If the drive is an NTFS or ReFS volume, you can use DiskXtender to move the files to media, such as an EMC Centera content addressable storage (CAS) system or tape in a library managed by DiskXtender MediaStor, without affecting the file listing as seen by the end user. When the users need to access the data again at a later time, they open the files normally and the files are recalled from the media. The user is not aware of the retrieval process. 26 EMC DiskXtender Microsoft Windows Version Administration Guide

19 Getting Started Starting the File System Manager Administrator The File System Manager Administrator window provides a user-friendly interface that enables you to configure the DiskXtender system. The Administrator can run on the same server where DiskXtender is installed. It can also be installed as a Remote Administrator on a different computer, which enables you to configure one or more DiskXtender servers on the network. The DiskXtender installation guide provides details on remote administration. To start the Administrator, open the Microsoft Windows Start menu and select Programs > EMC DiskXtender File System Manager > DiskXtender Administrator. The DiskXtender File System Manager Administrator window appears, as illustrated in Figure 1 on page 27. When you open DiskXtender for the first time, the Initial Configuration Wizard helps you set up a media service, add a media folder, and set up a basic move rule and migration. This enables you to take advantage of DiskXtender immediately after you have installed it. The Initial Configuration Wizard appears the first time you start DiskXtender, and every time until you have configured a media service. You can use the wizard to set up an EMC Centera or NAS media service. Figure 1 File System Manager Administrator window When the Administrator opens, it automatically connects to all registered DiskXtender servers. If this is a full installation of DiskXtender (as opposed to a Remote Administrator installation), the local server is automatically registered, and therefore automatically appears in the Administrator. Starting the File System Manager Administrator 27

20 Getting Started After these connections have been made, the Administrator displays information relevant to each connected service. The items that appear in the tree view depend on the DiskXtender configuration. You must create the extended drive, media folders, media groups, and rules, and add media to the system before these items are listed. 28 EMC DiskXtender Microsoft Windows Version Administration Guide

21 Getting Started Understanding the Administrator interface The main portion of the File System Manager Administrator window, which is illustrated in Figure 1 on page 27, is for navigation and information display, and is split into three panes: The left pane of the window, or the tree view, contains the tree-like structure from which most commands are performed. The top right pane, or the contents view, displays the contents of the item currently selected in the tree. The bottom right pane, or the description view, displays a description or detailed properties of the item selected. The Administrator window also contains several additional components that enable you to navigate through and configure DiskXtender: The menu bar contains the menu commands and can be found at the top of the window. The Computer drop-down list, which enables you to switch between DiskXtender servers that have been configured for remote administration, can be found just below the menu bar on the left. The toolbar, which contain toolbar icons that enable you to perform frequently used functions, can be found just below the menu bar to the right of the Computer drop-down list. The status bar, which displays information about the selected command or toolbar icon, as well as errors or warnings that occur, can be found at the bottom of the window. Advanced mode The Advanced mode is an option to display or hide advanced options and features available in the user interface. You can enable the Advanced mode from the Options tab of the Service Properties dialog box. When the Advanced mode option is enabled, all the advanced features and options in DiskXtender become available to the user, depending on the media services that are configured. By default, the Advanced mode option is disabled and should be enabled only for advanced users. Setting a timeout interval You can specify a timeout interval for the File System Manager Administrator interface. Understanding the timeout interval If the DiskXtender server is busy when you issue a command through the File System Manager Administrator, DiskXtender waits for the configured timeout interval. If the server is unable to respond to the command within the timeout interval, control is returned to the Administrator so that you can continue system configuration. DiskXtender uses the information from the last successful refresh to enable continued user input. Understanding the Administrator interface 29

22 Getting Started If the server remains unresponsive for two or more consecutive refresh cycles, a message appears in the tree view to indicate the condition. If the server remains unresponsive for 10 consecutive refresh cycles, the Administrator interface automatically disconnects from the server. In the event of a timeout error, you might be prompted to retry or cancel the request. Searching in the Administrator As you add multiple components, such as extended drives, media folders, and media, and multiple DiskXtender servers for remote administration to the system, the extended drive tree might become so large that it becomes difficult to locate a specific node of the tree. You can use the Find command in the Tree menu to quickly locate occurrences of text within the extended drives tree. By right-clicking the media group, you can select one of these options: View All Media View Only Full Media View Only Active Media If you choose View Only Full Media or View Only Active Media for a media group, then Full Media Only or Active Media Only are appended to the media group name in the tree view. 30 EMC DiskXtender Microsoft Windows Version Administration Guide

23 Getting Started Verifying the DiskXtender version The About dialog box, illustrated in Figure 2 on page 31, provides details on the version of DiskXtender that is installed, including the release number, as well as any service packs, patches, or hotfixes that have been applied. Version information is provided for both the client (the Administrator interface), as well as for each DiskXtender server/service that the Administrator is managing. Figure 2 About dialog box To access the About dialog box, open the Help menu in the Administrator and select About EMC DiskXtender Administrator. Verifying the DiskXtender version 31

24 Getting Started How to configure DiskXtender Configuring DiskXtender involves adding a media service, assigning an extended drive, selecting files that will be moved from the primary storage to secondary storage and setting up the migration schedule. You can use any of the following methods to configure DiskXtender: Configuration through the Initial Configuration Wizard on page 32 Configuration through menu options on page 32 Configuration through the Initial Configuration Wizard When you open DiskXtender for the first time, the Initial Configuration Wizard helps you set up a media service, add a media folder, and set up a basic move rule and migration. The Initial Configuration Wizard appears the first time you start DiskXtender, and every time until you have configured a media service. You can use the wizard to set up an EMC Centera or NAS media service. If you have not yet set up a media service, you can access the Initial Configuration Wizard from the Service menu > Initial Setup Wizard. Note: The Initial Configuration Wizard is disabled for DiskXtender installations on a cluster. The wizard does not pop up the first time the DiskXtender administrator window is opened on a cluster node, nor can it be accessed from the Service menu. Configuration through menu options To configure DiskXtender after installation: 1. Connect DiskXtender to a storage device: To use an EMC Centera cluster, provide the connection information to DiskXtender and create virtual pieces of media that simulate divisions of the EMC Centera cluster. Chapter 3, Connecting to EMC Centera, provides details. To use a share on a network-attached storage (NAS) device, create the shares, and connect DiskXtender to the shares. Each share is considered an individual piece of media in DiskXtender. Chapter 4, Connecting to Network-Attached Storage, provides details. To use a storage device with removable media such as tape or optical, connect DiskXtender to the software that manages the device (either EMC DiskXtender MediaStor or StorageTek ACSLS) and add media to the device. Chapter 5, Connecting to Optical and Tape Devices, provides details. To use an IBM Tivoli Storage Manager (TSM) system to store DiskXtender files, install the TSM client on the DiskXtender server, connect DiskXtender to the TSM server, and then create virtual pieces of media that simulate divisions of the TSM server. Chapter 6, Connecting to Tivoli Storage Manager, provides details. To use the cloud media to move files using DiskXtender, configure the cloud media (EMC Atmos) as a media service and create virtual cloud media. Chapter 7, Connecting to Cloud Storage, provides details. 2. Identify the Microsoft Windows NTFS or ReFS drives on the DiskXtender server to extend by using DiskXtender file migration services. These drives are called extended drives. Requirements and details on creating and managing these drives are available in Chapter 8, Extended Drive and Volume Mount Point Management. 32 EMC DiskXtender Microsoft Windows Version Administration Guide

25 Getting Started 3. Identify or create the folders on the extended drive that will contain the files to move to storage. These folders are called media folders in DiskXtender. A media folder can be a subfolder on the extended drive, or you can manage all the files on the extended drive by specifying the root of the drive as a media folder. Extended drive directory structure on page 65 provides details on planning for and creating media folders. 4. Configure file migration. In DiskXtender, the migration of a file is actually a copy. When DiskXtender moves a file to storage, it is really copying the file data to media and adding extended attribute information to the file on the drive. The file is then managed by DiskXtender, and the data resides both on the extended drive and on the media. To enable file migration, specify the criteria that files must meet before DiskXtender migrates them. You also must specify the media to which the files should be moved. Details on the available options for file migration are provided in Chapter 9, File Migration. 5. Configure file purging. When DiskXtender purges a file, it removes the file data from the extended drive and leaves behind a file tag on the extended drive. Purging files frees space on the extended drive, while maintaining the appearance that the files are still on the drive. A purged file is still accessible, and to a user machine, appears to reside on the extended drive. You can choose to automatically purge files from the extended drive either: Immediately after they are moved to storage When extended drive space is low Each night during the next background scan Chapter 12, Purging Files, explains how to develop an appropriate purging strategy and enable purging. 6. (Optional) If you are planning to index and search for files on the extended drive by using the DiskXtender Search Module, then configure file indexing. Guidelines for developing an indexing strategy and instructions for enabling indexing are available in Chapter 11, Indexing Files. 7. (Optional) Enable the DiskXtender Recycler to protect against the accidental deletion of files from the extended drive. If the DiskXtender Recycler is enabled and a user accidentally deletes a file, you can restore the file to the extended drive. The DiskXtender Recycler functions similarly to the Microsoft Windows Recycle Bin, but only affects files on a drive extended by DiskXtender. Recovering files from the DiskXtender Recycler on page 124 provides details on the Recycler. Note: The Recycler resides on the extended drive. As a result, the disk space used by the Recycler factors into the total available disk space for the extended drive. If you enable the Recycler, set a schedule for emptying the Recycler on a regular basis to help maintain an adequate amount of space on the extended drive. 8. (Optional) Configure automatic deletion of files. DiskXtender can automatically delete files from the extended drive based on certain file criteria by using delete rules. Delete rules can be used, for example, to help manage archival of particular files kept to comply with legal requirements. After the files no longer meet the criteria required to keep them (for example, the files reach a certain age), DiskXtender can automatically delete them in accordance with a delete rule. How to configure DiskXtender 33

26 Getting Started If you are protecting files with retention for a period of time, you may want to automatically delete the files when the retention expires and the files are no longer needed. Automatically deleting files on page 120 provides instructions on creating delete rules. 9. Develop a comprehensive backup and recovery strategy for the DiskXtender system. Chapter 16, Backup and Recovery, provides guidance on the aspects of the DiskXtender system that must be protected, as well as best practices for the tools needed for backup and recovery. 10. After you configure DiskXtender, monitor the system regularly by using the tools discussed in Chapter 15, Monitoring the System. 34 EMC DiskXtender Microsoft Windows Version Administration Guide

27 Getting Started Licensing DiskXtender licensing is capacity-based. A capacity-based license specifies the number of DiskXtender servers allowed, as well as the quantity of extended drive data that DiskXtender can manage. If you are installing DiskXtender in a Microsoft clustering environment or AutoStart domain, the license must account for each active File System Manager installation, and it must also be cluster-enabled. Additional licensing details are available in the DiskXtender installation guide. License Server Table 1 DiskXtender licenses are managed by the Xtender Solutions License Server product. The computer on which you install License Server depends on the environment: If License Server must manage only a DiskXtender license for a single DiskXtender installation, then it can be safely installed on the DiskXtender server without a significant impact on performance. If License Server must manage a DiskXtender license for multiple DiskXtender installations, then you may want to install it on a separate computer that is accessible to all of the DiskXtender servers. If License Server must manage licenses for both DiskXtender and multiple ApplicationXtender clients, then it should be installed on a separate computer to avoid an impact on DiskXtender server performance. The computer on which you install License Server determines the release of License Server that you need: If you plan to install License Server on a DiskXtender 6.5 SP 7 server, then use License Server 6.5 SP 7. If you plan to install License Server on a separate machine from DiskXtender, then the operating system on the machine determines the necessary License Server release. For a 32-bit operating system, use a 32-bit release of the License Server. For a 64-bit operating system, use a 64-bit release of the License Server. Table 2 on page 35 provides details on supported environments. Supported DiskXtender environments for License Server License Server release Supported DiskXtender environments 6.3 (32-bit) One or more DiskXtender 6.3 (32-bit) servers One or more DiskXtender 6.4 (64-bit) servers A mix of DiskXtender 6.3 and DiskXtender 6.4 servers in the same environment 6.4 (64-bit) One or more DiskXtender 6.3 (32-bit) servers One or more DiskXtender 6.4 (64-bit) servers A mix of DiskXtender 6.3 and DiskXtender 6.4 servers in the same environment 6.5 SP7 One or more DiskXtender 6.5 SP7 servers A mix of DiskXtender 6.3 and DiskXtender 6.4 servers in the same environment Licensing 35

28 Getting Started License Server 6.5 SP7 supports the following 32-bit operating systems: Microsoft Windows Server 2003 Standard, Enterprise, or Small Business Edition with SP2 Microsoft Windows Storage Server 2003 Standard or Enterprise Edition SP2 R2 Microsoft Windows Server 2008 Standard, Enterprise, or Small Business Edition SP2 License Server 6.5 SP7 supports the following 64-bit operating systems: Microsoft Windows 2003 Standard or Enterprise Edition with SP2 Microsoft Windows Server 2008 Standard or Enterprise Edition with SP2 and SP1 R2 Microsoft Windows 2012 Server Microsoft Windows 2012 Storage Server R2 The minimum system requirements for these operating systems are sufficient to run License Server. The License Server computer must belong to the same domain as all other DiskXtender servers. Before you install License Server, create a service account for License Server. The account should have access to the network and be a local administrator on the License Server computer. The necessary files to install License Server are available on the DiskXtender installation CD. A wizard leads you through the steps to install License Server. Evaluation licensing When you install File System Manager, you are prompted to select whether you are installing a licensed version or a 30-day evaluation version. If you choose a 30-day evaluation license, you must set up a license and point the program to the License Server before the 30 days expire, or the functionality of the product is disabled. Adding a license To obtain a product license and add the license for DiskXtender to use: 1. Determine the machine ID number for the License Server computer by opening the General tab of the Service Properties dialog box in the License Server Administrator after you install License Server. The machine ID is required to generate a product license. 2. Use the License Management process available on EMC Online Support, or contact the EMC Licensing Support team at licensing@emc.com. If you receive DiskXtender product license information in the form of a file with a.lic extension, you have a license file. If you receive DiskXtender product license information in the form of a series of numbers and letters, you have a license key. 3. Use the New License Wizard in the License Server Administrator to add the license file or key to License Server. To start the New License Wizard, open the Tools menu in the License Server Administrator, and select New License Wizard. 36 EMC DiskXtender Microsoft Windows Version Administration Guide

29 Getting Started 4. Connect DiskXtender to License Server: a. If License Server is installed on a different computer than DiskXtender, give the necessary user accounts the required privileges on the License Server computer: Add the DiskXtender service account to the local Administrators group on the License Server computer. If you log in to the DiskXtender server by using an account other than the DiskXtender service account, add the account to the local Administrators group on the License Server computer. b. Use the Edit product license information option on the DiskXtender setup wizard to point DiskXtender to the License Server installation with the license. Licensing in a cluster When you obtain a license for a DiskXtender installation in a cluster, ensure that the license is cluster-enabled and that the license accounts for each active DiskXtender installation in the cluster. In addition, decide whether to manage the DiskXtender license from within the cluster or on a server outside the cluster. When installed in the cluster, License Server runs actively on one node in the cluster. It can then fail over to one of the other nodes in the environment. In other words, License Server runs in active/passive mode in a cluster. You can assign the License Server cluster resources to the same virtual server group as the DiskXtender resources. This enables License Server to fail over to another node in the cluster at the same time as DiskXtender. Alternatively, you can create a separate virtual server for the License Server cluster resources. This virtual server could fail over separately from DiskXtender, if necessary, or it could fail over at the same time, if the node with both virtual servers fails. License Server runs active/passive even if DiskXtender is running active/active. This is true regardless of whether License Server resources are assigned to the DiskXtender virtual server. Licensing 37

30 Getting Started 38 EMC DiskXtender Microsoft Windows Version Administration Guide

31 3 Connecting to EMC Centera You can migrate files on a DiskXtender extended drive to an EMC Centera cluster. The following topics provide details on configuring and managing the environment: How DiskXtender works with EMC Centera Prerequisite steps to connect to EMC Centera Managing the lifecycle of EMC Centera media Expectations for files migrated to EMC Centera Adding custom metadata to files on EMC Centera Performance tuning for EMC Centera Connecting to EMC Centera 39

32 Connecting to EMC Centera How DiskXtender works with EMC Centera The following topics provide details on how DiskXtender communicates with EMC Centera, as well as how various features in the two products complement each other: Connecting DiskXtender and EMC Centera on page 40 Virtual EMC Centera media on page 42 Content addressable storage on page 43 Renaming EMC Centera folder on page 44 EMC Centera storage strategies on page 46 Communication threads on page 49 Compliance through retention and audit on page 54 Replication on page 56 Connecting DiskXtender and EMC Centera When you install or upgrade DiskXtender, the EMC Centera Software Development Kit (SDK) is installed automatically on the DiskXtender server to enable communication between DiskXtender and an EMC Centera cluster. When you create an EMC Centera media service in DiskXtender, you specify the connection string that the EMC Centera SDK uses to enable DiskXtender to connect to an EMC Centera cluster. The connection string for the primary EMC Centera includes information about the EMC Centera access nodes of primary EMC Centera, as well as the access profile that should be used for the DiskXtender connection. The connection string for the replica EMC Centera includes information about the EMC Centera access nodes of replica EMC Centera, as well as the access profile that should be used for the DiskXtender connection. DiskXtender connects with the access nodes by using a TCP/IP connection. Access nodes Each EMC Centera cluster contains two or more nodes with the access role. You should specify as many access nodes on the cluster in the connection string as possible. When the EMC Centera SDK attempts to open a connection to the cluster for DiskXtender, it attempts to connect to the first access node listed in the connection string. If the connection to the first access node is successful, the SDK queries for and internally stores information about all available access nodes in the cluster. Communication between EMC Centera and DiskXtender is then automatically load-balanced across this set of nodes. If the connection to the first access node fails, however, then the SDK attempts to connect to the second access node listed in the connection string. If the connection to the second access node fails, then the SDK attempts to connect to the third access node listed in the connection string, and so on. This process continues until the SDK is able to successfully connect to an access node. Once the connection is made, information about all available access nodes is stored by the SDK, regardless of the number of access nodes listed in the connection string. This connection process occurs each time the DiskXtender service restarts and each time the media service is set online. As a result, it is important to include as many access nodes on the connection string as possible, to ensure that DiskXtender can connect to EMC Centera even when one or more access nodes are offline. 40 EMC DiskXtender Microsoft Windows Version Administration Guide

33 Connecting to EMC Centera In addition, you cannot edit access node information in DiskXtender after you create the media service. As a result, you cannot add or remove node information for an existing media service. Because you cannot edit access node information in DiskXtender, equate each access node IP address with a node name alias before you create the media service. To configure node name aliases, either add the appropriate entries to a common DNS server or edit the local HOSTS file on the DiskXtender server. The connection string for a media service is limited to 512 characters. Keep this limit in mind when defining node name aliases. The aliases should be kept to a reasonable length. Otherwise, you may not be able to specify a sufficient number of access nodes on the connection string to enable reliable connections in the event of a node failure. You can then specify the aliases instead of the IP addresses when you create the media service. If you later need to change the access node IP addresses, you only need to change the HOSTS file or DNS entries. You might need to change the access node IP addresses if it becomes necessary to replace an access node. Note: If you add an access node to the EMC Centera cluster after you create the media service, you cannot add the address information to the media service connection string. However, DiskXtender can connect to the access node. This is because the EMC Centera SDK internally discovers and stores information about all available nodes after a successful connection is made to one of the current access nodes in the connection string. If you use EMC Centera replication, do not specify connection information for the access nodes of the target clusters. Replication failover on page 56 provides additional details. Pools and profiles EMC Centera security is based on pools and application profiles: An application pool, or virtual pool, is a logical area on an EMC Centera device where applications can store their data. Virtual pools enable you to logically separate data on an EMC Centera device. This is particularly useful if there are multiple applications with different security needs writing data to an EMC Centera device. Access profiles provide access to one or more EMC Centera pools. Pools grant capabilities to applications that are accessing EMC Centera by using the profile. For DiskXtender, the profile must have the Write, Read, Delete, and Query capabilities. To allow privileged deletes of retained files, the profile should also have the Privileged Delete capability. A.pea file, generated while creating or updating an access profile, is a clear text, XML-formatted, nonencrypted file that can be used by system administrators to communicate and distribute authentication credentials to application administrators. Each EMC Centera cluster can have multiple virtual pools and multiple access profiles. The EMC Centera online help provides information on pools, access profiles, and.pea files, which should be configured by an EMC Centera technical representative. When you create an EMC Centera media service in DiskXtender, specify a.pea file when you create the media service. You cannot edit a media service later to add a.pea file. There should be a single.pea file for each EMC Centera media service. You cannot specify multiple.pea files for a single EMC Centera media service. If the virtual pool is being replicated to another EMC Centera cluster, the.pea files for the virtual pools How DiskXtender works with EMC Centera 41

34 Connecting to EMC Centera and access profiles on the two clusters are merged. This merged.pea file, which enables access to both pools with a single profile, should be configured by an EMC Centera technical representative. If you have multiple, separate pools and access profiles each pair with its own corresponding.pea file then you can create multiple EMC Centera media services. Multiple media services enable you to further separate data you are writing through DiskXtender. If you do not specify a.pea file when you create an EMC Centera media service, then DiskXtender uses the Anonymous profile to connect to EMC Centera. Note: The Anonymous profile is disabled in EMC CenteraStar 3.1 and later. If CenteraStar 3.1 or later is installed on the EMC Centera cluster, you must use an access profile when you create a media service in DiskXtender. If you use an earlier release of CenteraStar with DiskXtender and you upgrade to 3.1 or later, you can continue to use the Anonymous profile. Virtual EMC Centera media EMC Centera media is virtual, meaning that a piece of media defined in DiskXtender does not specifically correspond to a physical piece of media (like a tape cartridge). Instead, the virtual media is designed to simulate divisions of an EMC Centera cluster. DiskXtender provides support for EMC Centera Virtual Access. You can use EMC Centera Virtual Access nodes with DiskXtender just like any other EMC Centera nodes. It is transparent to DiskXtender and all EMC Centera features continue to work same as before. The use of virtual media enables you to take advantage of the flexible file migration features available in DiskXtender. Each piece of virtual media can contain as much as 256 GB of file data and metadata, or can store up to 100,000 files. This is an either/or limit. It is possible that you can fill a piece of EMC Centera media with 100,000 files but remain well under the 256 GB size limit. When you create a piece of virtual media, the 256 GB of space is not reserved on the EMC Centera device. Space is used on the EMC Centera device only when a file is migrated from DiskXtender. Note: Because space is not reserved on the EMC Centera device for virtual media, if you meet the limit of 100,000 files before you meet the limit of 256 GB, the size of the piece of media is limited to the size required for the 100,000 files. In other words, the difference between the total size of the 100,000 files and 256 GB is not wasted space on the EMC Centera device. The size limit for EMC Centera media is imposed so that finding a file associated with a piece of media does not take an inordinate amount of time. When either threshold is reached, the media is considered full and cannot receive any more files, although files can be retrieved from the media when necessary. Full media appears with a blue label in the File System Manager Administrator. You can create as many pieces of virtual media as necessary. You are limited only by the total amount of space available on the EMC Centera device. Note: The size limit on EMC Centera media can be extended to 512 GB. To increase the size from the default 256 GB to 512 GB, contact EMC Customer Support. 42 EMC DiskXtender Microsoft Windows Version Administration Guide

35 Connecting to EMC Centera When a piece of virtual media is created in DiskXtender, a corresponding EMC C-Clip for the media is created on the EMC Centera device. Once the C-Clip is created, the content address (CA) of the C-Clip is sent back to DiskXtender and is stored in the Microsoft Windows registry on the DiskXtender server. When DiskXtender migrates a file to a piece of EMC Centera media, a C-Clip for the file is created on the EMC Centera device. Metadata about the piece of virtual media associated with the file is stored with the file on the EMC Centera device. In addition, DiskXtender creates a special C-Clip on the EMC Centera cluster daily at midnight (EMC Centera cluster time). This new C-Clip contains a list of all media C-Clips and folder rename C-Clips associated with those media currently on the system. One C-Clip is created for each EMC Centera media service. The CA for the C-Clip is stored in the DiskXtender event log when the C-Clip is created. If a disaster occurs, an EMC Customer Support Representative can retrieve the C-Clip to develop a list of media to restore to DiskXtender. There are two ways to create virtual EMC Centera media through DiskXtender: Automatically through the media group, either when free space in the media group falls below a certain number of megabytes or when the number of available pieces of media falls below a certain level. Manually through the media service. Use this method to create individual pieces of media. Then allocate the media to the extended drive and add it to a media group to make it available for file migration. This option is not recommended for most environments. To avoid confusion and simplify media naming conventions, use only one media creation method. If you no longer need the files that have been written to a piece of EMC Centera virtual media, you can run a Format media task on the media to clear the files from the EMC Centera device. The EMC Centera Garbage Collection feature then reclaims the space made available by the deleted files. Note: The Format task may take a significant amount of time to complete for EMC Centera media. Content addressable storage With content addressable storage (CAS), EMC Centera ensures that applications, such as DiskXtender, no longer have to track the physical location of stored information. Instead, with the CenteraStar software operating environment, EMC Centera creates a unique identifier, based on the attributes of the content, that applications use for retrieval. The unique identifier is called a content address (CA). The CA uniquely represents the object and its associated metadata, and is required to retrieve the object. The following topics provide details on how DiskXtender writes files to EMC Centera media, as well as how to customize file migration between DiskXtender and EMC Centera. How DiskXtender writes files to EMC Centera When DiskXtender writes a file to EMC Centera, a CA for the file (BLOB) is generated and stored in an XML file with standardized metadata from the DiskXtender system. The XML file is called a C-Clip Descriptor File (CDF). How DiskXtender works with EMC Centera 43

36 Connecting to EMC Centera Note: You can also add custom metadata to CDFs by using DiskXtender media groups. This custom metadata can be used to enhance EMC Centera Seek and Chargeback Reporter reports. Adding custom metadata to files on EMC Centera on page 70 provides details. Another CA for the CDF is generated, and then DiskXtender stores the CDF and the file data in a package on the EMC Centera cluster. This complete package, which contains both file data and metadata, is called a C-Clip and is illustrated in Figure 3 on page 44. C-Clip CDF (XML file) CA for CDF CA for file Metadata (filename, creation data, and so on) 3C08JM40C8AMMe0N8ATEJHC2DQN BLOB (file data) GEN Figure 1 EMC Centera C-Clip for a DiskXtender file The CA for the CDF is returned to DiskXtender and stored for the file. When the file is purged from the extended drive, this information can be used to retrieve the file data from EMC Centera. Renaming EMC Centera folder You can rename EMC Centera folder through the Microsoft Windows Explorer. The following topics provide details. What happens when you rename an EMC Centera folder Renaming an EMC Centera folder on the extended drive changes the folder name on the EMC Centera media. DiskXtender allows you to rename the folder and the subfolders on an EMC Centera media. Renaming of folders can be done any number of times. When you rename a folder, the transactions are logged to the folder rename master clip of each EMC Centera media associated with folder rename. When a media task such as compact, purge, fetch, prefetch, restore, or report is performed, the DiskXtender processes the folder rename transactions to restore files to the latest EMC Centera folder in the media. Note: Performance of EMC Centera media tasks may be affected as each of the media tasks involve processing the folder rename transactions log and restoring files to the latest folder. 44 EMC DiskXtender Microsoft Windows Version Administration Guide

37 Connecting to EMC Centera When the EMC Centera media is offline, you can continue to rename the folders in the extended drive. However, the synchronization of folder names on the EMC Centera will take place when the media is online. Because of the synchronization, there might be a delay in performing the media tasks as well as file fetching operations. Renaming EMC Centera folder uses the EMC Centera and DiskXtender time for logging the transaction and any change to the EMC Centera time or the DiskXtender server time may result in files retrieved to a different folder when a media task is performed. IMPORTANT Any system errors while processing folder rename transactions result in using the original EMC Centera folders. Note: While renaming a folder, if the recycler is enabled, and you delete or move the file under this folder to recycle bin. When you try to restore the same file, the file is restored to the original folder. How to enable EMC Centera folder rename To enable EMC Centera folder rename: 1. Right-click the extended drive in the DiskXtender Administration window. 2. Select Extended Drive Properties. Extended Drive Properties dialog box appears. 3. Select Options page. 4. Scroll down and select Centera folder rename option. 5. Select the Allow Centera folder rename radio button. 6. Click OK. IMPORTANT You must be a local administrator on the DiskXtender server to enable EMC Centera folder rename. To enable EMC Centera folder rename after an upgrade To be able to use the EMC Centera folder rename feature after upgrading to the latest release of DiskXtender, perform these additional post-upgrade steps: 1. Run the DxDmChk utility on the extended drive on which to enable folder renames. 2. In case of a cluster environment, run the DxDmChk/X option on all active cluster nodes.! IMPORTANT DxDmChk utility should be run after stopping DiskXtender service. Running DxDmChk utility without stopping the DiskXtender service can result in loss of data. How DiskXtender works with EMC Centera 45

38 Connecting to EMC Centera How to rename a EMC Centera folder To rename EMC Centera folder using Windows Explorer: 1. Open the Microsoft Windows Explorer and navigate to the EMC Centera folder on the extended drive. 2. Right-click the folder and select Rename. The folder name appears highlighted in the tree view. 3. Type a new name for the folder, and then press Enter. To rename EMC Centera folder using command-line interface, open the command-line interface and change path to the EMC Centera folder on the extended drive. Use the following command to rename the folder: c:\centerafolder> ren cenfolder1 cenfolder2 or c:\centerafolder> rename cenfolder1 cenfolder2 When the EMC Centera media is full and you rename an EMC Centera folder, the rename transaction is successful on the extended drive as well as on the EMC Centera media. Note: When multiple rename transactions are done on the extended drive and the user tries to fetch a file, there will be a delay in accessing the file. This is because, logging all rename transactions to the EMC Centera must be complete before fetching a file from the media. However you can change this behavior in the Extended drive option. When you rename a folder from a remote client that is being accessed in the local server, the folder rename transaction fails. Similarly, when you rename a folder from a local server that is being accessed from a remote client, the folder rename transaction fails. EMC Centera storage strategies Data can be stored on an EMC Centera device by using one of two storage strategies: Storage Strategy Capacity is designed to maximize the storage capacity of the cluster by taking advantage of single-instance storage. With single-instance storage, duplicate copies of a file are not stored on the cluster. Only a single copy of a file is stored. Storage Strategy Performance is designed to maximize file storage and retrieval performance at the cost of single-instance storage. With this storage strategy, smaller files with identical content may be stored multiple times. This is because it is faster to store duplicates than it is to perform the data comparison necessary to eliminate duplication. However, single-instance storage is still used to exclude duplicates of larger files. This is because the speed of the data comparison is equal to or faster than the amount of time necessary to store the file itself. Consult with an EMC Centera technical representative to decide which storage strategy is appropriate for your environment. Only qualified EMC Centera service personnel can change the storage strategy for a cluster. 46 EMC DiskXtender Microsoft Windows Version Administration Guide

39 Connecting to EMC Centera Note: DiskXtender also works with both EMC Centera data protection mechanisms; Content Protection Mirrored (CPM) and Content Protection Parity (CPP). The EMC Centera protection mechanism (CPM or CPP) is transparent to DiskXtender. DiskXtender options to tune CAS DiskXtender provides media service options that enable you to customize the way files are stored to EMC Centera, and therefore maximize performance: Collision avoidance Collision avoidance ensures that a unique CA is created for each file stored on EMC Centera, even if the file is a duplicate of another file stored on the cluster. If the file is edited and re-migrated, then a different unique CA is created. This feature is designed to prevent the unlikely event where the same CA is created for different files. If the files are purged and you attempt to fetch the second file, the data for the first file with the CA is returned instead, and the data for the second file is irretrievable. Collision avoidance should not be used in most environments because it prevents the primary benefit of single-instance storage the elimination of duplicate content so that only a single copy of each file is stored. It should be used only in environments where even the most remote possibility of data loss is unacceptable, or in environments where single-instance storage is forbidden, usually due to legal regulations. Note: If you enable collision avoidance, each file is stored uniquely, even if you use Storage Strategy Capacity (single-instance storage). Embedded BLOBs You can set a threshold for embedding small BLOBs (file data) in the CDF. Embedding file data in the CDF can decrease read and write times, since the overhead required to manage two objects (CDF and BLOB) is greater than the overhead required to manage a single object. Figure 4 on page 47 illustrates an embedded BLOB, while Figure 5 on page 48 illustrates a separate CDF and BLOB. CDF (XML file) CA for file Metadata (filename, creation data, and so on) DiskXtender server BLOB (file data) EMC Centera GEN Figure 2 Embedded BLOB for a file stored on EMC Centera How DiskXtender works with EMC Centera 47

40 Connecting to EMC Centera CDF (XML file) CA for file Metadata (filename, creation data, and so on) DiskXtender server BLOB (file data) EMC Centera GEN Figure 3 Separate CDF and BLOB for a file stored on EMC Centera To maximize the performance benefits of this feature, files smaller than 100 KB (the maximum value for the option) should be embedded in the CDF. If the EMC Centera is configured for Storage Strategy Capacity (single-instance storage), embedding BLOBs for files smaller than 100 KB will not allow you to realize the benefits of the storage strategy for these files. This is because the file content is embedded in the CDF. If you embed the BLOBs for files smaller than 100 KB, then single-instance storage is applied only to files larger than 100 KB. Note: The total file size (all streams and the stream tags) must be less than the embedded BLOB threshold before the BLOB is embedded in the CDF. Client-side ID calculation The Client-side ID (hash) calculation option enables you to control whether the EMC Centera API calculates the CA for a piece of data before the data is sent to the EMC Centera cluster. If the calculation results in the determination that the data already exists on the cluster, the data is not sent. Client-side ID calculation can improve performance and maximize storage efficiency under the following conditions: Files are larger than 10 MB in size. Identical data is likely to be sent to EMC Centera. You use Storage Strategy Capacity (single-instance storage). Client-side ID calculation is not beneficial under the following conditions: File data is embedded in the CDF (in other words, when you set an embedded BLOB threshold of greater than zero). Files are small (less than 10 MB). You use Storage Strategy Performance or collision avoidance and files are stored uniquely even if they are identical. Note: If you use CenteraStar or later, or or later, additional configuration steps are required on the EMC Centera device to enable client-side ID calculation. Contact an EMC Centera technical representative for assistance in enabling this feature if you use one of the specified CenteraStar versions. 48 EMC DiskXtender Microsoft Windows Version Administration Guide

41 Connecting to EMC Centera Garbage Collection When a file is deleted from the extended drive, a delete transaction is issued and flushed to the target storage device. Delete transactions can flush every minute, depending on what other transactions have been initiated. When EMC Centera receives a delete transaction from DiskXtender, the CDF is deleted from the EMC Centera device, and the BLOB is left there (orphaned). CDFs not under retention can be deleted from any type of EMC Centera device. If the EMC Centera Garbage Collection process is enabled, then the BLOBs with deleted CDFs are removed from the EMC Centera device and space is recovered. The file data for deleted files is removed from the EMC Centera device and is not recoverable. If Garbage Collection is disabled, however, then the BLOBs with deleted CDFs remain on the EMC Centera device and occupy space. Garbage Collection and the DiskXtender Recycler If the DiskXtender Recycler is enabled, files are placed in the Recycler when they are deleted from the extended drive. A delete transaction is sent to the EMC Centera cluster when the file is deleted from the Recycler. When a significant number of delete transactions are sent to the EMC Centera cluster, the completion of those transactions may take a long time. During that time, file fetches may be delayed or even canceled due to timeout issues. Do not empty the Recycler when users are most likely to access the system. Or, if the Recycler is disabled, do not delete a significant number of files from the extended drive when users are most likely to access the system. Communication threads Each access node of an EMC Centera has a certain number of available communication threads. The optimum performance level is reached when each EMC Centera access node processes approximately 20 simultaneous connections. You can exceed 20 connections for each access node. However, performance degradation typically results. When an application communicates with an EMC Centera cluster, it uses one or more of these threads to pass data to and from the EMC Centera cluster. With DiskXtender, these threads are used to read data from and write data to the EMC Centera cluster. It is important to understand the communication process between DiskXtender and EMC Centera so that you do not exceed the maximum number of available communication threads. The following topics provide additional details and guidance on configuring options that enable you to control the lines of communication and maximize performance. How DiskXtender uses the communication threads By default, DiskXtender can either read data from or write data to each piece of virtual media during any given time period. In other words, if DiskXtender is writing File G to a piece of media and a user requests File A from that same piece of media, then the user must wait until File G is written before File A is retrieved and displayed. The writing of File G and the reading of File A both would require a single thread, each at different times, as illustrated in Figure 6 on page 50. How DiskXtender works with EMC Centera 49

42 Connecting to EMC Centera Step 1: File G is migrated to Virtual Media 1 on the EMC Centera cluster Virtual Media 1 Extended drive File G Virtual Media 2 EMC Centera cluster Step 2: After File G is migrated, then File A, which resides on Virtual Media 1, can be fetched Virtual Media 1 Extended drive File A Virtual Media 2 EMC Centera cluster GEN Figure 4 Single thread per piece of virtual media In an active environment where file migration is necessary at the same time that file fetches are likely to occur, you can configure DiskXtender to both read from and write to a single piece of media at the same time. This functionality is available through the Enable simultaneous moves and fetches feature, which appears on the Options tab of the Service Properties dialog box. When simultaneous moves and fetches are enabled, two EMC Centera threads may be occupied by DiskXtender communication with a single piece of media one for moves and one for fetches, as illustrated in Figure 7 on page EMC DiskXtender Microsoft Windows Version Administration Guide

43 Connecting to EMC Centera File G is migrated to Virtual Media 1 at the same time that File A is fetched from Virtual Media 1 File G Virtual Media 1 Extended drive Virtual Media 2 File A Virtual Media 3 Virtual Media 4 EMC Centera cluster GEN Figure 5 Simultaneous moves and fetches With simultaneous moves and fetches enabled, DiskXtender will use only two threads for a piece of media if it is both moving and fetching files. It will not open two threads to a piece of media for moving files, nor will it open two threads for fetching files. If there are multiple pieces of virtual media in a media group and even multiple media groups with multiple pieces of media in each then a single DiskXtender installation can occupy numerous communication threads with an EMC Centera cluster at any one time. DiskXtender can initiate and maintain as many as 256 simultaneous connections to EMC Centera, regardless of the number of configured EMC Centera media services. DiskXtender will not reserve a thread to a piece of media if it does not need to. Communication threads to EMC Centera are initiated and maintained only if a file is requested from media or if there are files that need to be migrated and the file migration schedule is active. How DiskXtender works with EMC Centera 51

44 Connecting to EMC Centera DiskXtender writes files randomly to any piece of active media in a media group. This enables DiskXtender to continue writing files to media at the same time that it is fulfilling fetch requests for files on other pieces of media, as illustrated in Figure 8 on page 52. Communication threads File A EMC Centera cluster Media group 1 File B Virtual Media 1 File C Virtual Media 2 Media folder File D Virtual Media 3 Virtual Media 4 Extended drive Media group 2 Virtual Media 5 File E Virtual Media 6 File F Virtual Media 7 File G Virtual Media 8 File H GEN Figure 6 Random migration to EMC Centera You can configure DiskXtender to write files to media sequentially rather than randomly by using the Media fill method feature on the Options tab for each media group. However, sequential fill is strongly discouraged for EMC Centera media groups because it can lead to poor performance. Media group considerations for EMC Centera on page 71 provides additional information. Calculating the optimal number of available threads The equation for determining the optimal number of threads available for DiskXtender file migration and file fetches is as follows: where: (N x 20) - A - R = Available threads N is the number of access nodes in the EMC Centera cluster. 20 is the optimal number of threads per node. A is the number of threads used by other applications, including other DiskXtender installations. R is the number of threads used for EMC Centera replication, if replication is enabled. 52 EMC DiskXtender Microsoft Windows Version Administration Guide

45 Connecting to EMC Centera For example, if you are planning to write to an EMC Centera cluster with four access nodes, the cluster is dedicated to the DiskXtender installation, and replication is enabled and uses two threads, then the optimal number of threads available for file activity is 78, or: (4 access nodes x 20) = 78 The optimal number of available threads for DiskXtender can then be used when determining the number of pieces of active media that should be maintained in each media group. By controlling the number of active media, you are effectively controlling the number of threads that can be used for file migration. DiskXtender can automatically create EMC Centera media to maintain a specified count of active media in a media group. Active media are media that are still available for file writes. For EMC Centera media, this means that the media has not yet reached the maximum of 256 GB or 100,000 files. For example, if the DiskXtender installation has two extended drives, each extended drive has two media folders, and each media folder has one EMC Centera media group, then there are a total of four media groups targeting the EMC Centera cluster: 2 drives x 2 media folders x 1 media group (each) = 4 media groups If there are 78 available threads for the installation and you anticipate that file migration and fetch activity will be evenly distributed among the media groups, then you can divide the number of threads by the number of media groups: 78 available threads / 4 media groups = 19.5 pieces of active media Since it is not possible to have a half of a piece of media, you may want to round up to 20 pieces of active media for each media group. Note: In an active environment where simultaneous moves and fetches are enabled and are likely to occur, divide the number of active media by two. Remember that with simultaneous moves and fetches, two threads are used by each piece of media. Fine-tuning the equation to optimize performance The equation for determining the optimal number of available threads and ultimately the number of pieces of active media for each media group does not take into consideration the volume of file migration and file fetch activity to and from the EMC Centera cluster at different times of the day. As a result, you may need to adjust the DiskXtender or EMC Centera configuration to remain within the recommended number of available threads per access node and to maximize system performance. Consider the following tasks to fine-tune the usage of available threads: Increase the number of access nodes in the EMC Centera cluster to increase the number of available threads. When more threads are available, determine whether to increase the number of active media in each media group: If file migration is dominating the communication threads, then increase the number of active media in each media group to take advantage of the additional threads for file migration. If file fetches from full media (not active media) are dominating the communication threads, then you may want to leave the same number of active media in each media group so that the additional threads can be used for file fetches. How DiskXtender works with EMC Centera 53

46 Connecting to EMC Centera Schedule file migration so that it does not occupy the communication threads when they are needed for file fetches. By default, the file migration schedule is active at night, so that files are migrated when users are typically not fetching as many files. You may need to customize this schedule for your environment. Reduce the number of fetch requests from media so that they do not occupy the communication threads when they are needed for file migration. To reduce the number of requests for files on the EMC Centera cluster, leave as much active data on the extended drive as possible. Use DiskXtender purge rules to purge only file data that is no longer likely to be requested. Chapter 12, Purging Files, provides details on configuring file purging. Compliance through retention and audit EMC Centera is designed to facilitate compliance with externally driven regulations and internal governance requirements through its retention and audit features. File retention Table 1 EMC Centera can protect stored data through its retention feature, which is complemented by the DiskXtender retention feature. Through DiskXtender, you can apply a retention period automatically to files that qualify for move rules. You can also apply retention to and extend retention periods for specific files by using the DiskXtender Explorer Add-ons utility. Once DiskXtender applies a retention period to a file, the file cannot be edited or deleted from the extended drive until the retention period expires. (If you edit a retained file, you must save the file with a new filename.) If you have an EMC Centera GE or EMC Centera CE+ device, the file is also protected on the EMC Centera device. If you have an EMC Centera Basic Edition and you apply retention through DiskXtender, the file is protected on the extended drive but is not protected on the EMC Centera device. Table 3 on page 54 lists the retention options for files written to EMC Centera through DiskXtender. Types of retention Type of retention Fixed retention period Event-based retention Retention class Infinite retention Description A specific period of time (in days) during which file retention is enforced. Retention on a file is enforced when an event associated with the file occurs. A symbolic representation of a retention period. When the retention class is defined, you specify a name and a retention period (in days). If necessary, you can edit the retention period for a retention class by changing the class definition, thereby changing the retention period for a group of files. Retention classes on page 79 provides additional details, including a list of restrictions that apply when you use retention classes. Retention that can never expire. If you set global retention on the EMC Centera device and then you configure a different retention period through DiskXtender, the DiskXtender retention period applies. This is true even if the DiskXtender retention period is zero days (no retention). When you set a retention period of zero days through DiskXtender, then the file is not protected on the EMC Centera device, even if EMC Centera is configured for a global retention period of one or more days. 54 EMC DiskXtender Microsoft Windows Version Administration Guide

47 Connecting to EMC Centera DiskXtender always applies a retention period to files written to EMC Centera, even if the retention period is zero days (no retention). You cannot automatically apply the global retention period set on the EMC Centera device to files on the extended drive. To match the global retention period on EMC Centera with the retention set on files on the extended drive, specify the same retention setting in DiskXtender as on EMC Centera. Chapter 10, File Retention, provides details on setting and editing retention on files. Privileged delete If you have an EMC Centera Basic or EMC Centera GE device, you can delete retained files by using privileged deletes. To delete retained files on a CE+ device, contact an EMC Centera technical representative. To perform privileged deletes through DiskXtender, use the Privileged Delete option in the Explorer Add-ons utility. Note: When you perform a privileged delete, and the DiskXtender Recycler is enabled, the file is permanently deleted. It is not placed in the DiskXtender Recycler. When you delete a retained file through DiskXtender, you must enter an audit string. You can then view audit information by performing an EMC Centera query. To use the Privileged Delete option, first grant the right to perform a privileged delete to the profile that DiskXtender uses to connect to the EMC Centera device (either the anonymous profile or, as recommended, an access profile/.pea file). The EMC Centera online help provides additional information on profiles. Access profiles should be configured by an EMC Centera technical representative. In addition, privileged deletes are available only to users who are members of the DxAdministrators group on the DiskXtender server. Note: Consider the compliance regulations followed by your company before performing a privileged delete. Audited delete When you delete a file that is stored on an EMC Centera device, metadata about the deleted file remains on the cluster. This metadata is called a reflection or tombstone. Through DiskXtender, you can provide an audit string that is included in the reflection. An EMC Centera query then enables you to search for the reflection and view the audit string. If retention has never been set for the file, or if the retention period for the file has expired, the audit string can be provided automatically by DiskXtender through the media service. The audit string enables you to identify the data as deleted DiskXtender data. If the file is still under retention (and you are, therefore, performing a privileged delete through the Explorer Add-ons utility), then you must provide a custom audit string through the utility. If you provide an audit string through the media service but you delete the file by using a privileged delete, the audit string entered for the privileged delete is used instead of the media service audit string. An EMC Centera query enables you to search for deleted data and view the audit string. The EMC Centera documentation provides instructions on how to do this. How DiskXtender works with EMC Centera 55

48 Connecting to EMC Centera Replication The EMC Centera replication feature protects against data corruption and loss by automatically copying data from one EMC Centera cluster to another. As an EMC Centera cluster acquires new content from an application, the replication mechanism ensures that this new content is automatically and transparently transferred across a WAN or LAN to a designated EMC Centera in another location. Replication is used on an ongoing basis to keep two or more EMC Centera clusters synchronized with new content. In a typical replication setup, the EMC Centera clusters are geographically separate to ensure disaster recovery or to distribute the content for access from another location. For example, a company may replicate to a second EMC Centera cluster to enable recovery from the loss of the primary EMC Centera or to avoid multiple requests for the same content across a WAN connection. Files cannot be purged until all the files migrated to the media are replicated in the EMC Centera replica media. The majority of EMC Centera environments with replication are configured for unidirectional replication. With unidirectional replication, one EMC Centera cluster updates another cluster with its content. For example, if content is written to cluster A, then unidirectional replication transfers the content to cluster B so that it is located on both clusters. However, if content is written directly to cluster B, then the content is not transferred to cluster A. As a result, there may be additional content on cluster B that does not exist on cluster A. The EMC Centera online help provides additional information on replication. Replication and DiskXtender files Replication failover The replication process itself is transparent to DiskXtender. In other words, after DiskXtender migrates files to EMC Centera, the files are replicated from the source cluster to the target cluster without any DiskXtender involvement. When the source EMC Centera cluster fails, DiskXtender automatically attempts to set the failed media service online again. If DiskXtender is unable to set the media service online, the EMC Centera SDK provides a read-only connection to the target (replica) cluster after a brief pause. With read-only failover, files can be read from the target cluster, but additional files cannot be written to the target cluster. Once the source cluster is set online again, DiskXtender automatically resumes normal read/write activity with the source cluster. This scenario enables users to fetch files that have already been migrated and purged, even when the source cluster fails. However, new file migration activity (from new files that qualify for migration, as well as edits to and deletes of files that have already been migrated) must wait until the source cluster comes back online. Losing the read-only connection to the replica If the DiskXtender service restarts or if the media service is set offline while DiskXtender is connected to the target, then the connection to the target fails, and the media service is set offline. Purged files cannot be fetched until the source cluster is set online again. This is because DiskXtender passes the connection string to the EMC Centera SDK, and the connection string includes only addresses from the source cluster. If the source cluster is offline, then no connection can be made not even to the target cluster. The address information for the target cluster is passed to DiskXtender only after a successful connection to the source cluster. 56 EMC DiskXtender Microsoft Windows Version Administration Guide

49 Connecting to EMC Centera If the source cluster is offline for an extended period of time, contact EMC Customer Service. A Customer Support Representative can configure the environment to: Establish a read/write connection to the target cluster Ensure that files are replicated back to the source cluster once it is set online Replicate Delete If you use EMC Centera replication and you delete file data from an extended drive, you can choose whether the file data is deleted from only the source cluster or from all eligible clusters (including both the source cluster and all target clusters). When you delete file data from the extended drive and the Replicate Delete feature is enabled as part of the EMC Centera configuration, the file is deleted from the source cluster. The delete transaction is then placed in a queue, and eventually carried out on the target clusters. Replicate Delete is also known as delete propagation. When you enable Replicate Delete, the file data on all replicated clusters remains more closely synchronized. Note: When Replicate Delete is disabled, content deleted from the source cluster is not deleted from the target. This may be required in some circumstances, but will result in uneven capacity utilization. Synchronous deletion DiskXtender also enables you to control delete propagation to the target EMC Centera clusters. If it is necessary for deletions to occur almost simultaneously on both the source and target clusters, you can configure synchronous deletion through DiskXtender. When synchronous deletion is enabled, DiskXtender processes each file deletion on all eligible clusters. Note: If one or more of the clusters is not available, then the file deletion fails and an error occurs. When DiskXtender synchronous deletion is enabled, all eligible clusters must be available for a deletion to succeed. Enable synchronous deletion through DiskXtender only if the timing of delete transaction processing on the target clusters is important. Synchronous deletion may result in performance degradation when you delete files from the extended drive. This is because DiskXtender must process the deletion on all eligible clusters instead of on just the source cluster. If the timing of delete transaction processing on the target clusters is not important, enable only the EMC Centera Replicate Delete feature. This enables EMC Centera to process the deletion as system resources are available. To configure synchronous deletion through DiskXtender, use the Enable synchronous deletes on replicated Centeras option, which is available on the Options tab of the Service Properties dialog box. If you enable synchronous deletion through DiskXtender, enable EMC Centera Replicate Delete as well. Enabling both features provides additional assurance that the file is deleted on all eligible clusters. This is because EMC Centera may be able to delete the file even if DiskXtender is unable to do so. How DiskXtender works with EMC Centera 57

50 Connecting to EMC Centera Prerequisite steps to connect to EMC Centera Before you add an EMC Centera media service in the File System Manager Administrator, prepare the EMC Centera cluster and build the media service connection string. Preparing the EMC Centera cluster Work with an EMC Centera technical representative to install and configure the EMC Centera cluster before you attempt to connect DiskXtender to the cluster: 1. Connect all devices to a network connection that is accessible to the DiskXtender server. 2. Ensure that a supported version of CenteraStar is installed on the EMC Centera cluster. Table 4 on page 58 lists the supported versions. Table 2 Supported CenteraStar versions If you use this general CenteraStar version Ensure that this specific CenteraStar version is installed 3.2.x 4.0.x 4.1.x or later Certain CenteraStar versions do not properly support non-ascii characters (including Japanese, Chinese, Arabic, Cyrillic, and some Latin-based European characters). If one of these CenteraStar versions is installed on the EMC Centera device, the unsupported characters in file metadata are replaced with random data if the file is restored to the extended drive from EMC Centera media. File metadata includes the filepath and name. The files are stored correctly on the EMC Centera device. However, in file restore situations, files might be overwritten, resulting in data loss. You cannot create an EMC Centera media service unless you use a supported version of CenteraStar. If you attempt to create the media service and an unsupported CenteraStar version is detected, an error message appears and the media service is not created. 3. Consider the following EMC Centera features: Storage strategy Decide whether to use Performance or Capacity. Only qualified EMC Centera service personnel can change the storage strategy for a cluster. EMC Centera storage strategies on page 46 provides more information. Virtual pools Create one or more virtual pools to segregate data you write to EMC Centera through DiskXtender. An EMC Centera technical representative should perform this step. Pools and profiles on page 41 provides more information. Note: If you do not need to segregate data, you can write all data to the default pool. This practice is not recommended. 58 EMC DiskXtender Microsoft Windows Version Administration Guide

51 Connecting to EMC Centera Access profiles Create one or more access profiles to provide access to the pools. An EMC Centera technical representative should perform this step. The profile must have the Write, Read, Delete, and Query capabilities. To allow privileged deletes of retained files, the profile should also have the Privileged Delete right. This step should result in a.pea file. The EMC Centera technical representative should provide the.pea file to you. Copy the file to a location on the DiskXtender server. You then specify the.pea file when you add the media service. Retention Depending on the EMC Centera compliance model (Basic, GE, or CE+), decide whether to enable retention, and if so, which retention setting to use (fixed retention periods, retention classes, or infinite retention). To use retention classes, have an EMC Centera administrator configure them for you. File retention on page 54 provides more information. Replication and delete propagation If you enable replication of data between EMC Centera clusters, determine whether to use unidirectional or bidirectional replication. In addition, decide whether to configure deletion of files from both the source cluster and all target clusters. Replication on page 56 provides more information. Building the connection string When you create an EMC Centera media service in DiskXtender, you specify the connection string that the EMC Centera SDK uses to enable DiskXtender to connect to an EMC Centera cluster. The connection string includes information about the EMC Centera access nodes, as well as the access profile that should be used for the DiskXtender connection. To build the connection string for a single EMC Centera media service: 1. Collect the IP addresses for all access nodes on the primary EMC Centera cluster to which DiskXtender should write files. 2. icollect the IP addresses for all access nodes on the replica EMC Centera cluster which serves as the replica for the primary EMC Centera cluster. This information is generally available with the EMC Centera administrator. 3. Configure node name aliases for the access nodes. To configure node name aliases, use one of the following methods: (Recommended) Add the appropriate entries to a common DNS server. Edit the local HOSTS file on the DiskXtender server to include the IP addresses for the EMC Centera access nodes and the alias to use for each IP address. The connection string for a media service is limited to 512 characters. Keep this limit in mind when defining node name aliases. The aliases should be kept to a reasonable length. Otherwise, you may not be able to specify a sufficient number of access nodes on the connection string to enable reliable connections in the event of a node failure. On a single line in a text or word processor file, list the nodename aliases for the access nodes on the primary EMC Centera cluster or the EMC Centera replica address, separating the aliases with a comma. For example, if there are four access nodes on the primary cluster and the nodename aliases for the nodes are defined as Node1, Node2, Node3, and Node4, then type: Node1,Node2,Node3,Node4 Prerequisite steps to connect to EMC Centera 59

52 Connecting to EMC Centera 4. In the text or word processor file with the list of nodename aliases, add a question mark (?) after the nodename aliases, and then include the path on the DiskXtender server to the.pea file for the access profile that DiskXtender should use to connect to the EMC Centera cluster. For example, if the.pea file is located on the system drive at C:\Centera\DXProfile.pea, then the connection string would appear as: Node1,Node2,Node3,Node4?C:\Centera\DXProfile.pea Do not include commas in the path to the.pea file in the EMC Centera connection string. Using a comma may cause an authentication error, because the EMC Centera SDK considers the comma as a separator, and the remaining part of the connection string is truncated. 5. Ensure that the string with the nodename aliases and the path to the.pea file is fewer than 512 characters. 6. The string should be entered on the Centera Information page when you create the media service. Specifying EMC Centera media service options When you create an EMC Centera media service, there are several options that control how DiskXtender writes and reads files to and from EMC Centera. Figure 9 on page 60 illustrates the media service options. Figure 7 Media service wizard - Centera Information page 60 EMC DiskXtender Microsoft Windows Version Administration Guide

53 Connecting to EMC Centera Table 5 on page 61 lists the best practices for each EMC Centera media service option. Table 3 Best practices for EMC Centera media service options Media service option Pool Address Replica Address PEA File Name File Delete Audit String Embedded Blob Threshold Collision Avoidance Client-Side ID (hash) Calculation Best practice Type the connection string that you built in Building the connection string on page 59 to enable DiskXtender to connect to EMC Centera. Type the connection string that includes information about the EMC Centera access nodes of replica EMC Centera. Type the name of the PEA file for the access profile that DiskXtender should use to connect to EMC Centera Type the custom audit string to attach to file data when the file data is deleted from EMC Centera. The audit string is included in the reflection that remains on the EMC Centera cluster for the file. The audit string enables you to identify the data as deleted DiskXtender data if you perform an EMC Centera query to search for deleted data. If you have a significant number of unique, small files (less than 100 KB), type a threshold in KB. The file data for files that are smaller than the threshold are embedded in the CDF instead of being stored separately. Embedding file data in the CDF can decrease read and write times, since the overhead required to manage two objects (CDF and BLOB) is greater than the overhead required to manage a single object. Do not specify a threshold (in other words, leave the default of 0 KB) if most of your files are larger than the 100 KB maximum for the threshold, or if there are a significant number of duplicate files. If the EMC Centera is configured for Storage Strategy Capacity (single-instance storage), embedding BLOBs for files smaller than 100 KB will not allow you to realize the benefits of the storage strategy for these files. This is because the file content is embedded in the CDF. If you embed the BLOBs for files smaller than 100 KB, then single-instance storage is applied only to files larger than 100 KB. Enable collision avoidance only in environments where even the most remote possibility of data loss is unacceptable, or in environments where single-instance storage is forbidden, usually due to legal regulations. Collision avoidance ensures that a unique CA is created for each file stored on EMC Centera, even if the file is a duplicate of another file stored on the cluster. If the file is edited and re-migrated, then a different unique CA is created. This feature is designed to prevent the unlikely event where the same CA is created for different files. Collision avoidance should not be used in most environments because it prevents the primary benefit of single-instance storage the elimination of duplicate content so that only a single copy of each file is stored. Support for client-side ID calculation is being removed from DiskXtender in a future release. As a result, use of this feature is discouraged. Creating EMC Centera media When you configure a media group, you can enable the automatic creation of virtual EMC Centera media for that media group. This method of creating media is recommended in most environments because it ensures that media is available for file migration without requiring you to monitor each media group. On the Automation page for the media group, you can choose whether to automatically create media based on the amount of free space in the media group or on the number of active media (media that is not marked as full) in the media group. Figure 10 on page 62 illustrates the options to automatically create EMC Centera media. Prerequisite steps to connect to EMC Centera 61

54 Connecting to EMC Centera Figure 8 Options for automatic creation of EMC Centera media Creating media based on the number of active media is recommended because it enables you to optimize the number of media that DiskXtender writes to and reads from simultaneously. This optimization can be configured by using the Maximum media simultaneously receiving files option for the media group, which is available on the Options page for the media group. To create media based on the count of active media, select Auto-create virtual media to maintain a count of active media on the Automation page, and then specify the count to maintain. Calculating the optimal number of active media on page 62 provides guidance on how to determine the appropriate count. In addition, type a media naming convention and select the media service in which the media should be created on the Automation page. Calculating the optimal number of active media When you control the number of active media, you effectively control the number of EMC Centera communication threads that can be used for file migration. 62 EMC DiskXtender Microsoft Windows Version Administration Guide

55 Connecting to EMC Centera To calculate the optimal number of active media to maintain in a media group, first determine the number of EMC Centera communication threads that are available for DiskXtender file activity. The equation to determine the optimal number of threads available for DiskXtender file migration and file fetches is as follows: where: (N x 20) - A - R = Available threads N is the number of access nodes in the EMC Centera cluster 20 is the optimal number of threads per node A is the number of threads used by other applications, including other DiskXtender installations R is the number of threads used for EMC Centera replication, if replication is enabled For example, if you plan to write to an EMC Centera cluster with four access nodes, the cluster is dedicated to the DiskXtender installation, and replication is enabled and uses two threads, then the optimal number of threads available for file activity is 78, or: (4 access nodes x 20) = 78 Then, to determine the appropriate count of active media for each media group, divide the number of available threads by the number of media groups. For example, if the DiskXtender installation has two extended drives, each extended drive has two media folders, and each media folder has one EMC Centera media group, then there are a total of four media groups that target the EMC Centera cluster: 2 drives x 2 media folders x 1 media group (each) = 4 media groups If there are 78 available threads for the installation and you anticipate that file migration and fetch activity will be evenly distributed among the media groups, then divide the number of threads by the number of media groups: 78 available threads / 4 media groups = 19.5 pieces of active media Since it is not possible to have a half of a piece of media, you may want to round up to 20 pieces of active media for each media group. Note: In an active environment where simultaneous moves and fetches are enabled and are likely to occur, you should divide the number of active media by two. With simultaneous moves and fetches, two threads are used by each piece of media. Service options for EMC Centera on page 73 provides details on simultaneous moves and fetches. Fine-tuning the equation to optimize performance The equation to determine the optimal number of available threads and ultimately the number of pieces of active media for each media group does not take into consideration the volume of file migration and file fetch activity to and from the EMC Centera cluster at different times of the day. As a result, you may need to adjust the DiskXtender or EMC Centera configuration to remain within the recommended number of available threads per access node and to maximize system performance. Consider the following tasks to fine-tune the usage of available threads: Increase the number of access nodes in the EMC Centera cluster to increase the number of available threads. When more threads are available, determine whether to increase the number of active media in each media group: Prerequisite steps to connect to EMC Centera 63

56 Connecting to EMC Centera If file migration dominates the communication threads, then increase the number of active media in each media group to take advantage of the additional threads for file migration. If file fetches from full media (not active media) dominate the communication threads, then you may want to leave the same number of active media in each media group so that the additional threads can be used for file fetches. Schedule file migration so that it does not occupy the communication threads when they are needed for file fetches. By default, the file migration schedule is active at night, so that files are migrated when users typically do not fetch as many files. You may need to customize this schedule for the environment. Reduce the number of fetch requests from media so that they do not occupy the communication threads when they are needed for file migration. To reduce the number of requests for files on the EMC Centera cluster, leave as much active data on the extended drive as possible. Use DiskXtender purge rules to purge only file data that is no longer likely to be requested. Troubleshooting EMC Centera If you encounter unexpected errors, performance issues, or access problems between DiskXtender and EMC Centera, you can enable a log that traces EMC Centera events for DiskXtender. To enable the EMC Centera trace log: 1. Ensure that there is sufficient space for the log file on the system drive of the DiskXtender server. The log increases in size by as much as 1 GB each day. 2. Use Windows Explorer to navigate to the DiskXtender bin directory, drive:\program Files\EMC\DiskXtender\Bin, where drive is the system drive. 3. Right-click and select New > Text Document. 4. Name the new text file Dx_EmcService.log. 5. Click Yes on the pop-up message that warns you about changing file extensions. 6. Stop and restart the DiskXtender service. 7. After you finish troubleshooting and you no longer need the trace log, disable the log: a. Stop the DiskXtender service. b. Delete the Dx_EmcService.log file. 64 EMC DiskXtender Microsoft Windows Version Administration Guide

57 Connecting to EMC Centera Managing the lifecycle of EMC Centera media EMC Centera media is virtual, meaning that a piece of media defined in DiskXtender does not specifically correspond to a physical piece of media (like a tape cartridge). Instead, the virtual media is designed to simulate divisions of an EMC Centera cluster. The use of virtual media enables you to take advantage of the flexible file migration features available in DiskXtender. The following topics provide details on creating and managing virtual EMC Centera media throughout its lifecycle in DiskXtender. Understanding the EMC Centera media lifecycle The following steps detail the lifecycle process for virtual EMC Centera media: 1. You configure DiskXtender to create the virtual media and add it to the system: Automatically as needed based on certain criteria (recommended for most environments): DiskXtender creates the media. DiskXtender allocates the media to the extended drive. DiskXtender adds the media to the media group. Manually: a. Create each piece of media for the media service. b. Allocate the media to the extended drive. c. Add the media to a media group. Note: If there are multiple EMC Centera media services, you must create multiple media groups. A single media group cannot contain media from different EMC Centera media services. 2. DiskXtender moves files to the media until the media is considered full. 3. The full media remains in the media group. This enables continued access to the files on the media. 4. (Optional) If you no longer need the files that have been migrated to the media, or if you want to move the files to a different piece or type of media, remove the media from the media group. There are two ways to remove media from a media group: To copy the files on the media back to the extended drive and move the files to other media, run a Compact media task on the media. The media is automatically removed from the media group during compaction. Removing media but keeping the files on the drive on page 187 provides instructions. To remove the files on the media from the extended drive altogether, remove the media from the media group. 5. (Optional) If you removed the media from the media group and you do not want to keep the files on the EMC Centera cluster, run a Format media task on the media to clear the files. 6. (Optional) Deallocate the media from the extended drive, and delete it from the media service. 7. (Optional) Apply retention to a file through DiskXtender. File Retention on page 77 provides details. Managing the lifecycle of EMC Centera media 65

58 Connecting to EMC Centera For detailed instructions on managing the lifecycle of EMC Centera media, refer to EMC DiskXtender for Windows Online Help. Automatically creating virtual EMC Centera media When you configure a media group, you can enable the automatic creation of virtual EMC Centera media for that media group based on the amount of free space in the media group or on the number of media in the media group. This method of creating media is recommended in most environments because it ensures that media is available for file migration without requiring you to monitor each media group. On the Automation page for the media group, choose when DiskXtender should automatically create the media: Select Auto-create virtual media when free space falls below GB to automatically create the media and add it to the media group when the total number of free gigabytes on media in the media group falls below a specified value. Then type the number of gigabytes in the text box. (Recommended) Select Auto-create virtual media to maintain a count of active media to automatically create media and add it to the media group when the total count of available media (media that is not marked as full) in the group falls below a certain number. Then type the number in the text box. Maximum of 32 virtual media are allowed for auto-creation. When this total count of media is not sufficient to hold the data, another cluster of 32 virtual media is created automatically or virtual media are created as per the user-defined number. Note: This option is recommended for most environments. When you maintain a certain number of active media in each media group, you can optimize the number of media that DiskXtender writes to and reads from simultaneously. This optimization can be configured by using the Maximum media simultaneously receiving files option for the media group. Media group considerations for EMC Centera on page 71 provides details. If you selected the Auto-create virtual media when free space falls below GB option, then the media is created as needed based on when the amount of free space falls below the number of gigabytes specified. The amount of free space on media in the group, and therefore the need for media, is calculated after you create the media group. If you selected the Auto-create virtual media to maintain a count of active media option, then the media is created, one piece at a time, after the wizard is completed. After the specified number of active media are created, additional pieces are created as needed based on the number specified. Manually creating virtual EMC Centera media You can also manually create virtual EMC Centera media and add it to the system to make it available for file migration. 66 EMC DiskXtender Microsoft Windows Version Administration Guide

59 Connecting to EMC Centera Removing virtual EMC Centera media from the system You can delete any piece of EMC Centera virtual media, even if you applied retention to files that were written to the media. When you delete the media, you cannot restore the files on the media through DiskXtender. This is because the relationships between the file pointers used by DiskXtender and the file data on the EMC Centera device are removed when you delete the media. IMPORTANT Do not delete a piece of EMC Centera media unless you are absolutely certain that you no longer need the files on the media. If you think you might need the files, compact the media before deleting it. Compaction writes the files on the media back to the extended drive so that they can be migrated to another piece of media. Recovering deleted virtual EMC Centera media When you delete a piece of media that has files on it, the files and the media C-Clip remain on the EMC Centera cluster. If it is later necessary to restore the media to DiskXtender, contact EMC Customer Service by using the EMC Online Support. A Customer Support Representative can assist you in retrieving the media C-Clip information and adding the media back to DiskXtender. When you delete a piece of media that does not have any files associated with it-either because the media was formatted or because files were never written to the media-the media C-Clip is deleted from the EMC Centera cluster and cannot be re-created. Reclaiming deleted media space on EMC Centera If you no longer need the files that have been written to a piece of EMC Centera virtual media, you can run a Format media task on the media to clear the files from the media. The EMC Centera Garbage Collection feature then reclaims the space made available by the deleted files. Impact of file retention when formatting EMC Centera media If you have an EMC Centera GE or CE+ device, then you cannot format media if there are retained files on the media. If you assign a Format media task to a piece of media on a GE or CE+ device and there are retained files on the media, then the task fails and the media goes into an error state. If you have an EMC Centera Basic, then files can always be deleted when you format a piece of media. This is because retention is not applied to the files on EMC Centera, even if you set retention for the files on the extended drive. Managing the lifecycle of EMC Centera media 67

60 Connecting to EMC Centera Expectations for files migrated to EMC Centera After a file is moved to an EMC Centera device, it is the retention status of the file that primarily determines whether you can edit or delete the file. It is also the retention status of the files within a folder that determine whether you can edit or delete the folder. To keep the files on the EMC Centera device synchronized with the extended drive, certain operations are restricted on the extended drive. Table 6 on page 68 lists the extended drive operations that can be executed on specific types of files or folders. Table 4 Extended drive operations for EMC Centera files Type of file or folder Edit Rename Set attributes Delete Move within a media folder Move outside of a media folder Files with no retention Files with retention Only with a privileged delete Files with expired retention Folders with files under retention N/A N/A Folders with no files under retention N/A N/A What happens when you edit or move EMC Centera files Keep in mind the following important points when working with files that have been moved to an EMC Centera device, and the folders in which those files are located: When you edit a file on the extended drive, the new (edited) file is migrated to the EMC Centera device, and the original version of the file is deleted. In this way, only the newer version of the file remains on the EMC Centera cluster. If you attempt to move a folder outside of a media folder, the folder appears in its new location. However, none of the files in the original folder are moved to the new location. Instead, they remain in the original folder in the original location. The same thing occurs if you attempt to move a folder with retained files to another location in the same media folder. This is because the files within the folder cannot be moved to the new location, even if the folder itself can be. If you move a purged file to another location within the same media folder by dragging and dropping the file in Microsoft Windows Explorer, then the file is fetched to the extended drive. This behavior is caused by Explorer. If you move a purged file to another location within the same media folder by using the command line, then the file is not fetched. If you move a folder with purged files to another location within the same media folder, however, then the files in the folder are not fetched. 68 EMC DiskXtender Microsoft Windows Version Administration Guide

61 Connecting to EMC Centera What happens when you set attributes on EMC Centera files You can change the Read-only, Hidden, or System attributes for a non-retained file on the extended drive that has been migrated to EMC Centera media, but DiskXtender does not recognize the changes. Therefore, the attribute changes are not propagated to the files on the media. Note: If a file is updated on media for another reason, such as edits to file content or changes to other file attributes, then all attribute changes for the file are updated on the media. This restriction on recognition of attribute changes has been enforced because a new CDF is created on the cluster each time an attribute change is propagated. Older CDFs are not deleted, although the files on the extended drive no longer point to them. When EMC Centera Chargeback reports are run, the reports include all of the CDFs referencing the same data blob and cannot differentiate the old CDFs from the current CDF. This behavior is acceptable and has minimal impact when users change file attributes, which is a relatively infrequent event. However, when system tools periodically scan the file system and touch every file, the impact can be significant. The issue is compounded in environments with a significant number of files. If it is important in your environment to propagate attribute changes and you are certain that there are no system tools installed on the system that perform attribute changes during a scan, then you can enable recognition of file attribute changes by editing the Recognize file attribute changes setting on the Options page for an EMC Centera media group. Expectations for files migrated to EMC Centera 69

62 Connecting to EMC Centera Adding custom metadata to files on EMC Centera You can add custom metadata to each file that is written to EMC Centera media. The custom metadata, along with other standardized metadata, is added to the CDF. The standardized metadata enables you to use EMC Centera Seek and Chargeback Reporter with files written to an EMC Centera device through DiskXtender. The custom metadata can be used to enhance EMC Centera Seek and Chargeback Reporter reports. Understanding EMC Centera Seek and Chargeback Reporter EMC Centera Seek is a high-performance index and search engine that provides full-text indexing of the metadata files (CDFs) stored on the EMC Centera device. Chargeback Reporter is a layered software module used with EMC Centera Seek. Chargeback Reporter enables the assignment, management, and identification of customized categories of archived data based on metadata values stored on an EMC Centera device. EMC Centera Seek and Chargeback Reporter should be installed on a separate, dedicated server outside the EMC Centera cabinet. An EMC Centera technical representative should perform the installation and assist with configuration. For more information, contact an EMC Centera representative. 70 EMC DiskXtender Microsoft Windows Version Administration Guide

63 Connecting to EMC Centera Performance tuning for EMC Centera Review the following recommendations to improve DiskXtender performance with EMC Centera by adjusting the configuration of media folders, media groups, rules, file deletions, and service options. Media folder considerations for EMC Centera Use extra caution when you plan the directory structure and create media folders on the extended drive. When you use EMC Centera, folder renames are allowed. Folders can be renamed any number of times. When media tasks such as Prefetch, Report, Compact are performed, the files are restored to the latest Centera folder. When you perform media tasks, the folder rename transactions are processed and hence there could be a delay in restoring files. Media group considerations for EMC Centera When you are creating media groups, consider the following settings to optimize DiskXtender performance with EMC Centera: Enable automatic media creation based on the number of active media in the media group. On the Options page for the media group: Leave the Media fill method option set to Random. This is the default setting. It allows DiskXtender to connect to any available piece of media in the group and even multiple pieces of media at the same time when migrating files, rather than filling one piece of media and then moving on to the next one. Adjust the Maximum media simultaneously receiving files option as necessary. The default value for this option is the media count threshold that you type on the Automation page. DiskXtender should be configured to write to as many pieces as is appropriate for the best system efficiency. The number depends on the optimal number of media that should receive files at any given time, as discussed in Communication threads on page 49. Leave the attributes checkboxes on the Recognize file attribute changes option clear. This enables DiskXtender to ignore unnecessary file attribute changes resulting from scans by system tools. What happens when you set attributes on EMC Centera files on page 69 provides details. If there are a significant number of media transactions being flushed at the same time, consider enabling the Prioritize fetches over media transactions option. The option enables fetches to occur even while media transactions are in progress. As a result, users no longer need to wait to fetch a purged file during media transactions. If you use EMC Centera Seek and Chargeback Reporter and you want to enhance the reports, type custom metadata on the Metadata page for the media group. The metadata is added to the CDF for each file that is written to a piece of media in the media group. Adding custom metadata to files on EMC Centera on page 70 provides details. Performance tuning for EMC Centera 71

64 Connecting to EMC Centera If you are configuring multi-target migration through DiskXtender and the media in two different targets are two EMC Centera clusters, do not use EMC Centera replication. If you use both replication and multi-target migration, the CDF for each file may be stored on each EMC Centera cluster twice. If single-instance storage is disabled, then the file data will also be stored on each cluster twice. Move rule considerations for EMC Centera When you are creating move rules, consider the following settings to optimize DiskXtender performance with EMC Centera: To prevent files on the extended drive from being edited or deleted for a certain period of time, select a retention setting for the files on the Retention page of the move rule wizard. The retention applies when the file qualifies for the rule. Chapter 10, File Retention, provides details. On the Settings page for the move rule, do not select the options to purge files immediately after they are moved or to set Direct Read on the files. Instead, use the purge recommendations in Purge rule considerations for EMC Centera on page 72, and review the recommendations for when to use Direct Read in Reading files directly from media on page 115. Purge rule considerations for EMC Centera To maintain as many access node threads available as possible, limit the number of times that user requests for files are satisfied by having to fetch file data from the EMC Centera. In other words, leave as much file data on the extended drive as possible by waiting to purge files until it is absolutely necessary. To do this, configure an age delay for the purge rule. Select the Apply rule to files of age greater than option on the Age page of the purge rule wizard, specify the number of days (typically 30 or 60), and then select Last access time from the drop-down list. Also, on the Settings page of the purge rule wizard, select Do not force purges during background scans. This option prevents files from being purged until disk space is needed. File deletion considerations for EMC Centera When deleting files, consider the following to optimize DiskXtender performance with EMC Centera: When a significant number of delete transactions are sent to the EMC Centera device, the completion of those transactions may take a long time. During that time, file fetches may be delayed or even canceled due to timeout issues. Do not empty the DiskXtender Recycler when users are most likely to access the system. Or, if the Recycler is disabled, do not delete a significant number of files from the extended drive when users are most likely to access the system. If you use an EMC Centera Basic or EMC Centera GE device, you can delete retained files by using privileged deletes. To perform privileged deletes through DiskXtender, use the Privileged Delete option in the Explorer Add-ons utility. Privileged delete on page 55 provides details. 72 EMC DiskXtender Microsoft Windows Version Administration Guide

65 Connecting to EMC Centera You can use delete rules and retention periods to keep files protected under retention for a specified period of time, then automatically delete them when they are no longer needed. Automatically deleting files on page 120 provides instructions on setting up delete rules to automatically delete files after a certain period of time. You can prevent delete transactions performed on purged files on an extended drive, from being propagated to EMC Centera device. Prevent delete from media on page 121 provides details on how to control delete transactions done on an extended drive from being propagated to the EMC Centera device. Service options for EMC Centera On the Options tab of the Service Properties dialog box, consider adjusting the following settings to optimize DiskXtender performance with EMC Centera: For the Enable simultaneous moves and fetches option, select Enable simultaneous moves and fetches for Centera and NAS media. By default, simultaneous moves and fetches are disabled. When you enable simultaneous moves and fetches, DiskXtender can both read from and write to a single piece of media at the same time. This setting can improve performance in an active environment where file migration is necessary at the same time that file fetches are likely to occur. When simultaneous moves and fetches are disabled, and a fetch request occurs while a file is being moved, the fetch must wait until the move is complete. How DiskXtender uses the communication threads on page 49 provides additional details. When DiskXtender communicates with a piece of EMC Centera media to write files, it maintains the connection for at least five seconds. If necessary, you can adjust this connection time by using the EMC Centera timeslice - minimum mount option. The timeslice option controls the minimum amount of time DiskXtender keeps virtual media mounted in a virtual drive for file moves. For file fetches, DiskXtender maintains the connection for 1 second. You cannot edit the file fetch connection time. If you use replication, decide whether DiskXtender should perform synchronous deletion of files from all replica clusters. Synchronous deletion by DiskXtender is necessary only if the timing of delete transaction processing on the target clusters is important. Synchronous deletion may result in performance degradation when you delete files from the extended drive. This is because DiskXtender must process the deletion on all eligible clusters instead of on just the source cluster. To enable DiskXtender synchronous deletion, use the Enable synchronous deletes on replicated Centeras service option. Synchronous deletion is disabled by default. If the timing of delete transaction processing on the target clusters is not important, enable only the EMC Centera Replicate Delete feature. This enables EMC Centera to process the deletion as scheduled. Performance tuning for EMC Centera 73

66 Connecting to EMC Centera 74 EMC DiskXtender Microsoft Windows Version Administration Guide

67 4 Connecting to Network-Attached Storage You can migrate files on a DiskXtender extended drive to a share on a network-attached storage device. The following topics provide details on configuring and managing the environment: How DiskXtender works with NAS Creating the shares for NAS media Managing the lifecycle of NAS media Expectations for files migrated to NAS Performance tuning for NAS Recommendations to work with NAS media Troubleshooting NAS issues Connecting to Network-Attached Storage 75

68 Connecting to Network-Attached Storage How DiskXtender works with NAS DiskXtender considers network-attached storage (NAS) media to be any media available through a connection to a share on a network, including: A network share on a standard disk-based storage device, such as EMC CLARiiON, EMC Celerra, EMC Symmetrix, EMC Isilon, EMC VNX/VNXe series, or other RAID and NAS devices: RAID devices are storage devices that contain several high-capacity magnetic drives and manage storage of data to those drives in a way that is seamless to the user. NAS devices use an operating system (like Microsoft Windows) to function as the file system control that tracks the location of files within the NAS device. A shared folder on a server on the network. A shared media folder on another DiskXtender extended drive. The second extended drive must be located on a separate DiskXtender server. You cannot migrate files from a media folder on one extended drive to a location on the same extended drive. NAS media is considered virtual media because it does not correspond directly to a specific piece of media (like a tape cartridge). However, it does correspond to a specific location (the share) on a magnetic drive. When DiskXtender writes to NAS media, it is essentially copying files over the network from the extended drive to the shared location. DiskXtender does not impose a limit on the size of NAS media. The size of each piece of NAS media depends on the size of the partition on which the share is located. When you connect DiskXtender to a share on a NAS device by creating a piece of NAS media, you must choose the type of media you are creating. Table 7 on page 76 lists the available options. Table 1 NAS media types NAS media type Standard NAS on page 76 Aggregate NAS on page 77 Retained NAS on page 78 Type of share Standard NAS share on a disk-based storage device or server on the network Media folder on another DiskXtender extended drive as part of tiered migration Standard share on a retention-capable NAS device Standard NAS Standard NAS media can be created for any network share on any type of disk-based device, like a RAID or a NAS device, or a server drive on the network. DiskXtender treats Standard NAS media like any normal read/write magnetic media. There are no restrictions on editing or deleting files, or renaming files or folders. 76 EMC DiskXtender Microsoft Windows Version Administration Guide

69 Connecting to Network-Attached Storage Aggregate NAS Aggregate NAS is designed to be used in a tiered migration environment, in which you are aggregating files before moving them (a second time) to tape. Files are migrated from one DiskXtender extended drive to another extended drive, and then from the second extended drive to tape, as shown in Figure 11 on page 77. Extended drive on first DiskXtender server File Extended drive on second DiskXtender server File Tape GEN Figure 1 Tiered migration To accomplish this through DiskXtender, media folders on the second extended drive are configured as aggregate NAS media on the first DiskXtender server. The files from the first extended drive are migrated to the aggregate NAS media (the second extended drive). The DiskXtender installation on the second server then moves the files to tape media. When files are saved to a media folder on the first extended drive that contains a media group with aggregate NAS media, DiskXtender does not allow you to rename the files or the folders in which they reside. This behavior mimics the behavior of the second extended drive (the aggregate NAS media), because the second extended drive is writing to tape, which also does not allow file or folder renames. The restrictions on aggregate NAS media are designed to ensure that files and file tags on both extended drives remain synchronized, and that the files on the subsequent tape media are not orphaned. Note: DiskXtender does not restrict you from using the second extended drive to write to media other than tape. However, aggregate NAS media restricts file and folder renames, operationally imitating extended drives that write to tape media. Chapter 9, File Migration, provides instructions for configuring tiered migration. How DiskXtender works with NAS 77

70 Connecting to Network-Attached Storage Retained NAS Some NAS devices can be configured to contain variable retention volumes, including a Celerra Network Server with the File-Level Retention (FLR) file system and Network Appliance (NetApp) NAS devices with SnapLock software. With these retention-capable volumes, you can set retention on files written to the volumes. Through DiskXtender, you can apply a retention period automatically to files that qualify for move rules that write to these volumes. When DiskXtender applies a retention period to a file, the file cannot be edited or deleted from the extended drive until the retention period expires. You also cannot change the retention period that has been applied to a file until the original retention period expires. When the file is moved to media, the retention period you set through DiskXtender move rules is passed to the retained NAS device. As a result, the file is also protected on the NAS device until the retention period expires. Because retention periods are passed from DiskXtender to the retained NAS, do not configure retention through the NAS device. Instead, configure retention through DiskXtender. Note: You cannot create a piece of retained NAS media unless you use a supported device: either Celerra with FLR or NetApp with SnapLock. If you select the retained NAS option when creating NAS media for a share that does not reside on a supported device, then DiskXtender automatically converts the media to the Standard NAS media type. To write to a retention capable device share without retention, use the Standard NAS option when creating the NAS media that corresponds to the share. Chapter 10, File Retention, provides details on setting retention for DiskXtender files. Limitations for filepath lengths There is a limit to the total length allowed for a path and filename on both the extended drive and on NAS media. When you save or edit a file on the extended drive, the full path and filename can total as many as 259 UTF-16 characters. Most NAS file systems support filepaths of 259 UTF-16 characters. (Review the NAS device documentation for specific support information.) However, if DiskXtender is writing to a share on the NAS device, the character count of the full filepath on the device is included in the maximum number of characters allowed. In other words, you must consider the absolute path of the file on the device. To do this, total the character count of the path and filename on the extended drive and the full path to the share on the device. For example, assume that there is a share called \\Server\Share for the \\Server\Directory1\Directory2\Directory3\ location on a NAS device. Assume also that \MediaFolder\File.doc is on the extended drive writing to a piece of NAS media for \\Server\Share. The character count for the media derives from \\Server\Directory1\Directory2\Directory3\File.doc and not from \\Server\Share\File.doc. 78 EMC DiskXtender Microsoft Windows Version Administration Guide

71 Connecting to Network-Attached Storage What happens when the filepath length exceeds the limit If the character count for the media path exceeds the maximum allowed by the media file system, then the file is not migrated to media. Also, a warning is written to the DiskXtender event logs. This warning continues to appear in the logs every time the file is qualified against the move rules during a background scan. To avoid exceeding the maximum path length, ensure that the path from the NAS media share is equivalent to the absolute path. Do not create a share with a short name for a deep subfolder. Length limitations when restoring files from media Take filepath character counts into consideration when restoring files from media. The file restore may fail in the following scenario: 1. DiskXtender successfully migrates the file to media. 2. You remove the media from the media group. 3. You re-add the media to a media group in a different media folder, and the new media folder has a higher character count. 4. The combined character count of the file on the media and the new media folder exceeds the maximum allowed. File restore may also fail if you are creating a piece of NAS media. This occurs if the share already contains files, and the character count for the filepath exceeds the maximum allowed. If the file restore fails, a warning is written in the DiskXtender event logs. How DiskXtender works with NAS 79

72 Connecting to Network-Attached Storage Creating the shares for NAS media Before you add the NAS media service in the File System Manager Administrator, prepare the shares that will be identified as NAS media. NAS share requirements The shares for NAS media should meet the following requirements: The shares should not be located at the root of the storage device. Create shares for either folders or partitions on the device. The shares should be visible over the network to the DiskXtender server. The absolute filepath (in other words, the full filepath on the device plus the filepath of any files on the extended drive) must not exceed the maximum number of UTF-16 characters supported by both DiskXtender and the device. Limitations for filepath lengths on page 78 provides details. Only the DiskXtender service account and any accounts required for backups should have full access to the share that corresponds to a piece of NAS media. No other user accounts or applications should have write access to the share. NAS share security on page 82 provides details on why this is necessary. If you use Celerra FLR, the share should be located on a Celerra disk. If you use a NetApp device with SnapLock, the share should be located on a drive that has been specified as a SnapLock drive. If you set up tiered migration, configure tiered migration to prepare the DiskXtender installation with the media folder that will serve as a piece of aggregate NAS media. Deciding how many shares to create Each DiskXtender installation can have only one NAS media service. However, you can create as many as 256 pieces of NAS media for that media service, and each piece of media can write to a different share on a different device, as illustrated in Figure 12 on page EMC DiskXtender Microsoft Windows Version Administration Guide

73 Connecting to Network-Attached Storage G:\NTFS Extended drives H:\ReFS Microsoft Windows server with DiskXtender installed Standard network connection Shared directories A B C A B C A B C Partition/drive 1 Partition/drive 2 Partition/drive 3 NAS device GEN Figure 2 DiskXtender with NAS Note: You can create more than 256 pieces of NAS media, but no more than the maximum will be online at the same time. You cannot write files to or read files from the media that DiskXtender does not set online at initialization. Creating the shares for NAS media 81

74 Connecting to Network-Attached Storage Even if the NAS media service is writing to a single device, create several partitions and shares so that you can create multiple pieces of NAS media. This is important for the following reasons: DiskXtender performance improves when the system can write to and read from multiple media at once. Focusing all system activity on a single piece of media can cause a performance bottleneck. You can take advantage of the flexible file migration features available in DiskXtender. You can migrate data from multiple media folders to different locations (pieces of media) by using customized migration rules. More files remain available if an error or other system problem occurs. If a piece of media becomes inaccessible (for example, as a result of a failed transaction), the files on other pieces of media remain accessible while the inaccessible media is offline. The number of media to create depends on your environment and your tolerance for system downtime. For assistance in determining how many pieces of NAS media to create, contact EMC Professional Services. NAS share security Only the DiskXtender service account and any accounts required for backups should have full access to the share that corresponds to a piece of NAS media. No other user accounts or applications should have write access to the share. This restriction is critical to protect the data that DiskXtender manages. If a user saves, edits, or deletes a file directly on the NAS share instead of through the extended drive, the files and file tags on the extended drive are not synchronized with the files on media. If this occurs, users receive Access Denied errors when they attempt to open those files from the extended drive. If the file is renamed directly on the media, the file data (or file tag, if the file is purged) on the extended drive becomes orphaned. If you ever change the account that DiskXtender uses to log in as a service, you also need to change the access permissions to any network shares that are being used as NAS media. 82 EMC DiskXtender Microsoft Windows Version Administration Guide

75 Connecting to Network-Attached Storage Managing the lifecycle of NAS media NAS media is considered virtual media because it does not correspond directly to a specific piece of media (like a tape cartridge). However, it does correspond to a specific location (the share) on a magnetic drive. The following topics provide details on creating and managing virtual NAS media throughout its lifecycle in DiskXtender. Understanding the NAS media lifecycle The following steps detail the lifecycle process for virtual NAS media: 1. Create the media and add it to the system: a. Create each piece of media for the media service. b. Allocate the media to the extended drive. c. Add the media to a media group. 2. DiskXtender moves files to the media until the media is considered full. 3. The full media remains in the media group. This enables continued access to the files on the media. 4. (Optional) If you no longer need the files that have been migrated to the media, or if you want to move the files to a different piece or type of media, remove the media from the media group. There are two ways to remove media from a media group: To copy the files on the media back to the extended drive and move the files to other media, compact the media. The media is automatically removed from the media group during compaction. Removing media but keeping the files on the drive on page 187 provides instructions. To remove the files on the media from the extended drive altogether, remove the media from the media group. 5. (Optional) Deallocate the media from the extended drive, and delete it from the media service. The files remain on the NAS share when you remove it from the system. DiskXtender does not delete the files even when you remove the association between DiskXtender media and the NAS share. Managing the lifecycle of NAS media 83

76 Connecting to Network-Attached Storage Expectations for files migrated to NAS After a file is moved to NAS, the type of NAS media and for retained NAS media, the retention status of the file determines whether you can edit or delete the file. These factors also determine whether you can edit or delete the folder in which a file is located. To keep the files on the NAS synchronized with the extended drive, certain operations may be prohibited on the extended drive. The following topics provide details on the operations allowed for each type of NAS media. Expectations for Standard NAS files You can perform most operations on the extended drive for files that have been migrated to Standard NAS media. Table 8 on page 84 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to Standard NAS. Table 2 Extended drive operations for Standard NAS files Edit Rename Set attributes Delete Move within a media folder Move outside of a media folder Files moved to Standard NAS Folders with files moved to Standard NAS N/A N/A All changes to the files and folders on the extended drive are propagated to the files and folders on the Standard NAS media. Expectations for aggregate NAS files Files in media folders with aggregate NAS media groups behave similarly to files that have been migrated to tape media. That is because the aggregate NAS is associated with a media folder on a different extended drive that is writing to tape. Table 9 on page 84 lists the extended drive operations that can be executed on specific types of files or folders that are located within media folders that contain aggregate NAS. Table 3 Extended drive operations for aggregate NAS files Edit Rename Set attributes Delete Files moved to aggregate NAS Folders with files moved to aggregate NAS N/A N/A Move within a media folder Move outside of a media folder Note: You cannot rename files and folders in media folders that contain media groups with aggregate NAS media. This restriction applies even if the files have not yet been migrated to media. 84 EMC DiskXtender Microsoft Windows Version Administration Guide

77 Connecting to Network-Attached Storage Expectations for retained NAS files Table 10 on page 85 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to retained NAS. Table 4 Extended drive operations for retained NAS files Type of file or folder Edit Rename Set attributes Delete Files with retention Move within a media folder Move outside of a media folder Files with expired retention Folders with files under retention N/A N/A Folders with no files under retention N/A N/A What happens when you edit or move NAS files Keep in mind the following important points when working with files that have been moved to NAS, and the folders in which those files are located: When you edit a file on the extended drive, the new (edited) file is migrated to NAS, and the original version of the file is deleted. In this way, only the newer version of the file remains on the NAS device. If you move a purged file to another location within the same media folder by dragging and dropping the file in Microsoft Windows Explorer, then the file is fetched to the extended drive. This behavior is caused by Windows Explorer. If you move a purged file to another location within the same media folder by using the command line, then the file is not fetched. If you move a folder with purged files to another location within the same media folder, however, then the files in the folder are not fetched. Expectations for files migrated to NAS 85

78 Connecting to Network-Attached Storage Performance tuning for NAS Review the following recommendations for improving DiskXtender performance with NAS by adjusting the configuration of media folders, media groups, rules, file deletions, and service options. Media folder considerations for NAS If you use aggregate NAS or retained NAS, use extra caution when planning the directory structure and creating media folders on the extended drive. When you use aggregate NAS or retained NAS, folder renames are not allowed on the extended drive. After files are moved to media, you cannot change the directory structure. Media group considerations for NAS When you are creating media groups, consider the following settings on the Options page for the media group to optimize DiskXtender performance with NAS: Set the Media fill method option to Random. By default, the media fill method is set to Sequential. Setting the option to Random enables DiskXtender to connect to any available piece of media in the group and even multiple pieces of media at the same time when migrating files, rather than filling one piece of media and then moving on to the next one. This option can improve DiskXtender performance when writing to more than one piece of NAS media in a media group. Adjust the Maximum media simultaneously receiving files option to the total number of media that are in the media group. This enables DiskXtender to write to all media as necessary. Since you must manually create NAS media when it is needed, enable the Warn when group free space falls below option so that you are notified when the NAS media in the media group is starting to run out of space. You can then configure the warning to be sent out as an alert. Sending alerts for errors and warnings on page 131 provides instructions for configuring alerts. Move rule considerations for NAS When you are creating move rules, consider the following settings to optimize DiskXtender performance with NAS: If you use retained NAS, specify a retention period in days for the files on the Retention page of the move rule wizard. You must specify a retention period of one or more days for move rules that move files to a retained NAS media group. The retention applies when the file qualifies for the rule. Chapter 10, File Retention, provides details. On the Settings page of the move rule wizard, consider whether to purge files immediately after they are moved and whether to mark them for Direct Read. To decide, review the purge recommendations in Chapter 12, Purging Files, and review the recommendations for when to use Direct Read in Reading files directly from media on page EMC DiskXtender Microsoft Windows Version Administration Guide

79 Connecting to Network-Attached Storage Purge rule considerations for NAS File retrieval from NAS media is typically faster than from other media types. As a result, the purge strategy you select depends on other factors in your environment, such as the type of files on the extended drive and the volume of files that DiskXtender is managing. Review the purge recommendations in Chapter 12, Purging Files, to determine an appropriate purge strategy for your environment. File deletion considerations for NAS If you use retained NAS, you can use delete rules and retention periods to keep files protected under retention for a specified period of time, then automatically delete them when they are no longer needed. Automatically deleting files on page 120 provides instructions on setting up delete rules to automatically delete files after a certain period of time. Service options for NAS Although simultaneous move and fetch from a single piece of media is available for the NAS media service, its use is not recommended. The Enable simultaneous moves and fetches option on the Options tab of the Service Properties dialog box controls this behavior. Simultaneous moves and fetches are disabled by default. There is no performance advantage to the use of the feature with NAS. In addition, when you use the feature with NAS, multiple files may be kept open at the same time, leaving the system more vulnerable in the event of a system failure. Instead, leave the option disabled so that all activity to a piece of media, including reads and writes, are serialized, or performed in a single thread. Serial media activity prevents applications from backing up or restoring data to or from the same pieces of media at the same time. Note: Simultaneous moves and fetches are recommended for the EMC Centera media service. However, the feature is a global setting; you cannot configure it differently for EMC Centera than you do for NAS. If you are writing to both NAS and EMC Centera, carefully weigh the performance advantages of enabling this feature for use with EMC Centera against the failure risks of using it with NAS. Performance tuning for NAS 87

80 Connecting to Network-Attached Storage Recommendations to work with NAS media Use these recommendations while working with NAS media. Adding a NAS media service You can configure only one NAS media service for each DiskXtender server, but you can use multiple network shares as pieces of NAS media for that media service. Use the File System Manager Administrator to create the NAS media service. There are no options to configure for the media service. Creating and managing NAS media on page 88 provides guidance on creating the media for the media service. Creating and managing NAS media NAS media is considered "virtual" media because it does not correspond directly to a specific piece of media (like a tape cartridge). However, it does correspond to a specific location (the share) on a magnetic drive. When you create NAS media for the NAS media service, you specify: The path to the network share. Descriptive information about the media, which appears in the File System Manager Administrator to identify the media. The type of NAS media. In most cases, this is Standard NAS. Note: Aggregate NAS media is intended only when the share is a media folder on another DiskXtender extended drive as part of a tiered migration strategy. Retained NAS media is for a standard share on a retention-capable NAS device, such as a Celerra Network Server with the File Level Retention (FLR) file system and Network Appliance (NetApp) NAS devices with SnapLock software. If user uses NetApp device, it is required to select YES or NO from the message box Does the Configured Retained NAS share belong to NetApp device? After you create the media, allocate the media to the extended drive and add the media to a media group to make it available for file migration. Deleting NAS media Before you delete a piece of NAS media, you must remove the media from its media group. During this process, you have the choice of whether to remove the files on the media from the extended drive or to copy the files back to the extended drive: To copy the files on the media back to the extended drive and move the files to other media, compact the media. The media is automatically removed from the media group during compaction. To remove the files on the media from the extended drive altogether, simply remove the media from the media group. Finally, ensure that the media service is online and then deallocate and delete the media by using the File System Manager Administrator. 88 EMC DiskXtender Microsoft Windows Version Administration Guide

81 Connecting to Network-Attached Storage The files remain on the NAS share when you remove it from the system. DiskXtender does not delete the files even when you remove the association between DiskXtender media and the NAS share. Upgrading DX with Retained NAS Media DX service will not start after user upgrades to DX 6.5 SP7 if DX is already configured with Retained NAS media. DxDmChk utility is required to be run with /NetApp option to start the service. Recommendations to work with NAS media 89

82 Connecting to Network-Attached Storage Troubleshooting NAS issues The following information will help you avoid and, if necessary, resolve common problems when you use NAS with DiskXtender: If you are attempting to fetch files from and run media tasks (such as Prefetch and Compact) on aggregate NAS media at the same time, the operation may time out and the media is then set offline. In addition, Error 64 (NET_NAME_DELETED) may appear. Similar problems occur when you create a piece of NAS media for a drive on the DiskXtender server and then migrate a significant number of files to the media. The error is returned by the Microsoft Windows operating system, which sees that the resource (the NAS media) is busy. To work around this issue, retry the operation at a later time when the system is not as busy. In addition, schedule media tasks and other media activities for times of the least system activity. This allows system resources to remain available for file fetches. The share serving as NAS media should be accessible only to the DiskXtender service account and any account required for backups. However, if you encounter a situation where this security is compromised and a file written to the share is edited directly on the NAS device rather than on the extended drive, follow these steps to recover from the situation: a. Determine whether the original file on the extended drive or the edited file on the NAS share should be saved. b. If you decide to save the edited file on the NAS share, then save the file back to the extended drive, overwriting the original file on the drive. c. Delete the file on the NAS share. d. Re-migrate the file through DiskXtender. If you suspect that the files on the extended drive are not synchronized with the files on NAS media, use the DXNASVerify utility to compare the two. The utility verifies the file list in each location and lists any inconsistencies so that they can be resolved. The DXNASVerify utility is available on EMC Online Support. If NAS media transactions are failing, contact EMC Customer Service by using EMC Online Support for troubleshooting assistance. Failed media transactions are listed in the event logs and can be sent out as system alerts. Empty folders may appear on NAS media that has been added to a media group if you rename a directory in a media folder on the extended drive before you migrate files from the directory. This occurs because DiskXtender flushes the directory rename transaction to media even though the files have not migrated yet. 90 EMC DiskXtender Microsoft Windows Version Administration Guide

83 5 Connecting to Optical and Tape Devices You can migrate files on a DiskXtender extended drive to optical and tape devices. The following topics provide details on configuring and managing the environment: Setting up MediaStor Setting up StorageTek ACSLS Managing the lifecycle of removable media Expectations for files migrated to removable media Performance tuning for removable media Connecting to Optical and Tape Devices 91

84 Connecting to Optical and Tape Devices Setting up MediaStor You can migrate files on a DiskXtender extended drive to removable media in a library or standalone drive if the library or drive is managed by MediaStor. The following topics provide details on configuring and managing the environment: How DiskXtender works with MediaStor on page 92 Prerequisite steps for installing MediaStor on page 98 Running the setup wizard to install MediaStor on page 99 Starting the MediaStor Administrator interface on page 100 How DiskXtender works with MediaStor MediaStor is a DiskXtender for Windows device management component that manages the retrieval, mounting, and dismounting of removable media in a variety of libraries and standalone drives. DiskXtender architecture with MediaStor MediaStor must be installed on a Microsoft Windows server that is attached to the library or standalone drive to manage. MediaStor should be installed on a different server than DiskXtender to optimize system performance. The DiskXtender server (with File System Manager installed) connects to MediaStor by using an RPC connection. You can dedicate a MediaStor server with a single hardware device to a DiskXtender server, as illustrated in Figure 13 on page 92. Windows servers DiskXtender server File System Manager installed MediaStor server MediaStor installed Tape Library Extended drives G:\NTFS H:\ReFS GEN Figure 1 DiskXtender connection with MediaStor You can also set up a single MediaStor server to manage multiple hardware devices that are all available to a single DiskXtender server, as illustrated in Figure 14 on page EMC DiskXtender Microsoft Windows Version Administration Guide

85 Connecting to Optical and Tape Devices Tape library DiskXtender server MediaStor server Tape library DVD library GEN Figure 2 Single DiskXtender server connecting to a single MediaStor server Alternatively, a single DiskXtender server can connect to multiple MediaStor servers, each managing one or more hardware devices, as illustrated in Figure 15 on page 94. Setting up MediaStor 93

86 Connecting to Optical and Tape Devices MediaStor server 1 Tape library DiskXtender server MediaStor server 2 Tape library MediaStor server 3 DVD library GEN Figure 3 Single DiskXtender server connecting to multiple MediaStor servers Note: If there are multiple MediaStor servers in an environment, you can manage each server from any other server. You can also manage all servers from a separate computer without a MediaStor installation by using the MediaStor Remote Administrator. The DiskXtender installation guide provides details on remote administration. Multiple DiskXtender servers can also connect to a single MediaStor server that manages one or multiple hardware devices, as illustrated in Figure 16 on page 95. In other words, multiple DiskXtender servers can share a hardware device. This is because neither the library itself nor the drives within the library are assigned to DiskXtender. Instead, it is the individual pieces of media within the library that are assigned, or allocated, to an extended drive managed by a particular DiskXtender server. MediaStor manages requests for media from multiple DiskXtender servers, and mounts the media in drives as they become available. Note: Take into account the potential performance impact of sharing hardware devices among multiple DiskXtender installations. 94 EMC DiskXtender Microsoft Windows Version Administration Guide

87 Connecting to Optical and Tape Devices DiskXtender server 1 Tape library DiskXtender server 2 MediaStor server Tape library DiskXtender server 3 DVD library GEN Figure 4 Multiple DiskXtender servers sharing a MediaStor server Note: A single MediaStor installation must manage a hardware device. Multiple MediaStor installations cannot share a device, and the device cannot be shared with other non-diskxtender applications. DiskXtender interoperability with MediaStor You can install MediaStor 6.5 SP7 to format media with OTG file system as well as media with UDF file system. Formatting and labeling removable media on page 111 provides details on selecting a file system when you format media. Device requirements The DiskXtender for Windows Supported Device List, available on EMC Online Support, contains a complete list of supported storage devices. Adapter requirements The storage devices that MediaStor manages, such as drives or libraries, must be connected to the MediaStor server by using one of more of the following interfaces (Host Bus Adapter) depending on the device connection type: SCSI Fibre Channel iscsi (IP over SCSI) SAS Setting up MediaStor 95

88 Connecting to Optical and Tape Devices A separate adapter for each SCSI drive or library device improves MediaStor performance. If, for example, there are three drives or libraries on one adapter, then the three devices also share the adapter resources. If there is a 1:1 ratio, then all devices have equal and optimal throughput. The DiskXtender for Windows Supported Device List provides a list of the adapters you can use to connect hardware devices to the MediaStor server. iscsi device requirements The connection between an iscsi device and the MediaStor server is a point-to-point connection by way of an Ethernet port. The iscsi device is exclusive to the MediaStor server and cannot be shared by other servers. To manage an iscsi library with MediaStor, install and configure an iscsi HBA or the Microsoft iscsi Initiator. You must also configure the port in the iscsi device or, in some instances, from the front control panel. Usually the port is an RS-232 or LAN port. Refer to the vendor s documentation for instructions. EDL support DiskXtender can migrate files to a virtual tape library system, such as an EMC Disk Library (EDL) device, if the device is managed by MediaStor. The DiskXtender for Windows Supported Device List, available on EMC Online Support, contains a complete list of supported EDL devices. To configure an EDL device for DiskXtender, install and configure the device as discussed in the EDL documentation. Then, add the device as a library in MediaStor, and proceed with normal DiskXtender configuration. DiskXtender treats the media in the device as tape media. Drive considerations Carefully assess your hardware needs to determine what combination of storage devices (drives and libraries) provide the greatest efficiency. Number of drives If you plan to keep only the most recent file data on the extended drive and purge the rest, you might want to consider a library with many drives, so that multiple users can retrieve purged files simultaneously. However, if you plan to configure the system so that only older or infrequently accessed files are purged from the extended drive, you might not need frequent media mounts to access purged files, meaning that you might do better with a library with only a few drives. Mixing drives Do not mix drive types within a single library unless support for the specific combination is explicitly documented in the DiskXtender for Windows Supported Device List, which is available on EMC Online Support. DVD drive considerations Because different types of DVD media can be read and written in different types of DVD drives, ensure that the hardware is configured to enable drives to perform the appropriate functions. Keep in mind the following considerations: When you add the library to the MediaStor configuration, you select the drive type for the library. This involves selecting the type of media intended for the drive. 96 EMC DiskXtender Microsoft Windows Version Administration Guide

89 Connecting to Optical and Tape Devices As long as the drives in the library are the same type as the selected drive type, they function as read/write drives. If a drive in the library is of a different type (though still compatible with the library), the drive functions as a read-only drive. If the library contains combo drives, these drives are set to the drive type selected for the library. In addition, DVD media in the library that is not of the same type as the drive type selected for the library is set to be read-only media during library inventory. For example, if DVD-RAM media is in the library, but DVD-R is set as the drive type, all DVD-RAM media is read-only The same is true for DVD-R media in a DVD-RAM library. If there are combo drives, different drive types, or both, all media must be readable in all library drives. For example, if a library contains both DVD-R and DVD-ROM drives, you cannot insert DVD-RAM media into the library because DVD-RAM media is not readable in DVD-ROM drives. All read/write library media must be both readable and writable in all read/write library drives. Unfinalized DVD-R media cannot be mounted in a read-only drive (a drive not configured to write to media). If you intend to update copies of media before the originals are finalized, you must have at least two write drives in the system: one drive for the copy and one for the unfinalized original. This is not an issue if you plan to update the copy only after the original is finalized. If the library contains both a read drive and a write drive, you must ensure that the drives are on separate buses, according to typical manufacturer s instructions. Carefully follow all manufacturer recommendations for hardware before configuring hardware in a DiskXtender system. UDO drive considerations DiskXtender supports both UDO1 and UDO2 media and hardware. You can install UDO1, UDO2, and MO drives in a UDO library. When adding such a library to the MediaStor configuration, select UDO2 as the drive type for the library. This enables you to use UDO1, UDO2, and MO media in the library. If you select UDO1 as the drive type for the library, then you can use only UDO1 and MO media in the library. If you select MO as the drive type for the library, you can use only MO media in the library. In a UDO library with mixed drives, you can read from and write to MO media only by using MO drives. Similarly, you can read from and write to UDO1 media only by using UDO1 drives. A UDO2 drive can read UDO1 and UDO2 media and write to UDO2 media. However, a UDO2 drive cannot write to UDO1 media. If you insert UDO1 media into a UDO library that has only UDO2 drives, then the media is inventoried as read-only media. If you later add a UDO1 drive to the library to enable writes to UDO1 media, you must reinventory the media to reset the UDO1 media to read/write status. Setting up MediaStor 97

90 Connecting to Optical and Tape Devices MediaStor server requirements The server on which you install MediaStor should meet the requirements in Table 11 on page 98. Table 1 Minimum and recommended hardware requirements for MediaStor Requirement Minimum Recommended Processor 1.3 GHz 2.8 GHz Memory 1 GB 2 GB Free space on the system drive 200 MB 20% free Operating system Microsoft Windows 2003 Standard or Enterprise Edition with Service Pack 2 Microsoft Windows Server 2008 Standard or Enterprise Edition with Service Pack 1, or R2 with Service Pack 1 The EMC DiskXtender Release Microsoft Windows Version Installation Guide provides details on the requirements for the server to on which you will install MediaStor. Prerequisite steps for installing MediaStor Before installing MediaStor, review the information in the DiskXtender installation guide to ensure that the hardware and operating system meet or exceed the minimum system requirements. Before you run the setup wizard to install MediaStor, perform the following steps: 1. Ensure that the environment meets or exceeds the requirements listed in the following topics: Device requirements on page 95 Drive considerations on page 96 MediaStor server requirements on page If the operating system on the MediaStor and DiskXtender servers is Windows Server 2008, then install two Microsoft hotfixes on both servers: The first hotfix is required to resolve a memory leak that occurs with Windows Server 2008 during an RPC client callback function, which is used when DiskXtender and MediaStor communicate to perform media management tasks. To obtain the hotfix, contact Microsoft Support and reference KB article The second hotfix is required to resolve a truncated registry entry that DiskXtender and MediaStor use to retrieve device type information. To obtain the hotfix, look up KB article the Microsoft website. The hotfixes should be incorporated in a future Windows Server 2008 service pack. 3. Check EMC Online Support for patches to this release, and download the patches, if necessary. Note: The MediaStor release number must match the File System Manager release number. However, the patch number for each component may be different. 98 EMC DiskXtender Microsoft Windows Version Administration Guide

91 Connecting to Optical and Tape Devices 4. Check EMC Online Support for updates to the release notes for this release. Review the release notes for information on the following topics: Installation tips Known issues Descriptions of features that have been added to DiskXtender, or that are scheduled to be removed from the product in future releases Documentation discrepancies 5. Create a service account. The service account must meet the following requirements: Be a member of the Administrators group on the server Have the Log in as a service right Note: If you do not assign the Log on as a service right, the right is automatically assigned to the service account during the installation. For DFS in a Microsoft Windows AD domain environment, be a member of the Pre-Windows 2000 Compatible Access group in the Built-in folder of the Active Directory Users and Groups console on the domain controller Note: Assigning the Act as part of operating system right to the service account might be necessary if the service fails to log in to the server after the installation. 6. Log in to the server by using the service account. 7. If you are planning to install MediaStor on multiple servers at the same time, perform the following steps: a. Ensure that all of the servers for install are visible on the network. b. Ensure that the logged in user (the service account on the server from which you are running the install) is a member of the local Administrators group on all of the servers for install. Running the setup wizard to install MediaStor To install MediaStor: 1. Exit all applications on the server. If other software is running, the setup wizard might not be able to write to all necessary files. 2. Insert the DiskXtender installation CD into the drive. 3. From the Microsoft Windows Start menu, select Run. The Run dialog box appears. 4. Browse or type the path to the setup file (drive:\mediastor\setup.exe, where drive is the drive holding the installation CD). Update this path with each new release. 5. Click OK. The setup wizard appears, starting with the Installation page. 6. Click Next, and then proceed through the remaining pages of the setup wizard. The pages of the MediaStor setup wizard are the same as most of the pages of the File System Manager setup wizard. The DiskXtender installation guide provides details on each page. Setting up MediaStor 99

92 Connecting to Optical and Tape Devices 7. When you finish running the installation, remember to check EMC Online Support for any patches to this release. Download the patches and install them, if necessary, by using the instructions in the patch documentation. Starting the MediaStor Administrator interface The MediaStor Administrator interface, illustrated in Figure 17 on page 100, enables you to add and manage hardware devices and storage media. The MediaStor Administrator appears very similar to the File System Manager Administrator interface that you use to configure DiskXtender. Understanding the Administrator interface on page 29 provides details. To start the MediaStor Administrator, open the Microsoft Windows Start menu and select Programs > EMC MediaStor > Administrator. Figure 5 MediaStor Administrator Adding a library 1. Confirm that all hardware is properly configured, specifically SCSI devices: a. Ensure that all devices and the SCSI adapter are properly terminated. b. Power on all SCSI devices before starting Microsoft Windows. c. Install any necessary drivers for the devices. d. Test access to the devices through Microsoft Windows. 2. If you are adding an iscsi library, configure an iscsi HBA or the Microsoft iscsi Initiator when using a LAN card, and configure the port in the device. 3. If you are adding a tape library with automatic drive cleaning capability, disable the automatic drive cleaning function for the library. 4. Open the MediaStor Administrator, right-click the Hardware node, and select New to launch the Hardware Wizard. 100 EMC DiskXtender Microsoft Windows Version Administration Guide

93 Connecting to Optical and Tape Devices Refer to the MediaStor Administrator online help for details on proceeding through the pages of the wizard. 5. Restart the server. 6. Perform a drive configuration test to determine if the order of the library drives in the configuration is correct: a. Set the library offline. b. Right-click the library and select Modify. c. Proceed through the pages of the Hardware Wizard until you reach the Library Drives page. d. Click Test Config. 7. Set the library online, selecting the option to perform a full inventory of the library. Allocating MediaStor media to an extended drive When you configure a MediaStor media service in the File System Manager Administrator, all of the media in the hardware devices configured in MediaStor are available for allocation, as long as they have not already been allocated to another extended drive. Deallocating MediaStor media from an extended drive To deallocate media from a DiskXtender extended drive by using the MediaStor Administrator, select the media from its application pool and drag it to the Scratch Pool. For detailed instructions on allocating and deallocating MediaStor media, refer the EMC DiskXtender MediaStor Online Help. Setting up MediaStor 101

94 Connecting to Optical and Tape Devices Setting up StorageTek ACSLS If you have already invested in StorageTek Automated Cartridge System Library Software (ACSLS), you can use DiskXtender to migrate files from a Microsoft Windows server to tape media in libraries managed by the ACSLS server. The following topics provide details on configuring and managing the environment: How DiskXtender works with ACSLS on page 102 Preparing the ACSLS library on page 104 Gathering ACSLS drive information on page 104 How DiskXtender works with ACSLS The following topics explain how DiskXtender connects to and communicates with a StorageTek library managed by ACSLS. Connecting DiskXtender and an ACSLS library ACSLS is a UNIX-based device management product that manages StorageTek tape libraries and provides device-sharing capability between applications. For ACSLS to communicate with DiskXtender, StorageTek LibAttach must be installed on the DiskXtender server. LibAttach translates Windows-based product commands from DiskXtender into a syntax that the UNIX-based ACSLS understands. DiskXtender 6.5 SP 7 is supported with ACSLS 7.3 and LibAttach The ACSLS server is connected to the device controller for one or more StorageTek tape libraries. ACSLS controls the picker arm for each library device and manages retrieval of media within the library. However, to allow device sharing, StorageTek constructs some of the library devices in a way that allows individual applications to communicate directly with particular drives in a library. To enable this communication for DiskXtender, you connect the DiskXtender server and one or more drives in the StorageTek library by using a SCSI cable, as illustrated in Figure 18 on page EMC DiskXtender Microsoft Windows Version Administration Guide

95 Connecting to Optical and Tape Devices Extended drives DiskXtender server (Microsoft Windows) G:\NTFS File System Manager installed LibAttach installed H:\ReFS ACSLS installed UNIX server RPC connection SCSI connections StorageTek library GEN Figure 6 DiskXtender with ACSLS When you configure an ACSLS media service in DiskXtender, you designate the connected drives as the drives that should be used by DiskXtender. Other servers cannot access that drive. Certain pieces of media in the ACSLS system are assigned to DiskXtender by using ACSLS. When DiskXtender requires a particular piece of media, DiskXtender requests that media from ACSLS. ACSLS retrieves the media and places it in the drive connected to the DiskXtender server. DiskXtender then communicates directly with the drive and initiates all media-related tasks without any involvement from ACSLS. Setting up StorageTek ACSLS 103

96 Connecting to Optical and Tape Devices Understanding the components of an ACSLS library In a StorageTek library managed by ACSLS, two or more Library Storage Module (LSM) devices can be connected by using a pass-through connection. When two or more LSM devices are connected, they are called an Automated Cartridge System (ACS). In DiskXtender, an ACS, which contains one or more LSMs, is considered a library. When an LSM is set offline, the media within that LSM is still online in DiskXtender. Note: If an error appears indicating that the library (in other words, the ACS) is offline when DiskXtender tries to mount a piece of media in an LSM, clear the error state. This brings the media back online even if the LSM is offline. Preparing the ACSLS library To prepare the ACSLS library for use with DiskXtender: 1. Physically connect the DiskXtender server and the library drives with SCSI cables. 2. Power on the libraries. 3. Start the UNIX server. 4. If you have not already done so, install and configure ACSLS as described in the StorageTek documentation, including access control privileges. 5. Log in to the UNIX server as acsss. The ACSSS Console, the ACSSS Command Processor Window, and the Event Log Tail Window appear. Note: If the ACSSS Console does not appear automatically, right-click the background and select Terminal from the Tools menu. 6. Configure the library as described in the StorageTek documentation. Gathering ACSLS drive information When you configure an ACSLS media service in DiskXtender, you must specify the library drives that are connected to the DiskXtender server. Gather the device name and SCSI address for the drives before you configure the media service. To gather ACSLS drive information: 1. In the ACSSS Command Processor Window, type the following command and press Enter: query drive all All drives currently in the system are listed, as shown in the following example: :07:54 Identifier State Status Volume Type 0, 0, 1, 0 online available , 0, 1, 1 online available , 0, 1, 2 online available DLT7000 0, 0, 1, 3 online available DLT EMC DiskXtender Microsoft Windows Version Administration Guide

97 Connecting to Optical and Tape Devices The necessary drive information is in the first and last columns (the Identifier and Type columns, respectively). The numbers listed in the Identifier column represent the ACS, LSM, panel, and drive, in that order. The value in the Type column represents the drive name. 2. Record the information from those columns for the drives attached to the DiskXtender server. 3. On the DiskXtender server, open the Registry Editor: a. Open the Windows Start menu and select Run. The Run dialog box appears. b. In the Open text box, type regedit and click OK. The Registry Editor appears. 4. Navigate to HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\Scsi. 5. Select the SCSI key. 6. From the Edit menu, select Find. 7. In the Find What text box, type the drive name that you requested on the ACSLS server (for example, 9840) and click Find Next. 8. Use one of the paths that you find to construct the device name in the following format: SCSI.Port.Bus.Target.LogicalUnit where: Port is the value at the end of the Scsi Port key name. Bus is the value at the end of the Scsi Bus key name. Target is the value at the end of the Target Id key name. LogicalUnit is the value at the end of the Logical Unit Id key name. For example, to construct the device name for drive 9840, you might find the value 9840 in the following registry path: HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\Scsi\Scsi Port 2\Scsi Bus 0\Target Id 2\Logical Unit Id 0 The device name that you would construct from this path would be SCSI This is the device name that you type in the Device Name text box of the ACSLS Drive Properties dialog box when you configure the media service in DiskXtender. Setting up StorageTek ACSLS 105

98 Connecting to Optical and Tape Devices Prerequisites to adding an ACSLS media service You must perform the following tasks before you add an ACSLS media service: 1. If you have not already done so, install LibAttach and the necessary tape driver on the DiskXtender server. The EMC DiskXtender Software Compatibility Guide, available on EMC Online Support, provides a list of the supported versions of LibAttach. 2. If the operating system on the DiskXtender server is Windows Server 2008, perform the following steps: a. Make an exception in the Windows firewall to enable LibAttach to communicate properly with the ACSLS server. The LibAttach and Windows Server 2008 Firewall Configuration document, available on the Sun website at provides instructions on how to add LibAttach as an exception program for the Windows firewall. b. Install two Microsoft hotfixes on the DiskXtender server: The first hotfix is required to resolve a memory leak that occurs with Windows Server 2008 during an RPC client callback function, which is used when DiskXtender and MediaStor communicate to perform media management tasks. To obtain the hotfix, contact Microsoft Support and reference KB article The second hotfix is required to resolve a truncated registry entry that DiskXtender and MediaStor use to retrieve device type information. To obtain the hotfix, look up KB article on the Microsoft website. Both hotfixes should be incorporated in a future Windows Server 2008 service pack. For detailed instructions to add a ACSLS media service, add, edit or remove an ACSLS drive, refer to EMC DiskXtender Online Help. Allocating ACSLS media to an extended drive To allocate media in a Sun StorageTek library to an extended drive: 1. Log in to the ACSLS server as acsss. The ACSSS Console, the ACSSS Command Processor Window, and the Event Log Tail Window appear. Note: If the ACSSS Console does not appear automatically, right-click the background and select Terminal from the Tools menu. 2. In the ACSSS Command Processor Window, type the following command and press Enter: set owner "DX_ServerName_DriveLetter" VOL MediaRange where ServerName is the name of the DiskXtender server, DriveLetter is the letter of the extended drive to which you are assigning the media, and MediaRange is the range of media numbers to assign. The command should look similar to this: set owner "DX_XtenderServer_E" VOL EMC DiskXtender Microsoft Windows Version Administration Guide

99 Connecting to Optical and Tape Devices Deallocating ACSLS media from an extended drive To deallocate ACSLS media from an extended drive, reassign that media back to the ACSLS system (making ACSLS the owner of the media). Type all commands for deallocating ACSLS media on the UNIX server where ACSLS is installed. To assign media back to the ACSLS system: 1. In the ACSSS Console, change to the appropriate directory. Type the following command and press Enter: cd /export/home/acsss/data/external/volrpt 2. In the ACSSS Console, request owner information to find the media to deallocate. Type the following command and press Enter: volrpt f owner_id.volrpt a x The placeholder x represents the ACS ID. A list showing the owner of each piece of media appears, as shown in the following example: VOLUME REPORT UTILITY :06:52 TOTAL VOLUMES: 2 SEQUENCE: sort by volume identifier Volume Volume Owner Label: Status: ID: VOLUME_HOME SYSTEM VOLUME_HOME DX_DXSERVER_E In this example, the media with volume label is currently unassigned. The media with volume label is currently assigned to DX_DXSERVER_E (the extended drive E on the server DXSERVER). 3. In the ACSSS Command Processor Window, assign the media to the ACSLS system. Type the following command and press Enter: set owner "System" VOL MediaRange In this command, the placeholder MediaRange represents the range of numbers for the media to deallocate. The following example assigns the media numbered from to to the system, which deallocates it from the extended drive: set owner "System" VOL When the new assignment is completed, the following message appears in the ACSSS Command Processor Window: "Set: set completed, Success." Setting up StorageTek ACSLS 107

100 Connecting to Optical and Tape Devices Managing the lifecycle of removable media The following topics provide details on managing removable media throughout its lifecycle in DiskXtender: Understanding the removable media lifecycle on page 108 Inserting removable media into a MediaStor library on page 109 Supported removable media types on page 110 Formatting and labeling removable media on page 111 Renaming media on page 116 Reusing media on page 116 Finalizing DVD-R media on page 122 Removing media from a MediaStor library on page 123 Understanding the removable media lifecycle The following steps detail the lifecycle process for removable media: 1. Insert the media into the hardware device by using the instructions from the hardware manufacturer. If you use MediaStor, some libraries support the Insert Library Media function available in MediaStor. 2. Allocate the media to the extended drive: If you use MediaStor, you can use either the File System Manager Administrator or the MediaStor Administrator. If you use ACSLS, you allocate the media by using ACSLS. 3. Ensure that the media is formatted with a supported file system and labeled: If the media is blank, format and label the media. If the media has already been formatted with a supported file system, label the media. If the media has been formatted with a file system that is not supported, reformat the media and then label it. Formatting and labeling removable media on page 111 provides details. 4. Add the media to a media group. Note: To add DVD-ROM media to the system and make the files available on the extended drive, create a standard media group and select DVD-R as the media type for the group. 5. DiskXtender moves files to the media until the media is considered full. 6. The full media remains in the media group. This enables continued access to the files on the media. 7. (Optional) If necessary, rename the media. Renaming media on page (Optional) If the media is formatted with the UDF file system and it becomes corrupt, you can attempt to repair the media by using Check Disk. 9. (Optional) To reuse the media, compact and reformat the media. This may be necessary if there is an excessive amount of wasted space on the media. Reusing media on page 116 provides instructions. 108 EMC DiskXtender Microsoft Windows Version Administration Guide

101 Connecting to Optical and Tape Devices 10. If the media is DVD-R, finalize the media when it becomes full to stabilize the media and purge the files from the extended drive. Finalizing DVD-R media on page 122 provides details. 11. (Optional) If you no longer need the files that have been migrated to the media, or if you want to move the files to a different piece or type of media, remove the media from the media group. There are two ways to remove media from a media group: To copy the files on the media back to the extended drive and move the files to other media, compact the media. The media is automatically removed from the media group during compaction. Removing media but keeping the files on the drive on page 187 provides instructions. To remove the files on the media from the extended drive altogether, remove the media from the media group. 12. (Optional) Deallocate the media from the extended drive, and delete it from the media service. Deallocating MediaStor media from an extended drive on page 101 and Deallocating ACSLS media from an extended drive on page 107 provides instructions. 13. (Optional) Remove the media from the hardware device. Removing media from a MediaStor library on page 123 provides instructions. Inserting removable media into a MediaStor library Some library models have individual procedures for inserting media, such as opening up a door to place media into the shelves. This type of library also requires the library be taken offline while media is inserted, and then set online and inventoried after the media is in place. For information about inserting media into a library, refer to the individual library documentation. If the library supports it, you can insert media by using the Insert Library Media function available in MediaStor. Note: If you have a tape library, do not insert a data cartridge into a shelf that is reserved for cleaning cartridges. Managing the lifecycle of removable media 109

102 Connecting to Optical and Tape Devices Supported removable media types Table 12 on page 110 lists the high-capacity, removable storage media types that DiskXtender supports. Table 2 Supported removable media types Type of media Type of support Details DVD-R Read/write Data is written only once (sequentially), although it can be written in multiple sessions. Requires finalization when the media is full, which renders the media read-only. Because of the potential instability before DVD-R is finalized, files written to DVD-R cannot be purged from the extended drive until the media is finalized. DVD-RAM Read/write Data can be written to and read from randomly, accessing files and space wherever necessary. Does not require finalization. DVD-ROM Read-only Pre-manufactured media. DiskXtender cannot write to or erase data from DVD-ROM. Supported only with the UDF file system. Magneto-optical (MO) Read/write Also called erasable-optical. MO WORM Read/write Data is written only once (sequentially), although it can be written in multiple sessions. After data is written, it cannot be altered. Tape Read/write Data is written only once (sequentially), although it can be written in multiple sessions. Tape can be reformatted for reuse. Supported types of tape include LTO, SDLT, DLT, SAIT, AIT, 9840, 9940, Quantum DLT-S4, and Sun/STK Tape WORM Read/write Supported by drives that have firmware for WORM media. Contact the drive vendor to verify WORM support, and to acquire the firmware version, if necessary. Data is written only once (sequentially), although it can be written in multiple sessions. Tape WORM cannot be reformatted. After data is written, it cannot be altered. Ultra-density optical (UDO) Read/write Similar to MO, but contains more space and has a larger sector size (8k) than MO (4k). The differing sector sizes means that unlike DVD media, UDO and MO cannot use the same drives for reading/writing data. DiskXtender supports both UDO1 and UDO2 media. UDO WORM Read/write Data is written only once (sequentially), although it can be written in multiple sessions. After data is written, it cannot be altered. DiskXtender supports both UDO1 WORM and UDO2 WORM media. The MediaStor media service enables DiskXtender to write files to all supported removable media types. You can also write files to tape media in a StorageTek library by using the StorageTek ACSLS media service. Setting up MediaStor on page 92 and Setting up StorageTek ACSLS on page 102 provide details. 110 EMC DiskXtender Microsoft Windows Version Administration Guide

103 Connecting to Optical and Tape Devices Formatting and labeling removable media Removable media must be formatted with a supported file system and labeled by DiskXtender before you can add it to a media group and begin migrating files to the media. The following topics provide details: Understanding media formatting on page 111 Understanding media labeling on page 111 Supported file systems on page 112 Choosing a file system on page 114 Deciding whether to format or label media on page 114 Choosing a format and label method on page 115 Understanding media formatting Formatting verifies the integrity of a piece of removable media, and makes it available for use with system hardware. It prepares the media for file writes by creating the specified file system on the media. You can format new (unformatted) media, or reformat media that has previously been formatted for the same or another file system (except for DVD-R and WORM media types, which you can format only once). Low-level SCSI format When you format a piece of media, you must choose whether to perform a low-level SCSI format or a quick format by selecting or clearing the Force low-level SCSI format checkbox. Perform a low-level SCSI format when media has repeatedly failed for the current drive type due to media errors, or if the media is unformatted DVD-RAM that is not pre-certified. Perform a quick format (and leave the checkbox clear) if the media is preformatted or if the media has been previously low-level formatted for the current drive type. A quick format clears the file table of all pointers to files on the media, but not the actual information on the media. DiskXtender allows you to perform low-level SCSI format on the media that it supports. For media that does not support low-level SCSI formatting, such as DVD-R, this option is unavailable. Note: Formatting double-sided optical media in a stand-alone drive formats only one side of the media. You must dismount, flip, and format the reverse side as well. To avoid confusion, always format both sides of double-sided media at the same time. Understanding media labeling When you label a piece of media, you assign a name to the media. This label, along with the serial number that is automatically assigned to the media during labeling, enables DiskXtender to track the media. Use a unique label for each piece of media to enable easy identification of the media in the File System Manager Administrator. Media labels can include alphanumeric (a-z or 0-9) characters or the following special characters: $! - _ % and #. The recommended approach is to use a descriptive prefix (or set of prefixes) and a sequence number that represents the order of the media in the group. The sequence number could optionally reflect the side of media (for example, 1, 2, 3 or 1A, 1B, 2A, 2B, 3A, 3B). Managing the lifecycle of removable media 111

104 Connecting to Optical and Tape Devices Supported file systems Table 3 DiskXtender can write files to and read files from removable media that is formatted with either the OTG file system or the UDF file system. Table 13 on page 112 lists the file systems that DiskXtender supports for each type of media. FIle system support by media type Media type DVD-R Supported file systems UDF DVD-RAM OTG UDF Magneto-optical (MO) OTG UDF MO WORM OTG UDF Tape Tape-WORM OTG OTG UDO (UDO1, UDO2) OTG UDF UDO WORM (UDO1 WORM, UDO2 WORM)) OTG UDF Note: DVD-ROM can be read by DiskXtender only if it is written by using the UDF file system. DVD-ROM must be created in another system. DiskXtender cannot write to DVD-ROM. OTG file systems OTG file systems are installed with DiskXtender and are optimized for DiskXtender media performance. Table 14 on page 112 lists the two subsystems for the OTG file system and the media types that support them. Table 4 Subsystems for the OTG file system Subsystem Media types OSS (Optical Storage Subsystem) MO MO WORM UDO UDO WORM DVD-RAM TSS (Tape Storage Subsystem) Tape Tape-WORM OTG file systems are sequential, meaning that files are written contiguously from the beginning to the end of each piece of media, with single-seek read and write access. As a result, runtime overhead is very low and data is sequentially organized, enhancing overall performance. When you edit a file that has been written to media formatted with an OTG file system, DiskXtender does not edit the file data on the media. Instead, when a file is edited, the file is written as a new file to the next blank area on the media, and the file tag on the extended drive is updated to point to the new file. The old file data still 112 EMC DiskXtender Microsoft Windows Version Administration Guide

105 Connecting to Optical and Tape Devices remains on the media, but is orphaned. The only exception to this is when a file is renamed on rewritable media. In that case, the file trailer on the media, where the filename is kept, is updated to reflect the new filename. When you delete a file that has been written to media that has been formatted with the OTG file system, the file and its data are deleted only from the extended drive. They are not removed from the media. Deleting the file on the extended drive removes the file tag, which contains the location information for the file on the media. Without the file tag, DiskXtender can no longer track that file on the media (even though the data is still there), and the file on the media is orphaned. UDF file systems DiskXtender meets the specifications (version 2.01) laid out for the UDF (Universal Disk Format) file system by the Optical Storage Technology Association (OSTA), a nonprofit international trade association. For more information on OSTA, refer to the OSTA website. UDF is intended to enable file interchange between different operating systems. There are two types of UDF file systems: overwritable and sequential. Table 15 on page 113 lists the two UDF file systems and the media types that support them. Table 5 UDF file systems UDF file system Media types Overwritable MO UDO DVD-RAM Sequential DVD-R MO WORM UDO WORM With the overwritable UDF file system, DiskXtender writes files to any available location on the media and can overwrite and delete files. With the sequential UDF file system, DiskXtender writes files in sequential order on the media, and does not edit files. Instead, DiskXtender adds edited files as new files to the next blank space on the media (functioning much like the OTG file systems on page 112). In addition, when you delete a file that has been written to media formatted with the UDF sequential file system, the file and its data are deleted only from the extended drive and are not removed from the media. Deleting the file on the extended drive removes the file tag, which contains the location information for the file on the media. Without the file tag, DiskXtender can no longer track that file on the media (even though the data is still there), and the file is orphaned. Note: When writing to media that is formatted with the UDF file system, DiskXtender supports a shorter path and filename than the extended drive supports. DiskXtender supports 259 UTF-16 characters on the extended drive, but only 127 UTF-16 characters when writing files to media with the UDF file system. Path and filename length limitations on page 62 provides details. Managing the lifecycle of removable media 113

106 Connecting to Optical and Tape Devices Choosing a file system In general, if media portability (the ability to read from and write to media outside of DiskXtender) is a concern, choose the UDF file system. In addition, there are some operations (deletions, renaming of files and directories, and file attribute changes) that are allowed for media with one file system but not the other. Expectations for files migrated to removable media on page 124 provides details. Deciding whether to format or label media After you insert media into a hardware device and allocate it to an extended drive, the File System Manager Administrator assists you in determining whether the media needs to be formatted or labeled. Allocated media appears under a node in the Available Media tree, as illustrated in Figure 19 on page 114. Figure 7 Available Media tree Table 16 on page 114 lists the status of the media, depending on the Available Media tree node under which it is listed. Table 6 Status of media in the Available Media tree (page 1 of 2) Available Media tree node Media status Required steps for the media Original Copy Duplicate Media has been formatted and labeled. Media is a copy of a piece of original media, with the same data but a different serial number. Media has the same serial number as another piece of media. Media can be added to a media group for file migration and file fetches. DiskXtender updates the media as necessary to ensure that it matches the original. The copy can be promoted to an original if the original becomes unreadable. Copying removable media on page 162 provides instructions for copying media. The presence of duplicate media likely indicates a problem in the system. Contact EMC Customer Service by using EMC Online Support for assistance. 114 EMC DiskXtender Microsoft Windows Version Administration Guide

107 Connecting to Optical and Tape Devices Table 6 Status of media in the Available Media tree (page 2 of 2) Available Media tree node Media status Required steps for the media Blank Media has been formatted but not yet labeled. Label the media to make it a piece of original media, or label it as a copy of a piece of original media. You can also reformat the media. Note: DiskXtender automatically inventories new MO WORM media as blank media for the OTG file system. You can format new MO WORM media only to use the UDF file system. Foreign Unknown Media has been formatted for a file system that is not supported for that media type. DiskXtender does not recognize the media. If DiskXtender supports the type of media, format foreign media with a supported file system, and then label the media. The problem could be an unsupported media type or an unsupported file system. Ensure that the media type is supported, and format the media with a supported file system. Then label the media. Unformatted Media is not formatted. Format and label the media. Corrupt DiskXtender recognizes the media but cannot use it because of problems with the media itself. Media corruption most often occurs as a result of a power failure while files are being written to the media. Perform a Check Disk media task on the media to attempt to find and repair the errors. Choosing a format and label method There are several ways to format and label media in DiskXtender. The most efficient method depends on the status of the media and the number of media that need to be formatted and labeled: To format and label multiple pieces of unformatted media at once, use the Media Prepare Manager. Note: The Media Prepare Manager is available only for media in a library configured through the MediaStor media service. To format and label a single piece of unformatted media, assign the Format and Label media tasks to the media. To format but not label a single piece or multiple pieces of unformatted media, assign a Format media task to the media. You can then enable automatic labeling of the media through a media group, or you can label the media individually later by assigning a Label media task. To label a single piece of blank media, assign a Label media task. To set up automatic labeling and addition of blank media to a media group, use the Automation tab of the Media Group Properties dialog box. Repairing corrupt media If media is formatted with the UDF file system and the media becomes corrupt, you can run the Check Disk media task to attempt to repair the media. Corrupt media is listed in the Corrupt node of the Available Media tree in the File System Manager Administrator. Managing the lifecycle of removable media 115

108 Connecting to Optical and Tape Devices For MO, MO WORM, UDO, UDO WORM, and DVD-RAM media formatted with the OTG file system, the Check Disk media task performs a scan of DiskXtender file tags. Renaming media You can rename media through the Administrator without the Format and Label media tasks. The following topics provide details. What happens when you rename a piece of media Renaming a piece of rewritable media changes the label identifier on the media itself. Renaming a piece of write-once (non-rewritable) media only changes the reference that DiskXtender uses, and is therefore temporary. In other words, the name on the media itself does not change. Non-rewritable media reverts to its original label the next time you inventory the media. Renaming a piece of media does not affect the files or directories on the media. Note: Renaming original media does not rename any copy media associated with the original. The copies are updated, but the media names no longer match. In addition, you cannot rename copy media. Reusing media To ensure the most efficient use of media, you may want to eliminate wasted space on the media by compacting and reformatting the media. The following topics provide details: When media reuse is necessary on page 116 Understanding the media reuse process on page 116 When media reuse is necessary DiskXtender writes files sequentially to most removable media types. When you edit a file on the extended drive that has been written sequentially to rewritable media, DiskXtender does not edit the file data on the media. Instead, when a migrated file is edited, the file is written as a new file to the next blank area on the media, and the file tag on the extended drive is updated to point to the new file. The old file data still remains on the media, but is orphaned. This is true for DVD-RAM, MO, UDO, and tape media. Similarly, when you delete a file that has been written sequentially to these media types, the file and its data are deleted only from the extended drive and are not removed from the media. Deleting the file on the extended drive removes the file tag, which contains the location information for the file on the media. Without the file tag, DiskXtender can no longer track that file on the media (even though the data is still there), and the file is orphaned. The only exception to this is when a file is renamed on rewritable media. In that case, the file trailer on the rewritable media, where the filename is kept, is updated to reflect the new filename. The space taken up by older versions of files and deleted files is wasted space on the media. Understanding the media reuse process on page 116 provides details on how to reuse the media and minimize the amount of wasted space. Understanding the media reuse process You can reclaim this wasted space on media and use the media more efficiently by: 116 EMC DiskXtender Microsoft Windows Version Administration Guide

109 Connecting to Optical and Tape Devices 1. Returning all valid versions of files to the extended drive through compaction. 2. Rewriting the files to another piece of media. 3. Reformatting the media. 4. Writing new files to the media. When you compact a piece of media, DiskXtender evaluates each file on the media. If there is a file tag on the extended drive for a file on the piece of media, the file data is copied back to the extended drive and the migration information for the file is removed. In this way, the file appears as if it has never been migrated to media. IMPORTANT If you allow folder renames on media that does not support folder renames, then do not compact the media. Data loss might occur because DiskXtender will not find the corresponding file tags on the extended drive, so the files will not be copied back to the drive. DiskXtender then evaluates the file against configured move rules. Because the files have already been written to media once, they usually qualify for migration as soon as they are copied back to the extended drive. This means that the files are migrated back to media as soon as the file migration schedule is active (and a background scan occurs, if an age delay is configured for the assigned move rules). When DiskXtender finishes evaluating each file on a piece of media during compaction, the media is automatically removed from the media group to which it is assigned. The media can then be reformatted, labeled, and assigned to a media group. This enables DiskXtender to begin migrating new files to the media. Note: You can compact other removable media types, including DVD-R, MO WORM, UDO WORM, and tape-worm. However, you cannot reformat those media types. After the files are copied back to the extended drive through compaction and the media is removed from its media group, it remains assigned to the extended drive. The files also remain on the media. You can deallocate the media and remove it from the system. If you add the media to a media group again instead, the files are restored again to the extended drive, and DiskXtender continues writing files to the media from the point at which it last stopped. As a result, you cannot reuse these media types. You can configure DiskXtender to automatically reuse media based on the amount of wasted space on the media. You can also monitor the media by using DiskXtender reports, and then manually perform the steps to reuse the media at the appropriate time. Automatically reusing media If you use DVD-RAM, MO, UDO, or tape media in a device configured as part of a MediaStor media service, then you can configure DiskXtender to automatically compact a piece of media, reformat it, re-label it, and re-add it to a media group. This automatic reuse occurs based on the percentage of wasted space on the media, and is configured by using media group automation features. After the automatic compaction, media is reformatted with the same file system that was originally on the media. It is then labeled based on the configured label specifications, and added to the same media group. To configure automatic reuse of DVD-RAM, MO, UDO, or tape media: Managing the lifecycle of removable media 117

110 Connecting to Optical and Tape Devices 1. Ensure that there is sufficient free space on the extended drive to accommodate the files that are written back to the extended drive during compaction. At a minimum, the equivalent of one side of the media should be available. If you use media compression, then keep in mind that you may need as much as four times the amount of space on one side of the media. Purge inactive data to free space on the extended drive, as discussed in Chapter 12, Purging Files. 2. When configuring a media group with removable media, select Auto-label blank media when free space falls below MB in the Media Labeling section on the Automation page, as illustrated in Figure 20 on page 118. Figure 8 Media group Automation page 3. Choose whether to use a standard labeling method or barcodes: To use a standard method: a. Select Use standard method. b. In the Label Prefix text box, type a prefix for the media label. c. From the Number Width drop-down list, select the number of digits to be used to create the incremental numbering for the media label. The Sample Name text box provides an example of the label based on the prefix and number width criteria you set. For example, if you type Payroll as the prefix and select a number width of 3, the label for the first piece of media is Payroll001, the second is Payroll002, and so forth. To use barcodes, select Use barcode. 4. If there are multiple libraries configured and you want to limit the selection of blank media for the media group to a single library, select the Restrict blank media search to this library checkbox, and then select the library from the drop-down list. 118 EMC DiskXtender Microsoft Windows Version Administration Guide

111 Connecting to Optical and Tape Devices 5. In the Media Compaction section, select Auto-compact media when wasted space exceeds %, and the type the percentage of wasted space that must be on the media before DiskXtender automatically compacts the media. Note: If media compaction fails, some, all, or none of the files from the media might have been moved back to the extended drive. If you can determine the cause of the compaction failure, make the appropriate adjustments and reschedule the compaction. The compaction process starts over, but any files already copied back to the extended drive are not duplicated. In addition, all files that have been compacted are re-migrated to media as they qualify for movement. Monitoring and manually reusing media If you use DVD-RAM, MO, UDO, or tape media, then you can monitor the status of an individual piece of media. When the amount of wasted space on the media reaches a certain point, you can assign media tasks to compact the media, reformat it, and re-label it. You can then re-add it to a media group and begin migrating files to it. To monitor and manually reuse a piece of media: 1. Monitor the properties for the media on a regular basis to determine the amount of wasted space on the media: a. Right-click the piece of media and select Properties. The Media Properties dialog box appears. b. Click the Location tab. c. Make a note of the Byte Count value in the Extended Drive section. This is the amount of space in bytes on the extended drive that is occupied by files that have been written to the media. d. Click the Space tab. e. Make a note of the Used Space value in the Logical Capacity section. This is the amount of data in bytes that has been written to the media. f. Divide the byte count on the extended drive by the used space on the media to determine the percentage of wasted space on the media. g. If the percentage is high, proceed with the rest of this procedure to reuse the media. The default percentage of wasted space at which DiskXtender initiates media reuse as part of the media group automation process is 50 percent. 2. Right-click the piece of media and select Edit Tasks. 3. Select COMPACT from the Next Task drop-down list, and then click Add Next Task. The Compact task appears in the task list of the Media Tasks dialog box, as illustrated in Figure 21 on page 120. Managing the lifecycle of removable media 119

112 Connecting to Optical and Tape Devices Figure 9 Compact media task 4. Choose whether to process the media task as soon as possible or when the next media task schedule is active. By default, the media task schedule is active at the same time as the file migration schedule (daily from 8 P.M. to 8 A.M). 5. Select FORMAT from the Next Task drop-down list, and then click Add Next Task. The Format task appears below the Compact task in the task list of the Media Tasks dialog box, as illustrated in Figure 22 on page 120. Figure 10 Format media task after Compact media task 6. From the File System drop-down list, select the file system with which to format the media. 7. Leave the Force low-level SCSI format checkbox clear to perform a quick format on the media. A low-level SCSI format is not necessary. 120 EMC DiskXtender Microsoft Windows Version Administration Guide

113 Connecting to Optical and Tape Devices 8. If the library and file system support it, choose whether to enable compression for the media by selecting or clearing the checkbox. 9. To reformat a piece of tape media as tape-worm, select the Convert this media to WORM checkbox. This option is available only if you are formatting the media in a Quantum DLT-S4 drive. Note: Once tape has been formatted and converted to tape-worm, it cannot be converted back to tape. However, a tape can be reformatted and converted to tape-worm. 10. In the When To Process section of the Media Tasks dialog box, select the same scheduling option that you selected for the Compact media task. 11. Select LABEL from the Next Task drop-down list, and then click Add Next Task. The Label task appears below the Format task in the task list of the Media Tasks dialog box, as illustrated in Figure 23 on page 121. Figure 11 Label media task after Compact and Format media tasks 12. Type a label for the media in the Media Name text box. 13. In the When To Process section of the Media Tasks dialog box, select the same scheduling option that you selected for the Compact and Format media tasks. 14. Click Next. The Order Media in Task Queue page appears with the recently added media at the bottom of the media task queue list. 15. (Optional) Promote or demote the media in the list to change the order in which tasks are processed by clicking the up or down arrow to the right of the list box. 16. Click Finish. All media with tasks assigned appear in the Media Task Queue Manager until they are processed. Media with tasks selected to process as soon as possible may appear only briefly in the queue before the processing completes. 17. Add the media to a media group. Managing the lifecycle of removable media 121

114 Connecting to Optical and Tape Devices Troubleshooting a Compact media task When you run a Compact media task, you might encounter the following issues: Extended drive space issues If, during the processing of the Compact task, the extended drive becomes so full as to not be able to receive any more files, the Compact task goes into a suspended state for one hour. In addition, no other Compact tasks for other media are processed until the original hour (for which the attempted Compact task was suspended) has expired. After an hour has passed, DiskXtender retries the original compaction, and if the task is successful, processes any remaining Compact tasks. This suspension of all Compact tasks enables DiskXtender to move and purge files to clear necessary extended drive space, and it ensures that drives, which might be needed to complete other media functions, are not being used to attempt Compact tasks that cannot succeed until there is sufficient space on the extended drive. Compact task failure If a Compact media task fails, some, all, or none of the files from the media might have been moved back to the extended drive. If you can determine the cause of the compaction failure, make the appropriate adjustments and reschedule the compaction. The compaction process starts over, but any files already copied back to the extended drive are not duplicated. In addition, all files that have been compacted are re-migrated to media as they qualify for movement. Finalizing DVD-R media If you are finished writing files to a piece of DVD-R media, you can finalize the media. Finalization is a process of closing the media, making it read-only and preventing DiskXtender from writing any more files to it. Finalizing media makes the media more stable, better protecting the data on the media. It also enables the media to be taken out of the DiskXtender system and read on a computer with the Windows XP, Windows 7, Windows 8, Windows 2003, or Windows Server 2008 operating system. Finalizing media enables the files written to that media to be purged from the extended drive. Those files are not purged until the media is finalized, due to the potential instability of unfinalized media. There are two ways to set up finalization of media: Set up automated finalization of full media through the media group options. Assign the Finalize media task. If you have copies of the media, DiskXtender automatically finalizes the copy after it is updated. DiskXtender recognizes that the original is finalized, and therefore no more updates are made to the original, meaning no more updates are made to the copy. Automatically finalizing DVD-R media DiskXtender can automatically finalize DVD-R media when it is marked as full. Media is marked as full when it reaches the free space threshold configured on the Mark media full when free space falls below option for a media group. By default, this threshold is 1 MB. 122 EMC DiskXtender Microsoft Windows Version Administration Guide

115 Connecting to Optical and Tape Devices The Auto-finalize DVD-R media after marking full option, also available on the Options page for media groups, causes DiskXtender to automatically assign the Finalize media task after it reaches the free space threshold. You can also choose whether to automatically purge all files on the media after it has been finalized, as well as whether to mark the files for Direct Read. To enable automatic finalization: 1. When configuring a DVD-R media group, select Auto-finalize DVD-R media after marking full from the list on the Options page. 2. Select the Auto-finalization option. 3. (Optional) To automatically purge files after the media is finalized, select Purge files after auto-finalizing media from the list on the Options page, and then select the Purge option. If you do not automatically purge the files after the media is finalized, the files are purged according to configured purge rules. Chapter 12, Purging Files, provides details on choosing an appropriate purge strategy for your environment. 4. (Optional) To automatically mark purged files for Direct Read after the media is finalized, select Direct read files after auto-finalizing media from the list on the Options page, and then choose the Direct Read option. Direct Read is strongly discouraged in most environments. Review the recommendations in Reading files directly from media on page 115 before enabling this option. You can also manually finalizing DVD-R media by assigning a Finalize media task. See EMC DiskXtender for Windows Online Help for more detailed instructions. Removing media from a MediaStor library The method you use to remove a piece of media from a MediaStor library depends on whether you intend to reinsert the media at a later time: To permanently remove a piece of media, delete the media. To temporarily remove a piece of media, eject the media by using the instructions in Expectations for files migrated to removable media on page 124. Media that has been ejected remains listed in the MediaStor and File System Manager Administrators with an offline status. If a read or write request for the media occurs, MediaStor prompts you to reinsert the media. Deleting media from a MediaStor library When you delete a piece of media, the media is ejected from the hardware device and removed from the media list in the Administrator. Ejecting media from a MediaStor library When you eject a piece of media, DiskXtender continues to track the media. It is listed in the Administrator with an offline status, and you are prompted to reinsert the media if a read or write request occurs. Note: If a library has no mailslot, special considerations for ejecting media apply. Set the library offline before you manually remove media from the shelf or magazine. After you remove the media, set the library back online and reinventory the library. Managing the lifecycle of removable media 123

116 Connecting to Optical and Tape Devices Expectations for files migrated to removable media After a file is moved to removable media, the type of media and the file system on the media determine whether you can edit or delete the file. These factors also determine whether you can edit or delete the folder in which a file is located. To keep the files on the media synchronized with the extended drive, certain operations may be prohibited on the extended drive. The following topics provide details on the operations allowed for each type of removable media. Expectations for files migrated to MO and UDO media Table 17 on page 124 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to MO and UDO media formatted with either the OTG or UDF file system. Table 7 Extended drive operations for files migrated to MO and UDO media Edit Rename Set attributes Delete Move within a media folder Move outside of a media folder Files moved to MO and UDO media formatted with the OTG file system Folders with files moved to MO and UDO media formatted with the OTG file system Files moved to MO and UDO media formatted with the UDF file system Folders with files moved to MO and UDO media formatted with the UDF file system N/A N/A N/A N/A When the MO or UDO media is formatted with the UDF file system, all changes to the files and folders on the extended drive are propagated to the files and folders on the media. When the MO or UDO media is formatted with the OTG file system, file renames and attribute changes are propagated to the files on the media. However, file edits and deletions, as well as folder deletions, are not propagated. Instead, when a file is edited, the file is written as a new file to the next blank area on the media, and the file tag on the extended drive is updated to point to the new file. The old file data still remains on the media, but is orphaned. Similarly, when a file is deleted on the extended drive, the file and its data are deleted only from the extended drive. They are not removed from the media. Deleting the file on the extended drive removes the file tag, which contains the location information for the file on the media. Without the file tag, DiskXtender can no longer track that file on the media (even though the data is still there), and the file on the media is orphaned. Note: For files that are moved to MO WORM or UDO WORM devices, you can prevent the delete operations performed on the files in the extended drive from propagating to the media devices. Prevent delete from media on page 121 provides details. Supported file systems on page 112 provides details on system behavior in this scenario. 124 EMC DiskXtender Microsoft Windows Version Administration Guide

117 Connecting to Optical and Tape Devices Expectations for files migrated to MO WORM and UDO WORM media Table 18 on page 125 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to MO WORM and UDO WORM media formatted with either the OTG or UDF file system. Table 8 Extended drive operations for files migrated to MO WORM or UDO WORM media Edit Rename Set attributes Delete Move within a media folder Move outside of a media folder Files moved to MO WORM and UDO WORM media formatted with the OTG file system Folders with files moved to MO WORM and UDO WORM media formatted with the OTG file system N/A N/A Files moved to MO WORM and UDO WORM media formatted with the UDF file system Folders with files moved to MO WORM and UDO WORM media formatted with the UDF file system N/A N/A When the MO WORM or UDO WORM media is formatted with the UDF file system, file and folder deletions on the extended drive are propagated to the media. If you rename a folder or set file attributes on the extended drive, however, then the rename is not propagated. When the MO WORM or UDO WORM media is formatted with the OTG file system, file deletions and attribute changes are allowed on the extended drive. However, these operations are not propagated to the media. When a file is deleted on the extended drive, the file and its data are deleted only from the extended drive. They are not removed from the media. Deleting the file on the extended drive removes the file tag, which contains the location information for the file on the media. Without the file tag, DiskXtender can no longer track that file on the media (even though the data is still there), and the file on the media is orphaned. Supported file systems on page 112 provides details on system behavior in this scenario. Expectations for files migrated to tape or tape-worm media Table 19 on page 125 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to tape or tape-worm media. Table 9 Extended drive operations for files migrated to tape or tape-worm media Edit Rename Set attributes Delete Files moved to tape or tape-worm media Move within a media folder Move outside of a media folder Folders with files moved to tape or tape-worm media N/A N/A Expectations for files migrated to removable media 125

118 Connecting to Optical and Tape Devices Attribute changes and file and folder deletions are allowed on the extended drive. However, these operations are not propagated to the media. Supported file systems on page 112 provides details on system behavior in this scenario. Expectations for files migrated to DVD media Table 20 on page 126 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to DVD media formatted with either the OTG or UDF file system. Table 10 Extended drive operations for files migrated to DVD media Edit Rename Set attributes Delete Move within a media folder Move outside of a media folder Files moved to DVD-R media (which uses the UDF file system) Folders with files moved to DVD-R media (which uses the UDF file system) Files moved to DVD-RAM media formatted with the OTG file system Folders with files moved to DVD-RAM media formatted with the OTG file system Files moved to DVD-RAM media formatted with the UDF file system Folders with files moved to DVD-RAM media formatted with the UDF file system N/A N/A N/A N/A N/A N/A Files moved to DVD-ROM media (which uses the UDF file system) Folders with files moved to DVD-ROM media (which uses the UDF file system) N/A N/A Attribute changes and file and folder deletions are allowed on the extended drive for files migrated to DVD-R and DVD-ROM media. However, these operations are not propagated to the media. When DVD-RAM media is formatted with the UDF file system, all changes to the files and folders on the extended drive are propagated to the files and folders on the media. When DVD-RAM media is formatted with the OTG file system, attribute changes are propagated to the files on the media. However, file and folder deletions are not propagated. When a file is deleted on the extended drive, the file and its data are deleted only from the extended drive. They are not removed from the media. Deleting the file on the extended drive removes the file tag, which contains the location information for the file on the media. Without the file tag, DiskXtender can no longer track that file on the media (even though the data is still there), and the file on the media is orphaned. Supported file systems on page 112 provides details on system behavior in this scenario. 126 EMC DiskXtender Microsoft Windows Version Administration Guide

119 Connecting to Optical and Tape Devices What happens when you edit or move files that have been migrated to removable media Keep in mind the following important points when working with files that have been moved to removable media, and the folders in which those files are located: Supported file systems on page 112 provides details on system behavior when files are edited on the extended drive. This behavior depends on the type of media to which the file has been migrated, as well as the file system on the media. If you move a purged file to another location within the same media folder by dragging and dropping the file in Microsoft Windows Explorer, then the file is fetched to the extended drive. This behavior is caused by Windows Explorer. If you move a purged file to another location within the same media folder by using the command line, then the file is not fetched. If you move a folder with purged files to another location within the same media folder, however, then the files in the folder are not fetched. Expectations for files migrated to removable media 127

120 Connecting to Optical and Tape Devices Performance tuning for removable media Review the following recommendations for improving DiskXtender performance with removable media by adjusting the configuration of media folders, media groups, rules, file deletions, and service options. Media folder considerations for removable media Use extra caution when planning the directory structure and creating media folders on the extended drive. With some types of removable media, folder renames are not allowed on the extended drive. After files are moved to media, you will not be able to change the directory structure. Media group considerations for removable media When you are creating media groups, consider the following settings to optimize DiskXtender performance with removable media: To create a media group for DVD-ROM media, select DVD-R as the media type. To automatically label media and add it to the media group as space is needed, enable automatic labeling on the Automation page for the media group. To automatically compact media based on wasted space on the media, enable automatic compaction on the Automation page for the media group. On the Options page for the media group, review, and if necessary, adjust the options in Table 21 on page 128. Table 11 Media group options for removable media (page 1 of 2) Option Library mount - minimum free space on media (KB) Library mount - minimum number of files Media fill method Description This option controls the minimum number of KB of free space that must be available on a piece of media for DiskXtender to request that the media service mount that media to write files to it. The default value is 32 KB. This setting only controls whether DiskXtender requests the media be mounted specifically for media writes. If the media is already mounted when the file moves are activated, DiskXtender attempts to write to the media regardless of the amount of free space on the media. This option controls the minimum number of files that must be on the move list before the media in the media group is mounted for file writes. The default value is one file. This setting only controls whether DiskXtender requests the media be mounted specifically for media writes. If the media is already mounted when the file moves are activated, DiskXtender attempts to write to the media regardless of the number of files on the move list. Leave the media fill method set to sequential to fill media one at a time in the order they appear in the tree view of the File System Manager Administrator or on the Items tab of the Media Group Properties dialog box. Sequential fill is designed to maximize file retrieval times for library media. 128 EMC DiskXtender Microsoft Windows Version Administration Guide

121 Connecting to Optical and Tape Devices Table 11 Media group options for removable media (page 2 of 2) Option Auto-finalize DVD-R media after marking full Purge files after auto-finalizing media Description If you use DVD-R, this option controls whether you automatically finalize a piece of media after it is marked as full. Media is automatically marked full when the amount of free space falls below 1 MB, as configured on the Mark media full when free space falls below option for media groups. If you do not automatically finalize full media, you must manually assign a Finalize media task to the media to finalize it. Finalizing DVD-R media on page 122 provides details. If you choose to automatically finalize DVD-R media when it is full, you can automatically purge all files on the media from the extended drive after the media is finalized. Purge files automatically after finalization only if it is likely they will no longer be accessed. If users are likely to continue accessing the files on the finalized media, then configure purge rules instead to purge the files when they are no longer being used. Chapter 12, Purging Files, provides details. Note: Even if you enable automatic purging of files after finalization, you should still configure purge rules. If a purged file is fetched from finalized media, it cannot be purged again unless it qualifies for a purge rule. Direct read files after auto-finalizing media If you choose to automatically purge files after DVD-R is automatically finalized, then you can also mark those files for Direct Read. Direct Read is strongly discouraged in most environments. Review the recommendations in Reading files directly from media on page 115 before enabling this option. Move rule considerations for removable media When you are creating move rules, consider the following settings to optimize DiskXtender performance with removable media: With removable media, it is often best to set up an age restriction from last write time in the move rule. This limits the number of times frequently changing files are written and rewritten to the media. On the Settings page for the move rule, do not select the options to purge files immediately after they are moved or to set Direct Read on the files. Instead, use the purge recommendations in Purge rule considerations for removable media on page 130, and review the recommendations for when to use Direct Read in Reading files directly from media on page 115. If you use DVD-R media, the Purge files immediately after move and Mark files for Direct Read options on the move rule Settings page are dimmed because you cannot purge files until the media is finalized. Performance tuning for removable media 129

122 Connecting to Optical and Tape Devices Purge rule considerations for removable media When you are creating purge rules, consider the following settings to optimize DiskXtender performance with removable media: Because access to files on library media can be slower than other media types, limit the number of times that user requests for files are satisfied by having to fetch file data from the media. In other words, leave as much file data on the extended drive as possible by waiting to purge files until it is absolutely necessary. To do this, configure an age delay for the purge rule. Select the Apply rule to files of age greater than option on the Age page of the purge rule wizard, specify the number of days (typically 30 or 60), and then select Last access time from the drop-down list. Also, on the Settings page of the purge rule wizard, select Do not force purges during background scans. This option prevents files from being purged until disk space is needed. If you use DVD-R media, you cannot purge files until the media they are written to is finalized. Both the Finalize media task and the media group auto-finalization functions enable you to select purge files when the media is finalized. If you do not choose to purge files as a function of finalization, the data for those files remains on the extended drive until the files qualify for configured purge rules. Then it is the selections made in the purge rules that determine when the files are purged from the extended drive. Note: Even if you enable automatic purging of files after finalization, you should still configure purge rules. If a purged file is fetched from finalized media, it cannot be purged again unless it qualifies for a purge rule. File deletion considerations for removable media When deleting files, consider the following to optimize DiskXtender performance with removable media: When you delete a file from the extended drive, the type of media and the file system on the media control whether the file is deleted from the media. DiskXtender can successfully delete files from DVD-RAM, MO, and UDO media formatted with the UDF file system. With the media types in the following list, however, DiskXtender cannot delete the file from media. Instead, the file is marked for deletion. It still exists on the media, but it becomes an orphaned file and is no longer recognized or tracked by DiskXtender: DVD-R and DVD-ROM DVD-RAM formatted with the OTG file system MO WORM and UDO WORM MO and UDO formatted with the OTG file system Tape and tape-worm The deleted files are not actually removed from the media unless the media is compacted and reformatted. You can reformat DVD-RAM, MO, UDO, and tape media. 130 EMC DiskXtender Microsoft Windows Version Administration Guide

123 Connecting to Optical and Tape Devices When a significant number of delete transactions are sent to removable media in a library, the completion of those transactions may take a long time. During that time, file fetches may be delayed or even canceled due to timeout issues. Do not empty the DiskXtender Recycler when users are most likely to access the system. Or, if the Recycler is disabled, do not delete a significant number of files from the extended drive when users are most likely to access the system. You can prevent delete transactions performed on purged files on an extended drive, from being propagated to WORM media. Prevent delete from media on page 121 provides details on how to control delete transactions done on an extended drive from being propagated to the WORM media. Service options for removable media On the Options tab of the Service Properties dialog box, consider adjusting the settings listed in Table 22 on page 131 to optimize DiskXtender performance with removable media. Table 12 Service options for optical and tape media Media type Options DVD-R, DVD-RAM, and DVD-ROM DVD - drive saver DVD timeslice - minimum mount DVD timeslice - last fetch DVD timeslice - maximum mount MO, UDO, MO WORM, and UDO WORM Optical drive saver Optical timeslice - minimum mount Optical timeslice - last fetch Optical timeslice - maximum mount Tape Tape timeslice - minimum mount Tape timeslice - last fetch Tape timeslice - maximum mount Tape block size default value Use hardware compression Note: Some of the options listed above are available only when Advanced mode is turned on. The options apply at the service level. This means that you cannot customize the option for each media service configured. For example, if you configure multiple MediaStor media services with different DVD libraries connected to each media, the drive saver setting applies for all of them. Drive saver Timeslice - minimum mount The drive saver options enable you to specify how long DiskXtender waits after media becomes inactive before it automatically spins down the media. The default value is 5 minutes. Automatically spinning down media means that the media is still mounted but lies idle until the media is requested by DiskXtender or dismounted. The minimum mount timeslice options enable you to specify the minimum amount of time DiskXtender keeps removable media in a drive after it has been mounted. This means that the media is mounted for at least the amount of time you specify. DiskXtender does not automatically dismount the media, and you cannot dismount the media before the time limit is up. Performance tuning for removable media 131

124 Connecting to Optical and Tape Devices For DVD and optical media, the default value is 30 seconds. For tape media, the default value is 300 seconds. Timeslice - last fetch Timeslice - maximum mount Tape block size default value Use hardware compression The last fetch timeslice options enable you to specify the amount of time removable media must remain mounted in a drive after DiskXtender has fetched data from that media. This means that the media is not dismounted immediately after a fetch. Instead, DiskXtender waits at least the entered amount of time before dismounting the media. For DVD and optical media, the default value is 5 seconds. For tape media, the default value is 60 seconds. The maximum mount timeslice options enable you to limit the amount of time that removable media can stay mounted in a drive if it is inactive and has no pending fetches, and if there is other media that has pending fetches. This enables DiskXtender to dismount inactive media in favor of other media that is being requested. The default value for DVD and optical media is 60 seconds. The default value for tape media is 1,200 seconds. The tape block size default value enables you to configure the size settings DiskXtender uses for formatting tape and tape-worm media. The value you use dictates the default block size of tape and tape-worm media that DiskXtender uses to format the media when the Media Prepare Manager or the Format media task are run. The default value is 64 KB. If the tape hardware device you use supports data compression, you can configure DiskXtender to compress data with the Use hardware compression option. Data compression enables DiskXtender to store data in a format that requires less space than usual. 132 EMC DiskXtender Microsoft Windows Version Administration Guide

125 6 Connecting to Tivoli Storage Manager If you use IBM Tivoli Storage Manager (TSM) as a data storage or data backup system, you can migrate files on a DiskXtender extended drive to the TSM server. TSM can subsequently move the files to storage devices managed by the TSM server. The following topics provide details on configuring and managing the TSM environment: How DiskXtender works with TSM Installing the TSM Backup/Archive Client Managing the lifecycle of TSM media Expectations for files migrated to TSM Performance tuning for TSM Connecting to Tivoli Storage Manager 133

126 Connecting to Tivoli Storage Manager How DiskXtender works with TSM You can leverage the storage management capabilities of TSM with DiskXtender. You can configure a TSM media service in DiskXtender to move files on the extended drive to a TSM server. TSM can subsequently move the files to storage devices managed by the TSM server, as illustrated in Figure 1 on page 134. Extended drives G:\NTFS DiskXtender server (Microsoft Windows) DiskXtender installed Tivoli Storage Manager Backup/Archive Client installed H:\ReFS Tivoli Storage Manager Server installed and managing storage devices TSM server TCP/IP connection SCSI connection Storage device GEN Figure 1 DiskXtender with TSM After you create a TSM media service in DiskXtender to create the connection between DiskXtender and a TSM server, you must create virtual TSM media. The virtual media does not correspond to a physical piece of media in the TSM system, such as a tape cartridge. Instead, the virtual media is designed to simulate divisions of the larger repository (the TSM server) and corresponds to a filespace on the TSM server. The maximum size of a piece of virtual TSM media is 256 GB. This enables you to take advantage of the flexible file migration features available in DiskXtender. The TSM server manages the actual physical removable media in the storage device attached to the TSM server. Tasks like formatting, labeling, and copying media in the storage device attached to the TSM server are performed by TSM rather than by DiskXtender. 134 EMC DiskXtender Microsoft Windows Version Administration Guide

127 Connecting to Tivoli Storage Manager Installing the TSM Backup/Archive Client Perform the following steps to prepare TSM for use with DiskXtender: 1. Ensure that TSM Server 5.4, 5.5, or is installed and configured on a different server than the DiskXtender server. Both the Microsoft Windows and IBM AIX operating system versions of TSM server are supported with DiskXtender. 2. On the DiskXtender server, install and configure version 5.5 or of the TSM Backup/Archive Client. (TSM is required for Windows Server 2008.) Note: If you install DiskXtender in a Microsoft clustering environment, perform this step and the next step on all nodes of the cluster where DiskXtender is installed. 3. In the TSM Backup/Archive Client, set the Allow Backup Delete option to Yes. Otherwise, you receive an error when you attempt to create the TSM media service in DiskXtender (or when you try to set the service online, if the setting is changed after the service is created). 4. Copy the dsm.opt file from the BACLIENT subdirectory of the TSM client installation directory to the BIN subdirectory of the DiskXtender installation directory (Drive:\Program Files\EMC\DiskXtender\BIN, where Drive is the system drive on the DiskXtender server). If you are planning to connect to multiple TSM servers, or if you are planning to use multiple TSM media services based on different profiles (policies) created on the same TSM server, then create a uniquely named dsm.opt file (such as dsm1.opt and dsm2.opt) for each TSM server or media service. If you are preparing an active Microsoft cluster node, and there are multiple active nodes in the cluster, then create a uniquely named dsm.opt file for each node. This allows for cluster failover, where one dsm.opt file is failed over and moved to another node. Naming the files differently prevents each file from being overwritten. You do not need to prepare a dsm.opt file for passive nodes. 5. Edit the copied dsm.opt file and change the information in Table 1 on page 135 in that file. Table 1 DiskXtender edits for TSM dsm.opt file Field tcpserveraddress Edit required Type the IP address or symbolic DNS name of the TSM server. Note: If the tcpserveraddress information in the dsm.opt file on the TSM server changes, the change must also be made to the dsm.opt file on the DiskXtender server. The information in the dsm.opt files must be identical. Differences between the dsm.opt files impact whether DiskXtender can migrate files to and fetch files from the TSM server. nodename passwordaccess Type the client nodename configured on the TSM server for DiskXtender. If you are preparing an active cluster node, type the virtual server name. Type prompt. Installing the TSM Backup/Archive Client 135

128 Connecting to Tivoli Storage Manager In the following example, is the IP address of the TSM server, and DX-DXSERVER is the client nodename: *================================================== * tcp/ip *================================================== commmethod TCPIP tcpport 1500 tcpserveraddress nodename DX-DXSERVER passwordaccess prompt 6. Repeat step 5 for each option file in the BIN subdirectory of the DiskXtender installation directory. 7. Stop and restart the DiskXtender service to enable the changes to take effect. If you are adding the TSM media service to DiskXtender in a clustering environment, the dsm.opt file on each node is automatically replicated to all other nodes. 136 EMC DiskXtender Microsoft Windows Version Administration Guide

129 Connecting to Tivoli Storage Manager Managing the lifecycle of TSM media TSM media is considered virtual media because it does not correspond directly to a specific piece of media (like a tape cartridge). Instead, the virtual media is designed to simulate divisions of a TSM server and corresponds to a filespace on the TSM server. The use of virtual media enables you to take advantage of the flexible file migration features available in DiskXtender. The following topics provide details on creating and managing virtual TSM media throughout its lifecycle in DiskXtender. Understanding the TSM media lifecycle The following steps detail the lifecycle process for a virtual TSM media: 1. Decide how many pieces of TSM media to create. Deciding how many pieces of TSM media to create on page 137 provides guidelines for making the decision. 2. Create the media and add it to the system: a. Create each piece of media for the media service. b. Allocate the media to the extended drive. c. Add the media to a media group. 3. DiskXtender moves files to the media until the media reaches its maximum capacity of 256 GB, at which point the media is considered full. 4. The full media remains in the media group. This enables continued access to the files on the media. 5. (Optional) If you no longer need the files that were migrated to the media, or if you want to move the files to a different piece or type of media, remove the media from the media group. There are two ways to remove media from a media group: Compact the media to copy the files back to the extended drive and move the files to other media. The media is automatically removed from the media group during compaction. Removing media but keeping the files on the drive on page 189 provides instructions. To remove the files on the media from the extended drive altogether, remove the media from the media group. 6. (Optional) Deallocate the media from the extended drive, and delete it from the media service. Deciding how many pieces of TSM media to create You can create up to 512 pieces of TSM media for a media service. This maximum limit may be further limited by the number of client sessions allowed by the settings on the TSM server. Even if the TSM media service is writing to a single TSM server, you may want to create multiple pieces of TSM media. This is important for the following reasons: DiskXtender performance improves when the system can write to and read from multiple pieces of media at once. Focusing all system activity on a single piece of media can cause a performance bottleneck. Managing the lifecycle of TSM media 137

130 Connecting to Tivoli Storage Manager You can take advantage of the flexible file migration features available in DiskXtender. You can migrate data from multiple media folders to different locations (pieces of media) by using customized migration rules. More files remain available if an error or other system problem occurs. If a piece of media becomes inaccessible (for example, as a result of a failed transaction), the files on other pieces of media remain accessible while the inaccessible media is offline. The number of media to create depends on your environment and your tolerance for system downtime. There must be at least one piece of media for each media folder on the extended drive. For assistance in determining how many pieces of TSM media to create, contact EMC Professional Services. 138 EMC DiskXtender Microsoft Windows Version Administration Guide

131 Connecting to Tivoli Storage Manager Expectations for files migrated to TSM You can perform most operations on the extended drive for files that have been migrated to TSM media. Table 2 on page 139 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to TSM. Table 2 Extended drive operations for TSM files Edit Rename Set attributes Delete Move within a media folder Move outside of a media folder Files moved to TSM Folders with files moved to TSM N/A N/A All changes to the files and folders on the extended drive are propagated to the files and folders on the TSM server. Keep in mind the following important points when working with files that have been moved to TSM, and the folders in which those files are located: When you edit a file on the extended drive, the new (edited) file is migrated to TSM, and the original version of the file is deleted. In this way, only the newer version of the file remains on the TSM server. If you move a purged file to another location within the same media folder by dragging and dropping the file in Microsoft Windows Explorer, then the file is fetched to the extended drive. This behavior is caused by Windows Explorer. If you move a purged file to another location within the same media folder by using the command line, then the file is not fetched. If you move a folder with purged files to another location within the same media folder, however, then the files in the folder are not fetched. Expectations for files migrated to TSM 139

132 Connecting to Tivoli Storage Manager Performance tuning for TSM Review the following recommendations for improving DiskXtender performance with TSM by adjusting the configuration of media folders, media groups, rules, and service options. Media folder considerations for TSM You can rename and delete folders on the extended drive when you are migrating files to TSM. As a result, you have flexibility in creating the directory structure on the extended drive. Note: You cannot move files and folders outside of their original media folder. Use caution when designating media folders. Media group considerations for TSM When you are creating media groups, consider the following settings on the Options page for the media group to optimize DiskXtender performance with TSM: Set the Media fill method option to Random. By default, the Media fill method is set to Sequential. When migrating files, setting the option to Random enables DiskXtender to connect to any available piece of media in the group or even to multiple pieces of media at the same time. This option improves DiskXtender performance when writing to more than one piece of TSM media in a media group. Adjust the Maximum media simultaneously receiving files option to the total number of media that are in the media group. This enables DiskXtender to write to all media as necessary. Since you must manually create TSM media when it is needed, enable the Warn when group free space falls below option so that you are notified when the TSM media in the media group is starting to run out of space. You can then configure the warning to be sent out as an alert. Sending alerts for errors and warnings on page 247 provides instructions for configuring alerts. Move rule and purge rule considerations for TSM File migration and retrieval from TSM media is faster than other media types. As a result, the file migration and purge strategy you select depends on other factors in your environment, such as the type of files on the extended drive and the volume of files that DiskXtender is managing. Review the recommendations in Chapter 9, File Migration, and Chapter 12, Purging Files, to determine appropriate migration and purge strategies for your environment. File deletion considerations for TSM When deleting files, consider the following to optimize DiskXtender performance with TSM: 140 EMC DiskXtender Microsoft Windows Version Administration Guide

133 Connecting to Tivoli Storage Manager When a significant number of delete transactions are sent to the TSM, the completion of those transactions may take a long time. During that time, file fetches may be delayed or even canceled due to timeout issues. Do not empty the DiskXtender Recycler when users are most likely to access the system. Or, if the Recycler is disabled, do not delete a significant number of files from the extended drive when users are most likely to access the system. You can prevent delete transactions performed on purged files on an extended drive, from being propagated to TSM. Prevent delete from media on page 237 provides details on how to control delete transactions done on an extended drive from being propagated to TSM. Service options for TSM When DiskXtender communicates with a TSM server to write files, it maintains the connection for at least five seconds. If necessary, you can adjust this connection time by using the TSM timeslice - minimum mount option on the Options tab of the Service Properties dialog box. The timeslice option controls the minimum amount of time DiskXtender keeps virtual media mounted in a virtual drive for file moves. You can mount virtual media in as many as 256 virtual drives at one time. If that number exceeds, the additional mount requests are queued. The requests are then serviced as media is dismounted from the other virtual drives and the drives become available. For file fetches, DiskXtender maintains the connection for one second. You cannot edit the file fetch connection time. Performance tuning for TSM 141

134 Connecting to Tivoli Storage Manager 142 EMC DiskXtender Microsoft Windows Version Administration Guide

135 7 Connecting to Cloud Storage Cloud storage can be used as a data backup system or data storage and files can be migrated from the DiskXtender extended drive onto the cloud. The following topics provide details on configuring and managing the cloud environment: How DiskXtender works with cloud storage Configuring DiskXtender with EMC Atmos Managing the lifecycle of cloud media Expectations for files migrated to cloud media Performance tuning for cloud media Connecting to Cloud Storage 143

136 Connecting to Cloud Storage How DiskXtender works with cloud storage DiskXtender works effectively with the highly scalable virtual media delivered by the cloud storage system. You can configure cloud storage as a media service in DiskXtender and manage the cloud media through various rules. DiskXtender can be used to migrate files to cloud storage which serves as a back-end media. Similarly, files can be fetched from the cloud media back to the primary storage using DiskXtender. DiskXtender currently works only with EMC Atmos, the cloud storage infrastructure from EMC. 144 EMC DiskXtender Microsoft Windows Version Administration Guide

137 Connecting to Cloud Storage Configuring DiskXtender with EMC Atmos EMC Atmos, the cloud storage infrastructure and service, can be configured as a media service with DiskXtender. DiskXtender migrates files from the extended drive to the EMC Atmos cloud storage. Thereon, data is managed through the capabilities of EMC Atmos. How DiskXtender works with Atmos DiskXtender serves as a front-end and communicates with EMC Atmos using the REST API, as illustrated in Figure 1 on page 145. Data is migrated to Atmos as per the Move rules set in DiskXtender, and files are fetched back from Atmos to DiskXtender when requested. DiskXtender Server Figure 1 DiskXtender with cloud storage Steps to connect to Atmos Perform the following steps to prepare Atmos for use with DiskXtender: 1. Obtain the Atmos user credentials from the Atmos administrator. The Atmos administrator needs to provide an account using which the DiskXtender administrator can log in to the Atmos infrastructure. Multiple accounts can be created by the Atmos administrator for various DiskXtender users. EMC Atmos documentation provides details on how to create accounts. 2. Obtain details of the private/in-premises cloud settings if you are connecting to a private cloud media. Private or in-premises cloud network is an internal network which is installed and managed within an organization. Before configuring DiskXtender with a private cloud network, ensure you have obtained the hostname or an alias name of the cloud service, subtenant ID, user ID, and the shared secret from the Atmos administrator. 3. Obtain details of Atmos online/public cloud settings if you are connecting to a public cloud media. A Public cloud connection is the cloud infrastructure which is used by the organization, but managed by an external service provider. Obtain the token ID and shared secret from the Atmos online administrator before configuring DiskXtender with the cloud storage. 4. Perform the following tasks to configure DiskXtender with Atmos: Add the cloud media service Create cloud media Configuring DiskXtender with EMC Atmos 145

138 Connecting to Cloud Storage For detailed steps for the above tasks, refer EMC DiskXtender for Windows Online Help. Secure Atmos data with DiskXtender Algorithms used for encryption Encryption key management Validating the user password How does the encryption work? The data managed by DiskXtender on Atmos cloud can be protected from malicious access by encrypting data at source and then migrating it to the Atmos cloud. Atmos media service can be configured with a password to encrypt data. The password is used to generate a unique encryption key. During the migration process, the data is encrypted using the 32-byte key generated. However, the source data on the extended drive remains in an unencrypted form. The encrypted data on the cloud can be retrieved using DiskXtender by providing the password only. Therefore, the data is protected from unauthorized access. Data is encrypted using the Advanced Encryption Standard specification. The algorithm used is the symmetric-key algorithm where the same key is used for both encryption and decryption of data. EMC RSA libraries (RSA BSAFE Crypto - C Micro edition ) are used for encrypting the data. The key management mechanism used by DiskXtender is secure and cost effective. 12 to 16 characters length password is used to generate a 32-byte encryption key. The key is generated for each cloud media service and is stored in an external file named DxCrypt.log (Crypto file) in binary format under the bin folder. The Crypto file entries are guarded with cyclic redundancy check (CRC). It is recommended to backup the Crypto file during cloud media service creation. During disaster recovery, the availability of the Crypto file is critical to bring the cloud media service online. After the successful configuration of cloud media service (with encryption), a dialog box prompts you to provide a path to back up the Crypto file. An arbitrary string of characters is encrypted using the key generated for cloud media service. The cipher text generated for each cloud media service is stored in the cloud backend. In case the cloud media service is offline due to disaster or loss of the Crypto file, the media service can be set online by importing the valid Crypto file. When you try to reconfigure the media service with the password, the cipher text stored in the cloud backend is used to validate the entered password. The entered password is used to encrypt the arbitrary string of characters. The cipher text generated for the entered password is compared with the cipher text stored in the cloud backend. If the cipher texts are different, the media service is set to offline until the user enters a valid password. During service startup, the encryption keys are read from Crypto file and cloud media services are set online. These keys are used for encrypting and decrypting the data. In case of an unforeseen event, when the cloud media service is reconfigured, you are asked to enter the password. The entered password is validated against the earlier password. If there is a mismatch, you are asked to re-enter the password until the match is found. If you fail to enter the correct password, contact DiskXtender Support to access the older data and reconfigure it with a new password. This is because the cloud media service is configured with a password and the media service is confined to the password. Therefore, if the same media service is created on different DiskXtender installable, you must provide the same password to bring the media service online. Alternatively, you can import the Crypto file to bring the media service online. 146 EMC DiskXtender Microsoft Windows Version Administration Guide

139 Connecting to Cloud Storage Contact DiskXtender Support to change or reset the password. Troubleshooting EMC Atmos If you encounter unexpected errors, performance issues, or access problems between DiskXtender and EMC Atmos, you can enable a log that traces EMC Atmos events for DiskXtender. To enable the EMC Atmos trace log: 1. Ensure that there is sufficient space for the log file on the system drive of the DiskXtender server. The log increases in size by as much as 1 GB each day. 2. Use Windows Explorer to navigate to the DiskXtender bin directory, drive:\program Files\EMC\DiskXtender\Bin, where drive is the system drive. 3. Right-click and select New > Text Document. 4. Name the new text file Dx_AtmosService.log. 5. Click Yes on the pop-up message that warns you about changing file extensions. 6. Stop and restart the DiskXtender service. 7. After you finish troubleshooting and you no longer need the trace log, disable the log: a. Stop the DiskXtender service. b. Delete the Dx_AtmosService.log file. Configuring DiskXtender with EMC Atmos 147

140 Connecting to Cloud Storage Managing the lifecycle of cloud media Cloud media provides many petabytes of storage that can be distributed globally, but managed from one point. DiskXtender considers cloud media as a virtual media with unlimited storage because cloud media can be scaled as per requirements. The following topics provide details on creating and managing virtual cloud media throughout its lifecycle in DiskXtender. Understanding the cloud media lifecycle The following steps detail the lifecycle process for virtual cloud media: 1. Create the media and add it to the system: a. Create each piece of media for the media service. b. Allocate the media to the extended drive. c. Add the media to a media group. Cloud can also be added as one of the media in a multi-target media group. 2. (Optional) If you no longer need the files that have been migrated to the media, or if you want to move the files to a different piece or type of media, remove the media from the media group. There are two ways to remove media from a media group: To copy the files on the media back to the extended drive and move the files to other media, compact the media. The media is automatically removed from the media group during compaction. Removing media but keeping the files on the drive on page 189 provides instructions. To remove the files on the media from the extended drive altogether, remove the media from the media group. 3. (Optional) Deallocate the media from the extended drive, and delete it from the media service. Managing Atmos configuration Change shared secret Configuring cloud media service for encryption The Atmos connection parameters need to be updated in DiskXtender every time there is a change on the Atmos side. Whenever there is a change in the shared secret, the new shared secret must be updated in the cloud connection parameters, failing which, the connection to the cloud media cannot be established. You should receive the new shared secret periodically from the cloud administrator. You can enable data encryption for cloud during media service configuration. You are asked to enter and confirm the 8-character password. The password is encrypted and stored in the registry for future reference. Note: Data encryption is applicable only to the next set of files that are migrated to media under a media service. It does not apply to files that are already migrated without encryption enabled. The encryption feature can also be enabled by editing the properties of the media service. 148 EMC DiskXtender Microsoft Windows Version Administration Guide

141 Connecting to Cloud Storage Expectations for files migrated to cloud media You can perform most operations on the extended drive for files that have been migrated to cloud media. However, DiskXtender does not support retention on files moved to cloud media. Table 1 on page 149 lists the extended drive operations that can be executed on specific types of files or folders that have been migrated to cloud media. Table 1 Extended drive operations for files migrated to cloud media Edit Rename Set attributes Delete Move within a media folder Move outside of a media folder Files moved to cloud media Folders with files moved to cloud media N/A N/A N/A Considerations while naming files When you are naming files that will be migrated to Atmos, consider the following rules. The characters allowed in filenames are governed by both Atmos and HTTP URI rules: Filenames can contain any character in the printable ASCII character set except? and &. FIlenames can contain A-Z and a-z, 0-9, and / "-" / "." / "_" / "~" / "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "=" / ":". Expectations for files migrated to cloud media 149

142 Connecting to Cloud Storage Performance tuning for cloud media Review the following recommendations for improving DiskXtender performance with cloud media by adjusting the configuration of media folders, media groups, rules, and service options. Media service considerations for cloud media When configuring media services with multiple access nodes, it is recommended that the multiple nodes are geographically co-located. This is because, all the nodes participate equally while data is being migrated from DiskXtender to the cloud media; and physical distance between the nodes will hamper the performance. Atmos folder rename User can rename Atmos folder through the Microsoft Windows Explorer or command line. The following topics provide details Effects of Folder rename When media tasks such as compact, purge, fetch, prefetch, restore, or report is performed, the DiskXtender processes the folder rename transactions to restore files to the latest path. Note: Performance of Atmos media tasks may be affected as each of the media tasks involve processing the folder rename transactions log and restoring files to the latest folder. Renaming Atmos folder uses the Atmos and DiskXtender time for logging the transaction and any change to the Atmos time or the DiskXtender server time may result in files retrieved to a different folder when a media task is performed. How to enable Atmos folder rename To enable Atmos folder rename: 1. Right-click the extended drive in the DiskXtender Administration window. 2. Select Extended Drive Properties. The Extended Drive Properties dialog box appears. 3. Select Options page. 4. Scroll down and select Atmos folder rename option. 5. Select the Allow Atmos folder rename radio button. 6. Click OK. How to rename a Atmos folder 1. Open the Microsoft Windows Explorer and navigate to the Atmos folder on the extended drive. 2. Right-click the folder and select Rename. The folder name appears highlighted in the tree view. 150 EMC DiskXtender Microsoft Windows Version Administration Guide

143 Connecting to Cloud Storage 3. Type a new name for the folder, and then press Enter. 4. Open the command-line interface and change path to the Atmos folder on the extended drive. Use the following command to rename the folder: c:\atmosfolder> ren Atmosfolder1 Atmosfolder2 or c:\atmosfolder> rename Atmosfolder1 Atmosfolder2 5. When the Atmos media is full and you rename an Atmos folder, the rename transaction is successful on the extended drive as well as on the Atmos media. When multiple rename transactions are done on the extended drive and the user tries to fetch a file, there will be a delay in accessing the file. This is because, logging all rename transactions to the Atmos must be complete before fetching a file from the media. However user can change this behavior in the Extended drive option. Performance tuning for cloud media 151

144 Connecting to Cloud Storage 152 EMC DiskXtender Microsoft Windows Version Administration Guide

145 8 Extended Drive and Volume Mount Point Management An extended drive is an NTFS or ReFS volume whose storage capacity is extended by DiskXtender file migration services. A volume mount point is an unnamed drive mounted on the extended drive to help manage voluminous data for several clients. The following topics provide details on creating and managing a DiskXtender extended drive and volume mount point: Extended drive requirements Extended drive directory structure Support for volume mount point Background scans Virus scans Extended Drive and Volume Mount Point Management 153

146 Extended Drive and Volume Mount Point Management Extended drive requirements DiskXtender enables you to extend an NTFS or ReFS formatted hard drive by moving files to other storage media while making the files appear to remain on the drive. DiskXtender extended drives should meet the requirements listed in the following topics: General extended drive requirements on page 154 Changing the size of the extended drive on page 154 Extended drive data requirements on page 155 General extended drive requirements Each extended drive should be: Formatted as an NTFS or ReFS volume. Located on the DiskXtender server. This can either be a volume physically located on the DiskXtender server or a fibre-connected drive that has been mapped as a local drive. Not the system drive or a drive where applications are running. DiskXtender does not distinguish system or application files (like database binaries or application.dll files) unless you specifically configure the system to ignore these files. Avoid extending drives where system or application files reside to eliminate the possibility of accidentally purging or deleting these files. Dedicated to files that are managed by DiskXtender. DiskXtender monitors all file activity on the extended drive. Whenever files are saved to or changed on the extended drive, DiskXtender needs to identify and analyze them to determine whether to take action. System performance suffers if DiskXtender must monitor extra file activity on the drive in addition to its managed files. A minimum of 100 MB in size (10 GB or more is recommended), with sufficient space planned for future growth. The DiskXtender installation guide provides details on determining the appropriate size for the extended drive. A drive that contains (or will contain) no more than 100 million files. Use drive letters between C and Z. Drive letters A and B are not supported. Changing the size of the extended drive You can expand the size of the extended drive if the following conditions are true: The extended drive has been configured as a Dynamic Disk in Windows, and the drive is not part of a Microsoft clustering environment. The extended drive is a CLARiiON LUN, and you expand the drive by using either the concatenate or the stripe expansion method in EMC Navisphere. If the extended drive is part of a Microsoft cluster, use the Microsoft Cluster Administrator to set the DiskXtender service resource and the extended drive resource (the DiskXtender Disk resource, not the Physical Disk resource) offline before you expand the LUN. The extended drive is a Veritas Storage Foundation (VSF) 5.0 volume. Reducing the size of the extended drive is not supported in any of these scenarios. 154 EMC DiskXtender Microsoft Windows Version Administration Guide

147 Extended Drive and Volume Mount Point Management When configuring DiskXtender in these environments, the disk should be configured to enable future expansion before you add it as a DiskXtender extended drive in the File System Manager Administrator. For example, configure the disk as a Dynamic Disk in Windows before you add it as an extended drive. Extended drive data requirements DiskXtender supports a wide variety of file types. Because DiskXtender does not need to open a file in order to manage it, DiskXtender can manage most file types that can be saved on a Microsoft Windows NTFS or ReFS volume. However, DiskXtender is designed to serve as an archiving tool for fixed or unstructured data. As a result, adherence to the following guidelines ensures optimal system performance. Some of the most common file types used with DiskXtender include but are not limited to: Microsoft Office files (.doc,.ppt,.xls, and so on) Adobe Acrobat files (.pdf) Text files (.txt) HTML files (.htm or.html) XML files (.xml) ZIP archives (.zip) Image files, such as JPEGs (.jpg), TIFFs (.tif), bitmaps (.bmp), and GIFs (.gif) Macintosh files Personal Folder files (.pst) Note: The DiskXtender Search Module can index a majority of the common file types in this list. Supported file types for indexing on page 209 provides a complete list of the file types that can be indexed. Files that are frequently accessed or changed could result in the production of lock files or temporary files. A lock file is a file produced by an application to prevent write access when it is already open by another user. A large number of such files can clog the DiskXtender system or inadvertently fill the DiskXtender Recycler if it is enabled. Therefore, the following file and data types are not recommended for use with DiskXtender: Frequently accessed files, such as files or files in user home directories or temporary directories! IMPORTANT EMC SourceOne If you must manage these types of files with DiskXtender, then use a media type that allows renames and provides faster performance, such as NAS. Data that is part of a database Application files for programs installed on the extended drive Note: Installing applications on or running applications from the DiskXtender extended drive is not recommended. Extended drive requirements 155

148 Extended Drive and Volume Mount Point Management If the sixth character in a filename is a tilde and the eighth character is either null or a period (for example, filen~2.doc), then you can save the file to the extended drive, but DiskXtender does not move the file to media. This typically occurs in files with filenames that have been automatically shortened to an 8.3 format. Zipped folders and files that are managed by DiskXtender do not get purged, but are fetched back to the extended drive. This is seen when DiskXtender is installed on Microsoft Windows 2008, and is a behavior of files and folders zipped using Microsoft Windows Explorer. DiskXtender does not have control over this. Files managed by DiskXtender mapped to an extended drive on a Microsoft Windows 7 computer, do not get compressed into a folder when they are compressed using Windows Explorer. This is seen when DiskXtender is installed on Microsoft Windows Media folders, media names, media pool names, and file specifications in all the rules cannot contain these special characters:, + ; = [ ] However, files and folders with names containing the above characters are managed by DiskXtender without any exception. International character support Table 1 DiskXtender can manage files with Unicode characters in either the filename or the file data. However, characters may display incorrectly if the correct language code pages are not installed. In addition, the application that displays DiskXtender logs and reports (RtfPad) is not Unicode compliant. Filenames in logs and reports may display incorrectly, even though the file data remains intact. Some applications used with DiskXtender do not provide the same level of international character support. Table 1 on page 156 lists some of the supported applications with these limitations. Product interoperability and international character support Product AutoStart EMC NetWorker PowerSnap International character support Supports only ASCII characters. Supports file system backup if filenames and directory names are ASCII characters. Note: If you use the DiskXtender Search Module to index and search for files on a DiskXtender extended drive and the files include ANSI character encoding, such as Big5 (Chinese) or Shift-JIS (Japanese), then the corresponding code page must be installed and set as the default code page on the ISE server. If a different code page is set as the default on the ISE server, then perform the steps in the EMC DiskXtender Search Module Release 2.0 Release Notes to enable proper indexing and search of the ANSI-encoded files. Verify international character support statements for all applications installed on the DiskXtender server before you configure file migration. Path and filename length limitations The full path and filename for a file on the extended drive should total no more than 259 UTF-16 characters. The full path and filename includes the following components: Drive:\Directory1\Directory2\Filename.Extension 156 EMC DiskXtender Microsoft Windows Version Administration Guide

149 Extended Drive and Volume Mount Point Management where Drive is the assigned volume drive letter, Directory1 and Directory2 are optional folders on the drive, Filename is the name of the file, and Extension is the file extension. Note: Japanese path and filenames are limited to 145 characters or less. DiskXtender does not prevent users from saving a file to the extended drive when the character count for the path and filename exceeds 259 characters. However, files with excessive paths and filenames may not be migrated to media, depending on the type of media and the file system with which it is formatted. In addition, if the filepath for a file on the extended drive exceeds 259 characters and the DiskXtender Recycler is enabled, then you cannot delete the file. You must shorten the filepath or disable the Recycler in order to delete the file. Most types of media and media file systems support at least the same number of characters as DiskXtender. However, when writing to media that is formatted with the UDF file system, DiskXtender supports a shorter path and filename: only 127 characters. Most NAS file systems support filepaths of 259 UTF-16 characters. However, if DiskXtender is writing to a share on the NAS device, the character count of the full filepath on the device is included in the maximum number of characters allowed. In other words, the total character count includes the path and filename on the extended drive and the full path to the share on the device. Note: If the sixth character in a filename is a tilde and the eighth character is either null or a period (for example, filen~2.doc), then DiskXtender does not move the file to media. This typically occurs in files with filenames that have been automatically shortened to an 8.3 format. File stream support File streams contain the data that is written to a file and give more information about a file than attributes and properties. For example, you can create a stream that contains search keywords or the identity of the user account that creates a file. The NTFS file system uses file streams to store private data. Because NTFS supports file streams, many applications now take advantage of file streams to store their data. DiskXtender can manage files with file streams as long as the media supports file streams. All DiskXtender media types support file streams except for some NAS devices. In general, file streams are supported on NAS devices that use Common Internet File System (CIFS) or Server Message Block 2.0 (SMB). If a NAS device uses a different protocol, consult the manufacturer for guidance on file stream support. DiskXtender support for file streams ensures that you can protect all application data in files, not just the primary data. File streams are moved and fetched along with the primary file data. In addition, file streams can be restored from storage media, along with primary file data, for disaster recovery purposes. Note: You cannot directly read files with streams from media by using the Direct Read feature in DiskXtender. You can set the Direct Read attribute for these files. However, when the file is requested, it is fetched to the extended drive instead of being read directly from the media. In addition, the Direct Read attribute is removed. If a NAS device configured with DiskXtender does not support file streams, then the file is not moved to media. In addition, a warning message is listed in the event logs to notify you that the file could not be moved to media. Extended drive requirements 157

150 Extended Drive and Volume Mount Point Management Encryption Microsoft Windows offline files DiskXtender can manage encrypted files on the extended drive as long as the files are not encrypted with an on-the-fly encryption (OTFE) method, also called real-time encryption. This type of encryption prevents DiskXtender specifically the DiskXtender service account from gaining access to the files to manage them. DiskXtender supports the offline files feature available in several Windows operating systems. When users access the extended drive, they can configure files or folders on the extended drive as offline files. This enables users to access the files when their machines are not connected to the network. When the machines reconnect to the network, any changes that were made to the files are updated on the extended drive. When you configure a purged file as an offline file, the file is fetched. DiskXtender continues to migrate, fetch, index, purge, and otherwise manage files that are configured as offline files. The files are updated on media and on the ISE server, if necessary, when they are synchronized back to the extended drive and are no longer offline. Note: When you rename an offline file and the file contains file streams, then the file streams are lost when you reconnect to the network and update the file on the extended drive. This issue occurs regardless of whether DiskXtender is installed on the server. Microsoft Windows DFS links DiskXtender is supported in a Microsoft Windows Distributed File System (DFS) environment. However, DiskXtender manages only the files that are physically located on the server where it is installed. It does not manage files on other servers that are connected to it by DFS links. For example, the following configurations are supported: The DFS link is on a server where DiskXtender is not installed (Server A), and the link points to an extended drive on a server where DiskXtender is installed (Server B). Users can use the DFS link on Server A to access the files on Server B, even if the files have been purged. DiskXtender manages only the files on Server B. The DFS link is on an extended drive on a DiskXtender server (Server A), and the link points to another server (Server B). Users can use the DFS link on Server A to access the files on Server B. The DiskXtender installation on Server A manages all of the files that are saved locally to its extended drive. However, it does not manage the files on Server B. DFS is supported only in DFS non-replication mode. Attempting to use DiskXtender with DFS replication enabled might cause data loss. 158 EMC DiskXtender Microsoft Windows Version Administration Guide

151 Extended Drive and Volume Mount Point Management Extended drive directory structure You can configure DiskXtender to move all of the files on the extended drive to media and manage those files. Or, you can configure DiskXtender to move and manage only the files in certain directories. To select the directories on the extended drive that DiskXtender should manage, you configure media folders. You can create a media folder in DiskXtender for a directory that already exists on the extended drive, or you can create a new directory on the drive during the media folder creation process. Either way, media folders correspond to physical directories on the drive, and can be viewed as such through Microsoft Windows Explorer. The following topics provide details on planning the directory structure and managing media folders. Planning the directory structure You can create a single media folder on the extended drive, or you can create multiple media folders. If you create a single media folder, the media folder can either be at the root of the drive or it can be a single directory that DiskXtender should manage on the drive. Creating multiple media folders enables you to segregate the data on both the extended drive and the storage media. Individual pieces of media are assigned to media groups in the media folder. By organizing the files into the media folders, you can control the media to which files are written. You can even create a media folder for the subdirectory of another media folder to more precisely control how files are moved to media and otherwise managed. When a file is written to a piece of media, the directory structure on the media mirrors the extended drive directory structure. The root media folder, however, does not appear on the storage media. For example, if the Reports media folder is located at the root of extended drive E, then E:\Reports\Accounts.doc on the extended drive is stored as Accounts.doc on the media. Regardless of the number of media folders you create, limit the number of files on the extended drive that are outside of media folders. DiskXtender monitors all file activity on the extended drive, both inside and outside of media folders. Whenever files are saved to or changed on the extended drive, DiskXtender needs to identify and analyze them to determine whether to take action. System performance suffers if DiskXtender must monitor extra file activity on the drive in addition to its managed files. Keep in mind the following guidelines when planning the directory structure for the extended drive: If you use a type of media that does not allow folder renames, such as removable media formatted with the OTG file system, then use extra caution when planning the directory structure. After files are moved to media, you will not be able to change the directory structure. Do not use lengthy directory names, and minimize the depth of the directory hierarchy. The full path and filename of a file on the extended drive should total no more than 259 UTF-16 characters. Lengthy directory names can cause files to exceed the character count that DiskXtender supports for managing the files. Path and filename length limitations on page 156 provides details. Extended drive directory structure 159

152 Extended Drive and Volume Mount Point Management Setting media folder priority If you configure overlapping media folders, then order the folders to set the priority in which the files in the folders are evaluated against the rules assigned to each folder. Overlapping media folders occur if you identify both a directory and one of its subdirectories as separate media folders. If a file is located in overlapping media folders, the file is evaluated for migration, purge, deletion, or indexing based on the rules assigned to the media folder listed first in the tree view of the Administrator. If the file fails to qualify for action based on the rules assigned to the first of the overlapping media folders, the file is evaluated against the rules in the second media folder. If the file is excluded from action based on a rule in the first of the overlapping media folders, then the exclusion applies, even if the file qualifies for action based on a rule in the second media folder. You must also set the priority of the rules when you create them. Therefore, it is a combination of the order of the media folders and the order of the move rules that determines whether a file is migrated and to what media. Note: If you develop a complex file management strategy involving overlapping media folders and rules, use the Rule Evaluation Tool, available on the right-click menu for the rules, to verify how and whether a file will qualify for applicable rules. To edit the priority for a media folder, right-click the folder and select either Promote or Demote. 160 EMC DiskXtender Microsoft Windows Version Administration Guide

153 Extended Drive and Volume Mount Point Management Support for volume mount point When data grows enormously, extended drive limitation on the number of files it supports becomes a limitation especially when you have large numbers of files that run into the billions. This results in increased time for backup and background scan. In order to overcome this limitation, you can add volumes to systems without adding separate drive letters for each new volume. Volume mount points are robust against system changes that occur when devices are added or removed from a computer. A volume mount point can be placed in any empty folder of the host NTFS or ReFS volume. DiskXtender supports the use of a volume mount point as the extended drive. DiskXtender can fetch, move, delete, or purge files from a mount point. Benefits of using volume mount point Some of the advantages of supporting volume mount points are: Any number of windows partitions can be extended (limited only by the disk size). Allows better management of large data Demands of large number of clients can be managed easily Mount points recognized by DiskXtender DiskXtender recognizes the following volume mount points: Volume mount points created under a media folder path Volume mount points created for an unnamed NTFS or ReFS partition Volume mount points created prior to DiskXtender installation is recognized after the first full background scan is run for data archiving. Volume mount points created during DxDmService service execution is recognized instantaneously for data archiving. Volume mount points added as a Disk Resource to a cluster group, in case of cluster Note: On Windows 2012 systems, it is recommended to keep the same file system volume mounts under the extended drive. For instance, it is not advisable to create a NTFS volume mount point on a ReFS extended drive. Migration policy The migration policy that exists for the extended drive also applies to the volume mount point that is mounted. Unmounting of a volume mount point DiskXtender does not automatically recall purged files when you unmount a volume point. You must fetch these files manually before performing the unmount. Support for volume mount point 161

154 Extended Drive and Volume Mount Point Management Remounting of volume mount point DiskXtender does not support remounting of a volume mount point to a different folder. For instance, consider the following mount point: X:\Media\Mount_Folder1 where Mount_Folder1 is a mount point for an unnamed partition. After the initial data migration from Mount_Folder1, you cannot remount it to a different folder such as X:\Media\Mount_Folder2. Such a remount operation is not allowed. Delete operations on volume mount point A delete operation on the mount point uses the Recycler of the extended drive on which the volume point is mounted. Therefore, you can view and restore deleted files from the extended drive's Recycler. Note: If the extended drive is deleted, it will also result in the deletion of the DxLog files in the volume mount point folder. 162 EMC DiskXtender Microsoft Windows Version Administration Guide

155 Extended Drive and Volume Mount Point Management Background scans Background scans enable DiskXtender to verify that all files on the extended drive that need to be managed are being managed. There are two types of background scans - Full Scan and Incremental Scan. Full Scan Incremental Scan During a full scan, DiskXtender scans the entire extended volume to validate files for migration. The validations are done against the migration policies set by the user. During an incremental scan, DiskXtender scans only those folders that contain modified files since the previous scan and files that qualify. If for any reason, an incremental scan does not suffice, DiskXtender automatically resorts to a full scan. When only a full scan is run In the following scenarios, DiskXtender falls back to a full scan even if the user has chosen to perform an incremental scan: Rules or policies are added, modified or deleted The default exclude rule has changed The expiration policy in a multitarget group has changed Rules or policies are promoted or demoted A media is added or removed from a media group A media group is added or deleted In general, DiskXtender runs an incremental scan if there are no changes to rules or policies. Note: It is recommended that a full scan be scheduled to run periodically. The following topics provide details on when and how background scans occur, as well as the settings available to customize a background scan. What happens during a background scan During a background scan, DiskXtender scans the media folders on the extended drive, comparing the files in the media folders to the configured items set up through the File System Manager Administrator, such as move, purge, and delete rules. Background scans then perform the following general tasks: Write files that qualify for move rules to the move list, including files that were not added to the move list when appropriate due to file sharing issues and files that qualify for migration based on configured age delays Purge files that qualify for purge rules with the Force files to purge during background scans option enabled Write files to the purge list, if the files qualify for purge rules with the Do not force purges during background scans option enabled Background scans 163

156 Extended Drive and Volume Mount Point Management Delete files that qualify for delete rules Write files that qualify for index rules to the Index Transaction Log (ITL) Synchronize files on media in multi-target migration media groups Fetch files that are purged but that qualify for index rules and require indexing Remove media from media groups Updates EMC Centera folder names When background scans occur By default, background scans run daily, starting at midnight. The background scan continues until it has evaluated each file on the drive. The only system tasks that halt a background scan (or prevent a background scan from starting) are: Unavailability of at least 10 MB free space in the extended drive The stopping of the DiskXtender service Failover in a cluster environment or AutoStart domain Addition, editing, or deletion of a rule or media folder Addition or deletion of a media group Media compaction Initiation of a command to restore files from a piece of media to the extended drive Metadata export and import If a scan is interrupted or fails to start when it is scheduled, DiskXtender attempts to start another scan at the next scheduled time. Background scans are designed to run in the background so that you can continue to manage the system while they take place. Files are still accessible on the extended drive and on media, and you can continue to use the File System Manager Administrator to monitor and configure the system. However, background scans do consume processor resources, which may affect the performance of other processes. If necessary, you can adjust the speed of the background scan, altering the processor resources needed to run the scan. Risks of disabling background scans The most important function of the background scan is to provide DiskXtender with a comprehensive snapshot of all of the files on the extended drive, and of the system as a whole. Because the data in the system changes constantly, background scans are configured to run automatically. This provides DiskXtender with consistently updated information. DiskXtender allows you to disable background scans, but you should disable the scans only if you have specific reason to do so. Disabling background scans leaves the system vulnerable to a number of risks, including: Files may not be moved to media: 164 EMC DiskXtender Microsoft Windows Version Administration Guide

157 Extended Drive and Volume Mount Point Management File sharing issues or sharing violations can prevent files from being added to the move list (and therefore being moved to media) when appropriate. DiskXtender must have full access to a file to obtain the information required for the move list. If the file is open or otherwise being accessed by a program or user, DiskXtender cannot add it to the move list at that time. Background scans add these files to the move list when they become available. If you configure move rules with an age delay, files are qualified against these rules, and are therefore only written to the move list in preparation for migration, only during a background scan. Files may not be indexed. If you configure index rules with an age delay, files are qualified against these rules, and are therefore only written to the Index Transaction Log (ITL) in preparation for indexing, only during a background scan. Files are not qualified against purge rules or delete rules. DiskXtender evaluates files against these rules only during background scans. Files are not qualified against new or edited rules. If there are files already on the drive when you create or edit any rules, the files are not qualified against the rules until the next background scan runs. Tuning background scan speed You can adjust the speed of the background scan, altering the processor resources needed to run the scan. Slower scans use less resources, while faster scans use more. Running an advanced scan for troubleshooting For troubleshooting purposes, you can set advanced background scan configuration options to limit the tasks that a background scan performs. The advanced scan configuration options should be changed only in extraordinary situations. If configured incorrectly, key system functionality might be diminished or disabled. Background scans 165

158 Extended Drive and Volume Mount Point Management By default, background scans attempt to perform all of the listed tasks in Table 2 on page 166. Table 2 Background scan tasks Task checkbox Perform move rule evaluation Perform index rule evaluation Perform synchronization fetching Description Qualifies files against move rules. Qualifies files against index rules (when content indexing is enabled). Fetches files for either multi-target migration synchronization or content indexing (when content indexing is enabled): Multi-target migration sync-fetch If you add a media group to a multi-target media group that already contains other media groups, the purged files on the media in the other media groups are fetched to the extended drive and then migrated to the new group. Content indexing sync-fetch In most cases, files that qualify for index rules are indexed before they are purged. However, if a qualifying file is purged first, then the file must be fetched to the extended drive so that it can be indexed. Forcing a standard background scan If necessary, you can force a background scan to occur by using the default settings for all background scans. When a forced background scan may be necessary You might want to force a new background scan cycle to begin in the following situations: If you are planning any DiskXtender system maintenance, forcing a background scan and activating the file migration schedule is an excellent way to be sure all appropriate files are written to media before the maintenance is done. If you change any rule configurations (move, purge, delete, or index), these changes do not apply to existing files until a scan is run. Forcing a background scan will speed this process along. If you find you are unexpectedly running low on extended drive space, forcing a background scan and activating the file migration schedule also ensures that all appropriate files are written to media, and can be purged by the system (based on configured move and purge rules). Forcing a special background scan If necessary, you can now force a background scan to perform special maintenance tasks. Aside from the additional maintenance tasks, a special scan functions as any other scan would. When a special background scan is initiated, a full scan is performed. 166 EMC DiskXtender Microsoft Windows Version Administration Guide

159 Extended Drive and Volume Mount Point Management The special scan maintenance options are listed in Table 3 on page 167. Table 3 Scan maintenance options Option Build new sync-fetch list (truncate existing) Strip direct-read attribute from all files Strip indexed attribute from all files Description Creates a new sync-fetch list and deletes the existing sync-fetch list. The sync-fetch list contains a list of all purged files that must be fetched back to the extended drive so that they can either be indexed or synchronized between media groups in a multi-target media group. Removes the Direct Read attribute from any files managed by DiskXtender. Removes the Indexed attribute from any files managed by DiskXtender. The Indexed attribute identifies files that have qualified for index rules and have been full-text indexed. Note: This option should be used only if you no longer use the DiskXtender Search Module. Otherwise, all previously indexed files will be subject to re-indexing. Strip not indexed attribute from all files Recalculate licensed capacity usage Removes the Not Indexed attribute from any files managed by DiskXtender. The Not Indexed attribute identifies files that qualified for an index rule but could not be full-text indexed for example, because the ISE does not support a specific file type. This option is useful if you want to re-qualify files in the media folder against index rules for example, if the DiskXtender Search Module adds support for a certain file type. Recalculates the total license capacity used by all files managed by DiskXtender on the extended drive. The updated value is reflected in both DiskXtender and License Server. This option is used when the value of the total licensed capacity usage is known to be incorrect. The options available on the Scan Maintenance Options dialog box expose functionality that should be used only in extraordinary situations. Selection of one or more of these options can result in permanent, system-wide changes. Note: These settings apply only to this special scan. To apply these settings to any future scans, configure the options each time and force a special scan. Viewing background scan statistics You can view statistics relating to the current activity for a current background scan, as well as the statistics from the most recently completed scan: The Activity tab of the Extended Drive Background Scan dialog box provides statistics for the current scan. The Statistics tab of the Extended Drive Background Scan dialog box provides statistics for the most recently completed scan. The EMC DiskXtender for Windows Online Help provides details on the statistics listed on each tab. Background scans 167

160 Extended Drive and Volume Mount Point Management Virus scans Anti-virus protection is critical to any system. However, when you use an anti-virus program to scan files on a DiskXtender extended drive, review and follow the guidelines in the following topics to maximize DiskXtender performance by preventing unnecessary moves and fetches. Configuring virus scans of the extended drive Confirm that the anti-virus software is certified for use with DiskXtender. The EMC DiskXtender Software Compatibility Guide provides details and can be found on temc Online Support. Install the virus scan client on the DiskXtender server. Do not allow a user machine that is mapped to the DiskXtender server to run virus scans on the extended drive. In addition, minimize the impact of the virus scan on DiskXtender performance by using one of the following virus scan configuration strategies: Disable any real-time virus scan options. Instead, schedule periodic virus scans. The scheduled scans should not run concurrently with DiskXtender background scans. Instead, they should occur during times of low system activity. or If the anti-virus software allows it, add the DxDmService.exe file to the exclude list for virus scans. (DxDmService.exe is the program executable for the DiskXtender service.) In addition, configure the scan for inbound files only. In other words, only scan files that are written to the extended drive, not those that are read. The documentation for the anti-virus software should provide instructions for configuring these options. Configuring DiskXtender for virus scans The virus scan software should be included in the special application list in DiskXtender. When an application is on the list, DiskXtender can control whether purged files are directly read from media when accessed by the application, or if the application skips the purged files. In the case of anti-virus software, you might not want the anti-virus software to recall purged file data from media during a virus scan. Allowing DiskXtender to recall the file data would unnecessarily fetch purged files from media for virus scanning. This is not necessary if the files have been scanned prior to being moved and purged. Fetching files during a virus scan could significantly decrease DiskXtender performance. The special application list only applies to programs that run on the DiskXtender server. Because of this, user machines that are mapped to the DiskXtender server should not be permitted to run virus scans on the extended drive. Virus scans that are run from a user machine to a mapped extended drive will more than likely fetch purged files for scanning, defeating the purpose of the special application filtering option. 168 EMC DiskXtender Microsoft Windows Version Administration Guide

161 Extended Drive and Volume Mount Point Management Virus scan applications that are already filtered By default, the Special Applications List contains executables for the following anti-virus software: AVX Anti-Virus Dr. Solomon s Anti-Virus F-Secure Anti-Virus McAfee Anti-Virus Norton Anti-Virus Trend Micro Anti-Virus Symantec Anti-Virus Panda Anti-Virus Note: Not all anti-virus software is qualified for use with DiskXtender. The EMC DiskXtender Software Compatibility Guide provides details. DiskXtender prevents the executables for each of these programs from fetching purged files to the extended drive during a virus scan. Virus scans 169

162 Extended Drive and Volume Mount Point Management 170 EMC DiskXtender Microsoft Windows Version Administration Guide

163 9 File Migration The following topics discuss when and how to move files from the extended drive to storage media: Understanding the file migration process Developing a file migration strategy Enabling file migration File Migration 171

164 File Migration Understanding the file migration process The term file migration refers to the function of moving files from a local hard drive to one or more pieces of storage media. In DiskXtender, the migration of a file is actually a copy. When DiskXtender moves a file to media, it is really copying the file data to media and adding extended attribute information to the file on the drive. The file is then managed by DiskXtender, and the data resides both on the extended drive and on the media. The following topics discuss the file migration process. Prerequisite steps for file migration To enable file migration through DiskXtender, you must prepare the media to which files will be migrated, prepare the extended drive, and configure migration rules and schedules. To set up file migration: 1. Create a media service for the storage media to which files will be migrated, and add media to the media service. The following chapters provide details: Chapter 3, Connecting to EMC Centera Chapter 4, Connecting to Network-Attached Storage Chapter 5, Connecting to Optical and Tape Devices Chapter 6, Connecting to Tivoli Storage Manager Chapter 7, Connecting to Cloud Storage 2. Create the extended drive and the media folders on the drive. Chapter 8, Extended Drive and Volume Mount Point Management, provides details. 3. Create one or more media groups in each media folder. Media groups are specific groupings or pools of media within a media folder. 4. Allocate media from the media service to the extended drive. 5. Add the allocated media to the media groups. 6. Create move rules to specify: Which files should be moved to media Which files should not be moved to media Which media group the files should be moved to Whether to apply retention to the files so that the files cannot be edited or deleted for a certain period of time Note: Retention is available only when migrating files to EMC Centera or to retention-capable NAS devices. Chapter 10, File Retention, provides details. 7. If necessary, adjust the file migration schedule. By default, file migration takes place between 8 P.M. and 9 A.M. 172 EMC DiskXtender Microsoft Windows Version Administration Guide

165 File Migration How files qualify for file migration Files are qualified against move rules at the following times: When the files are saved to the extended drive in other words, when they are added, edited, renamed, or moved During a background scan DiskXtender checks the path length to the files in the extended drive. The absolute path length should be within 1024 Unicode characters of UTF-8 encoding with variable width upto double bytes. DiskXtender looks for files that did not qualify for a move rule when saved to the extended drive. For example: The file was already on the extended drive when the move rule was created. The move rules in the media folder may be configured with an age delay, so that files qualify only after they reach a certain age. You can also force all files to be evaluated against move rules during a background scan, regardless of move rule settings or when the files are saved to the extended drive. Disabling real-time rule qualification can improve performance on busy systems where a large number of files are saved to the extended drive in a short period of time. System resources can then be freed to service other requests, such as file fetches. To change when files are qualified against move rules, select Disable real-time moves for the Enable real-time moves option on the Options tab of the Service Properties dialog box in the File System Manager Administrator, as shown in Figure 1 on page 173. Be aware that the Enable real-time moves option applies to both move rules and index rules. Figure 1 Disabling real-time moves When a file meets the rule criteria for migration, the file is written to the move list for the media group. The move list contains a list of: New files that need to be migrated to media Edited files that need to be remigrated to media Understanding the file migration process 173

166 File Migration Even if a file meets the criteria for more than one move rule, the file is written to the move list based on only one of the move rules the one that is listed first in the File System Manager Administrator. DiskXtender, by default, excludes migrating the following files to the media during a file migration activity: Files of zero bytes size Windows system files (files whose system attribute is set) Files with extension.pst Executable files with extension.exe Desktop configuration file named desktop.ini These rules can be modified when Advanced mode is enabled. Note: For extended drives created by using previous versions of DiskXtender, these default exclude rules are disabled by default. However, they can be enabled. How and when files are actually migrated When the file migration schedule begins, DiskXtender begins processing the move list for each media group. The files on the move list are checked again to ensure that they still qualify for configured move rules. If they still qualify, they are written to storage media. Note: If the file migration schedule is active when the file qualifies for a move rule, the file is written to media as soon as the target media becomes available. In other words, the file migration process may occur quickly after the file is saved to the extended drive, depending on the system configuration and the availability of the media. The piece of media a file is written to is determined by the move rule for which the file qualifies. Each move rule must identify a target media group for the qualifying files. Files are written to the first available piece of media that belongs to the media group. Extended attribute information is added to files that are successfully migrated to media. This information includes details about the media to which the file was written. To determine whether a file has been migrated, use the DiskXtender Explorer Add-ons to view the properties for the file or to run a file report for the file. Explorer Add-ons on page 250 provides details on performing these tasks. Monitoring file migration You can use the Activity Monitor utility to see the status of multi-target synchronization. This is a command line utility that will display the number of files synchronized (migrated to all targets) and files to be synchronized of each media in a move group. This number is based on the last full background scan. For more details about the utility, see the Utility Guide. 174 EMC DiskXtender Microsoft Windows Version Administration Guide

167 File Migration Developing a file migration strategy The flexible file migration features in DiskXtender are designed to accommodate both simple and complex migration strategies. A simple migration strategy may involve migrating all files on the extended drive to a single back-end storage device. A more complex strategy may involve migrating different types of files, or files in different locations, to different types or even multiple pieces of media. DiskXtender file migration policies enable you to configure, very specifically, which files are migrated to which media. The following topics provide details. Choosing the files to migrate DiskXtender move rules enable you to target, very specifically, which files to migrate to media, and also which files should not be moved to media. The following topics provide details on developing a strategy for the move rules. Available criteria for move rules You can select files for migration or exclusion from migration by using the criteria in Table 1 on page 175. Table 1 Available criteria for selecting files to move Criteria Filename File size File age File attributes Details You can specify a particular filename, which may be useful for excluding a file from migration. You can also use the asterisk (*) symbol as a wildcard to substitute for one or more characters in a filename. For example: Specify *.* to target all files in a media folder. Specify *.doc to target all files with a.doc extension. Specify Payroll* to target all files with a filename that begins with the word Payroll, regardless of the file s extension. Specify *.tmp to target temporary files for exclusion from migration. You can target files smaller than a certain size in KB, larger than a certain size in KB, or within a range of sizes in KB. Age criteria enable you to specify the number of days that must pass since the file was created, last edited, or last accessed before DiskXtender migrates the file. You can target certain files with the Read-only, Archive, Compressed, Hidden, or System attributes. File attribute criteria are most effectively used to exclude files with certain attributes, such as the System attribute, from being moved. Simplifying move rules by grouping files in media folders If possible, group files into media folders based on the migration rule that should apply to the files. This enables you to simplify the rules for each media folder. For example, save all files that should be migrated to one type of media in one media folder, and all files that should be migrated to another type of media in a different media folder. Or, save all files that should be migrated immediately in one media folder, and all files that should be migrated after 30 days in a different media folder. Delaying file migration until files are finalized Specify an age delay on move rules to prevent DiskXtender from moving files until they are finalized and will no longer be edited. This may be especially useful with optical media because of the way files are written to optical media. It limits the Developing a file migration strategy 175

168 File Migration number of times frequently changing files are written and rewritten to the media. To specify an age delay, select the Apply rule to files of age greater than option on the Age page for the move rule, specify the number of days (typically 30 or 60), and then select Last write time from the drop-down list. Excluding unnecessary files from migration Create move rules to exclude any files that you do not want migrated to media. For example, never extend a system drive or a drive that contains application files. However, if there are system or application files on the extended drive, be sure to configure an exclude rule that excludes all files with the System attribute. This prevents the application and system files from being moved and subsequently purged. Rules that exclude files from migration are called exclusive move rules. To create an exclusive move rule, select the Exclude option on the Type page when creating the rule. Note: DiskXtender, by default, excludes migrating the following files to the media during file migration, if you have set the default exclude rule: - Files of zero bytes size - Windows system files (files whose system attribute is set) - Files with extension.pst - Executable files with extension.exe - Desktop configuration file named desktop.ini Prioritizing overlapping move rules Even if a file meets the criteria for more than one move rule, the file is migrated or excluded from migration based on only one of the move rules the one that is listed first in the File System Manager Administrator. For example, if you create a move rule that moves all.doc files (*.doc) to a NAS media group and another move rule that moves all files with the word Report in the filename (*Report*) to an EMC Centera media, then a file named Report.doc would technically meet the criteria for both rules. However, Report.doc would be moved only to the NAS media group. This is because it would qualify for the *.doc move rule, which would be listed first in the Administrator. After a file meets the criteria for one move rule, DiskXtender does not qualify files against additional move rules. In addition, even if two move rules are configured with the same file criteria but point to different media groups, files are written only to the media group specified in the move rule listed first in the File System Administrator. To move a file to multiple pieces of media at the same time, use a multi-target migration strategy. Multi-target migration on page 178 provides details. As a result, if you create multiple move rules in a media folder, you must order the rules to define how files are evaluated against the rules. To edit the priority for a move rule, right-click the rule and select either Promote or Demote. Keep in mind the following guidelines: If a file qualifies for two different move rules, the file is moved based on the rule that is listed first in the tree view of the Administrator. If the file fails to qualify for migration based on the first rule, the file is evaluated against the second rule. If the file is excluded from migration based on the first rule, then the file is not migrated, even if it qualifies based on the second rule. 176 EMC DiskXtender Microsoft Windows Version Administration Guide

169 File Migration If the file qualifies for migration based on the first rule, then the file is migrated, even if it is excluded from migration based on the second rule. As a result of the last two guidelines, exclusive rules and rules with more specific file criteria should generally appear first in the list of rules. This enables exclusions and special conditions to take effect, while more general criteria in the later rules in the list can capture any remaining files. Choosing the media for the files DiskXtender media groups enable you to select the media to which files in a media folder will be migrated. When you configure the rules that select files for migration, you also specify the media groups that will receive the files. Each move rule can target only one media group. If you create multiple move rules, you can create multiple media groups to receive the files that qualify for the move rules. However, a media group is not exclusively owned by a move rule. In other words, you can configure multiple move rules that all point to the same media group. Multiple media groups in each media folder are necessary only if there are multiple types of media or if it is necessary to segregate files on different pieces of media. For example, you may want to write frequently accessed files to a faster media type, such as NAS, and infrequently accessed files to a slower media type, such as tape. Or, you may want to write certain file types to a retention-capable media type, such as EMC Centera, while other file types do not require retention and can be written to NAS. The way that you configure the media groups determines how files are written to the media in the groups. The following topics provide details. Understanding the three file migration models You can migrate files to media through DiskXtender with one of three file migration models. These migration models are configured primarily through DiskXtender media groups. Standard migration With the standard migration model in DiskXtender, files that are written to a media folder on the extended drive are qualified against the move rules configured for the media folder. If the file qualifies for migration, the file is written to the move list for the media group to which the move rule points. (The media group contains only one type of media and is called a standard media group. The standard media group belongs to the media folder in which the file is located.) When the file migration schedule is active, the file is migrated to a single piece of media in the media group, as illustrated in Figure 2 on page 178. Developing a file migration strategy 177

170 File Migration Extended drive on DiskXtender server File Media GEN Figure 2 Standard migration Multi-target migration With multi-target migration, each file is migrated from the extended drive to multiple targets (pieces, and even different types of media), as illustrated in Figure 3 on page 178. Tape Extended drive on DiskXtender server File Disk array GEN Figure 3 Multi-target migration Multi-target migration provides improved data availability and reliability. In the event that one of the pieces of media is not available, a file can be retrieved from another piece of media in a different target. Multi-target migration is achieved through configuration of multi-target media groups. A multi-target media group is designed solely to contain media targets, which consist of standard media groups that contain the media to which to write files, as shown in Figure 4 on page EMC DiskXtender Microsoft Windows Version Administration Guide

171 File Migration Multi-target media group Standard media group with tape Standard media group with NAS Standard media group with DVD Standard media group with EMC GEN Figure 4 Multi-target media group A multi-target media group can contain as many as four standard media groups. When you create move rules for a multi-target environment, you select a multi-target media group to receive the qualifying files not the standard media groups within the multi-target group. When a file qualifies for movement to media based on move rules associated with a multi-target group, the file is moved to a piece of media in each target. Because the capacity and availability of the different types of media in each group are likely to vary, a piece of media in one target group is not likely to contain the same files as a piece of media in another target group. In other words, the media in target groups are not copies or duplicates of each other. Figure 5 on page 180 illustrates a scenario where File A and File B are migrated to the same piece of media in one media group, but to different pieces of media in another media group. Developing a file migration strategy 179

172 File Migration Standard media group NAS share 1 File A NAS share 2 Standard media group Tape 1 File B Tape 2 GEN Figure 5 Sample file migration pattern for multi-target migration A scenario such as the one illustrated in Figure 5 on page 180 is possible because different types of media have different maximum capacities. For example, the maximum capacity of a piece of tape media is likely to be smaller than the maximum capacity of a share on a NAS device, which is limited only by the size of the partition on which the share resides. As a result, File A and File B may be written to the same NAS share. However, File A may be the last file written to Tape 1 because it fills the tape, so File B would be written to Tape 2. Multi-target migration and media transactions The migration to media in a multi-target group actually takes place as the media in each target group becomes available, meaning that there might be some delay in a file being written to all targets. Similarly, if all of the media types in the multi-target group support file deletions, then a file that is deleted from the extended drive is deleted from the media as the media becomes available. 180 EMC DiskXtender Microsoft Windows Version Administration Guide

173 File Migration Expectations for files on media in multi-target media groups The most restrictive media type restrictions of all media types in the multi-target group apply to the files that have been written to media in the multi-target group. For example, if one target group contains a type of media that does not allow you to rename or delete files, you will not be able to rename or delete files in the folder, even if another group in the multi-target group does allow those transactions. Multi-target migration and purged files If a file qualifies for purging, the file is not purged until it has been moved to all targets within a multi-target group. If a user requests a file that has been purged, the file is fetched from the first target media group listed under the multi-target group. You can change the media priority from which DiskXtender will fetch the file by changing the order in which the standard media groups appear under the multi-target group. Multi-target migration and EMC Centera If the media in two different target media groups are two EMC Centera clusters, do not use EMC Centera replication. If you use both replication and multi-target migration, the CDF for each file may be stored on each EMC Centera cluster twice. If single-instance storage is disabled, then the file data is also stored on each cluster twice. In addition, if the media in two different target media groups are two EMC Centera clusters, then you cannot apply retention classes to files that are written to the media in the multi-target group. This restriction is intended to protect against the scenario where the retention class on one EMC Centera is applied to files that have been migrated to a different EMC Centera. Expiration of files from target media groups To delete files from one target media group to reclaim space, but leave the files on other target media groups to ensure continued access to them, you can configure an expiration policy for the target group. You can delete files from media in a target media group based on the age of the file since it was created, last written, or last accessed. For example, you may want to migrate files to both NAS and tape media. While the files are still frequently accessed, they can be fetched from the faster NAS media. However, once the files reach a certain age, they can be expired (deleted) from the NAS media and left only on the tape media for retrieval. The Automatically expire files setting on the Options page for the media group enables you to configure an expiration policy for a target media group. Files are evaluated against the expiration criteria during a background scan. Those files that meet the criteria are deleted during the scan. Files that are deleted based on expiration policies are not placed in the DiskXtender Recycler since the files are still active on the extended drive and on other media. You can define expiration policies for all but one of the target groups. In other words, you cannot expire files from all target groups in a multi-target group. To configure automatic deletion of files from the final target group, create a delete rule that targets the files and uses an appropriate age delay after the files are expired from the other targets. Keep in mind, however, that once a file is deleted from the final group, it can no longer be accessed because it is no longer on the extended drive. If a file has been migrated to EMC Centera or Retained NAS media and retention has been applied, then the file cannot be expired from the media until the retention expires. However, the file can be expired from media in other media groups that do not support retention. Developing a file migration strategy 181

174 File Migration Tiered migration With tiered migration, also called hierarchical storage management (HSM), each file is migrated twice: from one extended drive to another extended drive, and then from that extended drive to tape media, as illustrated in Figure 6 on page 182. Extended drive on first DiskXtender server File Extended drive on second DiskXtender server File Tape GEN Figure 6 Tiered migration To accomplish this through DiskXtender, media folders on the second extended drive are configured as Aggregate NAS media on the first DiskXtender server. The files from the first extended drive are migrated to the Aggregate NAS media (the second extended drive) by using a standard migration process. In other words, the Aggregate NAS media belongs to a standard media group in a media folder on the first extended drive. The DiskXtender installation on the second server then moves the files to tape media, also with a standard migration process. In other words, the tape media belongs to a standard media group in the media folder configured on the second extended drive. When files are saved to a media folder on the first extended drive that contains a media group with Aggregate NAS media, DiskXtender does not allow you to rename the files or the folders in which they reside. This behavior mimics the behavior of the second extended drive (the Aggregate NAS media), because the second extended drive is writing to tape, which also does not allow file or folder renames. The restrictions on Aggregate NAS media are designed to ensure that files and file tags on both extended drives remain synchronized, and that the files on the subsequent tape media are not orphaned. Tiered migration is supported only through the use of the Aggregate NAS media type and tape media. DiskXtender does not support tiered migration from one extended drive to another media type (such as EMC Centera) before a final migration to tape. Migration from Aggregate NAS to media other than tape (such as EMC Centera or optical) is also not supported. Media fill methods You can control the way in which DiskXtender selects the media in a media group when it writes files to the media. Media can be filled either sequentially or randomly. 182 EMC DiskXtender Microsoft Windows Version Administration Guide

175 File Migration With the sequential media fill method, DiskXtender fills media one at a time-based on the order in which they are listed for the media group in the File System Manager Administrator. In other words, DiskXtender continues writing files to a piece of media until it is full, and then it moves on to the next piece of media. Sequential fill is designed for use with removable media, such as optical or tape. With the random media fill method, DiskXtender writes files to any available media in the media group. Random fill is designed for use in environments where you are writing to disk-based media, such as EMC Centera, NAS, or TSM. With the random fill method, DiskXtender can write to multiple pieces of media at once. You can also select the number of media that DiskXtender can write to simultaneously. To select either the random or sequential media fill method for a media group, use the Media fill method setting on the Options page when you add or edit a media group. If you select the random fill method, you should also specify the number of media that can receive files simultaneously. This is configured with the Maximum media simultaneously receiving files setting, which is also on the Options page for media groups. Marking media as full When the amount of free space on a piece of media falls below a specified number of megabytes, then DiskXtender marks the media as full. When media is marked as full, DiskXtender no longer writes files to the media, although the media is left in its media group. This enables users to continue to access files on the media. Full media appears with a blue label in the File System Manager Administrator. By default, DiskXtender marks media as full when the amount of free space on the media falls below 1 MB. If you use EMC Centera, then the virtual media is marked as full when it reaches the free space threshold or when the maximum file count for the media (100,000 files) is reached. For NAS, remember that the capacity of a piece of media is limited by the capacity of the device on which the share configured as media resides. You can adjust the free space threshold for marking a media as full by configuring the Mark media full when free space falls below setting on the Options page for a media group. Note: Free space on media might fall below the configured level. This occurs because DiskXtender monitors media free space on a timed interval of every 15 minutes, and DiskXtender might continue to write files to the media in between the timed monitoring. After the timed verification occurs, the media is marked as full. However, at that point, the media free space might be less than the amount specified. You can also manually mark a piece of media as full by selecting the Media Full checkbox on the General tab of the Media Properties dialog box for the media. To access the Media Properties dialog box, right-click the media and select Properties. Developing a file migration strategy 183

176 File Migration Enabling file migration The following topics provide details on configuring file migration in DiskXtender: Configuring standard migration on page 184 Configuring multi-target migration in a new environment on page 184 Configuring multi-target migration in an existing environment on page 185 Configuring standard migration Perform the following steps to enable file migration in a standard environment, where DiskXtender moves files in a media folder to a single piece of media: 1. Create one or more standard media groups in each media folder. 2. Allocate media from the media service to the extended drive. 3. Add the allocated media to the media groups. 4. Create move rules to specify which files should and should not be migrated to media. Configuring multi-target migration in a new environment With multi-target migration, DiskXtender moves files in a media folder to multiple pieces of media. In a new environment, where DiskXtender is not yet migrating files to media, perform the following steps to enable multi-target file migration: 1. Create the multi-target media group in the media folder. 2. Create one or more standard media groups in the media folder. 3. Add the standard media groups to the multi-target media group. Note: If you are adding two or more EMC Centera media groups to a multi-target group, then the media in those media groups must be from the same media service. A multi-target media group cannot contain media groups with media from different EMC Centera media services. 4. Allocate media from the media service to the extended drive. 5. Add the allocated media to the standard media groups. 6. Create move rules to specify which files should and should not be migrated to media, selecting the multi-target media group as the target media group on the Settings page for the move rule. 7. (Optional) Configure an expiration policy to automatically delete files from standard media groups once the files meet certain age criteria. Specifying expiration policies for target groups To delete files from one target media group to reclaim space, but leave the files on other target media groups to ensure continued access to them, you can configure an expiration policy for the target group. Files are qualified against the expiration policy during a background scan. Files that meet the criteria are deleted from the media during the background scan. However, the files are not placed in the DiskXtender Recycler since they are still active on the extended drive and on other media. 184 EMC DiskXtender Microsoft Windows Version Administration Guide

177 File Migration Note: If a file has been migrated to EMC Centera or Retained NAS media and retention has been applied, then the file cannot be expired from the media until the retention expires. However, the file can be expired from media in other media groups that do not support retention. Configuring multi-target migration in an existing environment With multi-target migration, DiskXtender moves files in a media folder to multiple pieces of media. If DiskXtender is already migrating files to media by using a standard migration strategy, you can enable multi-target migration and begin migrating the files to one or more additional pieces of media. When you enable multi-target migration in an existing environment, the files that have already been written to media and purged must be fetched back to the extended drive so that they also can be written to the new media. The files are then purged again. This fetch process synchronizes the files on the first media with the files on the second media, and is called sync-fetch. Sync-fetch occurs during a background scan and is a system-intensive procedure. As a result, it is important to carefully plan sync-fetch to minimize the impact on user activity. The following topics provide details on enabling multi-target migration while taking into consideration the issues that affect performance. Preparing the environment for multi-target migration Perform the following steps to prepare the DiskXtender environment before enabling multi-target migration in an existing environment: 1. Connect DiskXtender to the additional storage device(s): To use an EMC Centera cluster, you must provide the connection information to DiskXtender and create virtual pieces of media that simulate divisions of the EMC Centera cluster. Chapter 3, Connecting to EMC Centera, provides details. To use a share on a network-attached storage device, create the shares, and connect DiskXtender to the shares. Each share is considered an individual piece of media in DiskXtender. Chapter 4, Connecting to Network-Attached Storage, provides details. To use a storage device with removable media such as tape or optical, connect DiskXtender to the software managing the device (either MediaStor or StorageTek ACSLS) and add media to the device. Chapter 5, Connecting to Optical and Tape Devices, provides details. To use an IBM Tivoli Storage Manager (TSM) system to store DiskXtender files, install the TSM client on the DiskXtender server, connect DiskXtender to the TSM server, and then create virtual pieces of media that simulate divisions of the TSM server. Chapter 6, Connecting to Tivoli Storage Manager, provides details. 2. Plan to enable multi-target migration during off-peak hours of user activity. 3. Ensure that the system is running without problems. For example: Review the DiskXtender event logs and the Windows event log. If DiskXtender is running in a cluster, review the cluster logs. If you use EMC Centera, run a health check on the device. Enabling file migration 185

178 File Migration If there are any issues, they must be resolved before enabling multi-target migration. 4. Ensure that there is sufficient space on the extended drive for files to be fetched for synchronization. At least 20 percent of the total size of the extended drive is recommended. To free space, purge files by either reducing the purge start watermark or manually purging files with the DiskXtender Explorer Add-ons. Chapter 12, Purging Files, provides details on purging. 5. If you use EMC Centera, ensure that there are sufficient communication threads available between DiskXtender and EMC Centera. Communication threads on page 49 provides guidelines for configuring the appropriate number of communication threads. 6. Minimize the activity on the extended drive by disabling as much system activity as possible while the sync-fetch process is in progress. For example: Disable anti-virus scans and backups. Disable the media task schedule. If the environment is configured for replication by using EMC RepliStor, EMC MirrorView, or EMC SRDF, then pause replication. Note: The file migration schedule must be active during the sync-fetch process. This enables files to be written to the tape media and re-purged. 7. Plan to enable multi-target migration for only one extended drive at a time. You may want to further limit the impact and duration of sync-fetch by setting a limit for the number of files that can be fetched and re-migrated. The steps for limiting sync-fetch are discussed in the topic that follows. Monitoring the sync-fetch process While DiskXtender fetches files from the original media and writes them to the new media, monitor the process to ensure that it proceeds successfully: Monitor the CPU usage on the DiskXtender server, keeping in mind that DiskXtender may also be writing new files to the original media while it performs sync-fetch for the new media. Monitor the DiskXtender event logs and the hardware devices to ensure that there are no unexpected errors. Configuring tiered migration Perform the following steps to enable tiered file migration, where DiskXtender moves files in a media folder to a second extended drive configured as Aggregate NAS media, and then from there to tape media: 1. On the second DiskXtender server (which is migrating files to tape media), add a MediaStor or StorageTek ACSLS media service, and add media to the media service. Chapter 5, Connecting to Optical and Tape Devices, provides details. 2. Create one or more extended drives, and create a media folder for each share that you intend to use as Aggregate NAS media on the first DiskXtender server. Chapter 8, Extended Drive and Volume Mount Point Management, provides details. 3. Create a standard media group that moves files to tape media for each media folder. In other words, select Tape from the Media Type drop-down list on the New Media Group page of the media group. 186 EMC DiskXtender Microsoft Windows Version Administration Guide

179 File Migration 4. Allocate tape media from the MediaStor or ACSLS media service to the extended drive. 5. Add the media to the media group. 6. Create a move rule that moves all files in the media folder to the tape media group. In other words, type *.* as the filename specification on the File Name page for the move rule, and select the tape media group from the Media Group To Receive Files drop-down list on the Settings page for the move rule. 7. Create a share for each media folder on the second DiskXtender server to use as Aggregate NAS media. Only the DiskXtender service account on the first DiskXtender server should have access to the share. 8. On the first DiskXtender server, create a NAS media service, and create Aggregate NAS media for the shares to the media folders on the second DiskXtender server. 9. Create one or more standard media groups in each media folder, selecting NAS as the type of media to add to the group. 10. Allocate the Aggregate NAS media from the media service to the extended drive. 11. Add the allocated media to the media groups. 12. Create move rules to specify which files should and should not be migrated to media, selecting the Aggregate NAS media group as the target media group on the Settings page for the move rule. Configuring media groups When you create a multi-target media group, you simply name the media group. After you create the multi-target media group, you add the standard media groups to it. When you create a standard media group, you name the media group and select the type of media you plan to add to the group. In addition, configure the following options for the media group: If you selected EMC Centera as the media type for the group, you can select options on the Automation page to automatically create virtual media for the media group based on the amount of available free space on the media in the group or based on the number of active (not full) media in the group. On the Metadata page, you can also specify custom metadata that is stored with each file that is written to the EMC Centera media in the group. The custom metadata can be used to enhance EMC Centera Seek and Chargeback Reporter reports. If you selected removable media such as tape or optical as the media type for the group, you can select options on the Automation page to automatically label blank media and add it to the media group based on the amount of available free space in the media group. You can also compact media, remove it from the media group, and reformat the media when the amount of wasted space on the media reaches a certain percentage. Enabling file migration 187

180 File Migration Consider the options in Table 2 on page 188, depending on the type of media you select for the group. Table 2 Recommended media group options for each media type (page 1 of 2) Media type Option Recommended setting EMC Centera Media fill method Random (the default). Maximum media simultaneously receiving files The default value for this option is the media count threshold that you type on the Automation page. You can tell DiskXtender to write to as many pieces as is appropriate for the best system efficiency. The number you enter depends on the optimal number of media that should receive files at any given time. NAS and TSM Media fill method Random. Note: For NAS and TSM media groups, the default is Sequential, so you must edit the setting. MO, UDO, tape, and WORM Maximum media simultaneously receiving files Warn when group free space falls below (MB) Media fill method Warn when group free space falls below (MB) Set this value to the total number of media that you plan to add to the media group. This enables DiskXtender to write to all media as necessary. Since you must manually create NAS and TSM media when it is needed, you should enable this option so that you are notified when the media in the media group is starting to run out of space. You can then configure the warning to be sent out as an alert. Sequential (the default). If you use the automation features for the media group to automatically label and add media to the group when it is needed, then you do not need to enable this option. However, if you do not enable automation, then you should enable this option so that you are notified when the media in the media group is starting to run out of space. You can then configure the warning to be sent out as an alert. 188 EMC DiskXtender Microsoft Windows Version Administration Guide

181 File Migration Table 2 Recommended media group options for each media type (page 2 of 2) Media type Option Recommended setting DVD-RAM and DVD-R Media fill method Warn when group free space falls below (MB) Mark media full when free space falls below (MB) Auto-finalize DVD+/-R media after marking full Purge files after auto-finalizing media Direct read files after auto-finalizing media Sequential (the default). If you use the automation features for the media group to automatically label and add media to the group when it is needed, then you do not need to enable this option. However, if you do not enable automation, then you should enable this option so that you are notified when the media in the media group is starting to run out of space. You can then configure the warning to be sent out as an alert. If you plan to enable auto-finalization of DVD-R, leave this option enabled. Otherwise, auto-finalization is not available. Choose whether to automatically finalize DVD-R when it is marked as full. Choose whether to purge all files on DVD-R from the extended drive after it is finalized. If you leave this option disabled, then files are purged based on configured purge rules. Choose whether to mark files for Direct Read after they are automatically purged after automatic finalization. Direct Read is strongly discouraged in most environments. Reading files directly from media on page 231 provides details. For all other options, the default value is recommended. The EMC DiskXtender for Windows Online Help provides details on each option. Temporarily disabling file migration You can temporarily disable file migration, if necessary by editing the file migration schedule for the extended drive. Keep in mind, however, that DiskXtender may continue to access storage media for file fetch activity when you disable the file migration schedule. To disable all media activity for example, to perform hardware maintenance set the media service or hardware device offline. In addition, files continue to qualify for move rules and are written to the move list when you disable the file migration schedule. You should only disable file migration for a limited period of time. When files cannot be migrated, then they also cannot be purged. As a result, extended drive space issues may occur. You can also permanently disable file migration for example, if you are preparing to uninstall DiskXtender by deleting all configured rules, media groups, and media folders, and finally, the extended drive itself. Removing media but keeping the files on the drive The media compaction process enables you to remove media from a media group while leaving files that have been migrated to the media on the extended drive. Enabling file migration 189

182 File Migration Understanding the compaction process DiskXtender performs the following steps when you compact a piece of media: 1. Strips the migration information from any fetched files (files that have been migrated to media but where the file data is on the extended drive). This enables the files to requalify for move rules and be remigrated to other media in the media group. 2. Copies the file data for any purged files back to the extended drive, and strips the migration information from the files. This enables these files to requalify for move rules and be remigrated to other media in the media group. 3. Removes the media from the media group. Compaction was initially designed to enable you to maximize media capacity usage by reclaiming deleted file space on the media. In that scenario, active files are returned to the extended drive and rewritten to other media, while the old media is reformatted and reused. The removable media automation feature for media groups enables you to automate this process based on the amount of wasted space occupied by deleted files on the media. However, the compaction process can also be used to remove a piece of media while leaving active files on the extended drive, and rewriting those active files to another piece of media. Important guidelines for compacting media Keep the following important guidelines in mind when compacting media: Ensure that there is enough free space on the extended drive to accommodate the return of the files from the compacted media. For the best compaction results, the equivalent of at least one side of the media should be available as free space on the extended drive. If the extended drive becomes so full during the compaction that it cannot receive any more files, the compaction goes into a suspended state for one hour. In addition, no compactions for other media are processed until the original hour (for which the attempted compaction was suspended) has expired. After an hour has passed, DiskXtender retries the original compaction, and if it is successful, processes any remaining compactions. This suspension accomplishes two things: it enables DiskXtender to move and purge files to clear needed extended drive space, and it ensures that drives, which might be needed to complete other media functions, are not being used to attempt other compactions that cannot succeed until there is sufficient space on the extended drive. If a compaction fails, some, all, or none of the files from the media might have been moved back to the extended drive. If you can determine the cause of the compaction failure, make the appropriate adjustments and reschedule the compaction. The compaction process starts over, but any files already copied back to the extended drive are not duplicated. In addition, all files that have been compacted are remigrated to media as they qualify for movement. If you allow folder renames on media that does not support folder renames, then do not compact the media. Data loss might occur because DiskXtender cannot find files on the extended drive that correspond to the files on the media. As a result, the file data for purged files cannot be copied back to the drive. 190 EMC DiskXtender Microsoft Windows Version Administration Guide

183 File Migration In addition, if there are one or more of these files on the extended drive during the compaction, the media is placed in an error state and is not removed from the media group after compaction is complete. If the number of these errors exceeds the maximum specified in the Number of non-fatal errors before task is aborted option for the extended drive, then the compaction is stopped and the media enters an error state. In either of these cases, you must resolve the issues with the files by using the following procedure before the compaction task can resume: 1. Remove the media from the media group. 2. Manually delete any remaining files that were migrated to the media from the extended drive. 3. Add the media back to the media group, restoring the files on the media to the extended drive. 4. Resubmit or restart the compaction task. Removing media and removing its files from the drive The Remove Media From Media Groups Wizard enables you to remove media from a media group and remove the files on the media from the extended drive. Note: You can also remove media from a media group by using a Remove from Media Group media task. The options available on the media task are the same as the options available in the wizard. The media task method is best used to remove a single piece of media from a media group in conjunction with other media tasks. To launch the media task process, right-click the piece of media and select Edit Tasks. Understanding what happens when you remove media In most cases, when you remove media from a media group, all files that have been migrated to that media are removed from the extended drive. However, the files remain on the media. This is true even if a retention period has been applied to a file. The file is removed from the extended drive, but it remains protected on the media and can be restored to the extended drive along with the rest of the files on the media, if necessary. There are two exceptions to this scenario: When you are removing media from a standard media group that belongs to a multi-target media group with other standard media groups, then files are not removed from the extended drive because they are also stored on another piece of media in the multi-target media group. If there is an unrecoverable problem with the media you are removing, then you cannot retrieve purged files from the media. These files are permanently lost, unless you can recover the full file data from a backup. When you are removing this damaged media, an EMC Customer Support Representative might recommend that you attempt to salvage as many files as possible from the media. During the salvage process, migration information is stripped from fetched files (files that have been migrated to media but where the file data is on the extended drive). Any purged files, where only a file tag for the file is on the extended drive, are deleted from the extended drive. This enables you to remove the media from the system and remigrate the salvaged files to another piece of media. Enabling file migration 191

184 File Migration 192 EMC DiskXtender Microsoft Windows Version Administration Guide

185 10 File Retention The term retention or file retention refers to the placement of an attribute on a file that restricts anyone from editing or deleting the file, effectively making the file read-only. The following topics provide details on retaining files in a DiskXtender environment: Understanding file retention Enabling file retention Changing file retention Deleting retained files Triggering retention events File Retention 193

186 File Retention Understanding file retention When you apply retention to a file through DiskXtender, the file cannot be edited or deleted on the extended drive until the retention expires. You also cannot rename retained files, change their attributes, or move them to another location on the extended drive. If you edit a retained file, then you must save the file with a new filename to save the changes. The following topics provide details: When you can use file retention on page 194 Types of retention on page 194 When files qualify for retention on page 196 Event-based retention on page 196 Retention restrictions for saving files to the extended drive on page 196 Delaying retention enforcement on page 197 Synchronizing retention on the extended drive and the media on page 197 What happens after retention expires on page 198 When you can use file retention Retention is available if you are migrating files to an EMC Centera cluster or to shares on a retention-capable NAS device, such as a Celerra Network Server with the File-Level Retention (FLR) file system or a supported Network Appliance device with SnapLock software. If you have an EMC Centera GE or EMC Centera CE+ device, then retained files are also protected on EMC Centera. If you have an EMC Centera Basic Edition and you apply retention through DiskXtender, retained files are protected on the extended drive but are not protected on the EMC Centera device. To enable retention for shares on a retention-capable NAS device, you must configure the shares as Retained NAS media in DiskXtender. Retained NAS on page 78 provides details. Retained files on Retained NAS media are protected on the media as well as on the extended drive. Types of retention Table 1 Table 1 on page 194 lists the three types of retention settings that are available in DiskXtender. Types of retention Type of retention Description Media types that support the retention type Infinite Retention Retention on a file that never expires. EMC Centera Retained NAS Fixed retention or Time-based retention Event-based retention A specific time period (in days) during which file retention is enforced. Retention period on a file is applied when the event occurs at some indeterminate time after the original C-Clip creation. EMC Centera Retained NAS EMC Centera 194 EMC DiskXtender Microsoft Windows Version Administration Guide

187 File Retention Retention period Retention classes A retention period is the numerical representation of the number of days for which the file cannot be modified or deleted. Although EMC Centera allows retention of files for less than one day, DiskXtender does not recognize the retention period less than one day. Hence a minimum retention period on EMC Centera using DiskXtender is one day. A retention class is a symbolic representation of a retention period. When you define the retention class on EMC Centera, you specify a name and a retention period (in days). If necessary, you can edit the retention period for a retention class by changing the class definition, thereby changing the retention period for a group of files. Because they can be edited albeit only by someone with administrative privileges on the EMC Centera cluster retention classes do not provide the same level of file security as a fixed retention period. If you have an EMC Centera Governance Edition device, an unscrupulous individual could edit a retention class to reduce the retention period so that retention expires early, thereby leaving important files vulnerable to editing or deletion. Use caution when assigning retention classes and EMC Centera administrative privileges. The list of retention classes and their definitions in DiskXtender is refreshed every time the EMC Centera media service is refreshed. This occurs approximately every two minutes. The list is stored and updated locally on the DiskXtender server (in addition to the definition on the EMC Centera). This enables retention enforcement on the extended drive even if the EMC Centera device is unavailable. Retention class restrictions There are several restrictions when you use retention classes to apply retention to files on a DiskXtender extended drive: If you have an EMC Centera CE+ device, then you can only extend the retention period definition for a retention class. Even though you can configure retention classes of less than one day on the EMC Centera cluster, you cannot use those retention classes in DiskXtender. Retention classes of less than one day are not listed in the DiskXtender interface. You cannot apply retention classes to files that are written to media in a media group if the media group contains media from two different EMC Centera media services. In other words, you must assign retention by using either a retention period in days, infinite retention, or no retention if you want to mix media from different EMC Centera media services in a media group. This includes media in both standard and multi-target media groups. If you use multi-target migration and there are two standard media groups in the multi-target group, each pointing to two different EMC Centera media services, then you cannot apply retention classes to files that are written to the media in the multi-target group. This restriction is intended to protect against the scenario where the retention class on one EMC Centera is applied to files that have been migrated to a different EMC Centera. Understanding file retention 195

188 File Retention When files qualify for retention Files on a DiskXtender extended drive qualify for retention in other words, retention begins when the files qualify for a move rule with retention enabled. If an age delay is configured for the move rule with retention, then qualifying files are not retained until after the delay has passed. DiskXtender adds the retention information to the attributes for the file, and the file is then protected on the extended drive. The retention setting is passed to the media when the file is actually moved. How files qualify for file migration on page 173 and How and when files are actually migrated on page 174 provide additional details. Note: Retention is applied on only those files that reside on the extended drive. When the file is set offline, retention settings do not apply to that file. Retention restrictions for saving files to the extended drive If files qualify for move rules with retention as soon as the files are saved to the extended drive (because there are no age delays configured for the rules and real-time moves are enabled), then users cannot create files directly on the extended drive. This is because the files are protected from further editing once they are saved or created on the extended drive. If this is the case, then users must create and edit files in another location and then move them to the extended drive when they are finished. Alternatively, you may want to disable real-time moves or specify an age delay for the move rule. When real-time moves are disabled, files only qualify for move rules during a background scan. When move rules contain an age delay, files do not qualify for the rules until a certain number of days have passed since the file was created, last accessed, or last written to (modified). Retention is applied and the file is moved to media after the age delay has passed. Event-based retention Retention is a powerful feature of EMC Centera and DiskXtender leverages that feature to retain files on media for a specific period. An event defined in the EMC Centera can be associated to a file. The retention on a file is applied when the specified event occurs. The event however must be triggered by a qualified DiskXtender administrator. The retention period is specified while setting retention on a file. An event can be associated to more than one file and can be triggered at different times. A file can have both time-based and event-based retention applied. Time-based retention is a fixed retention period. The retention on the selected file applies from the time the retention is set on that file till the retention period expires. Event-based retention uses either the retention period or the retention class defined in the EMC Centera. The file with event-based retention can be manually removed from the extended drive when both of the following conditions are met: Time-based retention associated with the file has expired Event associated with the file has occurred and the retention associated with the event has expired 196 EMC DiskXtender Microsoft Windows Version Administration Guide

189 File Retention Some key points before you set event-based retention: Once event-based retention is set on a file, the retention cannot be removed. An event can be triggered only once on a file. However, the same event can be triggered on a different file when associated with the event. Event-based retention cannot be set on a file associated with NAS media. The files whose event-based retention period has expired are not automatically deleted from the media. However, such files can be manually deleted. A DiskXtender administrator with privileged delete access can delete files under event-based retention. In a standard multi-target media group that contains media from two different EMC Centera systems, DiskXtender does not allow retention classes to be used for setting the event-based retention. Note: Event-based retention can be used only if the Advanced Retention Management feature in EMC Centera is enabled. Advanced Retention Management is a licensed feature of EMC Centera and is disabled by default. It can be enabled on GE and CE+ models, only by EMC service personnel. Delaying retention enforcement DiskXtender automatically delays the application of retention on a file for 10 seconds after the file is saved to the extended drive. Delaying the enforcement of retention allows programs that open and close files in the background when saving (like Microsoft Word) to save files to the drive unimpeded. If retention is applied to a file as soon as the data hits the drive, the program may not be able to finish saving the file because DiskXtender would deny any modifications to the file. If necessary, you can extend the number of seconds that retention enforcement is delayed. To edit the retention delay, use the Delay in seconds before retention period is enforced option on the Options tab of the Extended Drive Properties dialog box. To access the Extended Drive Properties dialog box, right-click the extended drive and select Properties. You can further delay retention enforcement, if necessary, by setting an age delay for the move rules. When move rules contain an age delay, files do not qualify for the rules and therefore, retention is not enforced until a certain number of days have passed since the file was created, last accessed, or last written to (modified). Keep in mind, however, that when you specify a retention delay, you must balance your company s requirements for data protection with the practices used to create and store files on the extended drive. Synchronizing retention on the extended drive and the media DiskXtender protects files on the extended drive as soon as they qualify for move rules with a retention setting. The retention is then passed to the media when the file is migrated. If you set global retention on the EMC Centera device and then you configure a different retention period through DiskXtender, the DiskXtender retention period applies. This is true even if the DiskXtender retention period is zero days (no Understanding file retention 197

190 File Retention retention). When you set a retention period of zero days through DiskXtender, then the file is not protected on the EMC Centera device, even if EMC Centera is configured for a global retention period of one or more days. Note: DiskXtender always applies a retention period to files written to EMC Centera, even if the retention period is zero days (no retention). You cannot automatically apply the global retention period set on the EMC Centera device to files on the extended drive. To match the global retention period on EMC Centera with the retention set on files on the extended drive, specify the same retention setting in DiskXtender as on EMC Centera. In the unlikely event that there is a delay between the time that the file qualifies for a move rule and the time that the file is actually migrated, then the total retention period is equivalent to the retention setting in the move rule plus the amount of time that the file was protected on the extended drive before it was migrated. For example, assume that the retention period on a move rule is 30 days. A file qualifies for the move rule as soon as it is saved to the extended drive on Monday at 9 A.M. The file is immediately protected on the extended drive starting at 9 A.M. However, the file is not migrated to media until Wednesday at 1 P.M. because the media is offline. When DiskXtender migrates the file on Wednesday, it passes the retention period of 30 days to the media. The file is then protected on the media for 30 days after it is migrated to the media. DiskXtender updates the retention information for the file so that it is synchronized with the media for 30 days of retention. However, the file has already been protected on the extended drive for two days. As a result, the file is protected on the extended drive for a total of 32 days, and not the 30 days specified in the move rule. What happens after retention expires After retention expires, users can manually delete the files. However, such files cannot be modified. DiskXtender does not automatically delete files after retention expires unless you configure delete rules. Automatically deleting files on page 236 provides details. DiskXtender retention compared with EMC Centera retention DiskXtender protects files on the extended drive as soon as they qualify for move rules with a retention setting. The retention is then passed to the media when the file is migrated. If you set global retention on the EMC Centera device and then you configure a different retention period through DiskXtender, the DiskXtender retention period applies. This is true even if the DiskXtender retention period is 0 days (no retention). When you set a retention period of 0 days through DiskXtender, then the file is not protected on the EMC Centera device, even if EMC Centera is configured for a global retention period of 1 or more days. Note: DiskXtender always applies a retention period to files written to EMC Centera, even if the retention period is 0 days (no retention). You cannot automatically apply the global retention period set on the EMC Centera device to files on the extended drive. To match the global retention period on EMC Centera with the retention set on files on the extended drive, specify the same retention setting in DiskXtender as on EMC Centera. 198 EMC DiskXtender Microsoft Windows Version Administration Guide

191 File Retention In the unlikely event that there is a delay between the time that the file qualifies for a move rule and the time that the file is actually migrated, then the total retention period is equivalent to the retention setting in the move rule plus the amount of time that the file was protected on the extended drive before it was migrated. For example, assume that the retention period on a move rule is 30 days. A file qualifies for the move rule as soon as it is saved to the extended drive on Monday at 9 A.M. The file is immediately protected on the extended drive starting at 9 A.M. However, the file is not migrated to media until Wednesday at 1 P.M. because the media is offline. When DiskXtender migrates the file on Wednesday, it passes the retention period of 30 days to the media. The file is then protected on the media for 30 days after it is migrated to the media. DiskXtender updates the retention information for the file so that it is synchronized with the media for 30 days of retention. However, the file has already been protected on the extended drive for two days. As a result, the file is protected on the extended drive for a total of 32 days, and not the 30 days specified in the move rule. Understanding file retention 199

192 File Retention Enabling file retention You can enable retention automatically on files by configuring retention settings on move rules. You can also set retention on individual files by using the DiskXtender Explorer Add-ons. Changing file retention After you apply retention to a file, the only way to change the retention setting for the file is by using the DiskXtender Explorer Add-ons. You cannot change the retention setting for a file by changing the move rule for which the file originally qualified. Once a file qualifies for a move rule with a retention setting of one or more days (or a retention class or infinite retention), then the retention on the file is not changed automatically, even if the file is qualified against the move rules again as part of a background scan. When you change retention for a purged file, the file remains purged. In other words, DiskXtender does not need to fetch a file to change the retention setting for the file. When you can change retention If you have Retained NAS media, then you cannot edit the retention period for a file until the original retention period expires. If you use EMC Centera, you can extend the retention period or change the retention class. You cannot, however, reduce a retention period or switch from a retention class to either a fixed retention period or infinite retention. If you have EMC Centera CE+, then you cannot reduce the time period assigned to a retention class. In addition, you cannot edit the name of a retention class. To change the name of a retention class, you must create a new retention class with the new name, and then switch the retention setting on the files to the new retention class. Table 2 on page 200 lists the options for editing retention on files migrated to EMC Centera. Table 2 Retention editing options If you use this retention option Then you can switch to this retention option A fixed retention period A different fixed retention period, as long as you are extending the retention period. A retention class, as long as the retention class definition is for a time period that is greater than the current fixed retention period. Infinite retention. A retention class A different retention class. Note: The new retention class definition can be a shorter time period than the previous retention class definition. Infinite retention Event-based retention Not applicable. If you use infinite retention, you cannot switch to a different retention option. Retention period on a file is applied when the event occurs at some indeterminate time after the original C-Clip creation. 200 EMC DiskXtender Microsoft Windows Version Administration Guide

193 File Retention Deleting retained files If you have an EMC Centera Basic or EMC Centera GE device, you can delete retained files by performing a privileged delete with the DiskXtender Explorer Add-ons. Ensure that the profile that DiskXtender uses to connect to EMC Centera has the Privileged Delete right. To delete retained files on a CE+ device, contact an EMC Centera technical representative. Note: Consider the compliance regulations followed by your company before you perform a privileged delete. The EMC Centera online help provides additional information on profiles. Access profiles should be configured by an EMC Centera technical representative. Triggering retention events If you have set event-based retention on a file, you can trigger a retention on that file by selecting trigger retention event with the DiskXtender Explorer Add-ons. Note: Once you trigger the event-based retention on a file, you cannot modify or remove the retention parameters, and you cannot re-trigger the event. In a multi-target media group, DiskXtender does not allow the use of retention classes for an event-based retention. Deleting retained files 201

194 File Retention 202 EMC DiskXtender Microsoft Windows Version Administration Guide

195 11 Indexing Files The DiskXtender Search Module, available separately from DiskXtender for Windows, enables you to index and search data on the extended drive. The following topics provide details on indexing files on the extended drive so that you can search for them: Understanding the indexing process Supported file types for indexing Developing an indexing strategy Enabling indexing Resynchronizing the index files and the extended drive Troubleshooting Indexing Files 203

196 Indexing Files Understanding the indexing process A thorough understanding of the indexing process is necessary for monitoring and troubleshooting purposes. Review the following topics to learn more about how and when files are indexed and available for searches. DiskXtender architecture for indexing The DiskXtender Search Module is installed separately from DiskXtender and includes the following two components: The Index and Search Engine (ISE) The user interface The ISE manages content indexing and is installed on a separate server from the DiskXtender server. A single ISE server can index files from as many as 10 DiskXtender servers, but 3 to 7 DiskXtender servers are recommended for each ISE server, for optimal performance. Further, each DiskXtender server should manage no more than five extended drives. The web-based user interface provides a way to search a collection of indexes for files managed by DiskXtender, and then to view the search results. It also enables you to perform several important administrative tasks on the ISE, such as index preparation tasks and the addition of index file storage. The user interface can be accessed through a http URL which can be obtained from the DSM administrator. You can log in using the credentials provided by the administrator and perform searches and administrative tasks. Figure 1 on page 205 illustrates a DiskXtender environment where the DiskXtender Search Module is installed. 204 EMC DiskXtender Microsoft Windows Version Administration Guide

197 Indexing Files Workstations saving and retrieving files Workstations through which the web interface can be accessed to perform searches and administer the ISE Microsoft Windows server with DiskXtender Indexing Indexing Extended drives Microsoft Windows server with the DiskXtender Search Module Index and Search Engine (ISE) Storage media such as EMC Centera, NAS, or tape GEN Figure 1 DiskXtender architecture for indexing You can install the DiskXtender Search Module along with DiskXtender version 6.3 or above (6.3, 6.4, 6.3 SP1, 6.4 SP1, 6.5, 6.5 SP1, 6.5 SP2, 6.5 SP3, 6.5 SP4, 6.5 SP5, 6.5 SP6) in a new environment. You can also install the DiskXtender Search Module in an environment where DiskXtender is already managing files, provided DiskXtender is upgraded to release 6.3 or later. The ISE can index new files that are saved to the extended drive, as well as files that were on the extended drive prior to the DiskXtender or DiskXtender Search Module installation. The DiskXtender Search Module installation guide provides detailed system requirements and procedure to log in to the web user interface. Prerequisite steps for indexing You must configure index collections on the ISE and enable indexing on the DiskXtender server before content indexing can begin. To set up indexing: 1. On the ISE, create an index collection for each extended drive that contains files to index. Additionally, if you are a non-administrator user, specify a share name and the IP address of the DiskXtender server. Understanding the indexing process 205

198 Indexing Files An index collection is a set of all index files for an extended drive. Index collection information is stored in a database on the ISE. 2. On the DiskXtender server, enable content indexing for each extended drive that contains files to index. You must specify: The ISE server that should index the files on the extended drive. The index collection that should receive the files. A schedule for when indexing should occur. Indexing can be scheduled to occur regularly on a daily, weekly, or monthly basis. You can also manually force indexing to occur once at a certain date and time. To maximize system performance, schedule indexing to occur at a time when there is minimal system activity. 3. On the DiskXtender server, in each media folder, create DiskXtender index rules to identify which files you do and which files you do not want to index. If you do not want to index any files in the media folder, do not create an index rule. The rule criteria you can use to select files for indexing or exclusion from indexing include file location, name, type, size, attributes, and age. How files qualify for indexing Files are qualified against index rules at the following times: When the files are saved to the extended drive in other words, when they are added, edited, renamed, or moved. During a background scan. This is true in cases where the file did not qualify for an index rule when it was saved to the extended drive. For example: The file was already on the extended drive when the index rule was created. The index rules in the media folder may be configured with an age delay, so that files qualify only once they reach a certain age. You can also force all files to be evaluated against index rules during a background scan, regardless of index rule settings or when the files are saved to the extended drive. Disabling real-time rule qualification can improve performance on busy systems where a large number of files are saved to the extended drive in a short period of time. System resources can then be freed to service other requests, such as file fetches. To change when files are qualified against index rules, select Disable real-time moves for the Enable real-time moves option on the Options tab of the Service Properties dialog box in the File System Manager Administrator, as illustrated in Figure 2 on page 207. Be aware that the Enable real-time moves option applies to both index rules and move rules. 206 EMC DiskXtender Microsoft Windows Version Administration Guide

199 Indexing Files Figure 2 Disabling real-time moves When a file meets the rule criteria for indexing, the file is written to the Index Transaction Log (ITL) for the extended drive. The ITL contains a list of: New files that need to be indexed. Edited files that need any corresponding index information to be updated. Deleted files that need any corresponding index information to be deleted. There is one ITL for each extended drive with indexing enabled. If a file is purged when it qualifies for indexing, the file is fetched to the extended drive before it is written to the ITL. File data must be present on the extended drive during the indexing process so that the ISE can scan the file. In existing systems with a significant number of files, the process of indexing purged files can significantly impact DiskXtender performance. Developing an indexing strategy on page 210 provides guidance on maximizing performance in such a scenario. Note: After the file is indexed, the file is automatically purged again. You do not need to wait until the extended drive reaches the purge start watermark or until the next background scan occurs (whichever is appropriate, depending on your purge settings). If a file qualifies for both indexing and purge at the same time, the file is indexed before it is purged. Files that qualify for index rules are not written to the purge list until after they are submitted for indexing even if they qualify for both an index rule and a purge rule at the same time (during a background scan). As a result, it is critically important that indexing takes place successfully on a regular basis. Otherwise, files may never be written to the purge list, meaning there may not be any files to purge if the extended drive begins to run out of disk space. How and when files are actually indexed When the indexing schedule begins, the DiskXtender server attempts to connect to the ISE to kick off the indexing process. If the connection is successful, the ISE processes the ITL. When it encounters a new file that needs to be indexed, or an edited file that requires an update to an index, the ISE scans the file and writes the appropriate index information to the ISE database and the working index files for the index collection on the ISE. When the ISE Understanding the indexing process 207

200 Indexing Files encounters a file on the ITL that has been deleted from the extended drive, it removes the file index information from the ISE database so that searches will no longer return the deleted file in their results. The ISE continues processing the ITL until there are no more entries left on the ITL, or until the process is interrupted (for example, if the indexing process is paused on the ISE server). Note: If users are saving files to the extended drive while indexing is taking place, and the files qualify for index rules, then the files are added to the ITL and are processed during the current active schedule. If the connection between the DiskXtender server and the ISE server is not successful for example, because there are already 25 other extended drives submitting files to the ISE server for indexing then the connection is retried after 30 minutes and the files are then indexed. This process is repeated till files from all the extended drives are indexed. Files that are successfully indexed are marked with the Indexed attribute on the extended drive. Files that have qualified for indexing and been submitted to the ISE for indexing, but that the ISE cannot index for example, because the file is not a supported file type are marked with the Do Not Index attribute. To determine whether a file is marked with the Indexed attribute or the Do Not Index attribute, or to manually mark a file with the Do Not Index attribute so that it is not indexed, use the DiskXtender Explorer Add-ons to view the properties for the file. The Indexed and Do Not Index options appear in the middle of the DiskXtender File Properties dialog box, as illustrated in Figure 3 on page 208. Figure 3 Indexing attributes on the DiskXtender File Properties dialog box Explorer Add-ons on page 250 provides details on how to view file properties. Note: Files that qualify for exclusive index rules, as well as files that have not qualified for any index rules at all, are marked with neither the Indexed nor the Do Not Index attribute. No content indexing attribute is assigned to these files. About fifteen minutes after all the files on an extended drive are indexed, the files are available for searches. This is because all the files have to be present on the ISE database to be available for searches. 208 EMC DiskXtender Microsoft Windows Version Administration Guide

201 Indexing Files Supported file types for indexing The ISE can index the following file types, as long as the files are smaller than 2 GB in size: Microsoft Word Microsoft RTF Microsoft Excel Microsoft PowerPoint Microsoft Visio Note: To index Visio files, you must install the Visio IFilter 2003 Add-in on the ISE server. Download the add-in from the Microsoft website, and install it before you begin indexing files. HTML JPEG (Exchangeable Image File Format, or EXIF, markers only) Adobe PDF (Portable Document Format) Text files Note: The ISE cannot index a text file unless it contains either several hundred characters, or a carriage return or line feed at the end of each line. TIFF (metadata only) XML ZIP Note: Within the ZIP file, ISE indexes only file types that are supported. However, all ZIP files are marked with the Indexed attribute, even if the ZIP file does not contain any supported file types. The ISE supports the indexing and search of files with Unicode characters in the filename, file metadata, and file data. If the files include ANSI character encoding, such as Big5 (Chinese) or Shift-JIS (Japanese), then the corresponding code page must be installed and set as the default code page on the ISE server. If a different code page is set as the default on the ISE server, then perform the steps in the EMC DiskXtender Search Module Release Notes to enable proper indexing and search of the ANSI-encoded files. Supported file types for indexing 209

202 Indexing Files Developing an indexing strategy Review the following topics for information on developing an indexing strategy that will maximize performance and avoid common problems based on the type of environment in which you are installing the DiskXtender Search Module. The following scenarios are covered: Indexing in a new DiskXtender environment with no files on page 210 Indexing in an environment with existing files on page 211 Adjusting an indexing strategy for ongoing maintenance on page 213 Indexing in a new DiskXtender environment with no files You have many indexing strategy options if you are installing DiskXtender on a new server that does not already have files. To maximize performance in a new environment: Exceed the minimum hardware and network requirements for the ISE and DiskXtender servers. Servers with a faster CPU speed, additional memory, plenty of disk space, and faster networks perform better than servers that meet only the minimum requirements. Review the list of Supported file types for indexing on page 209, and determine whether you are going to be storing any unsupported file types on the extended drive. Try to minimize the number of unsupported files that are submitted to the ISE for indexing. Indexing performance improves when you limit the number of files that the ISE is required to scan. Some strategies to accomplish this include: Save unsupported file types in a separate media folder than the supported file types. In the media folder with the unsupported file types, do not create any index rules. Create exclusive index rules to exclude unsupported file types from being indexed. (On the first page of the Index Rule Wizard, select Exclude. Then type *.extension on the File Name page, where extension is the file extension for the unsupported file type.) You must create one rule for each unsupported file type to exclude. In addition, exclusive index rules should be listed first in the list of index rules for a media folder. Ordering the rules this way ensures that the appropriate files are excluded from indexing. Selecting files for indexing on page 215 provides details.! IMPORTANT You may want to create separate purge rules for files that are excluded from indexing. Excluded files are eligible to be purged as soon as disk space is needed. They do not need to be indexed first. Review the guidelines in Adjusting an indexing strategy for ongoing maintenance on page 213. These guidelines also apply to a new environment that does not yet have a significant number of files. It is recommended that you do not schedule indexing of more than 50,000 files at a time, though there is no outer limit mandated. Batches containing files more than 50,000 might significantly affect the performance. 210 EMC DiskXtender Microsoft Windows Version Administration Guide

203 Indexing Files Indexing in an environment with existing files There are a number of important considerations for setting up indexing if you are installing the DiskXtender Search Module to index files in one of the following situations: In addition to installing the DiskXtender Search Module, you are installing DiskXtender in the environment. However, the server on which you are installing DiskXtender already contains a number of files. You are installing the DiskXtender Search Module in an environment where DiskXtender is already managing files. These indexing strategy considerations are designed to ensure that you are able to avoid errors and maintain an acceptable level of system performance while DiskXtender and the DiskXtender Search Module perform the system-intensive process of indexing (and migrating and purging, as necessary) the existing files on the extended drive. Once this initial processing is complete, you might be able to adjust the indexing strategy to more of a maintenance strategy one that needs to accommodate only new files saved to the extended drive on an ongoing basis. If there are a significant number of files on the extended drive when you install the DiskXtender Search Module, consider the following indexing strategy tips: Analyze the files on the extended drive, and try to minimize the number of unsupported files that are submitted to the ISE for indexing. Create exclusive index rules to exclude unsupported file types from being indexed. (On the first page of the Index Rule Wizard, select Exclude. Then type *.extension on the File Name page, where extension is the file extension for the unsupported file type.) You must create one rule for each unsupported file type to exclude. In addition, exclusive index rules should be listed first in the list of index rules for a media folder. Ordering the rules this way ensures that the appropriate files are excluded from indexing. Selecting files for indexing on page 215 provides details.! IMPORTANT You may want to create separate purge rules for files that are excluded from indexing. Excluded files are eligible to be purged as soon as disk space is needed. They do not need to be indexed first. Do not submit all files on the extended drive for indexing at once. Stagger the indexing of files over time. Some strategies to accomplish this include: Create index rules for one media folder at a time over the course of an appropriate time period. For example, create the index rules for one media folder on one night. The files in that media folder are then qualified against the rules during the next background scan and submitted for indexing based on the indexing schedule. On the next night, create the index rules for the next media folder, and so on. Target specific, high-priority files for the initial indexing effort by creating an index rule with a File Name specification based either on the file type or filename. For example, index all Microsoft Word files first. Then add index rules over time to include the remaining files on the extended drive. Index all the most recent or the oldest files on the extended drive first. Then change the age specification incrementally over time to include more files. For example, index all files that have been created within the last 30 days. (On the Age page of the Index Rule Wizard, select the Less than option, type Developing an indexing strategy 211

204 Indexing Files 30 in the Days text box, and select Create time from the Since drop-down list.) After all of the qualifying files have been indexed, change the rule to files created within the last 60 days, then 90 days, and so on. Use the Advanced Background Scan option to limit the number of files that are added to the ITL during a single scan. (Edit the Max pending files value next to the Perform index rule evaluation option on the Advanced Scan Configuration dialog box. The default value is 100,000 files.) If this is a new DiskXtender installation, wait to purge files until after they have been indexed. File data must be present on the extended drive to be indexed. If a file is purged when it qualifies for an index rule, the file is fetched to the extended drive before it is written to the ITL and submitted for indexing. The file is then automatically purged again after it is indexed. In systems with a significant number of files, this process can significantly impact performance. To delay purging, wait to configure purge rules until after all files in a media folder have been indexed. Alternatively, configure an appropriate age delay on the purge rules. Do not use the Purge files immediately after move option on move rules. Chapter 12, Purging Files, provides details on configuring purging. If this is an existing DiskXtender installation, prefetch purged files when you are ready to index them. During a prefetch, DiskXtender retrieves files from media and writes the file data to the extended drive. File data must be present on the extended drive to be indexed. For example, if you are indexing a single media folder at a time, create a prefetch request for all files in the media folder before the background scan occurs to qualify the files for indexing. Prefetch requests on page 231 provides instructions on creating a prefetch request. Determine the amount of available free space on the extended drive before prefetching files. You do not want to fill the extended drive with the prefetched files. If a single ISE server is indexing files for two or more DiskXtender servers, do not submit all files on all servers for indexing at once. Balance the processing requirements for the ISE server by staggering the indexing schedules. Index the files on only one server at a time. After the indexing process for the first server is complete, then index the files on the second DiskXtender server, and so on. Errors may occur if more than one DiskXtender server submits all files for the initial indexing process at the same time. The DiskXtender Search Module installation guide provides details on troubleshooting the errors. Note: Each ISE server can index files from no more than 10 DiskXtender servers. Each DiskXtender server submitting files for indexing should manage no more than five extended drives. Once the initial indexing process for all servers is complete, the ISE server can index files from no more than 25 extended drives at one time. (Requests from additional extended drives are denied. The connection between the ISE server and the additional extended drives is retried after 30 minutes.) In addition, errors can occur when the ISE server is required to process too many requests at the same time. 212 EMC DiskXtender Microsoft Windows Version Administration Guide

205 Indexing Files Adjusting an indexing strategy for ongoing maintenance After you finish indexing all existing files on the extended drive, consider the following tips for adjusting the indexing strategy for ongoing maintenance: Maintain or establish exclusive index rules to exclude common unsupported file types from indexing. Indexing performance improves when you limit the number of files that the ISE is required to scan. Ensure that files are indexed before they are purged: The best purge option for ensuring that files are indexed first is to purge files based on the purge watermarks (amount of used space) on the extended drive. If you are purging files during background scans, configure an age delay for the purge rule so that the purging takes place after indexing. If you configure an age delay on the index rules, ensure that the age delay on the purge rules is equal to or greater than the age delay on the index rules. Do not purge files immediately after they are moved with the move rule option for purging. Chapter 12, Purging Files, provides details on purging files. Ensure that indexing is taking place successfully on a regular basis so that indexed files can be purged as disk space is needed. Files that qualify for index rules are not written to the purge list until after they are submitted for indexing even if they qualify for both an index rule and a purge rule at the same time (during a background scan). If files are not indexed regularly, then the files may never be written to the purge list, meaning there may not be any files to purge if the extended drive begins to run out of disk space. You may also want to create separate purge rules for files that are excluded from indexing. Excluded files are eligible to be purged as soon as disk space is needed. They do not need to be indexed first. Balance the processing requirements for the DiskXtender server by adjusting activity schedules. System performance decreases as the DiskXtender server is required to manage more activities at the same time. These activities include: Users and applications saving files to the extended drive (and qualification of the files against move and index rules) Users and applications fetching purged files from media Moving files to media Background scans Media tasks Copy media updates Backups and DiskXtender metadata exports Virus scans Balance the schedules for the movement of files to media, background scans, media tasks, copy media updates, backups, metadata exports, and virus scans during the times when users are less likely to be accessing the files on the extended drive. To improve performance on busy systems where a large number of files are saved to the extended drive in a short period of time, disable real-time moves and indexing qualification. System resources can then be freed to service other requests, such as file fetches. By default, DiskXtender qualifies files against move Developing an indexing strategy 213

206 Indexing Files rules and index rules as soon as the files are saved to the extended drive (in real time). If there are a significant number of users saving files to and accessing files on the extended drive during a certain period of time, you can change this default behavior so that files are qualified for migration only during a background scan. To disable real-time moves, select Disable real-time moves option for the Enable real-time moves option on the Options tab of the Service Properties dialog box, as illustrated in Figure 2 on page 207. If a single ISE server is indexing files for multiple DiskXtender servers and extended drives, balance the processing requirements for the ISE server by staggering the indexing schedules. The ISE server can index files from no more than 25 extended drives at one time. (Requests from additional extended drives are denied. The connection between the ISE server and the additional extended drives is retried after 30 minutes.) In addition, errors can occur when the ISE server is required to process too many requests at the same time. The DiskXtender Search Module installation guide and online help provide details on troubleshooting the errors. Note: Each ISE server can index files from no more than 10 DiskXtender servers. Each DiskXtender server submitting files for indexing should manage no more than five extended drives. 214 EMC DiskXtender Microsoft Windows Version Administration Guide

207 Indexing Files Enabling indexing To enable indexing for files on an extended drive: 1. On the ISE, create an index collection for each extended drive that contains files to index. An index collection is a set of all index files for an extended drive. Selecting files for indexing on page 215 provides detailed instructions. 2. (Optional) If you plan to index Microsoft Visio files, download the Visio IFilter 2003 Add-in from the Microsoft website and install it on the ISE server. 3. In DiskXtender, enable content indexing for each extended drive that contains files to index. Specify the ISE that should index the files on the extended drive, and set a schedule for when indexing should occur. 4. Create DiskXtender index rules to identify which files you do and which files you do not want to index. You can index new files that are saved to the extended drive, as well as files that are already being managed by DiskXtender. Selecting files for indexing on page 215 provides detailed instructions. Note: Be sure that you have reviewed the guidelines in Developing an indexing strategy on page 210 before you enable indexing. The guidelines are designed to assist you in maximizing DiskXtender performance during indexing. Selecting files for indexing Index rules enable you to specify which files should and should not be indexed. You can configure index rules for each media folder. The criteria you can use to select files for indexing or exclusion from indexing include file location, name, type, size, attributes, and age. Developing an indexing strategy on page 210 provides guidance on setting up index rules and other indexing settings to maximize performance. Enabling indexing 215

208 Indexing Files Disabling indexing If necessary, you can disable content indexing either for a single extended drive or for all extended drives that are submitting files to an ISE for indexing. The following topics provide details: Disabling indexing for an extended drive on page 216 Disabling all content indexing on page 217 Disabling indexing for an extended drive There are two elements to content indexing that you can disable for an extended drive: The qualification of files against index rules and the writing of files to the ITL. To disable this indexing feature, clear the Enable content indexing of files on this extended drive checkbox on the Indexing tab of the Extended Drive Properties dialog box in the File System Manager Administrator, as illustrated in Figure 4 on page 216. Figure 4 Disabling index rule qualification The indexing schedule (processing of the ITL and submission of files to the ISE for indexing). To disable this indexing feature, select the Disable option in the Index Scheduling section on the Indexing tab of the Extended Drive Properties dialog box in the File System Manager Administrator, as illustrated in Figure 5 on page EMC DiskXtender Microsoft Windows Version Administration Guide

209 Indexing Files Figure 5 Disabling the indexing schedule Note: If you disable the indexing schedule while the schedule is active and processing the ITL, then DiskXtender does not stop the ongoing processing of the ITL. Once the ongoing process is complete, the indexing schedule is disabled and the ITL is no longer processed. To stop DiskXtender from processing the ITL while it is ongoing, clear the Enable content indexing of files on this extended drive checkbox. If you disable rule qualification, then the indexing schedule is also disabled. If you disable the indexing schedule but do not disable rule qualification, however, files continue to be written to the ITL as they qualify for configured index rules. This is useful to temporarily disable the indexing process; for example, to perform maintenance on the ISE. However, keep in mind that the ITL for each extended drive is kept as a file on the extended drive of the DiskXtender server. The file will continue to grow, occupying additional space on the drive, until you either delete the index rules or disable rule qualification as well. In addition, files that qualify for index rules cannot be purged. If the indexing schedule is disabled for a significant amount of time, extended drive space issues are likely to occur. Disabling all content indexing To disable indexing for all extended drives that the ISE indexes, you can pause the ISE. You may want to pause the ISE for system maintenance for example, to take a snapshot of the system for backup purposes. Do not disable indexing for a significant amount of time. Files on the DiskXtender extended drives continue to qualify for index rules, and therefore cannot be purged. Extended drive space issues are likely to occur unless indexing takes place regularly. The DiskXtender Search Module installation guide and online help provide details on pausing the ISE and disabling all indexing. Disabling indexing 217

210 Indexing Files Resynchronizing the index files and the extended drive DiskXtender and the ISE work together to ensure that the index files on the ISE are current with the files on the extended drive. However, in the unlikely event that the files on the extended drive are not synchronized with the indexes on the ISE, you can schedule a resynchronization process. The resynchronization process ensures that: There is corresponding index information on the ISE for all files on the extended drive that are marked as indexed. There are no orphan indexes on the ISE for files that have been deleted from the extended drive. IMPORTANT Resynchronization is designed to repair a damaged system. It is a system-intensive process and should be scheduled accordingly to accommodate the necessary system resources for an appropriate duration of time. When index resynchronization may be necessary Scenarios that may require resynchronization include: Files that have been indexed are deleted or renamed on the extended drive while the DiskXtender service is stopped. Files that have been indexed are deleted or renamed on the extended drive while indexing was disabled. An extended drive is damaged and an out-of-date backup is used to restore the drive. One or more index files is deleted and an out-of-date backup is used to restore the files. 218 EMC DiskXtender Microsoft Windows Version Administration Guide

211 Indexing Files Troubleshooting If indexing of files is taking too much time, make sure you have no more than 50,000 files in a batch. Before indexing files, make sure the files marked for indexing are fetched on to the extended drive. If you encounter ScanDoc error while index synchronization is in progress, and indexing stops, create a new index collection and map it to the extended drive. Restart all ISE services, remove indexed attribute from all the files that are already indexed, and start indexing again. Troubleshooting 219

212 Indexing Files 220 EMC DiskXtender Microsoft Windows Version Administration Guide

213 12 Purging Files The following topics discuss when and how to purge files from the extended drive, as well as how to manage user access to purged files: Understanding the purge process Developing a purging strategy Controlling user access to purged files Controlling third-party software access to purged files Purging Files 221

214 Purging Files Understanding the purge process When DiskXtender moves files, it is actually copying the files to the storage media. In other words, the file data exists on both the media and on the extended drive. When DiskXtender purges a file, it removes the file data from the extended drive and leaves behind a file tag on the extended drive. A file tag contains information about the file; including name, size, age, retention period (if applicable), and other attributes. Purging files frees space on the extended drive, while maintaining the appearance that the files are still present on the drive. In Microsoft Windows Explorer, the icons for purged files (file tags) appear differently than the icons for fetched or unmigrated files. Table 1 on page 222 provides details on file icons for unmanaged, fetched, and purged files. Table 1 Icons for files on the extended drive File type Icon Icon description Unmigrated or fetched file Standard Windows icon for the associated file type. Purged file (file tag) in Windows Vista and Windows Server 2008 The standard file icon appears to have a dimmed X in the lower left corner. Purged file (file tag) in earlier Windows versions The standard file icon appears to have a clock attached to the lower left corner. Although a purged file is still accessible, to a user machine, it appears to reside on the extended drive. When the user requests the file, DiskXtender accesses the media where the file is stored and retrieves the file data, displaying it for the user. When a file is migrated to a single piece of media, a file tag consumes between 0 bytes and 1 cluster of disk space. (On Microsoft Windows NTFS and ReFS systems, a cluster of disk space averages approximately 4 KB, although the size depends on a number of variables.) When a file is migrated to multiple pieces of media through multi-target migration, then a file tag is likely to consume 1 cluster of disk space. Note: You cannot decrease the size of a file tag by removing a file from media targets. For example, if you enable multi-target migration and later remove a file from one or more media targets so that the file is on only single piece of media, the file tag size remains unchanged. Although the size of a file tag is much smaller than the size of the full file data, the file and the folder in which the file is located appear with the original file size in Windows Explorer. To view the effects of purged files on disk space, view the properties for the entire extended drive through Windows Explorer. Because space and file retrieval requirements vary, DiskXtender provides several different ways to purge files from the extended drive, enabling you to be very specific as to when files are purged from the extended drive. Files cannot be purged from the extended drive until they are moved to media and indexed, if indexing is enabled. If a file is being moved to multiple pieces of media as part of a multi-target migration strategy, then the file must also be moved to all target media before it can be purged. 222 EMC DiskXtender Microsoft Windows Version Administration Guide

215 Purging Files However, once a file is moved to media, you can choose whether to automatically purge that file from the extended drive either: Immediately after it has been moved When extended drive space is low During the next background scan (after the file qualifies for a purge rule) You can also select specific files and purge them manually by using the DiskXtender Explorer Add-ons. There may be some files that you do not want to purge at all. The purge options you choose are determined by a number of factors, including the type of storage media, how much space is on the extended drive, how often files are accessed, and how long files are normally active. Developing a purging strategy on page 225 provides recommendations for selecting a purging strategy that is appropriate for your environment. Note: Retention on files does not affect whether a file can be purged. Files under retention can be purged from the extended drive because the data resides on the media; however, you cannot delete the file tag. Prerequisites for purging Regardless of the purge method you choose, files must be moved to media and indexed, if indexing is enabled, before they can be purged. The prerequisite steps for each available purge method listed in Table 2 on page 223 are also required before files are purged. Table 2 Prerequisite steps for purging Purge method Purging based on disk space Purging immediately after migration Purging during background scans Purging manually by using the Explorer Add-ons Purging files moved to DVD-R media Prerequisite steps 1. Configure purge rules that select the files to purge, and select the Do not force purges during background scans option for the rules. 2. Adjust the purge watermarks for the extended drive to specify when purging should begin and end to maintain a reasonable amount of free space on the extended drive. Select the Purge files immediately after move option for the move rules that are used to select files for migration. 1. Configure purge rules that select the files to purge, and select the Force files to purge during background scans option for the rules. 2. Adjust the purge watermarks for the extended drive to specify when purging should begin and end to maintain a reasonable amount of free space on the extended drive. No prerequisite steps are necessary. Finalize the media by using one of the following methods: Automatically based on media group settings Manually with a Finalize media task Understanding the purge process 223

216 Purging Files How and when purging takes place Table 3 on page 224 describes how files qualify for purging and when purging takes place. Table 3 Purge process overview by purge method Purge method Purging based on disk space Purging immediately after migration Purging during background scans How files qualify for purging and when purging takes place 1. Files qualify for purging when they qualify for purge rules during a background scan. 2. Qualifying files are written to the purge list in the order determined by the priorities set in the purge rules. 3. When the amount of used space on the extended drive reaches the purge start watermark, DiskXtender begins purging files on the purge list. 4. Purging continues until the amount of used space on the extended drive is reduced to the purge stop watermark. Files qualify for purging when they qualify for a move rule with the Purge files immediately after move option selected. After the file is successfully moved to media, the file is purged. 1. Files qualify for purging when they qualify for purge rules during a background scan. 2. Qualifying files are purged during the background scan unless they also qualify for indexing. If they qualify for indexing, then the files are indexed according to the indexing schedule. Once indexing of the file is complete, the file is automatically purged, even if the indexing takes place while a background scan is inactive. Note: Files might be purged before a background scan takes place (regardless of purge rule settings) if disk space is needed before the next background scan starts. Purging manually by using the Explorer Add-ons Purging files moved to DVD-R media 1. Files qualify for purging when you issue the purge command from the Explorer Add-ons. 2. Files are purged immediately. You can choose to automatically purge all files on the media when it is finalized. If you select this option for the finalization process, then files are purged as soon as the finalization process is complete. If you do not purge all files after finalization, the files must qualify for purge rules. They are then purged either when disk space is needed or during background scans, depending on purge rule settings. Note: 1. If you are migrating files to multiple media targets and the file qualifies for purging, the file is not purged until it has been moved to all targets within the multi-target group. 2. If you are migrating files to EMC Centera in a replication environment and the file qualifies for purging, the file is not purged until it has been moved to the replica EMC Centera. 224 EMC DiskXtender Microsoft Windows Version Administration Guide

217 Purging Files Developing a purging strategy Only inactive files, that is, files that are no longer being viewed or edited by users on the extended drive, should be purged. System performance improves when fewer files must be recalled from media for viewing and editing. Recalling files can be a system-intensive process, especially with media types such as removable media and EMC Centera. DiskXtender must locate the file on the media and copy the file back to the extended drive before the file is opened. For removable media, this also means mounting the media with the file in a drive before the file can be copied. However, you must balance the need to maintain file data on the drive for viewing and editing with the need to remove file data to maintain an adequate amount of free disk space for other files. In addition, consider backup requirements as you evaluate purge strategies. To delay purging, you can opt to maintain file data on the drive for a certain period to ensure that the full file data is included in a backup. DiskXtender provides several purge options so that you can customize a purge strategy for your environment. It may be necessary to implement several purge options to achieve the optimal balance of file availability and adequate disk space. The following topics provide guidance on selecting the right combination of purge options for your environment. Recommended purge methods for different environments Table 4 on page 225 lists the recommended environments for each purge method. Table 4 Recommended environments for each purge method (page 1 of 2) Purge method Recommended environments Purging based on disk space Environments with media types that have a slower file retrieval time, such as removable media, which must be mounted in a drive Environments with EMC Centera, which has a limited number of communication threads available for total file activity High-volume environments, where a significant number of files are being moved to media and then accessed for viewing or editing Environments where DiskXtender is managing user home directories Environments where the DiskXtender Search Module is indexing files on the extended drive Purging during background scans Environments with media types that have a faster file retrieval time, such as NAS or TSM Environments where DiskXtender is being used to archive files that are infrequently or never accessed again for viewing or editing Environments where the volume of file activity is such that file migration must be scheduled to take place at a different time than file purging (background scans) Environments that need to maintain a high volume of free space on the extended drive even if it means sacrificing system performance during file retrieval Developing a purging strategy 225

218 Purging Files Table 4 Recommended environments for each purge method (page 2 of 2) Purge method Purging immediately after migration Purging manually by using the Explorer Add-ons Automatically purging files moved to DVD-R media as soon as the media is finalized Recommended environments Environments with media types that have a faster file retrieval time, such as NAS or TSM Environments where DiskXtender is being used to archive files that are infrequently or never accessed again for viewing or editing Low-volume environments where file migration can take place at the same time as file purging with a minimal impact to system performance Environments that need to maintain a high volume of free space on the extended drive even if it means sacrificing system performance during file retrieval Small environments where an administrator is responsible for manually monitoring the capacity of the extended drive and can target certain inactive files for purging Environments with DVD-R media Environments where the files on the finalized media are infrequently or never accessed again for viewing Combining purge methods in a mixed environment Use this method if your environment contains a mix of file types. Some of the files may be used on a regular basis, while others are never accessed again after they are saved to the extended drive. In such an environment, target the immediately inactive files for immediate move and purge to free up space on the extended drive, while leaving the frequently accessed files on the drive forever (never purging them) or until it is absolutely necessary to recover space on the drive. DiskXtender rules enable you to target, very specifically, how to manage files. You can select files for different move and purge methods by using the criteria in Table 5 on page 226. Table 5 Available criteria for selecting files to purge Criteria Filename File size File age File attributes Details You can specify a particular filename, which may be useful for excluding a file from being purged. You can also use the asterisk (*) symbol as a wildcard to substitute for one or more characters in a filename. For example: Specify *.* to target all files in a media folder. Specify *.doc to target all files with a.doc extension. Specify Payroll* to target all files with a filename that begins with the word Payroll, regardless of the file s extension. Specify *.tmp to target temporary files for exclusion from purging. You can target files smaller than a certain size in KB, larger than a certain size in KB, or within a range of sizes in KB. Age criteria enable you to specify the number of days that must pass since the file was created, last edited, or last accessed before DiskXtender moves or purges the file. You can target certain files with the Read-only, Archive, Compressed, Hidden, or System attributes. File attribute criteria are most effectively used to exclude files with certain attributes, such as the System attribute, from being purged. 226 EMC DiskXtender Microsoft Windows Version Administration Guide

219 Purging Files Consider the following purge strategies for mixed environments: If possible, group different file types into different media folders. This enables you to simplify the rules for each media folder. For example, save all frequently accessed files in one media folder, and save all infrequently accessed files in another media folder. Then create a single purge rule for each media folder that applies to all files in the media folder. Create purge rules to exclude certain frequently accessed files from purging. These rules are called exclusive purge rules. To create an exclusive purge rule, select the Exclude option on the Type page when creating the rule. If there are other files on the extended drive that should not be purged, such as files with the System attribute, then create exclusive purge rules to prevent the files from being purged. Select the Force files to purge during background scans option for purge rules that target large, infrequently accessed files. This enables DiskXtender to clear space on the drive by purging large, inactive files quickly. Specify an age delay on purge rules to prevent DiskXtender from purging files until they are no longer needed. To specify an age delay, select the Apply rule to files of age greater than option on the Age page for the purge rule, specify the number of days (typically 30 or 60), and then select Last write time or Last access time from the drop-down list. This sets an age delay for purging files that may be viewed or edited for a limited duration of time after they are created. Keep in mind, however, that applications, such as backup and anti-virus applications, may also access files. Create multiple purge rules to select files for purging based on extended drive space, and set the priority for the purge rules so that certain files are written to the purge list and therefore purged first. For example, create a purge rule that targets all files larger than 100 KB for purge based on extended drive space, and then create a second purge rule that targets all other files for purge based on extended drive space. Set the priority for the first purge rule to First on the Settings page of the purge rule, and the priority for the second purge rule to Second. This way, files larger than 100 KB are purged first, which frees space more quickly on the extended drive when it reaches the purge start watermark. If you create multiple purge rules in a media folder, you must order them to define how files are evaluated against the rules. To edit the priority for a purge rule, right-click the rule and select either Promote or Demote. Keep in mind the following guidelines: If a file qualifies for two different purge rules, the file is purged based on the purge rule that is listed first in the tree view of the Administrator. If the file fails to qualify for purging based on the first purge rule, the file is evaluated against the second purge rule. If the file is excluded from purging based on the first purge rule, then the file is not purged, even if it qualifies based on the second purge rule. If the file qualifies for purging based on the first purge rule, then the file is purged, even if it is excluded from purging based on the second purge rule. As a result of the last two guidelines, exclusive rules and rules with more specific file criteria should generally appear first in the list of rules. This enables exclusions and special conditions to take effect, while more general criteria in the later rules in the list can capture any remaining files. Developing a purging strategy 227

220 Purging Files IMPORTANT Even if you plan to purge files immediately after they are moved by using the move rule option, you should still configure purge rules. If you do not configure purge rules and the purged files are fetched, the files may not be purged again. Special purge situations Review the following special purge situations to determine if they must be accommodated in your purge strategy: Purging and indexing on page 228 Purging files during DVD media finalization on page 229 Purging and indexing If you use the DiskXtender Search Module to index DiskXtender files so that you can search for them, then files are purged after they are indexed. The exception to this guideline is when extended drive free space is reducing. To free up additional space on the drive when necessary, files might be purged before they are indexed. If this occurs, the files are fetched back to the drive for indexing as soon as the indexing schedule and the drive space allows: If this is a new DiskXtender installation, wait to purge files until after they have been indexed. File data must be present on the extended drive to be indexed. If a file is purged when it qualifies for an index rule, the file is fetched to the extended drive before it is written to the ITL and submitted for indexing. The file is then automatically purged again after it is indexed. In systems with a significant number of files, this process can significantly impact performance. To delay purging, wait to configure purge rules until after all files in a media folder have been indexed. Alternatively, configure an appropriate age delay on the purge rules. Do not use the Purge files immediately after move option on move rules. Ensure that files are indexed before they are purged: The best purge option for ensuring that files are indexed first is to purge files based on the purge watermarks (amount of used space) on the extended drive. If you are purging files during background scans, configure an age delay for the purge rule so that the purging takes place after indexing. If you configure an age delay on the index rules, ensure that the age delay on the purge rules is equal to or greater than the age delay on the index rules. Do not purge files immediately after they are moved (by using the move rule option for purging). You may want to create separate purge rules for files that are excluded from indexing. Excluded files are eligible to be purged as soon as disk space is needed. They do not need to be indexed first. 228 EMC DiskXtender Microsoft Windows Version Administration Guide

221 Purging Files Purging files during DVD media finalization If you use DVD-R media, then you cannot purge files until the media they are written to is finalized. The Finalize media task and the media group auto-finalization functions both enable you to select a purge files option, which tells DiskXtender to purge all files that have been written to that media at the time the media is finalized. If you do not choose to purge files as a function of finalization, the data for those files remains on the extended drive until the files qualify for configured purge rules. Then it is the selections made in the purge rules that determine when the files are purged from the extended drive. Developing a purging strategy 229

222 Purging Files Controlling user access to purged files The following topics provide details on how to recall purged files from media when necessary: Fetching files from media on page 230 Disabling file fetches on page 230 Prefetching files from media on page 230 Reading files directly from media on page 231 Fetching files from media If a user attempts to access a file on the extended drive that has been migrated to media and purged, the file is retrieved from the media and copied back to the extended drive. The user can then open the file on the extended drive. Retrieving a purged file from media is called a fetch. If a user fetches a large file that exceeds the amount of available free space on the extended drive, then this activates the purge start watermark and DiskXtender attempts to purge files on the purge list to free space on the extended drive. If DiskXtender is not able to free enough space on the extended drive, then the fetch fails. If the environment is a multi-target migration environment, purged files are fetched from the first target media group listed under the multi-target group. You can change the media priority from which DiskXtender fetches the file by changing the order in which the standard media groups are listed under the multi-target group. When purge rules are configured, fetched files are qualified against the purge rules again during the next background scan, and purged again as appropriate based on purge rule settings. Disabling file fetches If necessary, you can prevent users from fetching files from media or directly reading files from media by disabling the file fetch schedule for an extended drive. You may want to disable fetches, for example, when you have to perform system maintenance. In this scenario, you may want to prefetch certain files from media. Prefetching files from media on page 230 provides details. Note: When you disable the file fetch schedule, you prevent only client fetches from occurring. Prefetch media tasks and prefetch requests continue. Prefetching files from media To reduce read requests from media during high traffic times, you can anticipate file retrieval needs and prefetch frequently used files. During a prefetch, DiskXtender retrieves files from storage media and writes the file data to the extended drive. DiskXtender provides two ways to prefetch files: If you know that you will need specific files, select the files by setting up a prefetch request through the Prefetch Request Wizard. Prefetch requests on page 231 provides details. 230 EMC DiskXtender Microsoft Windows Version Administration Guide

223 Purging Files To temporarily remove a piece of media from a device (for example, to perform maintenance), prefetch all files from the piece of media by assigning a Prefetch media task to the media. Prefetched files, regardless of method, remain in a migrated state and continue to be subject to purge rules, as with any other normally migrated files. Also, if you remove the media containing the files from the media group, DiskXtender removes the file data from the extended drive as it normally would, even though the files are prefetched. If you prefetch a file marked for Direct Read, the file is fetched and the Direct Read attribute is removed from the file. Depending on how you choose to mark files for Direct Read, the file might be marked for Direct Read again when it is purged. Reading files directly from media on page 231 provides details on Direct Read. Prefetch requests To select specific files for retrieval from storage media, you can set up either a direct or indirect prefetch request: Create a direct prefetch request to browse to and select certain files and folders. You can also select files for a direct request by using a text file that lists specific files, and save the list of files you select to a text file for use again later. Note: If the contents of a file list loaded for a direct request changes, DiskXtender does not register those changes for resubmission. For this reason, direct requests should not be used for recurring prefetches where file lists are regenerated each time. Instead, save the recurring file list for use later in an indirect prefetch request. Create an indirect prefetch request to load an existing file list. This option is intended for re-occurring batch prefetches where the list of files being prefetched might change. You designate the name of the file containing the file list, and then rebuild that list if needed. With an indirect request, DiskXtender registers only the name of the file that contains the list of files to be prefetched. As long as that list is saved under the specified filename, the contents of the list can change as often as required. Prefetch media task To prefetch all files from a certain piece of media, you can assign a Prefetch media task. This task is particularly useful if you know a specific piece of media will not be available for file retrieval for an extended period of time. It makes it possible to set the media offline and physically remove the media from the system without the risk that it might be needed to provide file data for client requests. Reading files directly from media In some situations, you may want to read purged files directly from media, rather than copying (fetching) the files back to the extended drive and opening them there. This behavior is called Direct Read in DiskXtender. Direct Read is strongly discouraged in most DiskXtender environments. In select situations, Direct Read can ease the strain on system resources caused by fetching certain files back to the extended drive. However, Direct Read requests can be as much as three times slower than normal fetch requests. Direct Read is not intended to preserve the amount of free space on the extended drive. When planning a file migration strategy, you must consider the amount of space required for files that will be fetched from media in addition to new files that are written to the extended drive. Use file purging features instead of Direct Read to manage extended drive free space. Controlling user access to purged files 231

224 Purging Files When to enable Direct Read Direct Read can be beneficial in archive environments with the following types of files: Large files that are accessed through applications that employ Byte Offset Read Note: Byte Offset Read enables an application to read a very small portion of a file rather than reading the entire file. Files with fixed content (read-only files, or files that are not edited once they are created) that are not accessed frequently Files that are permanently archived and are not accessed frequently When to avoid Direct Read Direct Read should not be used in environments where the extended drive is used as a file server for active and frequently accessed files. Avoid Direct Read with the following types of files: Files that are accessed frequently, such as files in user home directories or database index files. Files that are accessed through applications that must open the entire file. Also avoid Direct Read in the following environments: A significant number of users (more than 500) are accessing the same or different files. Files are moved to a slow media type, such as tape. Bottlenecks and media damage can occur if DiskXtender must repeatedly mount media to read files. Files are moved to multiple media groups (multi-target migration). If you attempt to open a file that is marked for Direct Read and the first target in a multi-target media group is unavailable, then the operation fails. DiskXtender does not open the file from the other targets. Setting the Direct Read memory cache If you enable Direct Read, the file data for Direct Read files must be fetched to a temporary cache for distribution to the requesting client. The Direct read memory cache option determines how much space to allow for that temporary cache. Calculating the optimal size of the cache Each open file consumes at least 64 KB of this cache. Configure the size of the cache based on the expected number of concurrently open files on the extended drive. The default value for this setting is 1 MB, which handles 16 concurrently open files (64 KB x 16 = 1 MB). The Direct Read cache should never exceed more than 10 percent of the available physical memory on the system. 232 EMC DiskXtender Microsoft Windows Version Administration Guide

225 Purging Files Direct Read exceptions There are some exception situations in which a file marked for Direct Read might be fetched anyway, or when the Direct Read attribute might be automatically removed from a file: When a file marked for Direct Read (that is not read-only) is opened with a write access application such as Microsoft Word, the file is fetched to the extended drive instead of being read directly from the media. DiskXtender cannot read a file with streams directly from the media. For this reason, when a streamed file marked for Direct Read is requested, the file is fetched and the Direct Read attribute is removed from the file. If you prefetch a file marked for Direct Read, the file is fetched and the Direct Read attribute is removed from the file. Depending on how you choose to mark files for Direct Read, the file might be marked for Direct Read again when it is purged. Controlling user access to purged files 233

226 Purging Files Controlling third-party software access to purged files You can configure DiskXtender to force a type of data access for purged files when those files are called by specific software applications. For example, you might not want anti-virus software to recall purged file data from media during a virus scan. The DiskXtender special application filtering option enables you to specify applications and whether DiskXtender should allow the application to read purged files directly from media or to ignore purged files altogether. Note: The special application filtering option applies only to applications that run on the DiskXtender server. Contact EMC Customer Service by using EMC Online Support for details on the limitations and special situation configurations for special application filtering. Considerations for special application filtering Here are some important considerations while working with the special applications list: The four TSM Backup executables in the Applications list (dsm.exe, dsmc.exe, dsmcsvc.exe, and damagent.exe) are set to Direct Read by default and should remain that way. If you use TSM as a media service, changing the action for these four files to No Recall can result in data loss. Deleting default applications set by DiskXtender might cause system problems, including deadlock, due to the application and DiskXtender attempting to run conflicting processes at the same time. However, if you know the listed application is not being used on the DiskXtender server, deleting the application might improve performance. When the list is reset, all of the applications you added since DiskXtender installation are removed, all of the applications you deleted are reinstated with their original settings, and all of the applications you changed are reset to their original values. A list of common anti-virus, backup, and security executables and the recommended action (either No recall or Direct read) is provided, including: AVX Anti-Virus Dr. Solomon s Anti-Virus F-Secure Anti-Virus McAfee Anti-Virus Microsoft Forefront Norton Anti-Virus Trend Micro Anti-Virus Panda Anti-Virus Tivoli Backup 234 EMC DiskXtender Microsoft Windows Version Administration Guide

227 13 Deleting Files The following topics discuss when and how you can delete a file from the extended drive, and if necessary, how you can restore the file: Deleting files from the extended drive Deleting files from media Recovering deleted files Deleting Files 235

228 Deleting Files Deleting files from the extended drive In certain situations, you might not want to keep files on the extended drive after a period of time. There are two ways to delete files from the extended drive: Manually (for example, through Microsoft Windows Explorer) Automatically, by using DiskXtender delete rules Manually deleting files In most cases, you can manually delete files from the extended drive by using Microsoft Windows Explorer. The only situation in which you cannot delete files is if the files have been moved to EMC Centera or Retained NAS media and retention is set. However, you may be able to delete even retained files if you have an EMC Centera Basic or Governance Edition device and you perform a privileged delete. If you enable the DiskXtender Recycler, files that are deleted manually are placed in the Recycler. You can then delete the files permanently or restore them to the extended drive, if necessary. If you delete a file from the Recycler and the file has been migrated to media, DiskXtender attempts to also delete the file from media. If you leave the DiskXtender Recycler disabled and you delete files from the extended drive, DiskXtender attempts to delete the file from media (if the file has been migrated to media). Files deleted through privileged deletes are not placed in the Recycler. Instead, they are deleted permanently. Automatically deleting files DiskXtender can automatically delete files from the extended drive based on certain file criteria by using delete rules. When a file qualifies for a delete rule, DiskXtender deletes the file from the extended drive and attempts to delete the file from media (if the file has been migrated to media). Files that qualify for delete rules are not placed in the Recycler. Instead, they are deleted permanently. Delete rules are processed and therefore, files are deleted automatically during background scans. Logging delete transactions provides details on logging delete transactions. Developing an automatic deletion strategy Delete rules can be used, for example, to help manage archival of particular files kept to comply with legal requirements. You could create a delete rule that deletes all files in a particular directory when you are no longer legally required to keep them. After the files no longer meet the required criteria (for example, the files reach a certain age), DiskXtender can automatically delete them in accordance with a delete rule. If you use EMC Centera or Retained NAS media, you can use delete rules and retention periods to keep files protected under retention for a period of time, then automatically delete them when they are no longer needed. 236 EMC DiskXtender Microsoft Windows Version Administration Guide

229 Deleting Files Never extend a system drive or a drive that contains application files. However, if there are system or application files on the extended drive, whose system attributes is set are excluded by default. This prevents the application and system files from being deleted. You can view file attributes through Microsoft Windows Explorer. Note: To clear space on the extended drive and yet leaving them accessible on the drive, purge the files instead of deleting them. Understanding delete rules Delete rules enable you to specify which files should and should not be deleted by DiskXtender. Delete rules do not, however, prevent users from manually deleting files. To prevent users from deleting files, you must use file retention. Chapter 10, File Retention, provides details. You can configure delete rules for each media folder. The criteria you can use to select files for deletion or exclusion from deletion include file location, name, type, size, attributes, and age. Developing an automatic deletion strategy on page 120 provides ideas for when to use delete rules. Logging delete transactions When a file is deleted from the extended drive, the delete transaction is recorded in the delete audit log for the extended drive. The log file enables you to troubleshoot unexpected file deletions. You can determine whether a file was deleted by a user or by the DiskXtender system, as well as whether the file was placed in the Recycler or permanently deleted. The audit log file, delete.log, is created and updated in the DxLogs\Transaction logs folder on the extended drive. To view the log, you must be a member of either the DxSecurityAdmins or the DxMonitors group on the DiskXtender server. Stop the DiskXtender service, and then use a text editor such as Notepad to open the file. The maximum size of the log file is 100 MB. When the log file reaches this size, 25 percent of the oldest entries in the log file are trimmed to reduce the size of the file. In the unlikely event that there is an insufficient amount of free disk space on the extended drive and DiskXtender cannot add entries to the delete log, a warning appears in the Event Viewer and an alert is sent, if alerts are configured for warnings. Delete logging is suspended until free space is recovered on the extended drive. You must manually restart logging delete transactions by using the Log file deletes setting on the Options tab of the Extended Drive Properties dialog box. Prevent delete from media By enabling the Prevent delete from media option, you can prevent delete transactions performed on migrated files on an extended drive, from being propagated to back end media. This is an Advanced Mode option and, by default, this option is set to No. Set this option to Yes to make sure that when a migrated file is deleted from the extended drive, only the file stub is deleted from the extended drive, and the file continues to reside on the media. Disable this option when you want to ensure that files deleted from an extended drive are deleted from the media too. Deleting files from the extended drive 237

230 Deleting Files This option is available for EMC Centera, WORM, and TSM media which support delete operation. This option is not available for multi-target media groups. If EMC Centera, WORM or TSM media which has the Prevent delete from media option enabled, is moved to be a part of the multi-target media, then the option will be automatically unavailable. Upon moving the media out of the multi-target group, the option has to be enabled again. When this option is enabled, delete transactions on files in the DiskXtender Recycler are not propagated to the media to which the file has been migrated to. With Prevent delete from media option enabled, deleting files from the Recycler will only delete the files from the Recycler, but the files continue to reside on the media. Such files can be restored from the media to the extended drive. Administrators should use this option judiciously. Leaving this option enabled might mean that even files that are not required and have been deleted from the extended drive will continue to occupy space on the media. Usage of this option should be reviewed from time to time. Retrieving files that are not deleted from the media When the Prevent delete from media option is enabled, files that are deleted from the extended drive, are not deleted from the backend media. These files can be retrieved from the media using these options: File restore - Use this media task to retrieve the file from the backend media to the extended drive. In case of a scenario where multiple files with same name have been deleted (after enabling Prevent Delete From Media), File Restore media task restores the latest file to the extended drive. The older files can be obtained from the Recycler (if Recycler is enabled). To recover files from Recycler Bin see Recovering deleted files on page 124. You can also retrieve files using stubs backed up before deletion. Restore the stubs from the back up to the extended drive, and double-click the stub to access a file. 238 EMC DiskXtender Microsoft Windows Version Administration Guide

231 Deleting Files Deleting files from media If a file has been migrated to media, DiskXtender attempts to delete the file from media when one of the following situations occurs: The DiskXtender Recycler is disabled and a file is deleted manually from the extended drive. The DiskXtender Recycler is enabled, the file is in the Recycler, and you empty or delete the file from the Recycler. The file qualifies for a delete rule during a background scan. DiskXtender can successfully delete files from the following media types: Standard and Aggregate NAS DVD-RAM, MO, MO WORM, UDO, and UDO WORM media formatted with the UDF file system EMC Centera with no retention or expired retention Note: If you have an EMC Centera Basic or Governance Edition device and you set retention, you can delete retained files by using a privileged delete. TSM With the media types in the following list, however, DiskXtender cannot delete the file from media: DVD-R and DVD-ROM DVD-RAM, MO, MO WORM, UDO, and UDO WORM media formatted with the OTG file system Tape and tape-worm Instead, the file is marked for deletion. It still exists on the media, but it becomes an orphaned file and is no longer recognized or tracked by DiskXtender. The deleted files are not actually removed from the media until the media is reformatted. IMPORTANT When a significant number of delete transactions are sent to some types of media, including EMC Centera or removable media libraries, the completion of those transactions may take a long time. During that time, file fetches may be delayed or even canceled due to timeout issues. Do not empty the DiskXtender Recycler when users are most likely to access the system. Or, if the Recycler is disabled, do not delete a significant number of files from the extended drive when users are most likely to access the system. Deleting files from media 239

232 Deleting Files Recovering deleted files The following topics discuss the options for recovering a file that has been accidentally deleted from the extended drive: Recovering files from the DiskXtender Recycler on page 124 Recovering files from a backup on page 125 Recovering files from the DiskXtender Recycler If the DiskXtender Recycler is enabled and a user accidentally deletes a file, you can restore the file to the extended drive. The restored file retains all of its migration information, if it was migrated to media. You do not need to re-migrate the restored file to media. The DiskXtender Recycler functions similar to the Microsoft Windows Recycle Bin, but only affects files on a drive extended through DiskXtender. When the deletion occurs, the file is placed in the Recycler instead of being permanently deleted. This occurs for files on the extended drive that are managed by DiskXtender (files that have qualified for movement to media) and files that are not managed by DiskXtender (files that do not qualify for movement to media). There is one DiskXtender Recycler for each extended drive. The Recycler for each extended drive appears under the extended drive node in the left pane (tree view) of the File System Manager Administrator. Note: You cannot rename a file or folder on the extended drive if there is a file or folder with the same name in the Recycler. An error message appears to notify you that the rename could not proceed. In addition, if the filepath for a file on the extended drive exceeds 259 characters and the Recycler is enabled, then you cannot delete the file. This is because DiskXtender cannot manage files with filepaths that exceed 259 characters. You must shorten the filepath or disable the Recycler in order to delete the file. IMPORTANT The Recycler is not a substitute for regular backups of the extended drive. It is intended only for short-term storage of deleted files to provide rapid recovery when an accidental deletion takes place. As a result, the Recycler is located on the extended drive, and it factors into the total amount of space on the extended drive. To avoid filling the drive with deleted files, empty the Recycler on a regular basis. You can schedule the Recycler to empty automatically, or you can manually empty the Recycler as needed. The following topics provide details on the DiskXtender Recycler. Excluding files from the Recycler If the Recycler is enabled, you can exclude certain files from being placed in the Recycler when they are deleted from the extended drive. Instead, the files are deleted permanently when they are deleted from the extended drive. These exclusions are intended to save space on the extended drive by preventing files from being placed in the Recycler when you are certain that they will never need to be restored. 240 EMC DiskXtender Microsoft Windows Version Administration Guide

233 Deleting Files You can exclude files based on the location and name of the file. Certain files, like Windows and shell temporary files, database lock files, and utility files, are excluded by default. In addition, you can specify whether files of a certain migration status (migrated, purged, or fetched) are excluded from the Recycler. Files that are automatically excluded from the Recycler By default, files deleted according to delete rules or by using EMC Centera privileged deletes are excluded from the Recycler. The files are permanently deleted. In addition, files with a filepath greater than 259 characters are excluded from the Recycler. You cannot delete the files. Certain files, like Windows and shell temporary files, database lock files, and utility files, are also excluded by default. Emptying the Recycler Because the DiskXtender Recycler resides on the extended drive, the disk space used by the Recycler factors into the total available disk space for the extended drive. Empty the Recycler on a regular basis to help maintain an adequate amount of space on the extended drive. When you empty the Recycler, the files in the Recycler are permanently removed from DiskXtender. In other words, the files are deleted from any media to which they have been written, if the media supports file deletions. Deleting files from media on page 123 provides more information. You can empty the Recycler either manually when needed or on a scheduled basis. If you schedule the Recycler to be emptied, you can choose whether to empty the entire Recycler or to remove only files that meet certain age criteria, such as files that were deleted more than seven days ago. You can also empty the Recycler on a file-by-file basis by deleting individual files from the Recycler. However, when the Prevent delete from media option is enabled, delete transactions on files in the DiskXtender Recycler are not propagated to the media to which the file has been migrated to. With this option enabled, deleting files from the Recycler will only delete the files from the Recycler, but the files continue to reside on the media. Such files can be restored from the media to the extended drive. Deleting files from the Recycler You can permanently delete a select file or group of files in the Recycler from DiskXtender. When you delete a file from the Recycler, the file is also deleted from any media to which the file has been written, if the media supports file deletions. Deleting files from media on page 123 provides more information. Restoring files from the Recycler If the DiskXtender Recycler is enabled and a file is deleted from the extended drive, you can restore the file to the extended drive. Recovering files from a backup When files on the extended drive are accidentally deleted, you may be able to restore the files from a backup of the extended drive, as discussed in Recovering from a backup on page 171. Recovering deleted files 241

234 Deleting Files Files should be restored to the same location from which they were deleted. Do not restore files to a different location. The type of media to which the files are migrated determines whether it is necessary to restore the file tag or the full file with all of its data. Table 16 on page 126 lists the media types that require recovery of the file tag or the full file. Table 1 File recovery requirements by media type Media types that allow recovery of the file tag DVD-R DVD-RAM formatted with the OTG file system DVD-ROM MO, MO WORM, UDO, and UDO WORM formatted with the OTG file system Tape Tape-WORM Media types that require recovery of the full file DVD-RAM formatted with the UDF file system EMC Centera without retention or after retention expires MO, MO WORM, UDO, and UDO WORM formatted with the UDF file system Standard and Aggregate NAS TSM These requirements are necessary because the type of media determines whether a file can be deleted from the media when it is deleted from the extended drive. Files that have been migrated to the media types in the first column of Table 16 on page 126 cannot be deleted from the media when they are deleted from the extended drive. In that scenario, the file remains on the media until the media is compacted and reformatted. Since the file remains on the media, you can restore the file tag from the backup, and the connection between the file tag on the extended drive and the file on the media is restored, as long as the backup software supports EAs. If a file is deleted from the media when it is deleted from the extended drive (which happens with the media types in the second column of Table 16 on page 126), then you must restore the full file with all of its data to the extended drive. The file can then be re-migrated to media and purged, if necessary. If you use a media type that requires recovery of the full file, then you must ensure that there is at least one backup set with a copy of the full file. In addition, you must be able to easily identify which backup set contains the copy of the full file. Backup sets from backup software that supports EAs (which includes most backup software) are likely to contain file tags for some files and the full file data for other files. This can make the process of tracking the backup location of the full data for each file quite complex. As a result, the use of the DiskXtender Recycler is recommended for restoring accidentally deleted files from the media types in the second column of Table 16 on page 126. IMPORTANT If the DiskXtender Recycler is enabled, do not restore a file from a backup set unless you are certain that the file is not in the DiskXtender Recycler and that the file in the backup set is a full version of the file (not a file tag). Data loss can occur when you restore a file tag from a backup set but the file data is no longer on the media. 242 EMC DiskXtender Microsoft Windows Version Administration Guide

235 14 Service Management Because DiskXtender functions as a Microsoft Windows service, part of administering the DiskXtender server includes administering the service. The following topics provide details: Understanding the DiskXtender service Service Management 243

236 Service Management Understanding the DiskXtender service DiskXtender functions as a Microsoft Windows service rather than as a user-mode application. As such, the DiskXtender service can continue to be active even after you log off Microsoft Windows, as long as the server is still running. IMPORTANT Because the service continues to run as long as the server is running, if you are planning to shut down the DiskXtender server, stop the DiskXtender service before doing so. This enables the service to complete whatever function it was performing before the system shuts down. When DiskXtender is writing a file to media, stopping the service before shutting down the server reduces the possibility of file write errors or media errors occurring during shutdown. Managing the service If necessary, you can pause, stop, and restart the DiskXtender service. You can also configure the service to start automatically upon the startup of the operating system. You can stop, start, and edit automatic startup for the service by using either the File System Manager Administrator or the Services and Applications option in the Microsoft Windows Administrative Tools console. The DiskXtender service is called EMC DiskXtender in the Microsoft Windows Services and Applications list. You must be a member of both the local Administrators group and the DxAdministrators group on the DiskXtender server to manage the service, including starting and stopping the service, editing startup settings, and editing the service account. Managing the service in a high availability environment If DiskXtender is installed in a clustered environment, the DiskXtender service should be stopped, paused, or started by using only the Microsoft Cluster Administrator. Do not manage services through the Service Manager or the Microsoft Windows Administrative Tools. If you stop the service in the Administrator or through the Administrative Tools, the system detects that the service failed and fails over to the other node in the cluster. 244 EMC DiskXtender Microsoft Windows Version Administration Guide

237 15 Monitoring the System DiskXtender provides several functions that enable you to monitor, diagnose, and troubleshoot the system. The following topics provide more information: Monitoring service events, errors, and warnings Monitoring file activity Monitoring media Monitoring licensing Monitoring performance Reports Audit logs Monitoring the System 245

238 Monitoring the System Monitoring service events, errors, and warnings DiskXtender provides built-in utilities for monitoring service events, errors, and warnings. The Event Viewer contains a listing of all DiskXtender events, errors, and warnings. This information is also logged to event logs. Errors and warnings are logged automatically, while DiskXtender must be configured to log other events. The Event Viewer and event logs provide a quick look at DiskXtender activities. Logs can help identify and solve potential problems during runtime that might otherwise become critical problems if ignored. If the Event Viewer and event logs become too large and cumbersome to navigate, you can clear them. A visual indicator appears in the Administrator window when a warning or error occurs. The Event Viewer icon on the toolbar changes from an informational icon to a triangular caution symbol to indicate a warning or an exclamation point to indicate an error, as illustrated in Table 1 on page 246. Table 1 Event Viewer icons for events, warnings, and errors Event Viewer toolbar icon Toolbar icon for warnings in the Event Viewer Toolbar icon for errors in the Event Viewer In addition, the number of errors and warnings since the service was last started is listed in the status bar at the bottom of the Administrator. For further notification, you can configure automatic communication of warnings and errors through alerts. Event Viewer The Event Viewer displays all events for the extended drive. If you do not want to automatically save a record of the events on the extended drive, you can disable event logging in the Service Event Configuration dialog box and monitor events in the Event Viewer. Configuring event logging on page 248 provides more information on disabling event logging. Event logs In addition to the viewing capabilities offered by the Event Viewer, there are three DiskXtender event logs: The All Events Log lists all activities for the DiskXtender service. Event logging is useful to trace service events of interest to you. You can configure which events to trace. The Errors Only Log lists only DiskXtender system errors and is a useful tool for detecting and diagnosing those errors. 246 EMC DiskXtender Microsoft Windows Version Administration Guide

239 Monitoring the System Note: You can use the System Error Lookup feature to obtain additional information about each error, including the error name and description, and to quickly translate error codes that appear in DiskXtender. Looking up errors on page 248 provides more information. The Warnings Only Log lists only DiskXtender system warnings to warn you of possible problems, such as: Requests for media (either to read or write to a file) that could not be satisfied because the media was not present in DiskXtender (not in any drive or library) Files that could not be migrated because media was full or because there were no more media available in a media group Disk full warnings for the NTFS or ReFS volume on the extended drive The purpose of this list is to provide a simpler means of discovering problems that are hindering DiskXtender performance. Errors and warnings also appear in the Event Viewer, but the Errors Only Log and Warnings Only Log provide a more precise focus on these important events. You can also configure errors and warnings to be sent out automatically as alerts. Sending alerts for errors and warnings on page 247 provides more information. Because they are necessary for system diagnostics, error and warning logging cannot be disabled. If technical support is needed, you might be asked to open these logs and specify the information reported. When opened, the logs slow down DiskXtender performance. When closed, the impact they have on performance is insignificant. The logs should be opened and used only for debugging purposes. Note: Because tracing events impedes system performance, no events are traced by default. Errors and warnings are logged automatically because of their necessity for troubleshooting. Configuring event logging on page 248 provides more information. To view a log, open the Service menu, select Event Logs, and then choose the log to view (All Events, Warnings Only, or Errors Only). The selected log appears in RtfPad. The log displays both the time and a message for each event. The most recent events are last on the list. (The list automatically scrolls the event viewer to display the most current events.) You can save, print, and the log from RtfPad. Sending alerts for errors and warnings You can configure DiskXtender to automatically send an alert to a specific address when a warning or error occurs. By default, DiskXtender does not send alerts. You can add, edit, and delete alerts for specific addresses. In addition to the automatic alerts that DiskXtender issues, there are two configurable alerts: Monitoring service events, errors, and warnings 247

240 Monitoring the System Extended drive free space You can configure DiskXtender to send an alert when the amount of free space on the extended drive falls below a specified threshold (in megabytes). The threshold is configured by using the Warn when free space is getting low option on the Options tab of the Extended Drive Properties dialog box. The default is 10 MB. When you receive this alert, you can take the necessary action to purge or delete files from the extended drive before it runs out of space. Media group free space You can configure DiskXtender to send an alert when the amount of free space in a media group falls below a specified threshold (in megabytes). The threshold is configured by using the Warn when group free space falls below option on the Options tab of the Media Group Properties dialog box. This alert is disabled by default. If you use automated media preparation features, such as automated EMC Centera media creation or automated labeling for removable media, you would not need this alert. In this case, DiskXtender can add media to the media group automatically as necessary. However, if you do not use automated media preparation, enable this alert to ensure that you are notified when it is necessary to add media to the media group so that DiskXtender has a ready supply of media available for file migration. Looking up errors Each DiskXtender error is displayed as an error number. The error lookup feature enables you to obtain additional information about the error, including the error name and description, and to quickly translate the error codes provided in DiskXtender messages. You can look up errors from either the DiskXtender Administrator or from the event log in RtfPad. Configuring event logging Event logging is necessary only when tracing events of interest to you. You can configure which events are traced. Because tracing events impedes system performance, no events are traced by default, though errors and warnings are logged automatically because of their necessity for troubleshooting. You can configure which events are logged, and whether to trace the local service events or remote procedure calls (remote administration events). For all logs, you can control the format of the log entries and adjust the maximum sizes for the log files. For the All Events Log, you can disable logging and enable the tracing of events upon startup. You can enable logging to the Windows Application Log for the Warnings Only Log and the Errors Only Log. Event tracing configuration You can configure which events are traced and reported to the Event Viewer and the Events Log. By default, all event tracing is disabled. Unless you are instructed to enable tracing of events by an EMC Customer Support Representative, you can leave the default settings in place. Enabling event tracing hinders performance and should be performed only when debugging is required. Note: Event tracing configuration affects events only; errors and warnings relating to events of the types listed are logged regardless of event tracing settings. 248 EMC DiskXtender Microsoft Windows Version Administration Guide

241 Monitoring the System Clearing the Event Viewer and event logs Event logs have a specific size limitation ranging from 1MB to 4095MB. The default size is 5MB. While the log size is regulated through automatic truncation ( Configuring event logging on page 248 provides more information), large logs are often cumbersome and difficult to navigate. You can manually clear event logs regularly to make it easier to find new information. Note: Before clearing the logs, you can retain historic events by saving the logs into RtfPad. You can also backup all the event log files using any of the native backup utilities that the Windows OS provides. Monitoring service events, errors, and warnings 249

242 Monitoring the System Monitoring file activity You can monitor statistics for the number of files pending migration, purge, and indexing. You can also monitor statistics indicating how many files have already been migrated, purged, deleted, and submitted for indexing. These statistics enable you to verify that these file activities are occurring. Note: Indexing statistics are available only if you use the DiskXtender Search Module and you have enabled indexing for the extended drive. Monitoring pending file activity Monitor the following locations for statistics on the number of files pending migration, purge, and indexing: Background Scan Properties, Statistics tab Provides the number of files qualifying for move rules and index rules during the last background scan. Description view (bottom right pane) of the File System Manager Administrator when an extended drive is selected Provides the number of files pending migration and purge for the extended drive. This is a count of the number of files currently on the move and purge lists. Note: Because the move list might contain multiple entries for any given file, the actual count for files queued for a move represents an approximation rather than an exact count. (Duplicate entries are discarded when moves are processed.) The statistics are intended to provide a window into DiskXtender system activity to indicate that file migration and purging is occurring. They are not intended to provide an exact count of files awaiting migration or purging. These statistics enable you to verify that files are qualifying for configured rules. You can also verify that files are qualifying for configured index rules by ensuring that the ITL file is being updated. To check for updates, monitor the date on the file. The ITL for an extended drive is located in the \DxLogs\Transaction Logs folder on the extended drive. 250 EMC DiskXtender Microsoft Windows Version Administration Guide

243 Monitoring the System Monitoring completed file activity The following topics discuss how to monitor completed file activity: Viewing summary statistics for completed file activity on page 251 Determining the status of a file on page 251 Viewing summary statistics for completed file activity Monitor the following locations for statistics on the number of files successfully migrated, purged, deleted, and submitted for indexing: Background Scan Properties, Statistics tab Provides the number of files purged and deleted during the last background scan. Extended Drive Properties, Statistics tab Provides the total number and size in bytes of files fetched, migrated, purged, submitted for indexing, and edited or deleted on the extended drive since the service was restarted or since the counts were manually reset. Media Group Properties, Statistics tab Provides the total number and size in bytes of files migrated to the media group since the service was restarted or since the counts were manually reset. These statistics enable you to verify that file migration, purging, and deletion is occurring after files qualify for configured rules, and that files are being submitted to the ISE for indexing. Determining the status of a file To determine the status of a file, run a DiskXtender file report. What is included in a file report A DiskXtender file report provides details on the status of each file selected, including the following information: Whether a file has been moved to media, and if so, the name of the media on which the file resides Whether the file is resident on the extended drive or if it has been purged Whether the file is marked for Direct Read Whether the file has been indexed (detailed file reports only) The retention setting applied to the file, if any (detailed file reports only) Tracking indexing activity To monitor that files are being submitted to the ISE for indexing and that indexing is taking place: Select the extended drive in the tree view of the File System Manager Administrator. The number of files that have been indexed is listed in the bottom right pane of the window. Note: The statistic for indexed files is reset each time the DiskXtender service is restarted. Check the DiskXtender Event Viewer. In the File System Manager Administrator, open the Service menu and select Event Viewer. Monitoring file activity 251

244 Monitoring the System Events that indicate that indexing is successfully occurring appear similar to the following examples: MSG 01/17 16:57:05 (0DA0) Index synchronization for extended drive D: has started. MSG 01/17 16:57:05 (0DA0) Index synchronization for extended drive D: has completed successfully. Note: To enable more detailed tracing of content indexing events on the DiskXtender server, select the Content Indexing option on the Trace tab of the Service Event Configuration dialog box. (From the Service menu in the File System Manager Administrator, select Event Settings.) Keep in mind, however, that enabling event tracing can impact system performance. Check the index root directory on the ISE server, and ensure that index files are being created and updated. Check the ISE event log. Use a text viewer to open the ISE000number.log file in the C:\Program Files\EMC\Index and Search\logs directory on the ISE server. (C:\Program Files\EMC\Index and Search is the DiskXtender Search Module installation directory, and number is a number such as 1, 2, 3, and so on. Open the log file with the highest number. This is the log file with the most recent events.) To monitor whether specific files have been successfully indexed, run a detailed file report or view file properties by using the DiskXtender Explorer Add-ons. Files that have been successfully indexed are marked with the Indexed attribute. Files that have qualified for indexing and been submitted to the ISE for indexing, but that the ISE cannot index for example, because the file is not a supported file type are marked with the Do Not Index attribute. Files that qualify for exclusive index rules, as well as files that have not qualified for any index rules at all, are marked with neither attribute. (In other words, no content indexing attribute is assigned to these files.) 252 EMC DiskXtender Microsoft Windows Version Administration Guide

245 Monitoring the System Monitoring media The following topics provide information on the DiskXtender options available for monitoring media and clearing errors when they occur: Media status indicators on page 253 Media task queue on page 253 Media group free space on page 254 Media logs on page 255 Media properties on page 255 Clearing media error status on page 256 Media status indicators The label for each piece of media automatically changes color in the tree view of the Administrator depending on its status, as listed in Table 2 on page 253. Table 2 Media status indicated by color Media color Black Green Blue Red Media status Online Online but with a task pending, in progress, or suspended Full Error Note: Clearing media error status on page 256 provides instructions for clearing media errors. Yellow Offline You can change tree view so that all media appears with a black label. To disable media status indicators, open the View menu and select Enable Color. This clears the checkmark next to the option. Media task queue If you assign media tasks to multiple pieces of media, you can monitor the status of the tasks with the Media Task Queue Manager. Note: Media tasks are assigned to media either individually, by using a right-click menu option, or in groups, by using features such as the Media Prepare Manager and the Copy Media Manager. Monitoring media 253

246 Monitoring the System You can change the processing order of the media in the queue by using the arrow buttons on the right side of the Media Task Queue Manager dialog box, which are selected in Figure 1 on page 254. Figure 1 Promote/demote arrow buttons Because all assigned tasks for a piece of media are processed together, moving a piece of media in the queue also moves all tasks assigned to that media. If necessary, you can stop, or abort, a media task that is currently in progress. To abort pending or suspended media tasks, use the Edit Tasks option and delete the appropriate tasks from the media. You can abort all media tasks with the exception of the Format, Finalize, Label, and Label Copy tasks. If the aborted task remains in the task queue as a failed task, clear the error status to process the remaining tasks (if any) for that piece of media. Clearing media error status on page 256 provides instructions. You can also remove pending tasks for a piece of media from the media task queue. When all tasks are deleted from a piece of media, the media is removed from the media task queue. Note: Because you cannot delete a task if it is already in progress, you must first abort any tasks that are currently processing before deleting them. Media group free space Automated media preparation features enable you to set up a variety of automated media functions and eliminate a time-consuming requirement of system administration: the need to manage media groups and the available space within them by creating, preparing, and/or compacting media when necessary. These features include automated EMC Centera media creation and automated labeling for removable media: EMC Centera automation enables you to automatically create virtual EMC Centera media and add it to the media group as space is needed in the group. 254 EMC DiskXtender Microsoft Windows Version Administration Guide

247 Monitoring the System If you use the MediaStor or StorageTek ACSLS media services, removable media automation enables you to automatically label and compact media as space is needed in the group. This enables you to load and format media (if applicable), and let other media maintenance activity be triggered by internal DiskXtender events. However, if you do not use automated media preparation, you can configure DiskXtender to send an alert when the amount of free space in the media group falls below a specified threshold (in megabytes). The threshold is configured by using the Warn when group free space falls below option on the Options tab of the Media Group Properties dialog box. When you enable this alert, the system notifies you when it is necessary to add media to the media group so that DiskXtender has a ready supply of media available for file migration. Sending alerts for errors and warnings on page 247 provides additional details on alerts. Media logs DiskXtender maintains a log of activity for each piece of media. These logs are useful for viewing task processing information and error codes/status for failed tasks. In addition, if a media task is suspended, an entry in the media log notes when the task will be retried. Media properties The Media Properties dialog box enables you to view detailed information about a piece of media, including the type, file system, location, number of files, amount of free and used space, number of file reads, writes, and errors, and the status of a media task, if one is in progress. To view media properties, double-click the piece of media. The Media Properties dialog box appears, with the General tab selected, as illustrated in Figure 2 on page 255. Figure 2 Media Properties dialog box Monitoring media 255

248 Monitoring the System Depending on the type of media and its status, the tabs in Table 3 on page 256 are available on the Media Properties dialog box. Table 3 Media Properties dialog box tabs Tab name General Location Space Statistics Progress Description Provides identifying information for the media, such as the name, serial number, type, file system, and status of the media. For original, copy, and duplicate media, you can use this tab to provide a brief description of the media. This description is then used for media reports. You can also use the General tab to manually mark the piece of media as full. When media is marked as full, DiskXtender no longer writes files to the media, although the media is left in the media group so that users can continue to access files on the media. Full media appears with a blue label in the Administrator interface. Provides information about the physical and logical location of the media. You can also provide an offline location for the media, which is used for mount requests and media reports. This location is also shown when a MEDIA NOT FOUND error appears. Provides statistical information on the physical and logical capacity of the media. Appears only for original and copy media, and provides statistics relating to mounts, file input and output, and errors for the media. Appears only during certain system activities, and provides the progress of the current activity. The EMC DiskXtender for Windows Online Help provides detailed information on the data available on each tab. Clearing media error status If the label for a piece of media is red in the Administrator, it means that an error has occurred on the media. This might occur when a task for a piece of media fails, or when an aborted task remains in the task queue as a failed task. It might also occur if DiskXtender encounters problems while writing files to the media. When a media encounters errors, you might need to clear the error status before any other activities for that media can take place. 256 EMC DiskXtender Microsoft Windows Version Administration Guide

249 Monitoring the System Monitoring licensing As files are added to the extended drive for DiskXtender to manage, you can monitor the DiskXtender license to ensure that you do not exceed the capacity allotted by the license. By default, DiskXtender sends a warning if the amount of data that DiskXtender manages exceeds 80 percent of the licensed capacity. The warning appears in the event logs and as an alert, if alerts are configured. If necessary, you can disable the alert by using the Enable high license capacity usage warning setting on the Options tab of the Service Properties dialog box. If you exceed the storage capacity of the license, a 90-day grace period commences. You must update the license with additional storage capacity before the grace period expires. Otherwise, file migration is disabled. Contact the EMC Licensing Support team at licensing@emc.com for assistance with updating the license. You can monitor license usage for a single DiskXtender installation from the File System Manager Administrator. The License Server Administrator enables you to monitor total license usage by all DiskXtender installations. You should also regularly review the License Server event logs to ensure that the License Server service and the licenses it manages are healthy and available to the DiskXtender servers that rely on them. Monitoring licensing from License Server The License Server Administrator interface displays the total licensed capacity and the amount of capacity currently in use when you select the license. You can also monitor the total capacity used by each DiskXtender installation, if there are multiple installations that use the same license. Monitoring licensing 257

250 Monitoring the System Monitoring performance The DiskXtender Performance Library enables DiskXtender administrators and EMC Customer Service to monitor the performance trends of the DiskXtender system. You can view the performance data using the Windows Performance Monitor utility. The DiskXtender Performance Library can be used in order to quantify the performance of DiskXtender against known system parameters. Bottlenecks in the system that can be mapped to performance degradation in DiskXtender can then be rectified to achieve optimal performance. DiskXtender Performance Library counters When the DiskXtender Performance Library is loaded, the performance counters are available under a DiskXtender Extended Drive object in the Windows Performance Monitor utility UI. Each extended drive is visible as an instance and various counters are available for each extended drive. Figure 3 on page 258 is a sample screenshot of the Add Counters dialog box with the list of counters that can be monitored on the DiskXtender extended drive. Figure 3 Add Counters dialog in Microsoft Windows 2008 Windows Performance Monitor References For information on using DiskXtender performance counters in Windows Performance Monitor utility, please see the whitepaper titled Performance Monitoring Library for EMC DiskXtender for Windows - Application and Usage onemc Online Support. For information on using the Windows Performance Monitor utility, refer Microsoft Management Console Help. 258 EMC DiskXtender Microsoft Windows Version Administration Guide

251 Monitoring the System Reports The DiskXtender reporting feature is a useful tool for tracking system statistics. The following topics provide a description of each of the reports available, as well as instructions for running a report. Types of reports Table 4 Table 4 on page 259 lists the reports available in DiskXtender. Reports Report name Extended Drive Information Media Media Files Description Displays information about the extended drives you select, including properties and assigned media. Full reports also display additional information, including schedule information, metadata export statistics, move statistics, and fetch statistics. Includes information on the selected media, either as a summarized list or a detailed description of properties. Information displayed in the detailed report includes location information, total used/free space, and read/write/mount statistics. Provides information about the files on the selected pieces of media. File Reports can also be viewed in CSV and XML format. These options can be selected in DX Admin GUI while generating the report through Report Generator. Note: User can also run a file report (for a selected group of files rather than for a piece of media) by using the Explorer Add-ons. The Explorer Add-ons online help provides more information. Media Services Media Tasks Product Registry Information Provides information on the media services configured on the selected DiskXtender servers. Displays a list of media tasks that are pending, are in progress, or that have failed for selected media on the selected extended drives. Provides a complete listing of all information contained in the Microsoft Windows registry about the DiskXtender service on the selected servers. Reports 259

252 Monitoring the System Selecting fields for Media Reports By default, Media Reports list following fields for each file: Date, Filesize, RetentionStart, RetentionPeriod, EBRTriggerTime, EBRPeriod, Encrypted, Filename, ContentAddress, Objectid, token If user is interested in specic fields, user can create DxReportField.txt file in DX installation directory (By Default Program Files\EMC\DiskXtender\Bin) and mentioning the required fields from the above list. Fields need to be specified as shown above else particular field will not appear in report. If the file exists but no field is specified then file is ignored and all fields are listed. 260 EMC DiskXtender Microsoft Windows Version Administration Guide

253 Monitoring the System Audit logs An audit log is a secure log file that functions independently of the existing DiskXtender logs (event and warning logs) and Microsoft Windows system logs. The purpose of the audit log is to provide a secure, access-controlled means for monitoring change events to the DiskXtender service, and whether each event succeeds or fails. An audit log enables you to monitor important DiskXtender events, including: Addition, editing, and deletion of configuration objects (extended drives, media folders, media groups, rules, and so on) Media management tasks Service stops and starts The performance impact caused by audit logging is minimal. Audit logging is available on both the DiskXtender server and on the MediaStor server. Enabling and configuring the audit log Audit logging is automatically enabled, and the DiskXtender service account is automatically added to the security administrators group (DxSecurityAdmins on the DiskXtender server and MsSecurityAdmins on the MediaStor server), during DiskXtender installation. However, you can also manually enable audit logging later by adding a user to the group. Deciding which events to audit You can choose the events to audit by selecting and clearing the Operations to Audit checkboxes on the Audit Log Configuration Settings dialog box. Audit logs 261

254 Monitoring the System Table 5 on page 262 lists the DiskXtender events that are logged when you select each checkbox. Table 5 Available events for audit logging Operation checkbox Creates Deletes Modifications Log On/Off Registry Log Saves Misc Operations Events logged when the checkbox is selected Creation of the following configuration items: Media services Media Media folders Media groups Rules (move, index, purge, and delete) Prefetch requests Report layouts and styles Deletion of the following configuration items: Media services Media Media folders Media groups Rules (move, index, purge, and delete) Prefetch requests Report layouts and styles Modification of the following configuration items: Media services Media Media folders Media groups Rules (move, index, purge, and delete) Prefetch requests Report layouts and styles Media tasks and queue Service configuration Audit log configuration Service stops and restarts DiskXtender updates to the registry log Miscellaneous operations, such as: Metadata import and export (and aborting a metadata export) Media allocation and deallocation Clearing of extended drive statistics, media statistics, media group statistics, and media errors Media copy promotion Media service being set online or offline DiskXtender service startup and status being set Special application list and media information being set Extended drive serial number being changed Controlling access to the audit log To gain read-only access to the audit log and its configuration settings, a user must be a member of the DxMonitors group on the DiskXtender server. (The corresponding group on the MediaStor server is MsMonitors.) 262 EMC DiskXtender Microsoft Windows Version Administration Guide

255 Monitoring the System DxAdministrators (or MsAdministrators in MediaStor) also have read-only access to the audit log and the audit log settings, but they can edit all other aspects of the DiskXtender configuration. To gain full access to view the audit log and edit audit log configuration settings, a user must be a member of the DxSecurityAdmins group on the DiskXtender server (or MsSecurityAdmins on the MediaStor server). Table 6 on page 263 summarizes the group membership required for access to the audit log and its configuration. Table 6 Access to the audit log for DiskXtender security groups Item to access MsUsers DxMonitors DxAdministrators DxSecurityAdmins Audit log None Read-only Read-only Read-only Audit log configuration settings None Read-only Read-only Read/write Only the service account is added to the security administrators group by default. No users are added to the monitors group by default. You can add users to either group by using Windows group management features. Audit log data Table 7 Table 7 on page 263 lists the information that is tracked for each event that is logged. Audit log fields Field Index Number (#) Date/Time User Description The index number for the logged entry. The date and time the event occurred. The user name or ID for the user initiating the event. Note: If the event is system-generated (for example, saving the registry log), "Internal" is displayed as the user. Action Item Type Item Name The action performed on the specified object, such as create, modify, delete, and so on. The object type accessed, such as move rule, media, extended drive, and so on. The name of the object accessed. Note: Not all entries include an item name. Item ID Parent ID Result Error Code The object ID (as a hexadecimal number, or a zero if the user is "Internal") used by DiskXtender. The extended drive the object is located on, if applicable. The success or failure of the specified action. The error code that corresponds with a failed action. Note: If you upgrade DiskXtender and then you open the audit log, the audit log displays the version number before the upgrade. To reset the version number to the current version, clear the audit log of all events. Audit logs 263

256 Monitoring the System Clearing the audit log If you are a member of the security administrators group (DxSecurityAdmins on the DiskXtender server and MsSecurityAdmins on the MediaStor server), you can reset the audit log. Resetting the audit log deletes all entries in the audit log. Resetting the log might be useful if you are short on disk space; however, creating a backup of the log before resetting is recommended. Exporting the audit log You can export the data from the audit log to a tab-delimited text file. The data can then be moved into a spreadsheet or database application. IMPORTANT Once you export the data, it is no longer protected by DiskXtender. In other words, users who are not members of the security administrators group can view and edit the information. You must implement other protective measures (such as password protection, available in many spreadsheet applications) to maintain security of the information. Backing up the audit log Frequent backups of the audit log are recommended. DiskXtender does not automate the backup of audit logs. Therefore, you must manually back up the file by using a dedicated backup software solution. 264 EMC DiskXtender Microsoft Windows Version Administration Guide

257 16 Backup and Recovery The following topics provide details on backing up and recovering the DiskXtender system: Developing a backup strategy Backing up the extended drive Ensuring storage redundancy Backing up the DiskXtender server Protecting against accidental deletion of files Recovering deleted files Recovering the system after a failure Backup and Recovery 265

258 Backup and Recovery Developing a backup strategy DiskXtender is an archival product that stores the primary copy of a file on storage media. Because it is the primary copy of a file that resides on storage media in DiskXtender not a backup copy of a file it is important to establish a comprehensive backup process to make a copy of the file. This copy can be used to recover the data if it is lost. A comprehensive backup strategy enables you to restore individual files, or even the full DiskXtender system, in the event of problems or an entire system shutdown. The strategy should protect the three basic components of the DiskXtender system: The extended drive from both the accidental deletion of individual files and the more complete destruction or corruption of the entire volume The DiskXtender server, including the DiskXtender installation The hardware devices and storage media to which files are migrated This chapter discusses the considerations for implementing a regular backup process for each one of these components. It also provides guidelines for the necessary tools that are available to every DiskXtender installation for backup and restore procedures. However, each DiskXtender environment is unique. Different hardware devices, network configurations, existing tools outside of DiskXtender, and business needs all contribute to the infinite variety of configurations that are available. It is important to take all of these variables into consideration when developing a backup strategy for your environment. Some of the procedures discussed will not be appropriate in every environment. Additional backup procedures may even be necessary, depending on your configuration. The goal of these topics is to provide you with the background to evaluate your own system and determine any additional points of possible failure so that you can design a strategy that meets your specific needs. If necessary, EMC Professional Services staff can assist you in choosing the appropriate backup software for your environment and developing a custom backup solution for you. The following best practices are recommended for backing up the DiskXtender system: Use qualified backup software to back up each extended drive and the system drive on the DiskXtender server. Qualified backup software is listed in the EMC DiskXtender Software Compatibility Guide on EMC Online Support. Use the Registry Log Wizard to change the location in which the DiskXtender registry backup is saved. The new location should be on a network drive that is backed up regularly. Enable the DiskXtender Recycler so that you can restore files that have been accidentally deleted from the extended drive. 266 EMC DiskXtender Microsoft Windows Version Administration Guide

259 Backup and Recovery Ensure that there is an additional copy of the secondary storage to which you are migrating files through DiskXtender. Table 1 on page 267 provides details for each type of storage. Table 1 Recommended media protection methods Type of storage EMC Centera NAS Removable media (tape or optical) TSM EMC Atmos Recommended protection method Enable EMC Centera replication to another EMC Centera cluster. Use qualified backup software to back up the NAS device. Create copies of each piece of media by using the DiskXtender Copy Media Manager, or configure multi-target migration to another type of media. (Available for both the MediaStor and StorageTek ACSLS media services.) Follow recommended TSM backup practices. Replication is enabled while creating resource management groups while configuring EMC Atmos. These best practices are discussed in the remainder of this chapter, as well as recommendations for alternative backup procedures when you are unable to follow these best practices. Note: Ensure high availability of the DiskXtender system through the use of Microsoft clustering or as discussed in the DiskXtender Installation Guide. Replicating the extended drive through products like RepliStor, SRDF, or MirrorView is also covered in the DiskXtender Installation Guide. Developing a backup strategy 267

260 Backup and Recovery Backing up the extended drive Most backup applications, including all of the applications qualified for use with DiskXtender, are capable of backing up a file on the extended drive and its metadata, including the extended attributes (EAs) that DiskXtender uses to store migration information. If a file is restored to the extended drive without its extended attributes, then DiskXtender treats the file as a new file and migrates it again. When backup software is EA-aware, backup performance is improved. This is because the backup software can back up file tags for managed files on the extended drive. Creating extended drive backups with software that is EA-aware on page 268 and Creating snapshot backups of the extended drive on page 271 provide recommendations for performing these types of backups. Note: Even if the backup software is capable of successfully backing up file tags, you may want to back up a full version of each file before it is migrated and purged to ensure that there is a full copy of each file available in one of the backup sets. When backup software is not EA-aware, such as IBM Tivoli Storage Manager, file tags are backed up as zero byte files, and the migration information for a file is not captured. Therefore, if you restore a zero-byte file from backup, the file is re-migrated, and data loss can occur. To prevent this situation, you must include the full file data in each backup. Purged files must be read from media so that they can be included in the backup. This slows system performance and requires more disk space for the backup set. If you use backup software that does not support EAs, follow the specific backup process to minimize the performance impact and ensure that you do not encounter data loss by restoring incomplete files. For details, refer Creating extended drive backups with software that is not EA-aware on page 272. The DiskXtender metadata export feature can be used to supplement backups from software that is not EA-aware. Metadata exports capture the file migration information that cannot be captured by backup software that is not EA-aware. Metadata exports on page 275 provides details. If necessary, EMC Professional Services staff can assist you in choosing the appropriate backup software for your environment and developing a custom backup solution for you. Creating extended drive backups with software that is EA-aware The most efficient, flexible, and reliable method of backing up the extended drive is to use a qualified backup application that is EA-aware. The following topics provide details: Qualified backup software that is EA-aware on page 268 What is included in an EA-aware backup on page 269 Installing and configuring EA-aware backup software on page 269 Configuring DiskXtender for EA-aware backup software on page 270 Qualified backup software that is EA-aware The EMC DiskXtender Software Compatibility Guide, available on EMC Online Support, provides a complete and updated list (including supported versions) of the backup software qualified for use with DiskXtender. The qualified applications include: NetWorker 268 EMC DiskXtender Microsoft Windows Version Administration Guide

261 Backup and Recovery Microsoft Windows NTBackup Symantec Backup Exec Symantec NetBackup HP OpenView Storage DataProtector CA ARCserve Backup BakBone NetVault CommVault Galaxy Enterprise Edition EMC Avamar Qualified backup software that performs snapshot backups is discussed in Creating snapshot backups of the extended drive on page 271. What is included in an EA-aware backup When a backup is performed by any of these programs, only file tags are backed up for purged files. In addition, you can choose whether to back up only file tags or the full file data for fetched files. (The full file data is backed up for files that are unmanaged, meaning that DiskXtender has not moved the files to media.) Backing up file tags for managed files significantly reduces the amount of time it takes to create the backup set. It also minimizes the size of each backup set to optimize storage capacity. If a file is being used by another application during the backup, the file may still be backed up, depending on the settings available in the backup software. Although EA-aware backup applications enable you to restore individual files from backup, review the information in Recovering deleted files on page 287 to determine whether this functionality or the DiskXtender Recycler should be used. Note: You may want to back up a full version of each file before it is migrated and purged to ensure that there is a full copy of each file available in one of your backup sets. To do this, configure purge rules with an age delay that is sufficiently longer than the time between backups. Installing and configuring EA-aware backup software Follow the recommended best practices of the qualified backup software to install the software and perform regular full, incremental, and differential backups according to your company s backup policies. In addition, the following general best practices are recommended: Install the client software for the backup program on the DiskXtender server. Install backup server software on a different computer. If you use Avamar, configure Avamar to back up files based on whether the Archive attribute is enabled. The Interoperability of Avamar and DiskXtender for Windows Technical Note, which is available on EMC Online Support, provides details. Back up the DxLogs directory on the extended drive, as well as all media folders. This ensures a more comprehensive recovery if you must restore the entire extended drive. Note: The DxLogs directory includes the DiskXtender Recycler and all of the files within the Recycler. The Recycler should be restored from a backup only if you must restore the entire extended drive. In other words, do not restore individual files from a backup to the Recycler. Otherwise, data loss may occur. Recovering deleted files on page 287 provides details on recovering individual files that have been accidentally deleted. Backing up the extended drive 269

262 Backup and Recovery Schedule backups for times of minimal system activity to minimize the impact to system performance. Consider all system activity on the DiskXtender server, including anti-virus scans, background scans, file migration, file fetch activity, and so on. Configuring DiskXtender for EA-aware backup software Consider the following DiskXtender configuration recommendations when you use EA-aware backup software to back up the extended drive: Choose whether to back up only the file tag or the full file data for fetched files. If you set retention on files that are migrated to EMC Centera or Retained NAS media, you may want to increase the delay period before retention is applied, as discussed in Increasing the retention delay on page 270. If the backup software backs up files based on whether the Archive attribute is enabled, such as Avamar, configure DiskXtender to enable the Archive attribute when the extended attributes for a file change. Enabling the Archive attribute for extended attribute changes on page 270 provides details. No additional configuration steps beyond these should be required in DiskXtender. Do not enable metadata exports of the extended drive. If the backup software relies on the Archive attribute to determine which files should be included in an incremental backup, then a metadata export can interfere. This is because the metadata export process clears the Archive attribute. If the attribute is cleared by a metadata export, then the file is not included in the next incremental backup by the backup software because the backup software sees the cleared attribute and skips the file, assuming that the file was backed up in a previously scheduled process. Increasing the retention delay If you are setting retention on files that are migrated to EMC Centera or Retained NAS media, you may want to increase the delay period before retention is applied. The default setting of 10 seconds may not be sufficient when you are restoring files with retention. This is because some backup programs restore a file in two passes: the first pass includes the file data, and the second pass includes file metadata, including EAs. If the delay is not sufficient, retention may be applied after the first pass, which prevents the second pass from being successful to complete the file restore. Increase the retention delay before retention is applied to files. If you increase the retention delay after retention is applied, the delay setting will not take effect when the file is restored. Enabling the Archive attribute for extended attribute changes You can configure DiskXtender to enable the Archive attribute for a file when the EAs for the file change. DiskXtender may change the EAs for a file when you edit retention on the file or when the migration (back-end media) information for a file changes, for example, as part of compaction or multi-target migration changes. Enabling the Archive attribute for EA changes may be useful to ensure that files are backed up by certain backup programs when these changes occur. Some backup programs, such as Avamar, may not recognize that the backup for a file needs to be updated after an EA change. As a result, you can configure DiskXtender to enable the Archive attribute, and then configure the backup program to back up the file based on the change to the Archive attribute. By default, DiskXtender does not enable the Archive attribute when the EAs change. 270 EMC DiskXtender Microsoft Windows Version Administration Guide

263 Backup and Recovery Creating snapshot backups of the extended drive Snapshot backup software takes a snapshot of the extended drive at a certain point in time. A backup application such as NetWorker can then create a backup that reflects the state of the drive at the time of the snapshot. So, if a snapshot backup begins at 8 P.M. and a file changes at 8:30 P.M. while the backup is still in progress, then the change to the file is not included in the snapshot backup that started at 8 P.M. It must be included in the next backup set. The following topics provide details on snapshot software for extended drive backups: Qualified snapshot backup software on page 271 What is included in a snapshot backup on page 271 Installing and configuring software for snapshot backups on page 271 Qualified snapshot backup software The following snapshot and backup software is qualified for creating a snapshot backup of the DiskXtender extended drive: Microsoft Windows 2003 Volume Shadow Copy Services (VSS) Note: The backup software used to back up the VSS snapshot, such as NetWorker or NTBackup, must be licensed and enabled for use with VSS. It also must be EA-aware. Do not use IBM Tivoli Storage Manager, which does not support EAs, to back up VSS snapshots. NetWorker PowerSnap Module for CLARiiON and the PowerSnap Module for EMC Symmetrix DMX! IMPORTANT EMC SnapImage is not qualified with DiskXtender when setting retention on files migrated to EMC Centera and Retained NAS media. Updates to this list (including supported versions) are provided in the EMC DiskXtender Software Compatibility Guide, available on EMC Online Support. What is included in a snapshot backup By definition, a snapshot reflects the entire extended drive at a certain point in time. Therefore, the entire extended drive is included in each snapshot backup. A snapshot backup does not fetch files from media, even if you change the 3rd party backup mode setting on the Options tab of the Extended Drive Properties dialog box to Full backup. In other words, file tags are backed up as file tags, and full files are backed up as full files. Snapshot backups typically include all files, even if the files are being used by another application during the snapshot. However, some snapshot backups can be used only to restore an entire volume. You may not be able to restore individual files from a snapshot backup. As a result, review the information in Recovering deleted files on page 287 to determine whether to restore files from the snapshot backup or the DiskXtender Recycler. Installing and configuring software for snapshot backups Follow the recommended best practices of the qualified snapshot and backup software to install the software and perform regular snapshot backups according to your company s backup policies. Backing up the extended drive 271

264 Backup and Recovery If you are setting retention on files that are migrated to EMC Centera or Retained NAS media, you may want to increase the delay period before retention is applied by performing the steps in Increasing the retention delay on page 270.No additional configuration steps should be required in DiskXtender. Do not enable metadata exports of the extended drive. If the backup software relies on the Archive attribute to determine which files should be included in an incremental backup, then a metadata export can interfere. This is because the metadata export process clears the Archive attribute. If the attribute is cleared by a metadata export, then the file is not included in the next incremental backup by the backup software because the backup software sees the cleared attribute and skips the file, assuming that the file was backed up in a previously scheduled process. Creating extended drive backups with software that is not EA-aware Backing up the extended drive with software that is not EA-aware, such as IBM Tivoli Storage Manager, presents several risks and configuration challenges. EAs are important in a backup because they store file migration information. If a file is restored to the extended drive without its EAs, then DiskXtender treats the file as a new file and migrates it again. This would result in data loss, as the correct version of the file with the full file data is overwritten on the media with the zero-byte file. Therefore, when you use software that is not EA-aware, you must back up the full file data. Unfortunately, backing up the full file data for each file slows system performance and requires a significant amount of disk space for the backup. The following best practices are recommended when you use backup software that is not EA-aware: 1. If there are files on the extended drive before you install DiskXtender, perform a full backup of the extended drive before the installation. 2. After you install DiskXtender, perform only incremental backups so that the backup captures only new or edited files. Avoid backing up files that have not changed. You can use differential backups to reduce the requirement for periodic full backups. 3. Delay file migration so that files can be captured by a backup before they are moved to media and extended attributes are added for the migration information. To delay file migration, set an age delay for the move rule by selecting the Apply rule to files of age option on the Age page of each move rule, and then choosing the age criteria. 4. Change the default setting for the 3rd party backup mode setting: a. In the File System Manager Administrator, right-click the extended drive and select Properties. b. Click the Options tab. c. Select the 3rd party backup mode for managed files option from the list. d. In the bottom portion of the dialog box, select Full backup - read all file data, including migration info. e. Click OK. f. Repeat these steps for each extended drive on the DiskXtender server. 5. Delay file purging until it is absolutely necessary. 272 EMC DiskXtender Microsoft Windows Version Administration Guide

265 Backup and Recovery This increases the likelihood that file data will be resident on the extended drive when the backup occurs. If the file data is on the extended drive, it does not need to be retrieved from media during the backup. a. Configure an age delay for purge rules by selecting the Apply rule to files of age option on the Age page of each purge rule, and then choosing the age criteria. b. Leave the default setting on the Settings page for each purge rule. With the default setting of Do not force purges during background scans, files are purged based on the extended drive used space percentages configured in the purge watermarks. 6. Configure a separate purge rule that prevents files with the Archive attribute enabled from being purged: a. On the Type page of the new purge rule, select Exclude. b. On the Attributes page of the purge rule, select Apply rule to files with the following attributes, and then select the Attributes checkbox. c. Complete the wizard. d. Reorder the list of purge rules so that the exclusive purge rule is listed first. This ensures that files are qualified against the exclusive purge rule before the other purge rules. 7. Enable the DiskXtender metadata export feature to periodically export files that are managed by DiskXtender (in other words, migrated to media): Note: The DxLogs and Recycler directories on the extended drive are not included in a metadata export. In addition, the size of a metadata export depends on the size of the extended drive and the data included in the export (files managed or not managed by DiskXtender), as well as the type of export (incremental or full). In general, the size of a metadata export is 1,500 bytes per file in the export. If the files on the extended drive have been migrated to a multi-target media group, then the size of the export increases to 2,000 bytes per file. a. Right-click the extended drive and select Properties. b. Click the Settings tab on the Extended Drive Properties dialog box. c. Click Meta-Data Export. The Extended Drive Meta-Data Export dialog box appears, as illustrated in Figure 1 on page 274. Backing up the extended drive 273

266 Backup and Recovery Figure 1 Extended Drive Meta-Data Export dialog box d. Click the Browse button to the right of the UNC Path text box, and browse to the folder in which to save the export sets. Or, type the path in the text box. The metadata export should not be stored on the extended drive. Instead, it should be stored at a network location. In addition, archive the metadata export as you would any other backup data for disaster recovery. e. In the Scheduling section, select the frequency with which incremental exports should be performed. f. To perform a full export the next time that the incremental export schedule is active, select the Force next export to be full checkbox. g. To perform regular full exports, select the Perform at scheduled interval checkbox. h. Click Full Export Frequency. The Full Meta-Data Export Frequency dialog box appears, as illustrated in Figure 2 on page 274. Figure 2 Full Meta-Data Export Frequency dialog box 274 EMC DiskXtender Microsoft Windows Version Administration Guide