GuardianOS Data Migration Tool Enables Ease of Snap Server Deployment and Data Consolidation WHITE PAPER

Transcription

1 GuardianOS Data Migration Tool Enables Ease of Snap Server Deployment and Data Consolidation WHITE PAPER

2 Introduction Powered by the GuardianOS operating system, the Overland Storage Snap Server family of NAS storage servers support numerous file access protocols that provide client access to files in Windows, UNIX, and AFP networks, as well as remote access via FTP and HTTP. Since Snap Servers are a cost-effective file server replacement for so many platforms, a utility that simplifies data migration to Snap Servers was designed as part of GuardianOS. This white paper will describe how to utilize and best leverage the data migration utility built into GuardianOS. Overview of the Data Migration Utility The data migration utility allows administrators to easily copy or move data from any file server that has an accessible Windows (CIFS/SMB) share or UNIX (NFS) mount. It can also preserve permissions in order to make the transition to a new Snap Server seamless to end users and data owners. Previously, only native tools were readily available to move data from older file servers. Now, for consolidating data from multiple sources or performing a technology refresh, the data migration utility simplifies the migration process and reduces the implementation time for the new Snap Server. The Old Way In the past, if an administrator was moving data from a Windows system he could simply drag-and-drop the data if he was unconcerned with preserving permissions. However, if he wanted to maintain any user and group permissions while migrating, he was forced to use other utilities, such as XCOPY, Robocopy, SCOPY, XXCOPY, etc. Each of these tools has its own set of subtle nuances that can make it difficult to work with and can produce inconsistent and unpredictable results. Not to mention that these tools lack the ability to verify the data transfers or restart without overwriting all data that was previously copied after a failed data movement. If the administrator was moving data from a UNIX-based operating system, the task was not as unpredictable, but consolidating multiple systems could, nevertheless, become incredibly cumbersome. In many cases, administrators are replacing or offloading data from existing network attached storage (NAS) systems that do not have local utilities or commands for migrating data from them, requiring the use of a third-party Windows or UNIX system to move that data. Permissions on both the source and target must be painstakingly set to allow the operation. The third-party system will become bogged down moving data instead of doing other productive work. This can especially be a problem if the administrator s own workstation is used. A Better Way Now, with an integrated data migration utility on GuardianOS-powered Snap Servers, administrators can easily move data, streamlining the integration of a new Snap Server. This makes it less daunting to offload shared data from servers that were not originally intended for file serving, perform a technology refresh on older file servers that cannot keep up with the current performance and capacity requirements, or even to centralize the storage of multiple file servers to one high-capacity and high-performance solution. 1

3 The benefits of using the embedded data migration utility over other copy or move tools are simplicity and centralization. The simplicity comes from having a centralized single interface for copying or moving data from any server or storage platform that can be mounted using either CIFS/SMB (Windows) or NFS (UNIX/Linux/Solaris/Mac OS X). Administrators will no longer have to learn any command-line switches to make sure that the permissions are included. No manual verification that the data was successfully migrated is required. Everything an administrator needs to successfully perform data migration from other servers is on one easy-to-use screen within the web browser administration tool of Snap Server systems. Since the data migration utility runs natively on the Snap Server, all migration processing is done locally on the Snap Server and does not require any other system to facilitate the migration of data from the source. Just getting the data where it needs to be on a new file server can be the most difficult part of integrating a new data storage device. With the data migration utility, administrators can now easily accomplish this task. Use Case Scenarios for the Data Migration Utility There are several scenarios where the data migration utility can simplify the task of getting data moved over to the new Snap Server. The following use case scenarios outline some of the ways that in which the data migration utility can be used. Figure 3.1 shows just two examples of these use cases. Snap Server 520, 620 or Snap Server with Expansion Figure 3.1 The data migration utility is designed to be a single-direction, single-instance tool for moving data from a source system to a target system. In the first usage case above (Performing a Technology Refresh with the Data Migration Utility) the usage of the data migration utility is illustrated as a single operation for each server that is being replaced with a more current Snap Server system. In the second usage case above (Consolidating Multiple File Servers with the Data Migration Utility) the data migration utility is run repeatedly for each source system that is being consolidated and/or replaced by the more current Snap Server system. 2

4 Use Case #1 Migrating Data from a Server Running Windows (CIFS/SMB) Migrating data from a Windows source server requires the existence of a Windows share to provide access to the data by the Snap Server. The source server can be in a Windows Workgroup, an NT Domain, or an Active Directory Domain. The data migration utility can retain permissions for servers in a Windows NT or Active Directory Domain as long as the Snap Server is joined to the domain prior to executing the migration. If the Windows source is part of a Windows Workgroup, permissions can not be retained as all local user and group accounts are only relevant on the source server. Local Windows user and group security identifiers (SIDs) are not transferable, and therefore, the permissions cannot be recreated. In all cases, a proper administrative user account will be required to access the data on the source machine in order to properly transfer the data and permissions. If the source data contains filenames with extended characters other than common Western European characters (CP 1252), it may be advisable to enable Unicode on the Snap Server. Then, convert the file system to the proper mode before migration in order to properly retain the extended characters. Note: SMB must be enabled on the Snap Server to perform migration using the Windows network protocol option. This is enabled by default in the Network Windows screen of the web browser administration tool. A valid Windows share must also exist that allows write access to the target path in order to properly preserve permissions and DOS attributes. Use Case #2 Migrating Data from a Server Running UNIX or Linux (NFS) When moving data from a UNIX-based file server via NFS, the source server must have a valid NFS mount point for access to the data by the Snap Server. The file server can be part of a NIS Domain, but does not have to be. If permission preservation is enabled, the data migration utility preserves permissions and ownership for all user identifiers (UIDs) and group identifiers (GIDs) verbatim, regardless of whether an equivalent UID/GID is known to the Snap Server. This means that if all UIDs/GIDs are created on the target and match the source, permissions will be properly retained. A proper administrative user account will be required to access the data on the source machine in order to properly transfer the data and permissions. The migration will take place with the UID of the local Snap Server user logged into the UNIX machine as the migration user (note that the name may not have an equivalent on the target machine only the UID is relevant). Filenames are transferred over NFS verbatim as a bag of bytes. Therefore, extended characters in filenames may not be properly retained if encoded on the source with a character set that does not match the character set used on the Snap Server's file system (ISO /CP 1252 if Unicode is disabled, or UTF-8 if Unicode is enabled). Note: When connecting to the NFS source, the Snap Server tries to connect using NFSv3 by default, but will fall back to NFSv2 if necessary. It will, likewise, try to use TCP as the default transport protocol, but fall back to UDP. Use Case #3 Migrating Data from a GuardianOS-Powered Snap Server Migrating data from another GuardianOS-powered Snap Server to a new Snap Server with the data migration utility works just as if you were moving data from any other file server source. Be sure to choose the appropriate protocol for the migration job with respect to the security model of the data being migrated. For example, if the data is in 3

5 a Windows SnapTree, use Windows, and if the data is in a UNIX SnapTree, use NFS to migrate the correct permissions. The rest works the same as any other Windows or UNIX data migration as described in Use Cases #1 and #2. To properly retain any extended characters in filenames, make sure the target Snap Server is configured for the same Unicode state (enabled/disabled) as the source Snap Server. Note: If the source location contains both Windows and UNIX SnapTrees and you want to preserve permissions, split the migration operation into multiple jobs, one for each SnapTree for the given security model, using the appropriate protocol per job. Use Case #4 Migrating Data from a SnapOS-Powered Snap Server Like GuardianOS-to-GuardianOS migrations, moving data off of a SnapOS-powered Snap Server is as simple as defining the data type (Windows or UNIX) that you will be migrating. Permission preservation, however, is generally not recommended when migrating from a SnapOS-powered Snap Server. SnapOS native file system permissions are only properly visible and configurable in the SnapOS web UI. Though modeled after Windows-style permissions, SnapOS permissions cannot be communicated over SMB or any of the file access network protocols. In addition, UNIX permissions reported over NFS by SnapOS servers are loosely interpreted from the SnapOS file system permissions, and do not necessarily properly represent them. Therefore, preserving them when migrating over NFS may not result in equivalent effective permissions on the target, even if preserved verbatim with the same UIDs/GIDs. SnapOS does not enforce a standardized character set for filenames and relies on DOS code pages for communicating filenames over SMB. Therefore, extended characters may not be properly retained using either SMB or NFS migration, depending on the method and protocol by which the filenames were originally written at the source. Because NFS migration transfers filenames verbatim as a bag of bytes, extended characters are likely to be copied improperly, especially if the target GuardianOS Snap Server is configured for UTF-8. Use Case #5 - Migrating Data from Mac OS X Mac OS X migrations can be done using either the Windows or NFS network protocol, but they cannot retain the extended ACLs or resource forks of OS X 10.4 s extended ACLs or resource forks. If neither of these is a concern, then the appropriate migration protocol should be selected based on the need to retain UNIX-style permissions, whether there are extended characters in filenames on the source, and whether the GuardianOS-powered Snap Server is in Unicode mode. A NFS migration can retain UNIX-style permissions from OS X, but characters in filenames are migrated verbatim as a bag of bytes. Extended characters on source filenames will, therefore, only be retained properly if Unicode is enabled on the Snap Server. A Windows migration cannot retain permissions from OS X, but it can retain common Western-European extended characters in filenames if Unicode is not enabled on the Snap Server target. It can retain all extended characters if Unicode is enabled on the Snap Server. Use Case #6 Migrating Data from Other Sources Migrating data from any other source is possible, as long as the source file server or NAS system has an NFS mount or CIFS/SMB share available. Moving data off of other sources only requires that you have an administrative login with appropriate access privileges. For customers whose NAS offerings are not performing adequately or meeting all of their 4

6 needs, the data can be easily migrated from these systems using the data migration utility. If NFS migration is chosen, note that filenames are transferred over NFS verbatim as a bag of bytes. Therefore, extended characters in filenames may not be properly retained if encoded on the source with a character set that does not match the character set used on the Snap Server s file system (ISO /CP 1252 if Unicode is disabled, or UTF-8 if Unicode is enabled). Note: Similar to a migration from GuardianOS to GuardianOS, migrating data from a location that has mixed Windows-style and UNIX-style file permissions e.g. a mixedmode qtree on a NetApp filer can only properly preserve permissions that correspond to the protocol used for the migration job. For example, if Windows is used, only Windows permissions will properly be preserved; and if NFS is used, only UNIX permissions will be properly preserved. Procedures for Using the Data Migration Utility The data migration utility can be found on the bottom of the main Administration web page of the web browser administration tool (see Screenshot 4-1) for any GuardianOSpowered Snap Server. The latest version of GuardianOS can be found at the support portal at Screenshot 4-1 It is important that, before any data migration, the administrator carefully plan out the process and allow enough time to move the data and test access prior to turning the new Snap Server live to the end users. Roughly estimating a data migration can be done by creating a dry run to copy (not move) the data; let the migration job run for some time; and then check the estimated time remaining on the Data Migration Status Page. The job can then be stopped and restarted when there is enough time to complete an uninterrupted migration. This will give a very rough estimate of the migration time and can in no way be used as an accurate gauge for the actual migration. See Screenshot 4-2 for a look at the Data Migration page. 5

7 Screenshot 4-2 Requirements: - GuardianOS or later - Valid Windows/CIFS share or NFS mount point with administrative access - Dedicated access to the source data (no clients accessing the source during migration) - Minimum 100Base-T network for both source and target on the same local LAN (Does not support WAN-based migrations) - Sufficient available capacity on the target volume on the Snap Server The following is an overview of the steps that should be followed to set up and execute a successful data migration: Step 1: Synchronize User Accounts If it is critical to preserve permissions, join the appropriate Windows Domain (with trusted domain support enabled, if applicable) or NIS Domain, and add any local users and groups that are necessary. Also make sure if there are any ID mappings that need to be created between Local or NIS users/groups and Windows users/groups, that this is done prior to beginning any migration jobs. ID Mapping instructions can be found in the GuardianOS Administrators Guide or in the online help. Note: All of the users and groups that own or have permissions assigned at the source need to be properly added and configured on the Snap Server. Step 2: Choose Network Protocol Decide which network protocol will be used for the data transfer. If you are transferring from a Windows file share, then you would choose the Windows option that uses the CIFS/SMB network protocol. If you are transferring data from a UNIX mount point, choose the NFS option. If either protocol is an option, choose whichever is more convenient, or, if preserving permissions, choose the protocol that matches the file permission type of the source data. Step 3: Identify Source User Account Identify an administrative user account that has the appropriate access on the source system to move the data. If performing a move operation, note that this user needs to be able to delete source data, and if preserving permissions, needs to be able to read permissions on source files. For Windows migration jobs, the user Administrator or an 6

8 equivalent account e.g. a domain administrator such as mydomain\administrator is recommended, as is the root account for NFS migration. If using root for NFS, make sure to use the no_root_squash option on the source export. In general, it is a good idea to test access to the source s share or mount point with the identified user account before proceeding. Step 4: Create the Security Model on the Snap Server Using the SnapTree feature, ensure that the appropriate security model for the migration type (Windows or NFS) is configured on the target path for the Snap Server. The following are some examples to help illustrate the importance of setting the proper security model: If moving Windows data, set up a Windows SnapTree at the target, and if moving UNIX data, set up a UNIX SnapTree. If the SnapTree type does not match the migration job s Network Protocol (e.g. a Windows job configured to migrate to a Unix SnapTree), a warning will be returned by the data migration utility. If preserving permissions when migrating Windows data, each file or folder written to a folder with the UNIX security model will result in a permission denied error output to the migration log. If the target location is a volume root directory that contains various SnapTrees of different security models, permissions will be properly maintained for files and folders placed in directories with the security model that matches the protocol of the migration job, whereas files and folders placed in directories with a mismatched security model will not be properly retained (and may return an error per the above). Step 5: Create a Share on the Snap Server Create the target share(s) or mount point(s) on the Snap Server exactly as you want them to be. Make sure that at least one share is available that can access the target path that gives write access to the Snap Server s local root user. Step 6: Move or Copy? Decide whether you want to copy the data (leave the data on the source) or move the data (delete the data from the source). This is a very important decision. For example: In your environment it may make the most sense to copy all of the data first and then to disable access to the old file server and activate the new Snap Server share. This would allow you to test and make sure everything is working as desired before removing the data from the old server. If you decide to move the data instead, it is highly recommended that you ensure there is no other user access to the source or target server during the migration. It is also highly recommended that you use the Verify option to ensure the data has been properly migrated. The Verify option will take twice as long, but will ensure that the data was moved or copied correctly. Note: If verification is enabled, files will only be removed from the source after successful verification; otherwise they ll be removed from the source immediately after copying. Step 7: Restrict Access to the Source Data Make sure that the source location is quiescent. In other words, at the very least no users or services can change or add data at the source. It is always best to restrict all other access except for the Snap Server system doing the migration. Step 8: Run the Migration Job Once all of the necessary information is gathered, configure and run the migration job. At a minimum, always restrict access to the source from which the data is being moved. It is always best to restrict all other access except for the Snap Server doing the migration. 7

9 Step 9: Check the Logs After the job runs, always check the log for errors by clicking on the View Log button at the bottom of the data migration screen. Step 10: Verify the Data Migration If you used the Verify option, the Data Migration Log will help you determine success for the migration. If you chose not to use the Verify option, make sure to use some other method to validate the success of the data migration. If you are executing another migration after a previous failed migration, it is crucial that verification is enabled, to ensure that any files that may be corrupt due to a migration failure are identified. Corrupted data can only be found by utilizing the Verify option. If a file comparison fails in the verification phase, the target file will be moved to a quarantine directory on the same volume. The log message for the comparison failure will indicate the path. Step 11: Take the Old Server Offline Once migration of the data has been successfully verified, bring the old file server shares or mount points offline. This will prevent new data from inadvertently being added to the old server. What Permissions are Preserved? The types of permissions retained will differ, depending on which of the following migration scenarios is applied: Migrating from a System Using a Windows Security Model: When migrating from a Windows server or any other server using a Windows Security Model to a Snap Server, permissions will be retained as follows: Known domain user owners are retained as owners if the Snap Server belongs to the same or trusting domain of users assigned permissions on the source data. To retain permissions for users of a trusted domain, Trusted Domain support must be enabled on the Snap Server. To enable, navigate to Network > Windows. Permissions for unknown users and groups and for local users and groups on the Windows server are NOT retained. Access Control Entries (ACEs) for users and groups will be applied on the target files approximately as they were on the source files. Due to differences between Windows Access Control Lists (ACLs) and POSIX ACLs, the effective permissions may not be those you expect. For more information, see How the GuardianOS and Windows Differ in Processing File-Level Access Permissions in the Snap Server Administrator Guide. For example: If there is no ACE for everyone on the source, it will be assigned No Access for all users who are not covered by higher POSIX ACL checks. If there is a Deny All ACE for everyone on the source, it will be applied to all ACEs on the ACL (that is, everyone is denied access, although the owner always has implicit change permissions). If there is a Deny All ACE for non-owning users or groups, it will be removed (i.e., enforced as a lack of permission rather than as an explicit Deny). If there are Deny ACEs on users, the denied bits are set as No Access rather than enforced as explicit Denies. If there are Deny ACEs on groups, the denied bits are set as No Access on the group ACE, and are also propagated as No Access bits to ACEs for individual group members (if any exist). 8

10 Migrating from a System Using a UNIX Security Model When migrating from a UNIX server using a UNIX Security Model to a UNIX SnapTree, UNIX permissions for UIDs/GIDs are copied exactly from the source to the target; thus, identities of the users and groups will be best retained if the Snap Server belongs to the same NIS domain as the UNIX server. If an NIS Domain does not exist all of the same UIDs/GIDs must exist and be configured properly as local users on the Snap Server to retain the permissions properly. Migrating between conflicting Security Models It is not recommended that permissions be preserved when migrating from a Windows Security Model to a UNIX SnapTree, or vice versa. Permissions will not be applied or enforced as expected. Migrating from a GuardianOS-Powered Snap Server When migrating from one GuardianOS-powered Snap Server to another, it is recommended that the same Security Model is used on the target server that was used on the source. If the source server uses a Windows SnapTree and has permissions assigned to Windows domain users, use a Windows connection for migration. Note: Permissions for local users on the source Snap Server will not be retained. If permissions or ownership is largely assigned to local users, it is not recommended to use the Preserve Permissions option because the permissions will all be dropped. If the source server uses a UNIX SnapTree and has permissions assigned to local or NIS users, use an NFS connection for migration. Generic POSIX ACLs or Solaris POSIX ACLs will not be retained. Only standard UNIX permissions (User/Group/Other) are maintained as part of data migration process. Note: Local users that have UNIX permissions on the source will not be created on the target with the same UIDs. These local user accounts and UIDs must be created manually. During a NFS migration, permissions, and ownership are preserved verbatim if local user name and/or group name equivalents for these UIDs/GIDs are needed, those users and groups with corresponding UIDs/GIDs need to be created manually. When Not to use the Data Migration Utility In some cases there are better options for moving or copying data to a Snap Server. The following describes some common scenarios that have better alternative methods for moving data: Data migration is not a replication replacement The data migration utility is not schedulable and therefore is not a good tool for repetitive data movement. The data migration utility also is not optimized by way of bandwidth throttling or copying bytelevel changes only, which are highly beneficial for WAN-based data movement. Purpose-built tools, such as Snap Enterprise Data Replicator (Snap EDR), are better suited for this type of task. Data migration is not a backup replacement For the same reasons that the data migration utility is not suited as an ongoing replication tool, it is not designed with data backup in mind. Features such as scheduling, incremental capabilities, versioning, and backup cataloging among many other things make the data migration tool a poor choice as a backup tool. 9

11 Mixed Windows and UNIX security models For any data that is accessed by end users from both Windows and UNIX machines, the data migration tool can only effectively retain the permissions of one or the other security model, based on the protocol used for the migration job. Therefore, it is recommended to choose the security model that is predominantly used by the end users. Once the data migration has completed, the other security model permissions will have to be manually adjusted. Note: When migrating from an older GuardianOS-powered Snap Server, Snap EDR can be used to make sure all Windows and UNIX permissions are properly retained during the data movement. Migration of AFP files from SnapOS-powered Snap Servers Macintosh files stored on a SnapOS-powered Snap Server may have resource forks associated with them that may not be properly accessible on the GuardianOS target after migration. Therefore, a manual copy or move of the data from a native Macintosh is the best option for migrating Macintosh files. Notable Caveats NFS data migration is very memory-intensive, scaling with the number of files and folders processed. Jobs should therefore be configured to divide up the source data into data sets not exceeding one million total files and folders per job. There are no limits for CIFS/SMB migration jobs. It is important to note the following behavior for migrating symbolic and hard links from UNIX-based source servers using NFS: Symbolic or soft links (also known as symlinks) are preserved verbatim, symbolically They are not followed (to prevent getting stuck in a recursive symlink loop). As a consequence, symlinks at the source migrated to locations outside of the data set may be broken at the target. Hard links are retained as hard links Multiple files pointing to the same inode at the source will result in those same files pointing to the same inode at the target. NetApp systems using a version of Data ONTAP older then version may not allow the Windows (CIFS/SMB) data migration engine to authenticate to the NetApp box via a Windows Domain. This was verified as fixed in version A NetApp bug has been noted during testing that can result in failure to migrate if NFSv3 is enabled and TCP is disabled on the NetApp server. If necessary, configure the NetApp server to either allow NFS over TCP or force NFSv2. 10

12 About Overland Storage Overland Storage provides affordable end-to-end data protection solutions that are engineered to store smarter, protect faster and extend anywhere across networked storage, media types, and multi-site environments. Overland Storage products include award- winning NEO SERIES and ARCvault tape libraries, REO SERIES disk-based backup and recovery appliances with VTL capabilities, Snap Server NAS appliances, and ULTAMUS RAID high-performance, high-density storage. For more information, visit WORLDWIDE HEADQUARTERS 4820 Overland Avenue San Diego, CA USA TEL FAX UNITED KINGDOM (EMEA OFFICE) Overland House, Ashville Way Wokingham, Berkshire RG41 2PL England TEL +44 (0) FAX +44 (0) Overland Storage, Inc. All trademarks and registered trademarks are the property of their respective owners. EDRDM-WP