Version 1.4.0 Admin Guide. Revision: B5

Version 1.4.0 Admin Guide Revision: B5 Document Number: 96-00320-001 October 2014

Important Information Information in this document is subject to change without notice and does not represent a commitment on the part of DataDirect Networks, Inc. No part of this manual may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, for any purpose other than the purchaser s personal use without the written permission of DataDirect Networks, Inc. 2014 DataDirect Networks, Inc. All rights reserved. DataDirect Networks, the DataDirect Networks logo, DirectOS, DirectProtect, DirectMon, EXAScaler, GRIDScaler, Information in Motion, NAS Scaler, NoFS, ObjectAssure, SATAssure, Silicon Storage Appliance, S2A, Storage Fusion Architecture, SFA, Storage Fusion Fabric, Storage Fusion Xcelerator, SFX, xstreamscaler, Web Object Scaler, WOS are registered trademarks or trademarks of DataDirect Networks, Inc. All other brand and product names are trademarks of their respective holders. DataDirect Networks makes no warranties, express or implied, including without limitation the implied warranties of merchantability and fitness for a particular purpose of any products or software. DataDirect Networks does not warrant, guarantee or make any representations regarding the use or the results of the use of any products or software in terms of correctness, accuracy, reliability, or otherwise. The entire risk as to the results and performance of the product and software are assumed by you. The exclusion of implied warranties is not permitted by some jurisdictions; this exclusion may not apply to you. In no event will DataDirect Networks, their directors, officers, employees, or agents (collectively DataDirect Networks) be liable to you for any consequential, incidental, or indirect damages, including damages for loss of business profits, business interruption, loss of business information, and the like, arising out of the use or inability to use any DataDirect product or software even if DataDirect Networks has been advised of the possibility of such damages by you. Because some jurisdictions do not allow the exclusion or limitation of liability for consequential or incidental damages, these limitations may not apply to you. DataDirect Networks liability to you for actual damages from any cause whatsoever, and regardless of the form of the action (whether in contract, tort including negligence, product liability or otherwise), is limited to the sum you paid for the DataDirect product or software. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 2

TABLE OF CONTENTS 1.0 WOS ACCESS OVERVIEW... 12 1.1 Product Description... 12 1.2 NFS Service... 12 1.2.1 NFSv4 ID Mapping... 13 1.3 CIFS Service... 13 1.4 Namespace Synchronization... 13 1.5 Disaster Recovery... 14 1.6 Services Relocation... 14 1.7 Web Administration User Interface Overview... 16 1.8 Network Configuration... 16 1.9 Sharing Files Between NFS and CIFS... 17 2.0 SOFTWARE INSTALLATION... 17 2.1 Operating System Installation... 17 2.2 Web Object Storage Prerequisites... 17 2.3 WOS-Access Appliances... 18 2.4 Default passwords... 18 2.5 Configuring WOS client I/O filters... 19 2.6 Verify WOS Access network connections between WOS Access Nodes and WOS Core... 19 2.7 Adding a Mellanox ConnectX-3 Networking Card... 19 3.0 WEB CONFIGURATION PROCEDURE... 19 3.1 WEB Configuration Steps... 19 3.1.1 Single WOS Access Server... 19 3.1.2 Multiple WOS Access Servers... 20 3.2 WOS Access Login Page... 20 3.3 WOS Storage Policy Configuration... 21 3.4 WOS Access Namespace Synchronization Configuration for Two Or More Servers Sharing the Same Namespace or for Service Relocation... 24 3.4.1 Configuring Namespace Synchronization... 24 3.4.2 Creating New Replication Group... 26 3.4.3 Join Existing Replication Group... 27 3.5 WOS Access NFS Configuration... 28 3.5.1 Creating NFS Export Points... 29 3.5.2 NFS Protocol Configuration... 30 3.5.3 NFS Security Configuration... 30 3.5.4 Delete NFS Export Point... 32 3.5.5 Start NFS Service... 33 3.6 WOS Access CIFS Configuration... 33 3.6.1 Changing CIFS Service Status... 34 3.6.2 Changing CIFS Service Configuration... 35 3.6.3 Creating, Modifying, and Deleting CIFS Export Points... 36 3.6.4 Join Active Directory Domain... 39 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 3

3.7 WOS Access Disaster Recovery Configuration... 40 3.7.1 Enable Disaster Recovery Service... 40 3.7.2 Scheduling a Full Backup... 41 3.8 WOS Access Services Relocation Service Configuration... 41 3.9 WOS Access Preferences... 44 3.9.1 Changing Email Alerts Preferences... 45 3.9.2 Changing Credentials... 47 3.9.3 Changing User Interface Preferences... 47 3.9.4 Changing Exporting syslog... 48 4.0 WEB ADMINISTRATION REFERENCE... 50 4.1 General Navigation... 50 4.2 Status Icons... 51 4.3 User Interface... 52 4.3.1 Summary Screen... 52 4.3.1.1 Service Status... 52 4.3.1.2 Storage Statistics... 53 4.3.1.3 System Charts: Protocols... 54 4.3.1.4 System Charts: WOS Storage... 54 4.3.1.5 System Charts: Synchronization... 55 4.3.1.6 System Charts: System... 55 4.3.1.7 System Charts: Controls... 56 4.3.1.8 Installed Packages Chart... 56 4.3.2 Synchronization Screen... 56 4.3.2.1 Nodes Section of Synchronization... 57 4.3.2.2 Total Section of Synchronization... 57 4.3.3 Services Screen... 57 4.3.3.1 WOS Access... 58 4.3.3.2 Synchronization... 58 4.3.3.3 Services-Relocation... 59 4.3.3.4 Backup... 59 4.3.4 Configuration Screen... 60 4.3.4.1 Export List... 60 4.3.4.2 NFS... 63 4.3.4.3 CIFS... 63 4.3.4.4 Active Directory Configuration... 63 4.3.4.5 WOS... 65 4.3.4.6 Services-Relocation... 65 4.3.4.7 Synchronization... 66 4.3.4.8 Adding A New Node to a Cluster Synchronization Group... 67 4.3.4.9 Backup... 68 4.3.5 Preferences Screen... 68 4.3.5.1 E-mail notifications... 68 4.3.5.2 Credentials... 69 4.3.5.3 User Interface preferences... 69 4.3.5.4 Export syslog... 69 4.3.6 Logs Screen... 70 5.0 ADVANCED CONFIGURATION PROCEDURE... 71 5.1 CIFS Service Shared Folder Configuration... 71 5.2 CIFS Service Security Configuration... 72 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 4

5.3 CIFS Users and Groups Management... 72 5.3.1 Add a new user... 72 5.3.2 Delete an existing user... 73 5.3.3 Modify user account attributes... 73 5.3.4 Add a new group... 74 5.3.5 Delete an existing group... 75 5.3.6 Modify a group membership... 75 5.3.7 List users and groups... 76 5.3.8 Check group membership... 76 5.3.9 CIFS Service Access Management... 77 5.3.10 CIFS Share Access Mode... 77 5.3.11 CIFS Service Current Usage Metrics... 77 5.3.12 Setup a Local CIFS User to Access CIFS Share... 78 5.3.13 Configuring CIFS UID/GID Values... 78 5.4 CIFS Service configuration in Active Directory (optional)... 79 5.4.1 Accessibility of Active Directory Domain Controller... 80 5.4.2 Join AD Domain... 80 5.4.3 Check AD Domain accounts... 81 5.4.4 Access to CIFS shares... 81 5.4.5 Leave AD Domain... 81 5.5 Namespace Synchronization Service... 81 5.5.1 Network requirements... 82 5.5.2 Join Synchronization Group... 82 5.5.3 View Synchronization Cluster Status... 85 5.5.4 Leave Synchronization Cluster... 85 5.6 WOS Access Services Relocation Service Configuration... 86 5.6.1 Network requirements... 87 5.6.2 Configure Cluster... 89 5.6.3 Cluster Installation... 92 5.6.4 Update Cluster Configuration... 94 5.6.5 Remove Cluster... 94 5.6.6 Failover Policy Details... 95 5.7 WOS Access Disaster Recovery Configuration... 98 5.7.1 Configuration steps... 98 5.7.2 Overview of backup process... 99 5.7.3 Scheduling backup... 100 5.7.4 Manual backups... 100 5.7.5 Restoring from backup... 100 5.7.6 References... 102 5.8 Mellanox ConnectX-3 configuration... 103 5.9 Enabling wos-fst logging dynamically... 103 5.10 Disk Space Monitoring... 103 5.11 Orderly Shut Down of the WOS Access Node... 103 5.11.1 WOS Access Shut Down Script... 104 5.12 Renaming Host Names... 105 6.0 QUOTAS... 106 6.1 Planning... 106 6.2 Setting Quotas... 106 6.3 Clearing Quotas... 107 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 5

6.4 Enabling Quotas... 108 6.5 Quota Reporting... 108 6.5.1 Quota Reports using quotactl... 109 6.5.2 Quota Reports from NFS clients... 109 6.5.3 Quota Reports via Email... 110 6.6 Functional Limitations... 110 6.6.1 Using quotas in a mixed CIFS/NFS environment... 111 6.7 Troubleshooting... 111 7.0 TRASHCAN FEATURE... 112 7.1 Overview... 112 7.2 TrashCan Utility tool (trashcanutil)... 112 7.2.1 TrashCan: A Simple Example... 113 7.2.2 Handling conflicts on recovery... 113 7.3 Background Delete... 114 8.0 WOS ACCESS UPGRADE PROCEDURE... 114 8.1 Preparing for Upgrade... 114 8.2 Upgrade... 115 8.3 Post-Upgrade... 115 9.0 REFERENCES... 116 9.1 NFS Configuration... 116 9.1.1 Overview... 116 9.1.2 Export... 116 9.1.3 OVFS_WOS... 117 9.1.4 OVFS_BACKUP... 119 9.1.5 CacheInode_Client... 119 9.1.6 NFS CORE Parameters... 120 9.1.7 Deprecated NFS Parameters... 120 9.2 Sync Configuration... 122 9.3 SR Configuration... 123 9.4 CIFS Configuration... 123 9.5 quotactl command... 125 9.5.1 quotactl on/off... 125 9.5.2 quotactl status... 125 9.5.3 quotactl cfg... 126 9.5.4 quotactl set... 126 9.5.5 quotactl get... 126 9.5.6 quotactl clear... 127 9.5.7 quotactl check... 127 10.0 TROUBLESHOOTING... 127 10.1 Troubleshooting Installation... 127 10.2 Troubleshooting NFS Service... 128 10.2.1 Server side errors... 128 10.2.2 Client side errors... 129 10.3 Troubleshooting Namespace Synchronization Service... 131 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 6

10.4 Troubleshooting RabbitMQ... 132 10.5 Troubleshooting Administration Web UI... 134 10.6 Troubleshooting Disaster Recovery... 135 10.6.1 Backup failure... 135 10.6.2 Backup uses up all the available disk space... 136 10.6.3 No backup mail notification... 136 10.7 Troubleshooting SR Cluster... 137 10.8 Troubleshooting CIFS Service... 138 11.0 SUPPORT... 142 11.1 Available Online Documentation... 142 12.0 APPENDIX... 142 12.1 List of Product Files... 142 12.2 List of TCP/UDP ports used by WOS Access... 142 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 7

LIST OF FIGURES Figure 1 WOS Access Network Configuration Example... 16 Figure 2 WOS Access Login Screen... 21 Figure 3 WOS Access Initial Configuration Screen... 21 Figure 4 WOS Configuration Section... 22 Figure 5 Password Prompt Box... 22 Figure 6 WOS IP Address Configuration... 23 Figure 7 WOS Policy Configuration... 23 Figure 8 Restart NFS/CIFS Services to apply WOS Parameters Message... 23 Figure 9 Synchronization option... 25 Figure 10 WOS Synchronization option... 26 Figure 11 Restart Notification Message... 26 Figure 12 Join Replication Group Window: Create New Replication Group... 27 Figure 13 Namespace Synchronization Started... 27 Figure 14 Join Existing Replication Group... 28 Figure 15 Synchronization Tab... 28 Figure 16 WOS NFS Configuration Section... 29 Figure 17 Restart Service Confirmation Screen... 29 Figure 18 NFS Export Point Sample Screen... 30 Figure 19 Creating NFS Export Point for All... 31 Figure 20 Granting Client and Root Access to those specified... 32 Figure 21 Edit/Delete NFS Export Points... 33 Figure 22 Start NFS Service... 33 Figure 23 NFS Service Status... 33 Figure 24 Export List section of the Configuration tab... 34 Figure 25 CIFS section of Configuration tab... 34 Figure 26 CIFS Service Running Status... 34 Figure 27 CIFS Service Configuration Screen... 35 Figure 28 Confirmation Message to confirm changes to CIFS Service Configuration... 36 Figure 29 Create New Export Point via CIFS... 37 Figure 30 Success Message... 38 Figure 31 Export list with created CIFS export point... 38 Figure 32 Delete CIFS share point confirmation window... 38 Figure 33 Join Active Directory Step 1... 39 Figure 34 Join Active Directory Step 2... 39 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 8

Figure 35 JoinActive Domain Result Message... 40 Figure 36 DC name... 40 Figure 37 Confirm Action to Leave Domain... 40 Figure 38 Enable Disaster Recovery Backup options Screen... 41 Figure 39 Disaster Recovery enabled... 41 Figure 40 Services Relocation Section... 43 Figure 41 Services Relocation Restart Notification Message... 44 Figure 42 Services Relocation Group Created... 44 Figure 43 WOS Access Preferences Tab Screen... 45 Figure 44 Email Alerts section on Preferences screen... 45 Figure 45 Email test status message... 46 Figure 46 Change Status Message... 46 Figure 47 Credentials section of Preferences screen... 47 Figure 48 Notification of Changes Message... 47 Figure 49 User Interface Preferences... 48 Figure 50 Notification of Changes Message... 48 Figure 51 Exporting syslog section of Preferences screen... 49 Figure 52 Notification of Changes Message... 49 Figure 53 WOS Access UI Screen... 50 Figure 54 Status Icons... 51 Figure 55 User Interface (UI) sample screen... 52 Figure 56 WOS Storage Protocols... 54 Figure 57 WOS Storage Chart... 55 Figure 58 System Synchronization Chart... 55 Figure 59 System Chart... 56 Figure 60 Synchronization Screen... 56 Figure 61 WOS Access Service tab screen... 58 Figure 62 Configuration tab screen... 60 Figure 63 Join AD Step 1... 64 Figure 64 Join AD Step 2... 64 Figure 65 Confirmation message... 65 Figure 66 Join Synchronization Group... 68 Figure 67 Preferences tab screen... 70 Figure 68 Sample Logs Screen... 71 Figure 69 Quota Notifications... 110 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 9

List of Tables Table 1 WOS Access Credentials... 18 Table 2 Administrative User Interface Tabs... 50 Table 3 Service Status Icons... 51 Table 4 Node Section Parameters... 57 Table 5 Total Section Parameters... 57 Table 6 WOS Access Status Parameters... 58 Table 7 Synchronization Parameters... 59 Table 8 Services-Relocation Parameters... 59 Table 9 Backup Parameters... 59 Table 10 Export List Parameters... 61 Table 11 CIFS Export List Parameters... 62 Table 12 NFS Parameters... 63 Table 13 CIFS Parameters... 63 Table 14 Join Active Directory Wizard... 64 Table 15 WOS Parameters... 65 Table 16 SR Service Parameters... 65 Table 17 Synchronization Service Parameters... 66 Table 18 Join To Synchronization Group Wizard.... 68 Table 19 Backup User Interface Options... 68 Table 20 Email Notifications Parameters (NFS/CIFS)... 68 Table 21 User Attribute Options... 72 Table 22 Modification Options of User Accounts... 73 Table 23 Group Attribute Options... 75 Table 24 Modifications Options for Group Membership... 75 Table 25 Users and Group Options... 76 Table 26 Group Membership Filters... 77 Table 27 Node management command-line tool... 83 Table 28 Nodectl Commands with optional parameters... 84 Table 29 SR Service Configuration Parameters... 87 Table 30 System Parameters... 91 Table 31 Policy description for tracked services... 96 Table 32 OVFS_BACKUP Parameters... 98 Table 33 OVFS_WOS Parameters... 99 Table 34 Backup_Ctrl Command Line Options... 102 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 10

Table 35 Backup Command Line Options... 102 Table 36 Restore Command Line Options... 102 Table 37 Block Name Usage Descriptions... 116 Table 38 Export... 117 Table 39 OVFS_WOS Block... 117 Table 40 OVFS_BACKUP... 119 Table 41 Cachelnode_Client Parameters... 119 Table 42 NFS Core Parameters... 120 Table 43 WOSAccess Sync Configuration Parameters... 122 Table 44 SR Configuration Parameters... 123 Table 45 WOSACCESS_CIFS Configuration... 123 Table 46 ClientProtocols key default values... 139 Table 47 Notable product directories and files... 142 Table 48 WOS Access TCP/UDP... 142 Table 49 Additional WOS Access TCP/UDP requirements for Active Directory Domain... 143 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 11

1.0 WOS Access Overview 1.1 Product Description DataDirect Networks WOS Access is a bundled server and software solution designed specifically to front the DDN Web Object Scaler (WOS) cluster with NFS and CIFS interfaces that provide a common, globally distributed namespace. With multiple WOS Access gateway nodes, distributed users can collaborate across geographic locations using NAS protocols. WOS Access supports two gateways to be configured for services relocation in an active/passive configuration. Up to eight gateway nodes (or four active/passive pairs) can be linked together to form a namespace synchronization group. Key features of WOS Access include: NFSv3 compatibility Basic NFSv4 compatibility (see the NFS Service section below for current NFSv4 limitations) CIFS (SMB 1.0, 2.0, and 2.1) compatibility Active Directory authentication support Support for local users and groups where Active Directory authentication is not available or desirable Synchronized namespace across remote sites (maximum of eight (8) gateway nodes) Multiple export points per gateway Local read and write cache Disaster Recovery (DR) protection Services Relocation WOS Access consists of the following key modules: NFS Service CIFS Service 1.2 NFS Service Namespace Synchronization Disaster Recovery (DR) Services Relocation (SR) Web User Interface (UI) The Network File System (NFS) Service is a network-based service allowing a user on a client computer to access files from a remote server in a manner similar to how local storage is accessed. WOS Access fully supports NFS version 3 (NFSv3) and has partial support for NFS version 4 (NFSv4). The following features of NFSv4 are not currently supported in WOS Access. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 12

Authentication mechanisms other than SYS_AUTH (i.e., AUTH_DH, Kerberos, LIPKEY, etc.) ID mapping (see below) Volatile file handles Access Control Lists (ACLs) o File Locking Permissions are stored only using NFS mode bits Delegations (Client-side caching) 1.2.1 NFSv4 ID Mapping A key part of any secure protocol is properly establishing identity of the users or groups that are attempting to use the protocol. In NFSv3, a common way of doing this is by using numeric identity values (i.e. UID/GID). However, this means that in order to share data between different NFS clients, all clients have to use the same UID/GID values. In some cases, this adds significant management overhead for system administrators. For this reason, and others, the NFSv4 protocol provides the ability to identify users and groups by name, rather than numeric ID. However, most UNIX-based systems require numeric IDs for file system attributes. This means that there must be a way to map NFSv4 user/group names to UID/GID values, consistently, on both the client and the NFS server. Currently, WOS Access does not support NFSv4 identity mapping and requires that NFS clients identify users and groups using UIDs and GIDs, respectively. Linux clients based on RHEL can be configured to do this by running the following command and rebooting the client: echo "options nfs nfs4_disable_idmapping=1" > /etc/modprobe.d/nfs.conf 1.3 CIFS Service The Common Internet File System (CIFS) Service is a network-based service allowing a user on a client computer to access files over a network in a manner similar to how a local Microsoft Windows shared folder is accessed. WOS Access CIFS fully supports Windows authentication via Active Directory or a local user/group database. Windows ACLs are fully supported. WOS Access CIFS supports SMB versions 1.0, 2.0, and 2.1. 1.4 Namespace Synchronization The Namespace Synchronization feature provides synchronization of the namespace metadata between multiple WOS Access gateways. This multi-master synchronization system is responsible for propagating the data modifications made by each node to the rest of the nodes in the same synchronization group or cluster. Namespace synchronization does not propagate the Active Directory (AD) state and changes across an existing cluster. This propagation can only be done when adding a new node to an existing synchronization group. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 13

1.5 Disaster Recovery WOS Access Disaster Recovery (DR) provides the ability to backup namespace metadata and the gateway configuration to WOS and to restore in the event of catastrophic failure. Backup can be executed manually or run according to a configurable schedule. 1.6 Services Relocation The Services Relocation (SR) feature provides an active-passive solution to relocate a Virtual IP address in the event of certain failures. When the IP address is relocated, applications that were performing I/O will need to restart their sessions, and CIFS clients may need to remap the export points. The following is a summary of the operations and support provided by Services Relocation and the Unified Namespace: NFS file data written to WOS Access is cached and flushed to WOS Core after 30 seconds (OVFS_WOS_CACHE_EXP_TIME) of file inactivity in order to optimize handling and converting NFS files into WOS Objects. This caching is performed on a file-per-file basis (I.e., any files inactive for 30 seconds will be flushed while any actives files will continue to be cached). WOS Access synchronizes its namespace database with other WOS Access and Service Relocation nodes every 10 seconds (SYNC_BATCH_INTERVAL) or 1000 name space changes (SYNC_BATCH_SIZE). Therefore, WOS Access effectively performs a checkpoint with other WOS Access nodes and WOS Core every 10 seconds for CIFS files and 30 seconds of inactivity for an NFS file. Upon failure of a WOS Access node, the state of files written to that node since the last successful checkpoint is undefined. When Services Relocation is configured, NAS Services will be relocated to a secondary node upon failure of the primary node, and any file data written to that failed node since the last checkpoint will need to be rewritten to the secondary node once the NAS Services have been relocated. Note that the Stale file handle error can occur on a client after fail-over of the NFS service, even if the file exists. The error will disappear when the NFS service is ready to work (relocation of NFS takes ~60 seconds). [DE13093] Services relocation does not guarantee high availability. While the active instance is being shut down and the passive instance being brought up, a brief interruption will be experienced. Once services relocation starts the services on the passive node, the NFS and CIFS shares, become available again but the data may not be fully available as the metadata synchronization from the Active node to the Passive node may not have taken place. Furthermore, NFS and CIFS clients would need to resubmit the write requests that were in flight at the time of services relocation. Any new NFS or CIFS requests will be satisfied by the node that has taken over the services. WOS Access supports these services relocation technologies: Service failure detection and automatic restart DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 14

Any services may die unexpectedly due to configuration or software problems. Services-Relocation cluster monitoring and restart services are available if a failure occurs. Clustering and Failover Clustering allows the components to be viewed functionally as a single entity from client-side for runtime processing and manageability. Server Migration Virtual IP (VIP) service can only have one instance running. If the active instance becomes unavailable, the service is automatically started on a different cluster member. NOTE: SR does not currently relocate the VIP to a secondary SR node in the event of a network connection failure between the WOS Access primary SR node and WOS Core. SR provides support for services relocation nodes pairing and four services tracking, so these tracked services can be reliably utilized with a minimum of downtime. The package remedies the situation, when system services fail, by detecting software faults, and immediately restarting the set of services on another node without requiring administrative intervention. Services Relocation (SR) does not support automatic procedure for the planned downtime. The planned downtime is a manual procedure in order to ensure that there is no data-loss. The procedure essentially comprises of manually stopping NFS/CIFS traffic going to the WOS Access active node; then allowing WOS Access NFS and CIFS on the active node to run for some time to allow the files on staging area to be pushed to WOS Core and the WOS Access namespace synchronization to synchronize the metadata to the Passive instance; and finally trigger an event on active node like shutting down the machine, network or services to let these services be relocated to the passive node. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 15

1.7 Web Administration User Interface Overview The WOS Access Administration User Interface (UI) provides simplified way to monitor and manage: Services Status, Storage Statistics, Installed Packages Synchronization Services Configuration Preferences Logs 1.8 Network Configuration Figure 1 below shows a possible network configuration for a multi-site WOS Access setup. In this example, there are three network connections specified: Protocol/external network used for connecting NFS and CIFS clients to the WOS Access gateway nodes. If the nodes are part of a Services Relocation pair then the connection should utilize the Virtual IP Address. Private network for storage provides the network connection to the WOS cluster. Heartbeat network If the nodes are part of a Services Relocation pair then the two nodes should be connected directly to each other on one or two dedicated interfaces. Figure 1 WOS Access Network Configuration Example DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 16

1.9 Sharing Files Between NFS and CIFS The following points should be considered when sharing files between NFS and CIFS protocols: On a Unix-based system, the filenames file and FILE are considered to be two different files. On Windows, the two file names are seen as the same file. So, if files with those names were created on NFS and then listed on Windows, you see both names but the content of only one of them. Windows doesn t support symlinks, so all files of that type are displayed as regular files (for example, a symbolic link to a directory created on the NFS mount using a Linux client). Due to incompatibility between NFS and CIFS protocols, not all file permissions created on an NFS mount will be available on the CIFS Share, and vice-versa. When using one export point for both NFS and CIFS, the NFS client will see an additional file within the directory as an artifact from CIFS. For example: drwx------ 3 root root 4096 Apr 26 13:02 :STREAM This extra file can by ignored by NFS users. In WOS Access, the Access time file attribute (i.e. atime ) is not supported on either NFS or CIFS. When reporting file or folder atime to NFS or CIFS clients, the mtime will be used instead. By default, WOS Access will auto-generate unique UID and GID values for CIFS users and groups. In an environment where WOS Access is being used for both CIFS and NFS access, these auto-generated ID values may conflict with NFS ID values, leading to unexpected permission issues and problems with using quotas. For a full explanation of the issue and potential workarounds, see Section 5.3.13. 2.0 Software Installation 2.1 Operating System Installation The WOS Access appliance ships with the operating system and product already installed. 2.2 Web Object Storage Prerequisites These WOS prerequisites are required by WOS Access: All nodes in the WOS cluster must be installed and configured. Please refer to the latest WOS Access Release Notes for WOS version requirements. At least one policy should be created. This policy will be used by WOS Access for storing data to the WOS cluster. If namespace synchronization will be used then a second, separate policy is required. The WOS Access gateway must be able to communicate through the network to at least one node in the WOS cluster. TCP ports 80, 8085, 8088, and 8090 are used by WOS Access to communicate with the WOS cluster. These ports must be accessible from all WOS Access nodes. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 17

NOTE: Contact DDN support for help in any of the above procedures. 2.3 WOS-Access Appliances If you purchased a WOS Access gateway appliance from DDN, please note: The default root password is DDNWOSaccess1". Please change this password immediately. Product installation logs may be found in /opt/ddn/nas/log. If the factory installation was successful, you should see a file in the log named wosaccess-install-successful. A detailed installation log is named wosaccess-install-<timestamp>.log. You must assign a hostname to your system and add it to/etc/hosts and /etc/sysconfig/network. o Ensure the hostname is not on the same line as localhost in /etc/hosts. o Ensure that the IP address is not localhost (127.0.0.1) and that you see 0% packet loss. o Try both ping commands to ensure correct hostname configuration. The ip addresses returned by both commands should match each other. For example, these commands produce the correct ping responses: [root@dell26 log]# ping -c 1 $(hostname ) PING dell26 (10.40.30.26) 56(84) bytes of data. 64 bytes from dell26 (10.40.30.26): icmp_seq=1 ttl=64 time=0.013 ms --- dell26 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.013/0.013/0.013/0.000 ms [root@dell26 log]# ping -c 1 $(hostname -i) PING 10.40.30.26 (10.40.30.26) 56(84) bytes of data. 64 bytes from 10.40.30.26: icmp_seq=1 ttl=64 time=0.008 ms --- 10.40.30.26 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.008/0.008/0.008/0.000 ms 2.4 Default passwords To ensure the security of your system, you should change the following default passwords immediately (Table 1). Table 1 WOS Access Credentials Feature User Password Notes Factory OS root DDNWOSaccess1 This is the default OS root password for the WOS Access gateway appliance. Use the passwd command to change it. CIFS Administrator Administrator DDNWOSaccess1 This is the default password for the CIFS Administrator user. To change this password, refer to section 3.9.2 for instructions. NFS n/a n/a NFS has no default user or password. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 18

Feature User Password Notes WebUI admin DDNWOSaccess1 Web interface access. To change this password, refer to please see section 3.6 for instructions. 2.5 Configuring WOS client I/O filters WOS Access is a NAS gateway to WOS storage. By default, WOS storage does not have any client I/O filters which means it allows any node to access the data given the Object identifier (OID). However to secure data access, WOS storage provides Administrator the ability to specify IP filters (IP address (or ranges) in CIDR format) in its management GUI to restrict hosts that can do I/O. As a security best practice you could set the I/O filters in the WOS to restrict the data access from the specific WOS Access nodes. 2.6 Verify WOS Access network connections between WOS Access Nodes and WOS Core A misconfigured network or networking issues between WOS Access server and WOS Core may lead to an overflow of staging area on NFS Server. Check and ensure networking settings on all servers and switches are properly configured and acceptable network connectivity and performance. 2.7 Adding a Mellanox ConnectX-3 Networking Card If the WOS Access Server does not have a Mellanox ConnectX 3 card installed during the WOS Access ISO install and it is added to the system, the Mellanox card will not be recognized as an Ethernet adapter. The fix is to execute the /sbin/connectx_port_config script from the shell prompt and select the Ethernet option. This configures the Mellanox card to be the Ethernet adaptor. Next, restart the Mellanox driver by executing /etc/init.d/openibd restart from the shell prompt. 3.0 WEB Configuration Procedure 3.1 WEB Configuration Steps The WOS Access configuration procedure has two variants: Single WOS Access server Multiple WOS Access servers. This includes Services Relocation. The multiple WOS sites have a synchronized name space 3.1.1 Single WOS Access Server If you have a single WOS Access server, perform: 1. WOS Storage Policy Configuration (Section 3.3) 2. WOS Access NFS Configuration (Section 3.4) 3. WOS Access CIFS Configuration (Section 3.6) DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 19

4. WOS Access Disaster Recovery Configuration (Section 3.7) 3.1.2 Multiple WOS Access Servers If you have multiple WOS Access servers (two or more), perform either step 1 or 2 and then step 3: 1. Create a new replication group (Namespace Synchronization) on first (head) server: a. WOS Access Configuration (Section 3.3) b. WOS Access NFS Configuration (Section 3.4) c. WOS Access CIFS Configuration (Section 3.6 ) d. WOS Access Namespace Synchronization Configuration (Section 3.4 ) 2. Join existing replication group (Namespace Synchronization): a. WOS Access Configuration (Section 3.3 ) b. WOS Access Namespace Synchronization Configuration (Section 3.4 ) c. Join Existing Replication Group (Section 3.4.3 ) d. WOS Access NFS Configuration (Section 3.4 ) e. WOS Access CIFS Configuration (Section 3.6 ) 3. After creating or joining existing replication group configure DR and SR: a. WOS Access Disaster Recovery Configuration (Section 3.7) b. WOS Access Services Relocation Service Configuration (Section 3.8) 3.2 WOS Access Login Page Access the administration user interface (Figure 2) with one of the recommended browsers by the web address http://<server_address>/. Default Web Administration credentials: user name - admin password - DDNWOSaccess1 NOTE: Although the default WEB UI account, admin, is PAM managed; you cannot use it to log in via SSH or TTY. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 20

Figure 2 WOS Access Login Screen Once you have logged in, you will need to configure WOS Access. Select the Configuration tab to access the configuration options (Figure 3). Figure 3 WOS Access Initial Configuration Screen 3.3 WOS Storage Policy Configuration To configure access to WOS, 1. Select the Configure button (can be grayed out) next to the WOS label on the Configuration screen (Figure 3). WOS configuration options are now available (Figure 4). NOTE: The user interface updates both NFS and CIFS configuration files with WOS parameters at the same time. However, if NFS and CIFS configuration files have different values set, NFS parameters are shown. (If the NFS and CIFS config files are different, you will see an error notification in the Web UI, and this needs to corrected please contact DDN support.) DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 21

Figure 4 WOS Configuration Section 2. On the WOS Configuration section of the screen, select the edit link next to the WOS IP label (Figure 4). You are prompted for an administration password (Figure 5). Figure 5 Password Prompt Box 3. Enter the password for the WOS Access Administrator and select the Edit button. 4. On the WOS Configuration section of the screen (Figure 4), enter the WOS IP address. 5. Click on the Refresh button (Figure 6) to update the list of available WOS policies. From the drop-down list, select the WOS policy (Figure 7)that will used to store file data in WOS. 6. Select the Apply (Figure 7) button and click Ok (Figure 8) to confirm. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 22

Figure 6 WOS IP Address Configuration Figure 7 WOS Policy Configuration Figure 8 Restart NFS/CIFS Services to apply WOS Parameters Message NOTE: If the list of WOS Policies does not appear, you should set WOS User and WOS Password using the same way described above for WOS IP field. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 23

7. If the WOS Policy in the CIFS and NFS configuration files are different, a warning symbol will appear next to the WOS Policy label. This suggests that one of the configuration files may have been edited manually. In this case, check each configuration file to ensure that the same WOS IP address and policy is being used in both files. 3.4 WOS Access Namespace Synchronization Configuration for Two Or More Servers Sharing the Same Namespace or for Service Relocation If you are deploying a single WOS Access server attached to a single WOS Cluster, this section is optional. If you are deploying WOS Access with Services Relocation (SR) or with a remote synchronized site, this section is mandatory. The steps needed to set up namespace synchronization are: 1. Configure namespace synchronization 2. Create or join a sync cluster. If creating a new namespace synchronization group, go to Section 3.4.2. To join an existing group, go to Section 3.4.3. The following requirements must be met before configuring namespace synchronization: All network interfaces that are used in the cluster configuration must be configured for IPv4. Each WOS Access server hostname must be resolvable by all other WOS Access servers in the replication group. This can be achieved in one of two ways: 1. Register the primary IP address of each WOS Access server with a centralized DNS server. This is the recommended method. 2. Create an entry for each WOS Access server in the /etc/host file on all servers in the replication group. All WOS Access servers must use NTP for time synchronization. The namespace synchronization scripts use SSH to get the replication states on remote servers. Each WOS Access server must be configured to allow root-level password-less SSH access from all other WOS Access servers. Please refer to your operating system documentation for information on how to configure password-less SSH access to a node. [DE12753] 3.4.1 Configuring Namespace Synchronization Before running the Namespace Synchronization (NS) service, which performs metadata replication, configure the NS service on each node using Synchronization option (Figure 9). For NS, you can specify these parameters: Batch Size Batch TTL (Days) Batch Interval WOS Policy DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 24

To change the WOS Policy: 1. See Section 3.3 for WOS Policy configuration. NOTE: You will use the same WOS Policy with CIFS and NFS services. You have to have a separate WOS Policy for Namespace Synchronization Service in order to keep the data related to the namespace synchronization separate from the file system data from CIFS and NFS. 2. Select edit from Synchronization pane (Figure 9), and enter in the WOS Access Administrator password (Figure 5), and press the Edit button. Figure 9 Synchronization option 3. Select your WOS Sycnronization Policy, which is different from the WOS Config Policy (Figure 10), and press the Apply button DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 25

Figure 10 WOS Synchronization option 4. On the Restart Notification Message (Figure 11), select the OK button to continue. Figure 11 Restart Notification Message 5. On the Synchronization screen (Figure 9), you can configure these parameters by selecting the edit link next to the parameter: Batch Size Batch TTL (Days) Batch Interval. 3.4.2 Creating New Replication Group Before a new replication group can be created, the following precondition must be met: The node cannot already be joined to a replication group. To create a New Replication Group: 1. On the Synchronization screen (Figure 9), select the Join button. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 26

2. On the Join Replication Group window, select the Create new Replication Group radio button. 3. Select the OK button (Figure 12). Figure 12 Join Replication Group Window: Create New Replication Group After creating a new group, The Stop button replaces the Start button in the New Replication Group Synchronization section (Figure 13). Figure 13 Namespace Synchronization Started 3.4.3 Join Existing Replication Group Before a WOS Access node can join an existing replication group, the following preconditions must be met: The NFS, CIFS and Synchronization services must be stopped on the local node. The database on the local node must be empty. This can be verified by running the following command and checking that the output matches: # psql -Uovfs -c "select count(*) from handle" count ------- 0 (1 row) If the command output does not match the above, then the database must be reinitialized before proceeding, using the following commands: # psql -U postgres c DROP DATABASE ovfs # /opt/ddn/nas/etc/ovfsdb_install_nfs.sh The CIFS, NFS, and Synchronization services must be running on the primary node in the existing replication group. The primary node is the node on which the replication group was created. To join an existing replication group: 1. On the Synchronization section (Figure 9), select the Join button. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 27

2. In the Join Replication Group window, select Join existing Replication Group radio-button (Figure 14). 3. In the text input box, enter the hostname of the primary node in the replication group. Only a hostname may be entered do not use an IP address. 4. Select the OK button. Figure 14 Join Existing Replication Group If the attempt to join the replication group fails, please refer to the Troubleshooting section of this guide for help in diagnosing the issue. After creating a new group, The Stop button replaces the Start button in the New Replication Group Synchronization section (Figure 12). To check the status of the replication group, select the Synchronization tab (Figure 15). Figure 15 Synchronization Tab NFS and CIFS services will need to be started. NFS mounts will not appear on the secondary nodes, you will need to either add those mounts on the secondary nodes, or configure an SR pair. 3.5 WOS Access NFS Configuration If your node needs to join an existing replication group, you must perform Namespace Synchronization Configuration before you configure NFS and CIFS service. Use the NFS section (Figure 16) to configure these parameters: Number of workers Start, stop, restart service DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 28

Figure 16 WOS NFS Configuration Section To configure NFS: 1. Enter the required number of workers. 2. Select the Apply button and click on Ok when the restart message appears (Figure 17). Figure 17 Restart Service Confirmation Screen 3.5.1 Creating NFS Export Points This section will review the steps necessary to manage NFS export points. NOTE: Unlike CIFS export points, NFS export points will not be synchronized across all members of a namespace synchronization group. NFS exported mounts are only synchronized between members of a Service Relocation pair. If the gateway node belongs to a synchronization group, then the export will need to be configured on the other nodes in the group as well, after time has been allowed for database synchronization to occur. You must specify these parameters to create the NFS export point: Export Name this is the name to be used when mounting the export from the NFS client File System Path this is the WOS Access namespace path associated with the export point, relative to /mnt To create the NFS export point: 1. On the Export List section under the Configuration tab (Figure 18), select the Add new button. 2. Enter the Export Name, for example, share. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 29

3. Enter the File System Path, for example, sharedfolder. 4. Select the Enable Export via NFS checkbox. Figure 18 NFS Export Point Sample Screen 3.5.2 NFS Protocol Configuration WOS Access allows administrators to specify NFS protocol version access on a per-export basis. The following protocols are currently supported: NFSv3 NFSv4 By default, NFS exports are configured for both NFSv3 and NFSv4. Use the checkboxes to add or remove protocol support. 3.5.3 NFS Security Configuration WOS Access NFS provides the following security capabilities: Namespace access at the export level Access authorization at the host/client level Root access at the host/client level Unix system authentication and authorization at the file level WOS Access provides the following options for securing access at the export level. These options apply to all users, groups and host systems. Read-Only File data and metadata is read-only, regardless of permissions. Read/Write (default) File data and metadata accessibility is dictated by file/folder permissions. Metadata Only Read-Only File data cannot be accessed in any way. File/folder attributes can only be read. Metadata Only - Read/Write File data cannot be accessed in any way. File/folder metadata accessibility is dictated by file/folder permissions. To configure the export-level security access, select the Access Type from the drop-down menu, as shown below: DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 30

Figure 19 Creating NFS Export Point for All DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 31

WOS Access also provides the ability to control export access at the host or client level. The default configuration allows any client to access the export but users attempting to access the export as root will have read-only access. Figure 20 Granting Client and Root Access to those specified Host entries can be specified as IP addresses or hostnames, using the * or? wildcards. Ranges can be specified using brackets (such as [3-9] ). Once all host entries have been added, click on the Apply button to create the export point. Click on the OK button in the dialog box to confirm creation and then restart the NFS service to make the export available to NFS clients. 3.5.4 Delete NFS Export Point To delete an NFS export, select the bold red next to the edit link (Figure 21) in the Export List. Confirm the deletion by selecting OK at the Delete Confirmation prompt. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 32

Figure 21 Edit/Delete NFS Export Points 3.5.5 Start NFS Service To start the NFS service, select the Start button on NFS section (Figure 22). Figure 22 Start NFS Service If there are no errors, the NFS section will indicate that NFS has started (Figure 23) by showing the option to stop the service. Figure 23 NFS Service Status 3.6 WOS Access CIFS Configuration If your node needs to join an existing replication group, you must perform Namespace Synchronization Configuration before you configure the NFS and CIFS services. The Configuration screen presents two sections: Export List section to create, modify, and delete CIFS export points (Figure 24) CIFS section to modify CIFS service status and configuration (Figure 25) Use the CIFS and Export List sections to: Check status of CIFS service Start, stop, or restart CIFS service Modify CIFS service configuration Join or leave Active Directory Domain DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 33

Add, modify, or delete CIFS export points Figure 24 Export List section of the Configuration tab Figure 25 CIFS section of Configuration tab 3.6.1 Changing CIFS Service Status NOTE: You must properly configure the WOS IP and the WOS policy before starting the CIFS service. On the CIFS service section (Figure 25): Select the Start button to start the CIFS service. Select the Restart button to restart the CIFS service. Select the Stop button to stop the CIFS service if it is running (Figure 26). Figure 26 CIFS Service Running Status As the CIFS service changes its status, the CIFS status icon automatically changes and presents the new status and valid buttons. For example, if the CIFS service is not running, you will not see a Stop button. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 34

3.6.2 Changing CIFS Service Configuration NOTE: CIFS service must be restarted to apply changes To change the default CIFS user password for Administrator (Figure 27): 1. Select the edit link next to the Administrator Password label. 2. Enter the new password (case-sensitive) in the revealed edit box. 3. Select the Apply button. Figure 27 CIFS Service Configuration Screen To change space for CIFS cache (Figure 27): 1. Either: a. Enter the new limit in the edit box next to the Staging Limit (MB) label. b. Or select the checkbox Unlimited. 2. Select the Apply button. 3. On the Confirmation Message popup window (Figure 28), select the OK button. To enable Data Integrity Check (Figure 27): 1. Select the appropriate checkbox. 2. Select the Apply button. 3. On the Confirmation Message popup window (Figure 28), select the OK button. To disable Data Integrity Check (Figure 27): 1. Select the checked Data Integrity Check checkbox to remove the check. 2. Select the Apply button. 3. On the Confirmation Message popup window (Figure 28), select the OK button. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 35

To cancel changes, select the Cancel button instead of the Apply button on the CIFS Service Configuration Screen. Figure 28 Confirmation Message to confirm changes to CIFS Service Configuration 3.6.3 Creating, Modifying, and Deleting CIFS Export Points This section will review the steps necessary to manage CIFS export points. NOTE: CIFS export points will automatically be synchronized across all members of a namespace synchronization group. Export points should only be created on one node of a synchronization group. To create a new export point via CIFS: 1. Select the Add new button on the Export List section of the Configuration tab screen (Figure 25). 2. Once this section expands, select the Enable Export via CIFS checkbox. 3. On the expanded CIFS configuration section (Figure 29), enter a CIFS export point name (case-insensitive) in the edit box under the Export Name label. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 36

Figure 29 Create New Export Point via CIFS 4. Enter new or existing relative path (case-insensitive) in the edit box under File System Path. 5. Choose the Access Type, select either Read/Write or Read Only radio button. 6. Check Enable Access Based Enumeration to enable access-based enumeration support. NOTE: Access Based Enumeration is set on a per-share basis and when enabled, displays only those files and folders to which a user has at least read-level permissions. This feature is disabled by default. To enable Access Based Enumeration, use the CIFS configuration section of the WOS Access Web GUI. The actual permissions setting on the files and folder is performed from Windows Client mounting the WOS Access CIFS share. 7. To add users to Allow Access to or Deny Access to the list of CIFS export points: a. Choose the appropriate new permissions radio button in the Access Permissions section. b. Enter a username (case-insensitive) in the text field under the Username label. 8. To add more than one user to the list: a. Select the bold green cross under the appropriate list. b. Enter additional usernames. 9. To remove a user from an access list, press the bold red cross next to the text field with username. NOTE: You may enter a domain username in the domainusername@fqdn format, for example: domain@mycompany.com. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 37

10. To clear appropriate Permissions List, select the Remove access list radio button. 11. After entering all data and selecting required options, select the Apply button to create the share. 12. On the Confirmation Message popup window (Figure 30), select the OK button. Figure 30 Success Message NOTE: Select the Cancel button to prevent the creation of CIFS export point. To modify a CIFS export point, click the edit link on the line of the CIFS export point you want to modify (Figure 31). The configuration of existing share is same as configuration of new share. Figure 31 Export list with created CIFS export point To delete a CIFS share, select the bold red next to the edit link (Figure 31). Confirm the deletion by selecting OK to the Delete Confirmation prompt (Figure 32). Figure 32 Delete CIFS share point confirmation window NOTE: More than one share can be created and modified at the same time. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 38

3.6.4 Join Active Directory Domain Since AD configuration is not propagated automatically to all nodes in an existing cluster, DDN recommends to setup AD configuration on primary synchronization group node before joining other nodes. When adding a new node to an existing synchronization group where all nodes already joined AD, the AD configuration will be propagated to new node automatically. In an existing synchronization group, an AD join operation should be performed on each node separately. To join Active Directory domain: 1. Select the Join button next to the Active Directory label at CIFS section of Configuration screen. 2. At the prompt, enter the Domain FQDN (case-insensitive) and IP address of Domain Controller (Figure 33). 3. Select the Next button. Figure 33 Join Active Directory Step 1 4. At the prompt, enter the Domain Administrator s username and its password. 5. Optionally, select the Synchronize Time checkbox to synchronize time with Domain Controller (Figure 34). 6. Select the Next button to join the Domain. Figure 34 Join Active Directory Step 2 7. If there are no errors, a success message appears (Figure 35); select the Close button. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 39

Figure 35 JoinActive Domain Result Message After the host is joined to the Domain, the DC name appears next to the Active Directory label and the Join button has changed to Leave button (Figure 36). Figure 36 DC name To leave the active directory domain, select the Leave button. At the prompt (Figure 37), select the Yes button to leave or Cancel button to stay. Figure 37 Confirm Action to Leave Domain 3.7 WOS Access Disaster Recovery Configuration 3.7.1 Enable Disaster Recovery Service WOS Access Disaster Recovery (DR) provides a feature to schedule full backups. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 40

The backup service by itself is quite I/O intensive. Therefore, if you run WOS Access SR setup with data replication between nodes, run the Backup service only on the passive node. Leave the Active SR node with backup running. DDN Recommendation: Only one active backup service per site To enable Disaster Recovery, select the Start button on the Backup option line (Figure 38) on the Configuration tab screen (Figure 3). Figure 38 Enable Disaster Recovery Backup options Screen If there are no errors, the Stop button replaces the Start button (Figure 39). Figure 39 Disaster Recovery enabled 3.7.2 Scheduling a Full Backup To schedule a full backup, from the drop-down menu on the Full Backup Schedule option line, select the number of days and then click the Apply button. 3.8 WOS Access Services Relocation Service Configuration Configure the Namespace Synchronization Service before configuring Services Relocation. The Services Relocation (SR) module configuration is only required if you need an active redundancy service. The SR module provides features to monitor and manage these services: Heartbeat network virtual IP of services relocation pair Database namespace database NFS service WOS Access service CIFS service WOS Access CIFS service NFS disk space service to monitor free space on a specific disk device, which used for NFS cache CIFS disk space service to monitor free space on a specific disk device, which used for CIFS cache DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 41

WOS service to monitor connection to WOS Storage nodes Before configuring SR, please ensure that these requirements are met: The two WOS Access gateway nodesthat will be paired together can be located in different subnets. However, they should still be visible to each other over the network. All network interfaces that will be used in an SR configuration should be configured for IPv4. Two UDP ports to be used for the Corosync service must be opened over the network. (Refer to AIS_UCASTPORT description in Table 27.) Hosts should be visible to each other over the network by hostnames. To achieve this requirement, the hostnames should either be configured by using the corporate DNS server, or be specified in /etc/hosts. Hosts should be accessible via SSH without password requirements. Primary Host has virtual network interface. Before configuring Services Relocation, first configure the nodes individually per the requirements above. All requirements stated above must be met prior to configuration. Configuration of Services Relocation is only required on one of the two nodes that will be paired together. The other node will be automatically configured. To configure Services Relocation: 1. Expand the Services Relocation section on the Configuration page of the Web UI (Figure 41). DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 42

Figure 40 Services Relocation Section a. Supported Services - select the NAS services that should be monitored NFS, CIFS, or both. b. Local Heartbeat Interface #1 select the network interface on the local node that will be used as the primary heartbeat link between the two nodes. c. Local Heartbeat Interface #2 select the network interface on the local node that will be used as the backup heartbeat link between the two nodes (this setting is optional). d. Remote Node specify the hostname of the other node in the SR pair. This must be the real hostname of the other node (as reported from the hostname command) not an IP address. After specifying the hostname, click on the Reload Interfaces button to load the remote network interfaces. e. Remote Heartbeat Interface #1 select the network interface on the remote node that will be used as the primary heartbeat link between the two nodes. f. Remote Heartbeat Interface #2 select the network interface on the remote node that will be used as the backup heartbeat link between the two nodes (this setting is optional). g. Virtual IP Address - enter the IP address that will be used for providing NAS services. If services fail on the primary node, this IP address will be migrated to the secondary node. h. Virtual IP Mask- enter the network mask to be used by the Virtual IP address, either as a number in the range [1..31] or in IP format (i.e. 255.255.255.0). i. Virtual IP Interface select the network interface that will be used to host the DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 43

Virtual IP Address. The interface must be present on both the local node and the remote node. j. Primary Node select the node that will host the Virtual IP Address when the SR service is started. k. Select the Apply button to save the configuration changes to both the local node and the remote node. 2. On the Services Relocation Restart Notification window (Figure 41), select the OK button to confirm and continue. Figure 41 Services Relocation Restart Notification Message To create the Services Relocation pair and start the service: 1. On the Services Relocation section, select the Start button. 2. After creating an SR pair, the Stop button replaces the Start button in the Services Relocation section on the Primary and Secondary hosts (Figure 42). Figure 42 Services Relocation Group Created 3.9 WOS Access Preferences Use the WOS Access Preferences screen (Figure 43) to configure the settings for: email alerts, credentials (login password), exporting syslog user interface DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 44

Figure 43 WOS Access Preferences Tab Screen 3.9.1 Changing Email Alerts Preferences To configure email notifications, access the Email Alerts section on the Preferences tab screen (Figure 44). Figure 44 Email Alerts section on Preferences screen DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 45

To configure email notifications about backup events, select the Backup Notifications checkbox. To configure email notifications about CIFS service events, select the CIFS Notifications checkbox. To receive email notifications: 1. In the Mail Server box, enter hostname (case-insensitive) or IPv4 address of mail server which accepts SMTP requests. Required. NOTE: You may use the hostname, localhost here if the Mail Server allows for unauthenticated message delivery and the MX DNS record points to an address that can be reached from the WOS Access node. 2. Optionally, to set the subject of email notifications, enter text in the Subject box. 3. In the From user box, enter the username (case-insensitive) to be used to send email notifications via mail server. Required. 4. In the Notification E-mail, enter the email address which will receive notifications. Required. 5. To add some additional addresses, select the Add new button in Email Alerts section and enter email address in the box which appears. 6. To remove email address from notification list, select the bold red x next to appropriate editbox. 7. To validate your email preferences settings, select the Send Test Email button. This generates a test email message using the values you have entered. A message window with the results of the test email sending operation is displayed (Figure 45). Figure 45 Email test status message To refuse to change email preferences, select the Cancel button on Email Alerts section. To save email preferences, select the Apply button. Message with results of changing email preferences appears (Figure 46). Select the OK button to continue. Figure 46 Change Status Message DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 46

3.9.2 Changing Credentials Note: The default password of WOS Access Web UI admin is DDNWOSaccess1. To change the password of WOS Access Web UI administrator: 1. On the Credentials section of the Preferences screen (Figure 47), enter the current password (case-sensitive) in the Current Password box. 2. Enter the new password (case-sensitive) in the New Password box. 3. Enter the new password (case-sensitive) in the Retype Password box. Figure 47 Credentials section of Preferences screen To keep the current password, select the Cancel button. To save the new admin password, select the Apply button. Notification of changes message appears(figure 48); select the OK button to confirm and continue. Figure 48 Notification of Changes Message 3.9.3 Changing User Interface Preferences To change the Auto-Refresh Data interval, select the appropriate interval from the dropdown menu on the User Interface Preferences section on the Preferences screen (Figure 49). DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 47

Figure 49 User Interface Preferences To keep the current auto-refresh data interval, select the Cancel button. To save the new auto-refresh data interval, select the Apply button. Notification of changes message appears (Figure 50); select the OK button to confirm and continue. Figure 50 Notification of Changes Message 3.9.4 Changing Exporting syslog To push syslog messages to a remote server using the syslog daemon, at the Exporting syslog section of the Preferences page (Figure 51), select the Add new button. In the revealed Facility and Severity drop-boxes, select the appropriate values. In the Export Address box, enter the IPv4 address of the remote machine that is running rsyslog. NOTE: More than one export address can be added at the same time. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 48

Figure 51 Exporting syslog section of Preferences screen To refuse to push syslog messages to export address, click the bold red cross next to appropriate edit box. To refuse to change exporting syslog preferences, click the Cancel button. To save exporting syslog preferences, select the Apply button. Notification of changes message appears (Figure 52); select the OK button to confirm and continue. Figure 52 Notification of Changes Message DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 49

4.0 Web Administration Reference You can access the administration user interface using any web browser client http://<server_address>/. Default Web Administration credentials: user name admin password DDNWOSaccess1. Recommended versions of browsers: Internet Explorer l0 Firefox 27 Chrome 33 These sections provide basic information regarding features exposed via the User Interface. refer to the administrative guides of respective services for the most detailed and up-to-date configuration and usage information. 4.1 General Navigation There are six areas in Administrative UI (represented on the tabs in Figure 53 and explained in Table 2): Summary Synchronization Services Configuration Preferences Logs Figure 53 WOS Access UI Screen Table 2 Administrative User Interface Tabs Service Summary Synchronization Services Configuration Preferences Document Provides summary view of the cluster statistics and node system information. Provides real-time view of synchronization process. Summary for all WOS Access services. WOS Access services can be configured and restarted from this page. Allows configuring preferences such as email notification and user settings. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 50

Service Logs Document Provides access to service logs. 4.2 Status Icons Administrative User Interface uses unified icons to display service statuses across different pages (Table 3). Administrator may see such icons as part of other icons (Figure 54), but they will still mean exactly the same status everywhere. Figure 54 Status Icons Table 3 Service Status Icons Service Icon Description Service is not installed. Service is started active. Service is stopped / inactive. Service is in idle state (only applies to Services-Relocation Service). Service state is unknown. Service reports errors. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 51

4.3 User Interface This section discusses the User Interface (UI). 4.3.1 Summary Screen A sample Summary tab screen is shown in Figure 55. Figure 55 User Interface (UI) sample screen 4.3.1.1 Service Status Service Status displays information about: Uptime and status of these services o NFS Server o CIFS Server o Services-Relocation o Synchronization o Backup Backup service also shows last time the file system backup was made. Synchronization services show the number of changes in the incoming and outgoing queue. The incoming queue keeps changes coming from other WOS Access servers and it keeps it there until they are applied to the file system on this node. The DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 52

4.3.1.2 Storage Statistics outgoing queue holds file system changes sent to all other nodes in the WOS Access cluster and they will be kept in the outgoing queue until target servers confirm that they processed those changes: o Low and steady numbers during operations with file systems indicate that file systems on different nodes are kept in sync with minimal timeout. o A high number in incoming queue indicates that current WOS Access node is behind the file system changes done on remote nodes. And it may take some time for it to get synchronized o A high number in outgoing queue indicates that some or all remote nodes have large numbers of changes, which they have not applied to their file systems yet. o A growing number in the incoming queue means that the local WOS Access node is lagging behind since it cannot process all changes from other nodes at the same rate that they arrive. o A growing number in outgoing queue means that remote nodes do not send confirmations for processed changes and that they are lagging behind. o Zeros in the queues indicate that there is no activity which changes a file system. It does not mean there is no activity at all. For example, reading a file does not change a file system. The Storage Statistics are represented with a pie chart. The legend at the left of the pie chart gives the colors meanings, and shows relative values for used and free space: Total/Active Nodes. The number of Total and Active Nodes Object Count. Displays number of objects stored Capacity / Used Capacity / Free Capacity. Displays storage usage information Cluster Data represents space used by file objects stored by WOS Access cluster Synchronization displays space used by Synchronization service to transmit file system changes between nodes in WOS Access cluster Other WOS Consumers shows space used by any other client applications, which may also use the same WOS Cluster. NOTE: The system distinguishes the objects stored by the different clients based on the WOS policy used. If all clients use the same policy Cluster Data, Synchronization and Other WOS Consumers show identical information. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 53

4.3.1.3 System Charts: Protocols The Protocols chart (Figure 56) displays CPU usage by WOS Access protocols (NFS and CIFS) and staging storage usage. Figure 56 WOS Storage Protocols 4.3.1.4 System Charts: WOS Storage The WOS Storage chart (Figure 57) displays performance characteristics for WOS Cluster: Rate Displays number of Read, Write and Delete operations per second going through WOS Cluster Throughput Displays Read, Write and Delete operations going through WOS Cluster Latency in milliseconds Transactions in thousands of operations DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 54

Figure 57 WOS Storage Chart 4.3.1.5 System Charts: Synchronization The System Synchronization chart displays number of messages in incoming and outgoing queues. Refer to page 52 in section 4.3.1.1 for queue details). Figure 58 System Synchronization Chart 4.3.1.6 System Charts: System The System chart displays the system characteristics: Memory usage CPU usage Incoming and Outgoing traffic DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 55

Figure 59 System Chart 4.3.1.7 System Charts: Controls There are two controls available for the above charts: Auto-Refresh Data Every Controls the update frequency Display Data For Controls how long the data will be visible on the chart. 4.3.1.8 Installed Packages Chart The Installed Packages chart displays any WOS Access packages installed on the node. Optionally, you may use the CLI command to display the same list: rpm -qa grep wos. 4.3.2 Synchronization Screen The Synchronization screen displays the status of each node in the WOS Access cluster (Figure 61). Figure 60 Synchronization Screen DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 56

4.3.2.1 Nodes Section of Synchronization From the Synchronization tab screen, select the Nodes button to navigate to the Nodes view (Figure 61). The Nodes section provides synchronization status for each node in the WOS Access cluster. Parameters and descriptions are listed in Table 4. Table 4 Node Section Parameters Parameter Name Memory Disk Space Uptime Description The name of the node within the synchronization group. System memory used by synchronization (SI units) System disk space available to synchronization (SI units) Displays synchronization service uptime. The Node section uses color coding for parameters. Green areas mean no problems detected. Place the cursor over the red areas to see the reason for the alarm. 4.3.2.2 Total Section of Synchronization From the Synchronization tab screen, select Total to navigate to the Total view. The Total section displays current snapshot of messages going through this node. Parameters and descriptions are listed in Table 5. Table 5 Total Section Parameters Parameter Ready Unacknowledged 4.3.3 Services Screen Description Number of messages ready for delivery Number of messages waiting for confirmation from remote nodes. Total Total number of messages on this node: Ready + Unacknowledged. Message rate Total rates for all queues The Services tab screen (Figure 61) is a dashboard displaying the status of WOS Access Services. Each section has an icon; place the cursor on this icon to see a tool tip displaying current service status. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 57

Figure 61 WOS Access Service tab screen 4.3.3.1 WOS Access Table 6 lists the WOS Access Status parameters for NFS and CIFS services. Table 6 WOS Access Status Parameters Parameter IP Address NFS Service CIFS Service WOS IP WOS Policy Export List Description Displays the IP address of this WOS Access Node. This IP can be used to access it; however it will not guarantee relocation services provided by Services-Relocation. Use the virtual IP address displayed in Services- Relocation section for fail safe WOS Access cluster use. Displays status of NFS service. Displays status of CIFS service. Web Object Scaler cluster s IP. WOS policy name to be used for file objects stored via WOS Access. Displays a list of export points: protocols, export point names and local folder name that are exported. 4.3.3.2 Synchronization The Synchronization section displays the status for WOS Access namespace synchronization across gateways on different sites. Table 7 lists the WOS Access Synchronization parameters and descriptions. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 58

Table 7 Synchronization Parameters Parameter Node Name Sending Service Status Receiving Service Status Sync Port WOS Policy List of member servers Description Node name is shown in brackets after Service Status title. This service is responsible for sending locally produced changes to other nodes. This service is responsible for receiving and processing changes produced by other nodes in cluster. Port that is used by synchronization process to communicate between WOS Access nodes. WOS policy name that is used to store and retrieve synchronization messages. List of Member Servers shows a list of nodes in the synchronization cluster and their statuses. Queue size displays number of messages in each node s incoming and outgoing queue. 4.3.3.3 Services-Relocation Table 8 represents parameters displayed inside the High-Availability section. Table 8 Services-Relocation Parameters Parameter Virtual IP Service Status List of Failover Servers Description The IP address that gets relocated between nodes in a Services- Relocation pair in the event of certain failures. When the IP address is relocated, applications that were performing I/O will need to restart their sessions, and CIFS clients may need to remap the export points. Shows service statuses for all controlled services (IP Address, Metadata Database, and NFS service). The status can be either Running, Stopped, or Unknown. Shows a list of nodes in the Services Relocation cluster and their statuses. The possible status values are Running, Stopped, or Idle. The active node is indicated by Active and the standby node in indicated by Passive. The active node is the one that is currently serving requests from the WOS Access clients. 4.3.3.4 Backup The Backup section shows information about the status of the metadata database backup service and information regarding the last few backup executions. Table 9 displays this backup information. Table 9 Backup Parameters Parameter Backup Schedule Last 5 backup results Description Displays scheduled backups Information shown includes: Backup date and time DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 59

Full backup Backup file size Type of backup file: full Status of performed backup operation. Information about when the last full backup was executed successfully is displayed just below last five backup results table in case there are no successful full backups shown among those five results. 4.3.4 Configuration Screen Access the Configuration tab screen (Figure 62) to modify WOS Access Services parameters. NOTE: Any changes made to these service parameters, once saved, are not applied automatically. You must restart the Service for these changes to take effect. NOTE: Parameters with edit links are password protected. To enable editing, enter the UI login password. Figure 62 Configuration tab screen 4.3.4.1 Export List The Export List section of the Configuration screen (Figure 63) displays the list of exports for the CIFS and NFS protocols. Use the Enable checkbox to enable/disable the Exports for each protocol. Table 10 lists the export list parameters. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 60

Table 10 Export List Parameters UI option Responsible Variable Description Name EXPORT:Tag Export point name. This name is used when mounting the share on the client. File System Path EXPORT:Path WOS Access file path associated with export point. This is path relative to /mnt. NFS Export Point Settings Access Type EXPORT:Access_Type Used to limit what clients can do with information available via an export point: Read Only Mode. File system is read-only (for data and metadata); Read/Write Mode. All read/write operations are allowed on file system; Metadata Only (Read/Write) Mode. Read and write data operations are forbidden. However, all metadata operations are allowed (mkdir, create, remove); Metadata Only (Read Only). File system is read-only for metadata. Both Data Write and Data Read operations are forbidden. Client Access Root Access EXPORT: CLIENT {Clients =... ; Squash=Root} EXPORT: CLIENT {Clients =... ; Squash=None} Provides access to a list of nodes, networks, and groups. UNIX authentication and authorization applies. The hostname may be defined as a valid IPv4 address or fully qualified network name, and may contain wildcards (*?). Grants access to the local root account s file system to a specific list of nodes, networks, and groups. The hostname may be defined as a valid IPv4 address or fully qualified network name, and may contain wildcards (*?). DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 61

Table 11 CIFS Export List Parameters UI option Responsible Variable Description Access Type Access Permissions Not applicable Not applicable Used to create an export point where all read/write or read-only operations are permitted. Used to enable Access Based Enumeration: defining new permissions for a user (Set new permissions), removing access list (Remove Access list), or leave the access list unchanged. Access Based Enumeration is set on a per-share basis and, when enabled, displays only those files and folders to which a user has at least read-level permissions. This feature is disabled by default. To enable Access Based Enumeration, use the CIFS configuration section of the WOS Access Web GUI. The actual permissions settings on the files and folder is performed from a Windows Client when mounting the WOS Access CIFS share. Username Not applicable Used to specify a list of users for the Allowed or Denied lists. These commands can be used to check user accounts known to CIFS server, run the command: /opt/ddn/nas/bin/lw-lsa findobjects Administrator@FQDN. Where FQDN is a fully qualified domain name of CIFS server (if you would like to check local administrator account) or domain controller (if you would like to check account for domain administrator). Active Directory users will only be available after the CIFS server has joined the domain. This command returns the full list of users known to CIFS server: /opt/ddn/nas/bin/lw-lsa enumobjects NOTE: CIFS does not keep exports in a configuration file. CIFS exports can only be managed via CLI or WOS Access UI interface. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 62

4.3.4.2 NFS 4.3.4.3 CIFS Table 12 lists the configurable parameters in the NFS section. Table 12 NFS Parameters UI option wosnas.nfsd.conf variable Description Number of Workers NFS_Core_Param:Nb_Worker Table 13 specifies the parameters in the CIFS section. Table 13 CIFS Parameters Defines the number of worker threads which are responsible for data transfer between the file staging area in an NFS and WOS Cluster. In general, a higher number of threads may give you better data throughput. However, it depends on multiple factors such as: Client and WOS Access server performance; Number of active client connections. Twenty two worker threads is the recommended value. UI option env.conf variable Description Active _ Allows joining a specified domain. See Directory Active Directory Configuration for details. Staging Limit Data integrity Check CACHE_SIZE 4.3.4.4 Active Directory Configuration DATA_INTEGRITY_ENABLED Limits the amount of data, specified in megabytes or as a % of total disk space on the node, that will be used when staging files for modification or when the WOS cluster is inaccessible. To use all available disk space, specify -1 for this value. Allows turning on MD5 sum calculation on write operations so that data can be validated on read operation. Enabling this option will affect both read and write operations. Use the Active Directory wizard to specify the parameters necessary to join the specified domain allowing available users to access the node. Table 14 describes configuration process. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 63

Table 14 Join Active Directory Wizard Steps UI option Description 1 Domain Fully qualified domain name that WOS Access node needs to join. IP Address IP address for DNS server that should be used to resolve name provided in Domain parameter. 2 Administrator Username Password Synchronize time 3 Confirmation Message Figure 63 Join AD Step 1 Domain user with enough permission to add new servers into domain. Password for specified user. This parameter allows synchronizing time on a WOS Access node prior to joining a domain to prevent possible user authentication issue due to tokens, which might be treated as expired based on current time. Message with successful/unsuccessful information. Figure 64 Join AD Step 2 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 64

Figure 65 Confirmation message 4.3.4.5 WOS Table 15 lists the parameters and descriptions for the WOS section. Select the Configure button to access the WOS Cluster UI screen. Table 15 WOS Parameters UI option wosnas.nfsd.conf variable Description WOS IP OVFS_WOS:OVFS_WOS_CLUSTER Web Object Scaler Cluster address WOS Policy OVFS_WOS:OVFS_WOS_POLICY Web Object Scaler Policy name that needs to be used for file storage WOS User OVFS_WOS:OVFS_WOS_USER Username to access Web Object Scaler cluster WOS Password OVFS_WOS:OVFS_WOS_PASSWORD Password to access Web Object Scaler cluster 4.3.4.6 Services-Relocation Table 16 lists the Service Relocation Service parameters. Table 16 SR Service Parameters UI option cluster.conf variable Description Supported Services Local Heartbeat Interface #1 Local Heartbeat Interface #2 SUPPORT_SERVICES AIS_NETADDR AIS_NETADDR_RESERVE Specifies the NAS services that will be monitored for failover. Should be nfs, cifs or nfs,cifs. Network interface to be used by Corosync for unicast transmission for local host. Only IPv4 is supported. Secondary network interface to be used by Corosync for unicast transmission for local host. Only IPv4 is supported. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 65

UI option cluster.conf variable Description This is optional. Remote Node Remote Heartbeat Interface #1 Remote Heartbeat Interface #2 Virtual IP Address Virtual IP Mask Virtual IP Interface 4.3.4.7 Synchronization PRIMARY_NODENAME or SECONDARY_NODENAME AIS_REMOTE_NETADDR AIS_REMOTE_NETADDR_RES ERVE VIP VIP_MASK VIP_INTERFACE The hostname of the remote node in the SR pair. Whether this node is primary or secondary depends upon the setting of the Primary Node option. Network interface to be used by Corosync for unicast transmission for remote host. Only IPv4 is supported. Secondary network interface to be used by Corosync for unicast transmission for remote host. Only IPv4 is supported. This is optional. Virtual IP address shared between nodes. Only IPv4 is supported. The netmask of the Virtual IP Address, specified either as a number in the range [1..31] or in IP format (for example, 255.255.255.0). The NIC card interface (can be a bonded interface) where the Virtual IP will be hosted. Synchronization Service only allows changing minimum set of parameters (Table 17). These settings appear in /opt/ddn/nas/etc/wosnas-cluster.conf. Table 17 Synchronization Service Parameters UI option wosnas-cluster.conf variable Description Synchronization Batch Size SYNC_BATCH_SIZE Defines number of file data or attribute updates, which have to be accumulated in the synchronization queue of this node prior to sending those changes across other nodes from the synchronization cluster. Batch size = (Batch timeout) * (Expected NFS share load, files per sec) * 5 (each file creation is 4 DB update, plus 1 additional). Batch timeout is a maximum DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 66

UI option wosnas-cluster.conf variable Description replication system s response time; that is, the timeframe when local changes are being replicated to all nodes in the replication cluster. For example, (Batch timeout) = 10 sec, (Batch size) = 10 sec * 50 files/sec * 5 = 2500. Batch Interval SYNC_BATCH_INTERVAL_SEC Defines timeout in seconds. The synchronization service will send changes for synchronization after each such interval. Batch TTL SYNC_BATCH_TIMETOLIVE_D AYS Defines number of days within which synchronization batches will be kept. These batches allow incremental recovery of other nodes if that becomes necessary. Batches, which are older than Batch TTL, will be removed and the only way to send such old changes to other nodes in case they need to recover will be to send full database dump. WOS Policy SYNC_WOS_POLICY Defines WOS Cluster policy name to be used for communicating synchronization messages across the WOS Access cluster. It is recommended to use two different policies between NFS and CIFS. 4.3.4.8 Adding A New Node to a Cluster Synchronization Group The Join to the Synchronization group, the wizard adds a new node to a synchronization group. A replication group is a logical set of nodes which replicate the local changes of a database to all other nodes in the group. If a new node joins the replication group, the other nodes add the new node to a list of known replication nodes. You do not need to manually join the nodes to each other. A node cannot be joined to a replication group: if there is no existing DB; if the tables in DB are not empty (for all new nodes except first); if the specified node ID is already used by some other nodes in replication group. In each case, an additional notification with short description is printed. Table 18 describes the configuration process. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 67

Table 18 Join To Synchronization Group Wizard. Step UI option Description 1 Create new Replication Group Join existing Replication Group Create new replication group. Short hostname of destination host which is already in replication group. This must be a hostname, not an IP address. 2 Confirmation If we successfully entered the replication group no error messages should appear and Join button should also disappear. Figure 66 Join Synchronization Group 4.3.4.9 Backup Access the Backup section to specifying the number of days between performing a full backup (Table 19). Table 19 Backup User Interface Options UI option wosnas.nfsd.conf variable Description Full Backup Schedule 4.3.5 Preferences Screen OVFS_BACKUP:OVFS_BACKUP_DB_FUL L_HOURS Amount of time between full backups. The Preferences tab screen (Figure 68) allows changing e-mail notification settings, admin password; change automatic data refresh frequency and allows setting up Log file export parameters. 4.3.5.1 E-mail notifications This section allows configuring e-mail notifications for certain WOS Access cluster events and testing e-mail settings, once they re entered, by sending a test message (Table 20). Table 20 Email Notifications Parameters (NFS/CIFS) UI option wosnas.nfsd.conf variable Description Supported NFS Events Mail Server OVFS_BACKUP:OVFS_MAIL_ SERVER Subject OVFS_BACKUP:OVFS_MAIL_ SUBJ Mail server to send messages from Subject of the message DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 68

UI option wosnas.nfsd.conf variable Description From User Notification E- mail Backup Notifications OVFS_BACKUP:OVFS_MAIL_ USER_FROM OVFS_BACKUP:OVFS_MAIL_ ADDR_TO OVFS_BACKUP:OVFS_BACKUP_MAI L_NOTIFICATION Local user to be set as sender Destination address (may be multiple, in this case addresses are merged using comma as a separator before writing to the configuration file) Set to yes to enable. Set to no to disable. Supported CIFS Events UI option CIFS Notification /opt/ddn/nda/share/config/env. conf variable CIFS_MAIL_NOTIFY Description Set to y to enable. Set to n to disable. 4.3.5.2 Credentials This section allows changing admin password. Actually runs just runs CLI utility passwd to change password for UNIX user admin. Current password is required for setting a new one. New password and its confirmation must match each other. 4.3.5.3 User Interface preferences This section allows setting data refresh frequency. If configured information on Summary, Synchronization and Service pages refreshed with selected frequency. 4.3.5.4 Export syslog This section allows setting up pushing selected log events to specified remote node. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 69

Figure 67 Preferences tab screen 4.3.6 Logs Screen Figure 68 shows an interface for viewing logs written by four services: NFS, CIFS, Backup, Sync and SR. Specified number (1...200) of strings from the end of the specified log file are shown in direct order. To automatically enable/disable update log lines on user s screen every 5 seconds, select the Auto-refresh checkbox. To open a modal pop-up that contains a directory listing with logs, select the Export button. The files are grouped by their names in order to get rotated logs together. Select the Export button under the list to pack selected logs into a tarball to download it. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 70

Figure 68 Sample Logs Screen 5.0 Advanced Configuration Procedure 5.1 CIFS Service Shared Folder Configuration After the installation, the WOSACCESS CIFS package has only one default share (IPC$) to be used for administrative tasks through the Microsoft Management Console. To setup additional shares in WOSACCESS CIFS, you should use the lwnet utility. When creating shares, use the value of the CIFS_ROOT_SHARE_PATH parameter as the root folder. For example, to create a shared folder/test when $CIFS_ROOT_SHARE_PATH=/mnt, you should actually use /mnt/test: # /opt/ddn/nas/bin/lwnet share add test="c:\mnt\test" After the creation of shares, enable the share to allow the user to access it. For example, enable access for Administrator on share test: # /opt/ddn/nas/bin/lwnet share set-info test --allow Administrator When a share is created for administrator of a specified domain, the user name should contain the Fully Qualified Domain Name (FQDN). For example: # /opt/ddn/nas/bin/lwnet share set-info test --allow Administrator@domain DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 71

NOTE: The share is created for access only from the accounts of the administrators group. The access for the user accounts should be granted by the administrator. To set the security permissions for specific folders inside the share, DDN recommends using the Security Attributes manager in Windows Explorer. To delete an existing share, run the command: # sudo /opt/ddn/nas/bin/lwnet share del <shareid> where <shareid> is the share to be deleted. For example, to delete the test share: # sudo /opt/ddn/nas/bin/lwnet share del test 5.2 CIFS Service Security Configuration The CIFS Service provides authorization on the host and the file system levels and supports both Unix system authentication and authorization and MS Windows authentication and authorization, including Active Directory. 5.3 CIFS Users and Groups Management 5.3.1 Add a new user To add a new user to the local authentication database, run the command: /opt/ddn/nas/bin/lw-lsa add-user --uid 2500 <account> where <account> is the new user account name. You can set the user attributes with additional options (Table 21). Table 21 User Attribute Options Option home shell uid group sid Explanation Home directory Login shell User id (uid) Primary group name Security identifier (SID) Add the options in the command before the <account> name. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 72

For example, to add a user with specific user identifier 2500: 5.3.2 Delete an existing user /opt/ddn/nas/bin/lw-lsa add-user --uid 2500 <account> To delete an existing user account, use the command: /opt/ddn/nas/bin/lw-lsa del-user <account> where <account> is the user account name. To delete a user with a specific UID: /opt/ddn/nas/bin/lw-lsa del-user --uid <UID> where <UID> is the specified user ID. For example, to delete user with UID=2500: /opt/ddn/nas/bin/lw-lsa del-user uid 2500 5.3.3 Modify user account attributes You can modify a user's account settings in the local authentication database, including an account's expiration date and password. You can perform these tasks: enable a user disable a user unlock an account add a user to a group remove a user from a group. The general syntax of the command for account modifications is: /opt/ddn/nas/bin/lw-lsa mod-user {modification options} (<user login id> --uid <uid>) where: {modification options} determine user s account attributes, and (<user login id> --uid <uid>) specifies user account to be modified. Table 22 lists the options available for modification. Table 22 Modification Options of User Accounts Option --disableuser Description Disable user account. A disabled account is not used in authentication procedures. For example: /opt/ddn/nas/bin/lw-lsa mod-user --disable-user newuser DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 73

Option --enableuser --setpassword -- passwordneverexpires -- passwordmustexpire --setaccountexpiry --add-togroups --removefromgroups setprimarygroup Description Enable user account. An enabled is used in authentication procedures. For example: /opt/ddn/nas/bin/lw-lsa mod-user --enable-user newuser Set the password for the user in plain text form. For example, to set password 01234567 for user newuser: /opt/ddn/nas/bin/lw-lsa mod-user --set-password 01234567 newuser Set the user password to never expire. For example: /opt/ddn/nas/bin/lw-lsa mod-user - password-never-expires newuser Set the user password to expire. For example: /opt/ddn/nas/bin/lw-lsa mod-user --password-must-expire newuser Note: You must define the date of expiration for the user account. Set the expiration date for this account in YYYY-MM-DD format. For example: /opt/ddn/nas/bin/lw-lsa mod-user --set-account-expiry 2012-08-27 newuser Add a user to the specified groups. The format of the name of the group is NT4 style: DOMAIN\\group. For local groups, only use the group account name. For example to add user newuser in local group testgroup: /opt/ddn/nas/bin/lw-lsa mod-user --add-to-groups testgroup newuser Remove a user from the specific groups. The format of the name of the group is in NT4 style: DOMAIN\\group. For local groups, only use the group account name. For example to remove user newuser from local group testgroup: /opt/ddn/nas/bin/lw-lsa mod-user --remove-from-groups testgroup newuser Set the primary group for the user account. For example, to set primary group for newuser to group with UID 2500: /opt/ddn/nas/bin/lw-lsa mod-user --set-primary-group 2500 newuser 5.3.4 Add a new group To add new group in local authentication database, run the command: /opt/ddn/nas/bin/lw-lsa add-group <account> where <account> is new group account name. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 74

You can set group attributes with additional options (Table 23). Table 23 Group Attribute Options Options --gid --sid Descriptions Group identifier Security identifier (SID) Set the options in the command before the <account> name. For example, to add group with specific group identifier 3500: /opt/ddn/nas/bin/lw-lsa add-group - uid 3500 <account> 5.3.5 Delete an existing group To delete an existing group account, use the command: /opt/ddn/nas/bin/lw-lsa del-group <account> where <account> is group account name. To delete a group with a specific GID, use the command: /opt/ddn/nas/bin/lw-lsa del-group --gid <GID> For example, to delete group with GID=3500: /opt/ddn/nas/bin/lw-lsa del-group --gid 3500 5.3.6 Modify a group membership You can modify a group member list by adding or removing a user from a group. The general syntax of the group modifications command is: where: /opt/ddn/nas/bin/lw-lsa mod-group {modification options} ( <group name> --gid <gid> ) {modification options} determine operation with user, and ( <group name> --gid <gid> ) specifies group to be modified. Group membership options available for modification are listed in Table 24. Table 24 Modifications Options for Group Membership Option --add-members --remove-members Description Add the members to the specific group. Specify the name of the user in NT4 format: DOMAIN\\username. For local users, only use the user account name. For example: /opt/ddn/nas/bin/lw-lsa mod-group --add-members newuser testgroup Remove the members from the specific group. Specify the name of the user in NT4 format: DOMAIN\\user. For local users, only use the user account name. For example: DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 75

Option Description /opt/ddn/nas/bin/lw-lsa mod-group --remove-members newuser testgroup 5.3.7 List users and groups To list objects in a local authentication database, use the command: /opt/ddn/nas/bin/lw-lsa enum-objects [ --<object type> ] [ <flags> ] [ --provider name ] The command without any additional arguments lists all known objects (users and groups) in the local authentication database. For more exact output, use the options listed in Table 25. Table 25 Users and Group Options Option --<object type> --domain name --provider name Description Two types of objects are available, users and groups. To restrict command output to show only users or only groups, set this option. For example to list all known users: /opt/ddn/nas/bin/lw-lsa enum-objects user For example to list all known groups: /opt/ddn/nas/bin/lw-lsa enum-objects group Enumerate only objects for specified domain. For example, to restrict output for domain TEST: /opt/ddn/nas/bin/lw-lsa enum-objects --domain TEST The request is to the specific database provider, either lsa-local-provider, or lsa-activedirectory-provider 5.3.8 Check group membership To check user membership in a group, use the command: /opt/ddn/nas/bin/lw-lsa enum-members [ --<object type> ] [ -- by-<key> ] [ <flags> ] [ --provider name ] groups For example, to check members in group testgroup: /opt/ddn/nas/bin/lw-lsa enum-members testgroup To specify an object type, use the <object type> user or group. To filter the command s output, use <key> with the options listed in Table 26. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 76

Table 26 Group Membership Filters Key --by-dn --by-sid --by-nt4 --by-alias --by-upn --by-unix-id --provider name Description Specify the groups by the distinguished name Specify the groups by SID Specify the groups by NT4-style domain-qualified name Specify the groups by alias (must specify object type) Specify the groups by user principal name Specify the groups by UID or GID (must specify object type) The request is to the specific database provider, either lsa-local-provider, or lsa-activedirectory-provider To check if a user is a group member, use the command: /opt/ddn/nas/bin/lw-lsa query-member-of [ --<object type> ] [ --by-<key> ] [ <flags> ] [ --provider name ] objects For example, to check group membership for user Administrator: /opt/ddn/nas/bin/lw-lsa query-member-of --user Administrator The resulting list of groups shows those in which the Administrator account is included. 5.3.9 CIFS Service Access Management Access for user accounts should be granted by an administrator. For example, to allow access for share test for user user1, use the command: /opt/ddn/nas/bin/lwnet share set-info test --allow user1 5.3.10 CIFS Share Access Mode CIFS shares can be configured in one of two modes. This is controlled by the lwnet utility. These access modes are available: RW: All read/write operations are allowed on shared folder file system. RO: File system is read-only (for data and metadata). For example, to set access mode for share test as read-only: /opt/ddn/nas/bin/lwnet share set-info test --read-only After this command has been run, all allowed users can only read the share content unless their account has been configured for write operations too. 5.3.11 CIFS Service Current Usage Metrics CIFS Service provides these usage counters: Active Session statistics Open Files statistics Current sessions can be enumerated with the command: DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 77

/opt/ddn/nas/bin/lwnet session In an emergency, it s possible to close a session with the command: /opt/ddn/nas/bin/lwnet session //host delete For example, to close sessions established to the CIFS Service from the host02 computer: /opt/ddn/nas/bin/lwnet session //host02 delete To list opened files, use the command: /opt/ddn/nas/bin/lwnet file In an emergency, to cancel the file operation, use the command: /opt/ddn/nas/bin/lwnet file fileid close For example, to list open files: /opt/ddn/nas/bin/lwnet file File [1] Id: 4 Pathname: c:\mnt\test\testfile.txt Username: hostname01\administrator Number of locks: 0 Permissions: 0x00000006 For example, to close this file: /opt/ddn/nas/bin/lwnet file 4 close 5.3.12 Setup a Local CIFS User to Access CIFS Share Shown below are the steps to setup a new local user and give the user the access to the CIFS share. 1. Add user using the following syntax: /opt/ddn/nas/bin/lw-lsa add-user --uid 2500 [user_name] 2. Enable user using the following syntax: /opt/ddn/nas/bin/lw-lsa mod-user --enable-user [user_name] 3. Set the password for user using the following syntax: /opt/ddn/nas/bin/lw-lsa mod-user --set-password [password] [user_name] 4. Allow user access to the share using the following syntax: /opt/ddn/nas/bin/lwnet share set-info [share_name] --allow [user_name] 5. Add a user to the Administrators group using the following syntax: /opt/ddn/nas/bin/lw-lsa mod-group --add-members [user_name] Administrators NOTE: You will need to log off and log in for client using this local user account. 5.3.13 Configuring CIFS UID/GID Values In WOS Access, NFS users and groups are identified by numeric identifiers (UID and GID, respectively), whereas CIFS users and groups are identified by a Security ID (SID) a multicharacter string that uniquely identifies a Windows user or group across domains. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 78

When files or folders are created via CIFS, WOS Access will set the owner UID and group GID of the file/folder to a numeric value that is calculated using the last few digits of the SID. In an environment where WOS Access is being used for both CIFS and NFS access, these auto-generated CIFS ID values may conflict with NFS ID values, leading to unexpected permission issues and problems with using quotas. A workaround to this issue is to force the WOS Access CIFS server to always use the same ID value for owner and group permissions. This can be done by using the two commands below: /opt/ddn/nas/bin/lwregshell add_value [HKEY_THIS_MACHINE\\services\\lwio\\parameters\\drivers\\woscifs] "VirtualOwner" REG_DWORD <ID> /opt/ddn/nas/bin/lwregshell add_value [HKEY_THIS_MACHINE\\services\\lwio\\parameters\\drivers\\woscifs] "VirtualGroup" REG_DWORD <ID> The value used for <ID> should be chosen so as not to conflict with existing NFS users. NOTE: Changes to the ID values will not affect the owner/group IDs on existing files and folders. NOTE: The ID changes must be applied to each node in the namespace synchronization group. The changes will not take effect until the CIFS service is restarted on each node. 5.4 CIFS Service configuration in Active Directory (optional) NOTE: When running these instructions in your environment, you should replace FQDN with your fully qualified domain name in capital letters (for example, CIFS.DOM) and fqdn with your fully qualified your domain name in lower case letters (for example, cifs.dom). NOTE: Check the iptables configuration and make sure that ports used by Active Directory Domain are open (see Table 53). [DE5797] These steps describe the common operations to use AD domain accounts with wosaccesscifs package. 1. Ensure the file /etc/resolv.conf contains these strings: search <fqdn> nameserver <your_domain_controller_ip> Example of /etc/resolv.conf: cat /etc/resolv.conf search cifs4.ddn nameserver 10.10.135.40 2. After changing resolv.conf, ensure that this file cannot be altered by a different process when CIFS server is joined into AD domain. To prevent any changes, use the command: DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 79

chattr +i /etc/resolv.conf 3. To allow changes in the reslov.conf file, use the command: chattr -i /etc/resolv.conf 4. Ensure that the hostname of WOSACCESS CIFS host is properly configured and contains a fully qualified domain name. For example, if the name of your domain is CIFS.DOM and you will work on the host named MYCIFSHOST, then configure the hostname as MYCIFSHOST.CIFS.DOM. To check the name of your host in the domain, use the command: /opt/ddn/nas/bin/lw-lsa get-status For example, the hostname is displayed in the string:... Domain: MYCIFSHOST... 5.4.1 Accessibility of Active Directory Domain Controller You can test accessibility of the Domain Controller with either of these commands: /opt/ddn/nas/bin/lw-get-dc-list FQDN /opt/ddn/nas/bin/lw-get-dc-name FQDN The result is the name and the IP address of the domain controller host. 5.4.2 Join AD Domain 1. Before you run the command to join the domain, DDN recommends that you synchronize the host time with DC time with either command: ntpdate -s DC_IP ntpdate s FQDN 2. Check the time on the DC server with the command: /opt/ddn/nas/bin/lw-get-dc-time FQDN NOTE: If the time on the CIFS server and DC are different, and NTP is used on the CIFS server, DDN recommends that you switch NTP off before you run the command to join the AD. 3. If the join AD and resolve time synchronization is a problem for both hosts (DC and CIFS server), then you should initialize kerberos with this command: /opt/ddn/nas/bin/kinit Administrator@FQDN 4. Join the domain with the command: /opt/ddn/nas/bin/lw-lsa join fqdn Domain_Administrator_Name 5. You must enter the password for the user account. 6. After joining the domain, update the DNS records with the command: /opt/ddn/nas/bin/lw-update-dns DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 80

5.4.3 Check AD Domain accounts To check the user account in the CIFS server s local provider database, run the command: /opt/ddn/nas/bin/lw-lsa find-objects Administrator@FQDN The output for the Administrator account is listed. To change the user properties, use lw-lsa utility. The local provider database is used when the AD domain is not available. To check the AD user account on WOSACCESS CIFS, join the domain and run the command: /opt/ddn/nas/bin/lw-lsa enum-objects tail The command will show a set of known accounts from the AD database. 5.4.4 Access to CIFS shares Establish access to the WOSACCESS CIFS share from the Windows hosts by mapping a network drive with the appropriate user's credentials. On client host: 1. Start > Computer > Right click > Map Network Drive > Fill the form with share host name or IP address > Enter User credentials. 2. Start > Run > Enter: compmgmt.msc 3. Right-click on tree view on "Computer management" label > Connect to another computer > Enter share host name or IP address. 4. Select : Computer Management > System tools > Shared folders > Shares > Select Your share from step 1 > right-click > Properties > Share permissions. 5. Select the checkboxes with permissions that you need for the user specified in step 1. 6. Try make actions for access rights setup against the mounted drive specified in step 1. 5.4.5 Leave AD Domain To leave the AD domain from the WOSACCESS CIFS host, use the command: /opt/ddn/nas/bin/lw-lsa leave NOTE: If you leave an AD domain to join a new one, restart the lwsmd service. To refresh the state of the CIFS server after a join or leave AD domain operation, use the command: /opt/ddn/nas/bin/lw-lsa ad-reload-join-state This command updates the status of authentication providers based on the joined domain information. 5.5 Namespace Synchronization Service If you are deploying WOS Access with high availability or with a remote synchronized site, this section is mandatory. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 81

If you are deploying a single WOS Access server attached to a single WOS Cluster, this section is optional. You can add multiple WOS Access servers on the same site. The Namespace Synchronization (NS) module provides the ability to synchronize namespace data. Modifications of metadata that are stored in the database are transmitted to the other nodes of the cluster. At the same time, modifications that are done on the remote node are received by the current node and applied to the local database. All modifications are packed into batches (varying in size) and are sent using a messaging service to reduce the network load. The NS solution consists of these modules: The producer and consumer which perform the sending process and the receiving updates process. A node management tool which is responsible for: tracking the status of, joining a node to, and 5.5.1 Network requirements removing a node from the synchronization cluster. Ensure these network requirements are enforced: Hosts that will be clustered together can be located in different subnets, but they should still be visible to each other over the network. All network interfaces that are used in the cluster configuration must be configured for IPv4. Hosts must be visible to each other over the network by their hostnames. To achieve this requirement, all hostnames should be configured using the corporate DNS server. All nodes must have the system clock synchronized using NTP service and they should use the same NTP server. 5.5.2 Join Synchronization Group The NS module contains a node management command line tool (Table 27), which is able to join a node to a replication group or to remove a node from a replication group. A replication group is a logical set of nodes, which replicate the local changes of a database to all other nodes in the group. If a new node joins the replication group, the other nodes add the new node to a list of known replication nodes. You do not need to manually join the nodes to each other. Once a node joins a cluster successfully, use the nodectl -s command to get a list of all nodes. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 82

Table 27 Node management command-line tool Syntax: nodectl command [options] Commands Abbreviation Options: Abbreviation Join -j quiet -q Leave -l dbuser -u Forces -f dbbase -d Status -s nodeid -n Help -h Version -v Usage: nodectl -j <remote_node_short_hostname> <db_name>] [-u <db_user>] [-q] [-n node_id] [-d Where: <remote_node_short_hostname> Is a short hostname of the destination host, which is already in a replication group. The value of the "hostname" for the command "join" is casesensitive. To check the value of the short hostname, use the command, `hostname -s` at destination host. <node_id> <db_name> <db_user> The unique node id [0,255]. It is used during the generation of the handleid sequence for each node. The database name which is monitored and replicated. The database user which is used by the NFS layer. The required database name and database user is taken from the replication settings file /opt/ddn/nas/etc/wosnas-cluster.conf (DDNPG_DB_NAME and DDNPG_DB_USER). If it is necessary to install the solution to the database with the other name, change the corresponding options in the wosnas-cluster.conf file or use flags -d and -u to override them. If no nodes exist in the replication group, specify only one local node, use the command: nodectl -j If you want to use any custom node ID, use the n option: nodectl -j n <Node_ID> NOTE: When adding a new node to the replication group, use the nodectl tool on the new node, but not on any node already included in the replication group. To confirm the node is added to the replication group, print the information of the node and replication groups with the command: nodectl s DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 83

To leave the replication group and disable replication for the local database, use the command: nodectl l IMPORTANT: A node cannot be joined to a replication group: if there is no existing DB if the tables in DB are not empty (for all new nodes except the first) if the specified node ID is already used by some other nodes in the replication group. In each case, an additional notification with a short description is printed. If the database is not empty, only the first node can be joined to the replication group if these conditions are met: The database has a valid handleid sequence with an incremental step 256. If the step is not in 256 and the database is not empty, the access to the replication group will fail with a corresponding notification. If the database is not empty, but the database has a valid handleid sequence, the only determined node ID can be used for the first node. The determined node ID should be the same as the result of a modulo operation between existing values in the handleid field in the handle table with an increment step (256). Usually the determined node ID for the first node is 1. Joining a replication group can be successfully performed even with the wrong sequence increment only if the database is empty. In such a case, the correct handleid sequence is set to 256 automatically. Table 28 Nodectl Commands with optional parameters Command nodectl s nodectl l nodectl h Explanation Print replication status details related to the replication group, e.g. whether replication is enabled for local database, node ID, replication cluster status etc. Leave replication group and disable replication on local database Print usage information Example 1, there are no nodes already added in replication group, the first node is added to it: # nodectl -j Generating node ID Node ID: 1 Configuring RabbitMQ cluster Declaring queues Installing SQL trigger and stored procedures Starting wosnas-replication service Done DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 84

Example 2, some nodes are in the replication group, one more is added to it: # nodectl -j node_01 Generating node ID Node ID: 2 Configuring RabbitMQ cluster Declaring queues Installing SQL trigger and stored procedures INFO: Database is empty and needs to be synced with one of cluster nodes INFO: Applying database dump from host node_01 Sync process finished Starting wosnas-replication service Done 5.5.3 View Synchronization Cluster Status To monitor the status of a replication group, use the command: For example: # nodectl s # nodectl -s Node ID: 2 Nodes: node_01 [online, synced] node_02 [online, synced] Queues: Hostname Message Bus Incoming Outgoing Exception node_01 0 0 0 0 node_02 0 0 0 0 ovfs_mb (pid 31962) is running... Done 5.5.4 Leave Synchronization Cluster To leave the replication group, use the command: # nodectl l The system issues a warning about stopping synchronization service and prompts you to confirm continuation. For example: WARNING: You're about to drop synchronization service on the node. WARNING: Metadata modifications will not be synchronized on the node. WARNING: Are you sure you want to proceed? [Yes/No] Yes Stopping replication service Stopping RabbitMQ cluster Removing SQL trigger and stored procedures Done If it is necessary to perform a non-interactive leave from a cluster, use option -f. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 85

5.6 WOS Access Services Relocation Service Configuration Configure Namespace Synchronization Service before configuring Services Relocation Service (SR). The Service Relocation (SR) module configuration is only required if you need an active redundancy service. SR module provides features to monitor and manage these services: Heartbeat network virtual IP of high availability pair Database namespace database NFS service WOS Access service CIFS service WOS Access CIFS service NFS disk space service to monitor free space on a specific disk device, which is used for NFS cache CIFS disk space service to monitor free space on a specific disk device, which is used for CIFS cache To monitor the status of services, these init.d-scripts are used: Heartbeat network /etc/init.d/network status Database /etc/init.d/wosaccess-postgresql status Service is installed during WOS Access SR installation. It is wrapper agent for postgresql service. NFS service /etc/init.d/wosaccess-nfsd status Service is installed during WOS Access SR installation. It is wrapper agent for wosnasnfsd service. NFS service monitor /etc/init.d/wosaccess-mon status Service is provided by WOS Access HA installation. This agent is provided for restarting wosnas-nfsd service on both nodes during failover to resolve troubles with bonding VIP on active node after failover. NFS disk space /etc/init.d/wosaccess-nfs-diskspace status Service is installed during WOS Access HA installation. CIFS service /etc/init.d/wosaccess-cifs status Service is installed during WOS Access SR installation. It is the wrapper agent for the lwsmd service. CIFS Disk space /etc/init.d/wosaccess-cifs-diskspace status Service is installed during WOS Access SR installation. It checks disk space for specific device. All init.d-scripts provide next return codes for status: 0 service running; DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 86

3 service stopped; Any other codes indicate that service is active but failed. In such a case, this tells the cluster that, before moving the resource to another node, it should stop it on the existing one first. For example: # service postgresql status; echo result "$?" postmaster (pid 17021) is running... result 0 # service postgresql status; echo result "$?" postmaster dead but pid file exists result 1 # service postgresql status; echo result "$?" postmaster is stopped result 3 Although the Heartbeat, Database and NFS services are deployed by external packages, the wosaccess-sr package provides an additional service, wosaccess-dev, which is used to monitor usage disk space on a specific device. 5.6.1 Network requirements Ensure these requirements are met: Hosts, which are going to be paired, can be located in different subnets, but they should still be visible to each other over the network. All network interfaces that are used in cluster configuration should be configured for IPv4. Two UDP ports, which are used for Corosync must be opened over the network. (Refer to AIS_UCASTPORT description in Table 29.) Hosts should be visible to each other over the network by hostnames. To achieve this requirement, the hostnames should either be configured by using the corporate DNS server, or be specified in /etc/hosts. Before running the application which will create a services-relocation cluster, configure the application for specific nodes which are going to be paired. All mandatory parameters should be set because they are necessary for cluster operability (Table 29.). Table 29 SR Service Configuration Parameters Parameter Name Description Required Cluster configuration parameters: VIP Virtual IP address shared between nodes, IPv4 only supported yes VIP_INTERFACE NIC card interface (can be bonding) which the VIP is hosted yes VIP_MASK Netmask for the interface in CIDR format (e.g. 24 and not 255.255.255.0) no CLUSTER_NAME Resources group name which is used in the pacemaker configuration no WOS NODES IP addresses of WOS Core nodes, which yes DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 87

Parameter Name Description Required should be monitoring (comma-separated) PRIMARY_NODENAME Hostname of the primary node ( use "hostname --long" or "uname -n" to get yes name of host ) SECONDARY_NODENAME Hostname of the secondary node ( use "hostname --long" or "uname -n" to get yes name of host ) AIS_NETADDR Network interface or IP which is used by corosync for unicast transmission for local yes host, IPv4 only supported AIS_NETADDR_RESERVE Backup network interface or IP which is no used by corosync for unicast transmission for local host, IPv4 only supported. AIS_REMOTE_NETADDR Network interface or IP which is used by corosync for unicast transmission for yes remote host, IPv4 only supported AIS_REMOTE_NETADDR_R Backup network interface or IP which is no ESERVE used by corosync for unicast transmission for remote host, IPv4 only supported AIS_UCASTPORT Unicast port (default=5450). corosync uses two UDP ports ucastport (for ucast receives) and ucastport - 1 (for ucast no sends). AIS_UCASTPORT_RESERVE Backup unicast port (default=5454). no corosync uses two UDP ports ucastport (for ucast receives) and ucastport - 1 (for ucast sends). SUPPORT_SERVICES Services are monitored by pacemaker ( should contain either "cifs", or "nfs" or both (comma-separated). No other services yes affected Wosaccess-dev configuration parameters: DEV_LIMIT Maximum value for disk space used (in percentage) no CHECK_INTERVAL Defines the interval between checks for free space (in seconds). no CHECK_COUNT Defines the number of tests service will do once it identified that disk usage limit has been exceeded. Service will make CHECK_COUNT verifications with CHECK_INTERVAL period. no Notes: You may configure a services relocation cluster only on one node. Any configuration changes you make will expand on another node automatically. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 88

Paired hosts can be located in different subnets, but they must be visible to each other over the network. To achieve this requirement, configure the hostnames by using the corporate DNS server or specify them in /etc/hosts. Add the hostnames and their IP addresses of both of pairing nodes to /etc/hosts on each host. Example of /etc/hosts file: 127.0.0.1 host1.mycompany.com localhost 10.6.19.51 host2.mycompany.com node01 10.6.18.151 host3.mycompany.com node02 5.6.2 Configure Cluster Enable or disable the selinux and iptables services on both nodes; otherwise, the traffic between the nodes is blocked. Two UDP portsto be used for Corosync must be opened.) Refer to the description of AIS_MCASTPORT in Table 27.) Disable the NetworkManager because it changes the entries in /etc/hosts after reboot and then host names overwrites them incorrectly. All configuration parameters can be set in a single command line during installation. To see all available options, use the command: # SR-cluster.sh h Usage: SR-cluster.sh command [options] Commands: Description -h, --help This message -e, --get-nic value -i, --install Get list of available NICs for the specified host Create the cluster -u, --update Update the configuration of the cluster -r, --remove Remove the cluster Options: --vip=value --vif=value --routing-prefix value -g, --group value Description: Virtual IP address shared between nodes #Required The NIC card interface (can be bonding) which the VIP will be hosted #Required The netmask for the interface in CIDR format (e.g., 24 and not 255.255.255.0) The name of resources group which will be used in pacemaker configuration -n, --node value Hostname of the remote node ( use \"hostname -- long\" or \"uname -n\" to get name of host ) #Required -w, --wos-nodes IP addresses of WOS Cluster nodes (commaseparated), which should be monitored -p, --primary Flag to define remote node as primary node DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 89

--ais-addr value --ais-addr-res value --ais-raddr value --ais-raddr-res value --ais-port value Reserve network interface or IP which will be used by corosync for unicast transmission for local host -l, --dev-space-limit value -s, --check-interval value -c, --check-count value Additional options: --config For example, to install the cluster with a configuration file: Network interface or IP which will be used by Corosync for unicast transmission for local host #Required Reserve network interface or IP which will be used by corosync for unicast transmission for local host Network interface or IP which will be used by Corosync for unicast transmission for remote host #Required Reserve network interface or IP which will be used by corosync for unicast transmission for remote host Unicast port for Corosync, default=5450. Corosync uses two UDP ports ais-port #Required (for ucast receives) and ais-port - 1 (for ucast sends) Reserve network interface or IP which will be used by corosync for unicast transmission for local host Maximum value for disk space used (in percentage) Interval between checks for free space Number of tests service will do once it identified that disk usage limit has been exceeded Descriptions: All values for installation will be taken from /opt/ddn/nas/etc/cluster.conf. Working only with --install command. # /opt/ddn/nas/bin/sr-cluster.sh --install config For example, to install the cluster without a configuration file: # /opt/ddn/nas/bin/sr-cluster.sh --install --vip 192.168.0.1 --vif eth0 --node MYHOST.mycompany.com --ais-addr eth0 --ais-raddr eth0 For example, to update the cluster s configuration: # /opt/ddn/nas/bin/sr-cluster.sh --update --vif eth0 --routing-prefix 20 For example, to remove the cluster: # /opt/ddn/nas/bin/sr-cluster.sh -remove Save the updates to the cluster in the configuration file /opt/ddn/nas/etc/cluster.conf. For all of the system s parameters, refer to Table 31. Refer to Table 30 for the descriptions and contextual information for all supported flags and configuration elements. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 90

Table 30 System Parameters VIP Parameter Variables VIP_INTERFACE VIP_MASK CLUSTER_NAME SUPPORT_SERVICES WOS_NODES PRIMARY_NODENAME SECONDARY_NODENAME AIS_NETADDR AIS_NETADDR_RESERVE AIS_REMOTE_NETADDR AIS_REMOTE_NETADDR_R ESERVE AIS_UCASTPORT AIS_UCASTPORT_RESERVE Instructions Choose any IP address for virtual IP not used in current subnets. On a current node run: # ip a Choose a network interface on which the VIP is hosted. This network interface must be present on both nodes. Choose any routing prefix what is used for VIP. Choose any name for the resources group which is used in the pacemaker configuration and update. The name should contain only simple ASCII characters [a-za-z0-9_\-] and have a maximum length of 64 characters. Services are monitored by pacemaker should contain either "cifs", or "nfs" or both (comma-separated). No other services are affected. Comma-separated IP addresses of WOS Storage nodes To get the host name on the node that is expected to be a primary one, run: #uname-n To get the host name on the node that is expected to be a secondary one, run: #uname n On a current node run: #ip a Choose the network interface to be used for communication with the other node. On a current node run: #ip a. Choose the network interface to be used as backup interface for communication with the other node. On a current node run: #ip a Choose the network interface to be used for communication with the other node. On a remote node, which will be used to create SR cluster run: #ip a. Choose the network interface to be used as backup interface for communication with the current node. Choose any UDP port which is not used for applications and services. If default is set instead of a number, then Corosync will work via ports 5450 and 5449. IMPORTANT: Corosync uses two UDP ports AIS_UCASTPORT (for mcast receives) and AIS_UCASTPORT - 1 (for mcast sends). Choose any UDP port which is not used for applications and DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 91

Parameter Variables DEV_LIMIT Instructions Choose the maximum value for the disk space used (in percentage) for the two devices PRIMARY_DEV_NAME and SECONDARY_DEV_NAME. CHECK_INTERVAL CHECK_COUNT Choose the interval between the checks for free space (in seconds). Choose the number of tests the service will perform when the disk usage limit has been exceeded. Save all changes in the configuration file /opt/ddn/nas/etc/cluster.conf. For example: 5.6.3 Cluster Installation VIP=192.168.153.251 VIP_INTERFACE=eth0 VIP_MASK=20 CLUSTER_NAME=wosaccessnfs_cluster PRIMARY_NODENAME= HOST1.mycompany.com SECONDARY_NODENAME= HOST2.mycompany.com SUPPORT_SERVICES=nfs,cifs WOS_NODES=10.100.100.1,10.100.100.2 AIS_NETADDR=eth0 AIS_REMOTE_NETADDR=eth0 AIS_MUCASTPORT=5450 AIS_NETADDR_RESERVE=eth1 AIS_REMOTE_NETADDR_RESERVE=eth1 AIS_UCASTPORT_RESERVE=default DEV_LIMIT=85 CHECK_INTERVAL=30 CHECK_COUNT=2 For cluster installation: Script uses SSH to copy configuration files and run commands on the secondary node, assuming SSH is pre-configured. SSH should allow a root login and have keys properly set up; otherwise, SSH will ask for password every time it needs to run a command on a remote host. All errors and warnings that occur during the install/update/uninstall SR cluster and all failures of wosaccess-dev service are written to the log file /opt/ddn/nas/log/wosaccess-sr.log. To install a cluster on a configured node, use the command: # /opt/ddn/nas/bin/sr-cluster.sh --install --config Optionally, to install a cluster with configuration options on any node, run: # SR-cluster.sh --install --vip 192.168.0.1 --wos-nodes 10.10.10.1 --vif eth0 --node MYHOST.mycompany.com --ais-addr eth0 --ais-addr-res eth1 --ais-raddr eth0 --ais-raddr-res eth0 This command: Runs Corosync and Pacemaker on both nodes, DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 92

Creates a cluster with PRIMARY_NODENAME and SECONDARY_NODENAME, and Starts monitoring a group of resources on both nodes. After the installation, all configuration parameters are saved in the /opt/ddn/nas/etc/cluster.conf file. For example: # /opt/ddn/nas/bin/sr-cluster.sh --install --config INFO: Running cluster install -- Checking corosync status on "MYHOST1.mycompany.com" corosync is stopped INFO: Corosync stopped on node "MYHOST1.mycompany.com" -- Checking corosync status on "MYHOST2.mycompany.com" corosync is stopped INFO: Corosync stopped on node "MYHOST2.mycompany.com" -- Generate corosync authkey Corosync Cluster Engine Authentication key generator. Gathering 1024 bits for key from /dev/random. Press keys on your keyboard to generate entropy. Press keys on your keyboard to generate entropy (bits = 128). Writing corosync key to /etc/corosync/authkey. -- Copying corosync authkey from "MYHOST1.mycompany.com" to "EVBYMINSD1209.mycompany.com" -- Get ip from interface "eth0" -- Get ip from remote interface "eth0" -- Copying "/etc/corosync/corosync.conf" from "MYHOST1.mycompany.com" to "MYHOST2.mycompany.com" -- Starting corosync on "MYHOST2.mycompany.com" Starting Corosync Cluster Engine (corosync): [ OK ] -- Adding corosync services for management on "MYHOST2.mycompany.com" -- Waiting when primary node is online -- Configuring cibadmin -- Starting corosync on "MYHOST1.mycompany.com" Starting Corosync Cluster Engine (corosync): [ OK ] -- Adding corosync services for management on "MYHOST1.mycompany.com" -- Waiting when 2 nodes are online INFO: Cluster with nodes "MYHOST2.mycompany.com" and "MYHOST1.mycompany.com" successfully created To validate the installation look at log file /opt/ddn/nas/log/wosaccess-sr.log and run command below to get status of cluster: #crm_mon -1 Example of a successful installation: # crm_mon -1 ============ ============ Online: [ MYHOST2.mycompany.com MYHOST1.mycompany.com ] failover-ip (ocf::heartbeat:ipaddr2): Started MYHOST1.mycompany.com failover-mon (lsb:wosaccess-mon): Started MYHOST1.mycompany.com Clone Set: cl_wosaccess_cluster [wosaccess_cluster] Started: [MYHOST1.mycompany.com MYHOST2.mycompany.com ] Sample of log output: # cat /opt/ddn/nas/log/wosaccess-sr.log Tue Nov 29 16:32:37 EET 2011 MYHOST2.mycompany.com ha-cluster:[8325] INFO: Running cluster install Tue Nov 29 16:32:37 EET 2011 MYHOST2.mycompany.com ha-cluster:[8325] INFO: Corosync stopped on node "MYHOST2.mycompany.com" DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 93

Tue Nov 29 16:32:38 EET 2011 MYHOST2.mycompany.com ha-cluster:[8325] INFO: Corosync stopped on node "MYHOST1.mycompany.com" Tue Nov 29 16:33:55 EET 2011 MYHOST2.mycompany.com ha-cluster:[8325] INFO: Cluster with nodes "MYHOST1.mycompany.com" and "MYHOST2.mycompany.com" successfully created Example of a successful installation but one of the services has broken down with all services moving to a secondary node: # crm_mon -1 ============ ============ Online: [ MYHOST2.mycompany.com MYHOST1.mycompany.com ] failover-ip (ocf::heartbeat:ipaddr2): Started MYHOST1.mycompany.com failover-mon (lsb:wosaccess-mon): Started MYHOST1.mycompany.com Clone Set: cl_wosaccess_cluster [wosaccess_cluster] Started: [ MYHOST1.mycompany.com MYHOST2.mycompany.com ] Failed actions: failover-nfs-diskspace_monitor_30000 (node=myhost1.mycompany.com, call=30, rc=7, status=complete): not running Example of a failed installation: # crm_mon -1 Attempting connection to the cluster... Sample of log output: # cat /opt/ddn/nas/log/wosaccess-sr.log Thu Dec 8 12:18:40 EET 2011 MYHOST1.mycompany.com ha-cluster:[11402] INFO: Running cluster install Thu Dec 8 12:18:40 EET 2011 MYHOST1.mycompany.com ha-cluster:[11402] INFO: Corosync stopped on node "MYHOST1.mycompany.com" Thu Dec 8 12:18:40 EET 2011 MYHOST1.mycompany.com ha-cluster:[11402] INFO: Corosync stopped on node "MYHOST2.mycompany.com" Thu Dec 8 12:18:41 EET 2011 MYHOST1.mycompany.com ha-cluster:[11402] ERROR: AIS_REMOTE_NETADDR "" is incorrect Thu Dec 8 12:18:41 EET 2011 MYHOST1.mycompany.com ha-cluster:[11402] ERROR: Creation of cluster with nodes "MYHOST2.mycompany.com" and "MYHOST1.mycompany.com" failed 5.6.4 Update Cluster Configuration To update a cluster configuration, run: #/opt/ddn/nas/bin/sr-cluster.sh -u --vip 169.168.0.1 --routingprefix 24 See Table 30 for detailed information. For example: 5.6.5 Remove Cluster # /opt/ddn/nas/bin/sr-cluster.sh -u --vip 169.168.0.1 --routing-prefix 24 INFO: Running cluster update -- Checking corosync status on "MYHOST2.mycompany.com" corosync (pid 31789) is running... INFO: Corosync running on node " MYHOST2.mycompany.com" INFO: Cluster with nodes " MYHOST2.mycompany.com" and " MYHOST1.mycompany.com" successfully updated To remove a cluster, run on any node: DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 94

#/opt/ddn/nas/bin/sr-cluster.sh --remove This command: removes the tracked services from the pacemaker configuration, deletes the corosync configuration files, and stops the cluster. Example: #/opt/ddn/nas/bin/sr-cluster.sh --remove INFO: Removing cluster -- Checking corosync status on " MYHOST2.mycompany.com" corosync (pid 31789) is running... INFO: Corosync running on node " MYHOST2.mycompany.com" -- Checking corosync status on "MYHOST1.mycompany.com" corosync (pid 25303) is running... INFO: Corosync running on node "MYHOST1.mycompany.com" -- Try to get node id formyhost1.mycompany.com -- Switch off corosync services on "MYHOST1.mycompany.com" -- Stop corosync onmyhost1.mycompany.com Signaling Corosync Cluster Engine (corosync) to terminate: [ OK ] Waiting for corosync services to unload:.[ OK ] -- Remove corosync config files on "MYHOST1.mycompany.com" -- Remove wosacess-dev config files on "MYHOST1.mycompany.com" -- Remove node "MYHOST1.mycompany.com" -- Remove node status "MYHOST1.mycompany.com" -- Switch off corosync services on "MYHOST2.mycompany.com" -- Stop corosync on "MYHOST2.mycompany.com" Signaling Corosync Cluster Engine (corosync) to terminate: [ OK ] Waiting for corosync services to unload:.. [ OK ] -- Remove corosync config files on "MYHOST2.mycompany.com" -- Remove wosacess-dev config files on "MYHOST2.mycompany.com" INFO: Cluster successfully removed 5.6.6 Failover Policy Details Pacemaker logic is based on the return codes of init-scripts. This checks the status of tracked services and then decides if it is necessary to migrate the services to another node. But not all services will migrate. To migrate, use these set of services working on both nodes: wosaccess-nfs-diskspace wosaccess-cifs-diskspace wosnas-nfsd lwsmd (cifs service) postgresql VIP and wosaccess-mon services have only one instance in the SR Cluster. To minimize data loss, PostgreSQL and wosnas-nfsd services are always running and must never be stopped during migration resources. Wrapping resource agents wosaccesspostgresql and wosaccess-nfsd are implemented to use this logic. These agents contain a modified function stop, which does not allow any action and always returns a positive status (that is, the function stop always returns a value of 0). Functions start and status DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 95

of the wrapping resource agents call the native functions of postgresql and wosnas-nfsd services. To keep the system working, the cluster is monitoring all nodes for free space. It uses the wosaccess-nfs-diskspace and wosaccess-cifs-diskspace services for this. Details of the service for the wosaccess-nfs-diskspace and wosaccess-cifs-diskspace policy: On "start", the device is checked only one time, and it returns an error if the used space is more than the set limit. On "status", the device is checked and if the used space is more than the set limit, a lock file with a timestamp is created and the current configuration parameters and count of failed checks are logged. On the next try, if the used space is still not free, the logged items will be compared to the current number of failed checks and CHECK_COUNT. If the current number of checks is less than the CHECK_COUNT, the service will increase the count of checks in the locked file. If the current number of checks is more than or equal to the CHECK_COUNT, the service returns a negative return code. On stop, the file with a timestamp is removed. A failure of any tracked services triggers the system to stop the set of services on a current node and to start the migration of the virtual IP to another node without requiring administrative intervention. All services are checked every 10 seconds. The wosaccess-dev service is checked every CHECK_INTERVAL seconds (Table 29..) All services have a default timeout on each operation (for example, 15 seconds for the start operation). If the pacemaker does not get any status and operation times out, then the system displays the "unknown exec error" message. All information about these failures are cleaned from the system 60 seconds after the fail (failure-timeout). This allows the possible return to the virtual ip on the node, which initially failed. Table 31 Policy description for tracked services Services Instance in SR Cluster Fail trigger fail-over Monitor interval (seconds) Failure migrate VIP Single yes 10 Two failures in 60 sec wosaccess-mon (NFS Monitor) postgresql wosaccess-nfsdiskspace (check space for NFS cache) wosnas-nfsd (NFS Server) Single. Must be together with VIP Always run on all nodes Always run on all nodes Always run on all nodes yes 10 Two failures in 60 sec yes 10 Two failures in 60 sec used>85 % 30 Two failures in line (CHECK_ COUNT parameter) yes 10 Two failures in 60 sec DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 96

Services lwsmd (CIFS Server) wosaccess-cifsdiskspace (check space for CIFS cache) Instance in SR Cluster Always run on all nodes Always run on all nodes Fail trigger fail-over Monitor interval (seconds) Failure migrate yes 10 Two failures in 60 sec used>85 % 10 Two failures in 60 sec NOTE: Failover count for service is 2 by default (migration-threshold=2). These Pacemaker sequences of steps list the reasons for any service to result in failover: Failure on start : Example 1: service postgresql start failure (return code not equal 0) service postgresql stop done (return code 0) failover Example 2: service postgresql start done (return code 0) service postgresql status failure (return code not equal 0) service postgresql stop done (return code 0) service postgresql start failure (return code not equal 0) service postgresql stop done (return code 0) failover Failure on stop : Example: service postgresql start done (return code 0) service postgresql status failure (return code not equal 0) service postgresql stop failure (return code not equal 0) failover Failure on status : Example. 1: service postgresql start done (return code 0) service postgresql status failure (return code not equal 0) service postgresql stop done (return code 0) service postgresql start done (return code 0) service postgresql status failure (return code not equal 0) service postgresql stop done (return code 0) failover Example. 2: service postgresql start done (return code 0) service postgresql status failure (return code not equal 0) service postgresql stop done (return code 0) service postgresql start done (return code 0) service postgresql status done (return code 0) service postgresql status failure (return code not equal 0) (second failure has occurred) service postgresql stop done (return code 0) failover DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 97

5.7 WOS Access Disaster Recovery Configuration WOS Access Disaster Recovery provides these features: Scheduled backups Email notifications of the successful backups and backup failures Possible WOS cluster storage of the backup data Possible recovery on the reserve system in case of physical damage to the WOS Access node (Requires data to be stored on the WOS cluster and email notification enabled). Configurable recovery of the database only or configuration only Possible recovery of an arbitrary backup The backup service by itself is quite I/O intensive. Therefore, if you run WOS Access SR setup with data replication between nodes, run the Backup service only on the passive node. Leave the Active SR node with backup running. 5.7.1 Configuration steps DDN Recommendation: Run only one active backup service per site The WOS Access disaster recovery tool configuration resides inside the file wosnas.nfsd.conf, which also contains the configuration of the wosnas-nfs tool. The primary parameters controlling backup are in the OVFS_BACKUP section of this file (Table 32). There are also some relevant parameters in the OVFS_WOS section of this file (Table 33). Table 32 OVFS_BACKUP Parameters Value OVFS_BACKUP_SCHEDULE OVFS_WOSNAS_PATH OVFS_BACKUP_CONFIG_DIRS OVFS_BACKUP_LOG_DIRS OVFS_BACKUP_UPLOAD_PATH Description The schedule of backups. The schedule format is similar to the format of the entry in the system cron table. See manual page for crontab (5) for the details of the format. Path to WOS Access installation. Configuration directories (under OVFS_WOSNAS_PATH) that should be backed up (comma separated list). Log directories (under OVFS_WOSNAS_PATH) that should be backed up (comma separated list). Path to the place where backup will be stored in case of local backup, or where the temporary files will be placed in case of WOS stored backup. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 98

Value OVFS_BACKUP_STORE OVFS_BACKUP_STORE_CMD OVFS_BACKUP_DB_FULL_HOURS OVFS_BACKUP_KEEP_DAYS OVFS_DB_DATA_PATH OVFS_DB_ARCH_PATH OVFS_BACKUP_MAIL_NOTIFICATION OVFS_MAIL_SERVER OVFS_MAIL_ADDR_TO OVFS_MAIL_USER_FROM Description If this parameter is set to yes, the backups are stored to WOS cluster specified in OVFS_WOS section. Otherwise the backups are stored in OVFS_BACKUP_UPLOAD_PATH Usually contains a path to storefio utility. Time interval between full backups of the database. Storage duration of backups in OVFS_BACKUP_UPLOAD_PATH (in case of WOS backup disabled). Directory where the database files are stored. Usually it is /var/lib/pgsql/data Directory where the database write ahead log (WAL) segments will be stored. This directory can grow large so be sure to place it on the filesystem with plenty of storage space. If this parameter is set to yes, the mail notifications are sent for each backup. Mail server to relay the messages of backup notification. Address of the user who receives backup notifications. Mail account name from which the notifications are sent. Table 33 OVFS_WOS Parameters Value OVFS_WOS_CLUSTER OVFS_WOS_POLICY OVFS_WOS_USER OVFS_WOS_PASSWORD Description WOS cluster to store data, the same one as for WOSNAS filesystem Policy for this cluster Username for WOS cluster Password for WOS cluster 5.7.2 Overview of backup process Each time a backup is run, it stores: Configuration files specified by OVFS_BACKUP_CONFIG_DIRS and OVFS_WOSNAS_PATH Log files specified by OVFS_BACKUP_LOG_DIRS and OVFS_WOSNAS_PATH DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 99

Database data directory specified by OVFS_DB_DATA_PATH and database Write Ahead Log (WAL) files, which are stored under OVFS_DB_ARCH_PATH. The write ahead log files are copied to the corresponding directory automatically by the database when database archiving is enabled. When the OVFS_BACKUP_STORE parameter is set to yes, compressed backups are sent to the Web Object Scaler (WOS) storage which is specified in the OVFS_WOS section of the configuration file. Otherwise these backups are stored locally under the location specified by the OVFS_BACKUP_UPLOAD_PATH parameter. The frequency of full backups is determined by the OVFS_BACKUP_DB_FULL_HOURS parameter. Backup generates restore information, which is stored in local DB in 'backup' table. This 'backup' table is replicated across WOSNAS sites. The same restore information is sent as an email attachment (restore.txt) file in email notification message if email notifications are enabled. That is, OVFS_BACKUP_MAIL_NOTIFICATION is set to yes. The most recent restore.txt file can be found under OVFS_WOSNAS_PATH/var/ovfs-backup. Database backups are scheduled as cron job. This scheduling as well as enabling the backup on the database level is performed by backup_ctrl utility. 5.7.3 Scheduling backup To schedule backup, issue this command: sudo /opt/ddn/nas/bin/backup_ctrl -q This schedules backups as a cron job and enable the database archiving. 5.7.4 Manual backups To perform a manual backup: 1. You must first enable backup on the database level with this command: sudo /opt/ddn/nas/bin/backup_ctrl -e 2. After that, each time you want to perform a manual backup, issue this command: sudo /opt/ddn/nas/bin/backup 5.7.5 Restoring from backup Restore reads the restore information from the backup table in the database or from the restore.txt file found in the default location or specified on the command line. The restore process then downloads all the necessary files from the WOS storage and restores them to the given location. NOTE: Restore does not read the configuration file; therefore, provide all the nonstandard options on the command line. NOTE: Backups created prior to WOS Access 1.4.0 cannot be restored automatically manual intervention is required. Please contact DDN Support for assistance. Restoring from the most recent backup DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 100

Run sudo /opt/ddn/nas/bin/restore without parameters. It will look up the restore.txt file under /opt/ddn/nas/var/ovfs-backup/restore.txt and it will use this information to restore the database and config files. Note that /opt/ddn/nas/var/ovfsbackup/restore.txt will be deleted after a successful restore. Restoring from old backup Copy restore.txt from the corresponding e-mail notification into the default location (/opt/ddn/nas/var/ovfs-backup) and run /opt/ddn/nas/bin/restore without parameters. You can also use command line option to specify path to restore.txt if it s not in the default location: sudo /opt/ddn/nas/bin/restore -i /path/to/restore.txt Restoring from backup database To restore using the database, use the command: sudo /opt/ddn/nas/bin/restore -t <timestamp in the restore.txt> -w <database name> -y <database user name> -z <database password> You need to know the restore timestamp of the corresponding backup. For example: sudo /opt/ddn/nas/bin/restore -t "2011-09-16T18:14:05" -w ovfs -y ovfs -z ovfs Some other useful restore options: o o Use the -r option to restore the configuration and log files not at the default path but at a different path. Use the -u option to specify the backup files upload path if the backup files are stored locally. When restoring a node from backup, a replication group needs to be recreated. The restore process does not replicate restored data to a replication group. o Use these steps to restore the database for the replication group: 1. Leave the replication group on each node. 2. Restore the necessary backup on the target node. 3. Create a new replication group where the target node will be the first node because the other nodes should copy the restored database from this node. Take care that the databases on all nodes in the replication group, except for the target node, are cleaned. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 101

5.7.6 References Table 34 Backup_Ctrl Command Line Options Option Meaning -c Check if backup is scheduled -e Enable backup at the database level -d Disable backup at the database level -q Schedule backup and enable backup at the database level -w Remove backup schedule Table 35 Backup Command Line Options Option Meaning -b <file> Use <file> instead of /opt/ddn/nas/etc/wosnas.nfsd.conf as a configuration file. -o <file> Use <file> as output file -s <path> Use command <path> instead of /opt/ddn/nas/storefio to store the backups Table 36 Restore Command Line Options Option Meaning -a <path> Use the specified path instead of the default for database archive instead of /var/lib/pgsql_archive -c <WOS cluster> Use the specified WOS cluster instead of the one in the restore.txt -d Do not restore database. Restore config file only. -g <path> DB data path. Use the specified path instead /var/lib/pgsql/data/ as a database data path. -i <file> Use the specified file instead of /opt/ddn/nas/var/ovfsbackup/restore.txt -l <hostname> Read restore data for <hostname> in the backup table -n <DB name> Use <DB name> instead of ovfs for -p <WOS policy> Use <WOS policy> to retrieve data from WOS -r <path> Restore at the <path> instead of default path -u <path> Use <path> as a backup upload path (this option should be used in case of driverless backup). -x <host> Use <host> instead of localhost as a source for backup table -y <user> Use <user> to connect to the database -z <password> Use <password> to connect to the database DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 102

5.8 Mellanox ConnectX-3 configuration If the WOS Access Server does not have a Mellanox ConnectX 3 card installed during the WOS Access ISO install and it is added to the PCI box later, the Mellanox card is not recognized as an Ethernet adaptor and is not configured as such. The fix is to the execute the "/sbin/connectx_port_config" script from the shell prompt, and select the Ethernet option. This configures the Mellanox card to be the Ethernet adaptor. Following that, restart the Mellanox driver by executing "/etc/init.d/openibd restart" from the shell prompt. 5.9 Enabling wos-fst logging dynamically The dynamic logging of the wos-fst module can be dynamically enabled or disabled. The wos-fst logging helps in diagnosis and analysis of communications between the WOS Access Gateway node and the WOS cluster. Logging of wos-fst is disabled by default. Logging can be enabled using the steps below: 1. Obtain the pid of WOS Access NFS and/or WOS Access CIFS (wos_worker) ps ef grep wosnas.nfsd ps ef grep wos_worker 2. Send the SIGALRM signal to enable/disable fst logging from WOS Access NFS and/or WOS Access CIFS kill s SIGALRM <pid of wosnas.nfsd> kill s SIGALRM <pid of wos_worker> 3. The fst messages are logged in the following files /opt/ddn/nas/log/wosfs-nfs.log /opt/ddn/nas/log/wosfs-cifs.log The fst logging is enabled and disabled by following the same procedure. If logging is enabled, then sending SIGALRM disables it and vice versa. 5.10 Disk Space Monitoring To prevent the possibility of data corruption, a service is started to monitor /opt/ddn disk space and is configured to shut down nfs and cifs if the filesystem passes 95% capacity. This configuration file, /opt/ddn/nas/etc/monit.d/ddn_disk.conf, may be edited or commented out according to your local requirements. To make any configuration changes effective, execute the shell command "monit reload." 5.11 Orderly Shut Down of the WOS Access Node The orderly shutdown of WOS Access is a manual procedure in order to ensure that there is no data loss. The procedure essentially consists of manually stopping NFS/CIFS traffic going to the WOS Access active node: allowing WOS Access NFS and CIFS to run for sometime to allow the files on the staging area to be pushed to WOS Core, allowing the WOS Access namespace synchronization to synchronize the metadata, and finally shutting down the node safely. Use the following procedures for an orderly shutdown. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 103

1. Ensure that no NFS or CIFS client are mounting the filesystem through WOS Access gateway and writing data to it. It is required to stop writing data to the WOS Access gateway or even un-mounting/disconnecting the share from the client as shown: umount l /mnt/wos (for example, how to umount an NFS client, executed on the client machines) Disconnect Network Drive for Windows share from Windows explorer 2. Ensure that no files on NFS staging area are waiting to be pushed to WOS Core. # ls -l /opt/ddn/nas/var/ovfs-cache/ Retry in 30 seconds if any files are reported. 3. Ensure that no files on CIFS cache area are waiting to be pushed to WOS Core. #ls -l /opt/ddn/nas/var/cifs-cache/ Retry in 30 seconds if any files are reported. 4. Ensure that WOS Access node does not have unsent Namespace Synchronization messages Outgoing -column should be zero for the maintained WOS Access node. # nodectl -s Retry in 10 seconds if Outgoing-value is other than zero. 5. Stop all WOS Access services. # service wa stop-all 6. Shutdown the WOS Access node. /sbin/shutdown -h now 5.11.1 WOS Access Shut Down Script Below is a short example of the wa script. Sample of output of utility: # service wa status Status WOS Access services: NFS-Server [Inactive] CIFS-Server [Inactive] Namespace-Sync [Inactive] Status WOS Access dependent services: RabbitMQ [Not Installed] PostgreSQL [Inactive] Redis [Inactive] # service wa stop-all Stop WOS Access services: Service-Relocation [Done] Namespace-Sync [Failed] NFS-Server [Done] CIFS-Server [Failed] Stop WOS Access dependent services: RabbitMQ [Not Installed] PostgreSQL [Done] Redis [Done] DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 104

5.12 Renaming Host Names When a WOS Access gateway server is given a new host name, it is important to perform the following steps to maintain data integrity. Failing to do so will cause the error "Rabbit MQ did not start" from the WEB UI when starting the namespace synchronization group. After renaming a WOS Access gateway, the following steps need to be done: Stop the RabbitMQ service and ensure no rabbitmq processes are running. o service rabbitmq-server stop killall u rabbitmq Ensure that there are no erlang instances running. o ps aux grep beam kill <pid of beam> Change hostname. o echo newhostname > /etc/hostname echo 127.0.0.1 rabbit >> /etc/hosts hostname F /etc/hostname Remove existing mnesia directory. o rm rf /opt/ddn/nas/lib64/mnesia/wosnfs* Start the RabbitMQ server. o service rabbitmq-server start DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 105

6.0 Quotas By default, WOS Access puts no limits on the amount of storage a user or group of users can consume (other than the physical limits of the underlying storage). In some cases, however, storage administrators will want to control how resources are allocated and utilized. Quotas allow administrators to place limits on both storage capacity and the number of files and folders consumed by users and groups. WOS Access supports both soft and hard quotas. Hard quotas are an absolute limit and a Quota Limit or I/O error messsage (depending upon the client) will be returned when usage exceeds this limit. Soft quotas, on the other hand, can be exceeded. When a soft quota is exceeded, a grace period (seven days by default) begins and if the storage usage is not reduced below the soft quota within the grace period then the soft quota becomes a hard quota.at that point, all subsequent storage allocations will fail until usage is reduced to below the soft quota. WOS Access quotas are managed from the command-line using the quotactl tool.this tool must be run as the root user on a WOS Access gateway server. For a complete overview of the quotactl utility, refer to the References section of this document. 6.1 Planning Before configuring and enabling quotas in WOS Access it is helpful to know the total number of users that may access the system as well as the total usable storage capacity. The total storage capacity available to WOS Access is determined by the capacity available in the WOS cluster, taking into account replication and other policy settings. Once the capacity to be allocated is known, a per-user quota can be calculated by dividing the allocated capacity by the expected number of system users. In general, it is good practice to start with low quota values. Quota limits can be raised easily but reducing them may require users to delete data. 6.2 Setting Quotas When setting quotas, it is good practice to start with a default user quota. This will enforce the same quota limits for all users of the system and provides a baseline for resource allocation. From that point on, limits can be adjusted on a per-user or group basis, as needed. Note that default quotas can be configured for groups as well. NOTE: Quotas can be set at any time, regardless of whether quotas are enabled or disabled. To set a default quota for all users, run the following command: quotactl set { u -g} -d {<softcap> <hardcap> <softinode> <hardinode>} Example: $> quotactl set u d 400M 500M 10K 15K $> quotactl get u d h Capacity Limit (Soft/Hard) Inode Limit (Soft/Hard) DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 106

--------------------------------------------------------------------------- User Default 400.00M / 500.00M 10.00K / 15.00K In the example above, default quota limits are set for all users (per the -u flag). The limts are specified as: 400MB soft capacity limit 500MB hard capacity limit 10,000 soft inode limit 15,000 hard inode limit Quota limits can be configured on both capacity and inodes. Capacity limits indicate the total size, in bytes, of all files and folders that a user or group can allocate. Inode limits indicate the total number of files and folders that a user or group can allocate. A limit value of 0 is used to indicate that no quota is enforced for that particular setting. NOTE: Quota limits are synchronized across all nodes in a synchronization group and only need to be set on one node in the group. To set a quota for a specific user or group, use the i flag followed by a UID or GID, as shown below: quotactl set {-u -g} -i <id> {<softcap> <hardcap> <softinode> <hardinode>} Example: $> quotactl set u i 5002 1G 1.2G 0 0 $> quotactl get u i 5002 -h UserID Capacity-Used Limit (Soft/Hard) Grace Time Remaining Inode-Used Limit (Soft/Hard) Grace Time Remaining ---------------------------------------------------------------------------------- -------------------------------------------------------------- 5002 0.00 1.00G / 1.20G - 0.00 0.00 / 0.00 - In the example above, a specific quota is set for the user with UID 5002. Quotas will be enforced on capacity only, not on inodes (per the 0 for both softinode and hardinode). 6.3 Clearing Quotas In some cases, it may be desirable to revert a specific user or group back to the default quota. WOS Access provides the quotactl clear command for this purpose, as shown below: quotactl clear { u -g} {-d i <id>} Example: $> quotactl clear u i 10000 Quota cleared for UID 10000 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 107

Note that clearing the quota setting for a user or group quota is different from specifying 0 for all limits. A value of 0 indicates no limit should be enforced for that setting whereas clearing a quota will revert a user or group to the limits of the default quota. 6.4 Enabling Quotas Quotas can be enabled at any time. However, when enabling quotas on a system with existing data, the following procedure should be used: 1. Before enabling quotas, stop both the NFS and CIFS services on all nodes in the synchronization group. 2. On each node in the synchronization group, run the following commands: a. Run quotactl on to enable quotas b. Run quotactl check to update usage statistics. 3. Start NFS and CIFS services on all nodes Be default, quotas are disabled in WOS Access. While disabled, quotas will not be enforced and I/O to WOS Access will not be counted for users or groups. Quotas can be enabled or disabled using the following command: quotactl {on off} Example: $> quotactl on Quota has been set to 'on $> quotactl off Quota has been set to 'off' NOTE: Quotas must be enabled or disabled on each node in a synchronization group, individually. After enabling quotas, it is always recommended to run the check command to update the quota database with current data. The command can be run as follows: quotactl check Example: $> quotactl check 0% 10 20 30 40 50 60 70 80 90 100% ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- *************************************************** Finished. NOTE: The check command must be run on each node in a synchronization group, individually. 6.5 Quota Reporting WOS Access provides three methods for reporting quota information: using quotactl on the WOS Access gateway, using the quota command on NFS client or via periodic email reports. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 108

6.5.1 Quota Reports using quotactl To report current quota usage on a WOS Access gateway, run the following command: quotactl get {-u -g} [-d -i <id>] [-h] Example: $> quotactl get u i 5002 -h UserID Capacity-Used Limit (Soft/Hard) Grace Time Remaining Inode-Used Limit (Soft/Hard) Grace Time Remaining ---------------------------------------------------------------------------------- -------------------------------------------------------------- 5002 0.00 1.00G / 1.20G - 0.00 0.00 / 0.00 - $> quotactl get u d h Capacity Limit (Soft/Hard) Inode Limit (Soft/Hard) --------------------------------------------------------------------------- User Default 400.00M / 500.00M 10.00K / 15.00K The following flags can be used: -u Report quotas for users -g Reports quotas for groups -d Report default quota settings -i <id> - Report quota settings for a particular user or group -h Report quotas using human-readable values For users or groups that have exceeded a soft quota, a grace period value will be displayed indicating the time remaining, in seconds, until the soft quota is converted to a hard quota. The grace period will be reset only when the capacity or inode usage falls below the soft quota limit. NOTE: The default grace period is 7 days (604,800 seconds). This can be modified using the quotactl cfg command. 6.5.2 Quota Reports from NFS clients Most UNIX-based NFS clients will have the quota utility command installed. This command can be used to display quota values for the current user, as shown below: Example: user1@nfsclient $> quota Disk quotas for user user1 (uid 10000): Filesystem blocks quota limit grace files quota limit grace wos 14 5120 10240 3 5 10 System usage (blocks) and the soft and hard capacity values (quota and limit, respectively, in the example above) are reported in 1K blocks. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 109

See the quota man page on your NFS client for more information about how to use this utility. 6.5.3 Quota Reports via Email WOS Access also provides the ability to receive periodic emails containing a report of current quota usage. To enable this feature, select the Quota Notifications checkbox on the Preferences tab in the Web UI, as shown below: Figure 69 Quota Notifications By default, quota reports are generated every 12 hours and are sent to the Notification E- mail list configured on the Preferences tab (as shown above). The time period is specified in the root user s crontab on each WOS Access gateway server. To modify the time period, use the crontab e command. 6.6 Functional Limitations The following limitations apply to WOS Access quotas: WOS Access currently supports monitoring usage and checking quotas only through NFS. Files or folders created through CIFS will not be accounted for in the quota usage statistics and will not be subject to quota limits under normal operating conditions. However, files and folders created through CIFS will be accounted for by the quotactl check command. In some cases this can lead to unexpected behavior. See the section below for more information on using quotas in a mixed-mode environment. Quotas can only be managed from the command-line, running as root directly on a WOS Access gateway. There is currently no Web UI support for managing quotas. Quotas apply to the entire WOS Access namespace. Quotas cannot be set on a per-directory or per-file basis. Note that files and folders in the TrashCan do not affect quotas. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 110

Quotas can be viewed from NFS clients but they cannot be modified (for example, using setquota or other tools). In some cases, hard quotas may be exceeded when namespace synchronization is used. Consider the case where two users from the same group create large files on separate gateways. Individually, the files do not exceed the group quota but together they will push used capacity beyond the hard quota. Only after the gateways namespaces have been synchronized will WOS Access recognize that the hard quota has been exceeded. 6.6.1 Using quotas in a mixed CIFS/NFS environment WOS Access quotas are assigned to users and groups using UID/GID values. These are the same UID/GID values of users and groups that use WOS Access via NFS. Unlike NFS, CIFS users and groups are identified using a Security ID (SID) a multicharacter string that uniquely identifies a Windows user or group across domains. When files or folders are created via CIFS, WOS Access will set the owner UID and group GID of the file/folder to a numeric value that is calculated using the last few digits of the SID. In some cases, the calculated UID/GID values may be the same ID values used by other NFS users or groups. When the quotactl check command is run, it will scan the entire namespace, gathering usage statistics for all files and folders, regardless of whether they were created via NFS or CIFS. In the case where CIFS UID/GID values intersect NFS UID/GID values, the reported usage for a user or group may be more than expected and may potentially exceed quota limits. To workaround this issue, two actions may be required: 1. Configure CIFS to use a single ID for all users and groups (see Section 5.3.13).This will ensure that all new CIFS data will be created with predictable ID values. 2. For pre-existing data, it may be required to manually update the owner and group of each file created via CIFS to use the single ID configured in the previous step. For assistance with this procedure, please contact DDN Support. 6.7 Troubleshooting If issues arise when using quotas, consider the following: Quota settings are synchronized using the same time periods as namespace synchronization. Therefore, it may take some time for settings to be synchronized across all nodes in a synchronization group. Quotas must be enabled/disabled on each node in a synchronization group. The quotactl check command should always be run immediately after quotas are enabled. NFS and CIFS services should be stopped while the command is being executed. Note that on systems with millions of files, the quotactl check command can take a significant amount of time to complete. The quotactl check command must be run on each node in a synchronization group when updating quota usage statistics. The following logs can be consulted when debugging quota-related issues: /opt/ddn/nas/log/wosnas-nfs.log DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 111

/opt/ddn/nas/log/quota.log 7.0 TrashCan Feature 7.1 Overview The TrashCan feature in WOS Access makes permanent deleting of files or folders (directories) a two-step process. When a user deletes a file or folder via CIFS or NFS, it is hidden and no longer visible to the user, but it is not immediately deleted from the underlying storage. Instead, the files or folders are placed into the WOS Access TrashCan. From there, they may be recovered and restored by an administrator or they may be permanently deleted after a certain amount of time has elapsed. The TrashCan feature is intended to support recovery and restoration of files or folders that may have been inadvertently deleted by users. The WOS Access database keeps a record of all deleted files or folders in a TrashCan folder. This folder is synchronized across all nodes in a synchronization group, however it is not visible through NFS or CIFS, preventing users from recovering files or folders on their own. In order to restore the deleted file or folder the WOS Access Administrator uses the command line utility trashcanutil from the WOS Access server see the section on the TrashCan utility below. Starting in version 1.4.0, WOS Access now automatically removes files and folders from the TrashCan after a certain period of time. By default, this occurs after a file or folder has been in the TrashCan for 7 days. However, this value is configurable see the section on Background Delete below. 7.2 TrashCan Utility tool (trashcanutil) The trashcanutil tool is distributed with WOS Access and is located at /usr/bin/trashcanutil. The WOS Access Administrator executes this utility to list, restore, or permanently delete the files or directories that are contained in the TrashCan folder of the WOS Access database. The following is the list of options supported by the TrashCan utility: -v, --version -q, --quiet --verbose -h, --help -l, --list <pattern> -d, --delete <pattern> -r, --restore <pattern> -t, --timestamp -f, --force Print version information Runs quietly Print verbose status information, works only with the 'status' option Print this help menu List all deleted files and directories that match the given pattern Permanently delete all matching files or folders from the TrashCan Restore all matching files or folders from the TrashCan Specify a file timestamp. Used to determine which file should be listed/deleted/restored in the case of multiple files or folders with the same name Ignore errors from WOS when deleting file objects DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 112

7.2.1 TrashCan: A Simple Example The following example shows the contents of the "TrashCan" folder after an NFS/CIFS user has deleted three files test1, test10, and test11 from the mount point. After having deleted these files the NFS user will no longer have access to these files. #> /usr/bin/trashcanutil l path type uid gid date rmtimestamp ---------------------------------------------------------------- /mnt/wos/test1 file 0 0 2014-01-09 10:52:06 1389293526 /mnt/wos/test10 file 0 0 2014-01-09 10:52:12 1389293532 /mnt/wos/test11 file 0 0 2014-01-09 10:52:14 1389293534 In case the user wishes to restore these files, the user needs to contact the WOS Access Administrator with the path name of the deleted files and the WOS Access Administrator can use the "-r" option of trashcanutil utility to restore the files. Below is the command to restore /mnt/wos/file1: #> /usr/bin/trashcanutil -r *test1 Object '/mnt/wos/test1' is restored as '/mnt/wos/test1' After restoration the file appears in its original location and the user can again access it via CIFS or NFS. The WOS Access Administrator can permanently delete the files or directories by using the "-d" option of the trashcanutil.pl utility. Below is the command to permanently delete test10. #> /usr/bin/trashcanutil -d *test10 Shown below are the contents of the example "TrashCan" table after having restored and deleted the files as shown above: #> /usr/bin/trashcanutil l path type uid gid date rmtimestamp ---------------------------------------------------------------- /mnt/wos/test11 file 0 0 2014-01-09 10:52:14 1389293534 7.2.2 Handling conflicts on recovery In some cases, file or folder recovery may overlap with existing changes to the namespace. For example, a user may delete a file named foo and while the file was in the TrashCan, a new file named foo was created under the same folder as the previous one. In this case, attempting to recover foo from the TrashCan will collide with the new foo. WOS Access handles this scenario by appending a unique identifier to the name of the restored file to differentiate it from the existing file. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 113

7.3 Background Delete By default, WOS Access will automatically delete files and folders from the TrashCan after 7 days. To prevent automatic deletion of TrashCan files and foldes, or to change the time period, do the following: On each WOS Access node in the synchronization group, open the file /opt/ddn/nas/etc/wos_deleter.conf Find the configuration entry DELETER_STORAGE_TIME and specify one of the following values: * - Disable automatic deletion files and folders will remain in the Trashcan indefinitely or until they are manually deleted using the trashcanutil.pl utility. 0 Immediately delete files and folders from the TrashCan. 'XXd Specify the number of days a file or folder will remain in the TrashCan before being auomatically deleted. For example, a value of 8d would delete files or folders that have been in the TrashCan for 8 days or more. 'XXh Specify the number of hours a file or folder will remain in the TrashCan before being auomatically deleted. For example, a value of 32h would delete files or folders that have been in the TrashCan for 32 hours or more. Restart the background delete daemon by running service wos_deleter restart 8.0 WOS Access Upgrade Procedure This section outlines the procedure for upgrading from previous versions of WOS Access (1.2.3 and above). For upgrading from older versions of WOS Access (1.2.2 or below) please contact DDN Support for assistance. NOTE: Refer to Section 2.5 of this guide for more detailed information about starting and stopping services, and adding nodes from Synchronization and HA (or SR) group. NOTE: A full backup of each WOS Access gateway should be performed prior to upgrading it. If a backup schedule has not been established, then a manual backup can be performed. See section 5.13 for more information. 8.1 Preparing for Upgrade Before executing the upgrade procedure, please perform the following steps: 1. Stop the Services Relocation service on all gateways in each synchronization group. 2. Stop NFS and CIFS services on all gateways in each synchronization group. 3. Wait until namespace synchronization has completed across all gateways. This can be accomplished by running the command nodectl s from any gateway node in the synchronization group and waiting for the Incoming and Outgoing values to drop to 0. 4. Stop the remaining services (Synchronization and Backup) on each gateway node. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 114

8.2 Upgrade Complete the steps below to upgrade each WOS Access gateway node: 1. Obtain the WOS Access ISO file from DDN Support and copy it to your local PC or desktop machine. 2. Copy the ISO file from your local machine to the WOS Access gateway node using the following command: scp <iso file> root@<wosaccess gateway>:/tmp 3. Login to the WOS Access gateway as root and run the following commands to perform the upgrade: mkdir p /mnt/upgrade mount o loop /tmp/<iso file> /mnt/upgrade bash /mnt/upgrade/wosaccess.d/wosaccess-<version>/wosaccess_upgrade.sh NOTE: The above commands can be run in parallel on each WOS Access gateway node, if desired. 4. Once the upgrade is complete, run the following command to reboot the gateway node: reboot n 8.3 Post-Upgrade Once all nodes have completed upgrading and have successfully rebooted, please do the following: 1. From the WOS Access UI Summary page on each gateway node, verify that all rpms report the correct version. 2. On each gateway node start the Namespace Synchronization service and wait until all nodes are synchronized by monitoring the output of nodectl s. 3. Select one gateway node in each namespace synchronization group and start the NFS and CIFS services. 4. After five minutes, start the NFS and CIFS services on all remaining gateway nodes. 5. If configured, start the Services Relocation and Backup services on all nodes. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 115

9.0 References 9.1 NFS Configuration 9.1.1 Overview The configuration file contains the sets of parameter blocks that control and define each component of the server. For example: <FEATURE_BLOCK> { Parameter name = Parameter value; } # comments 9.1.2 Export Not all parameter blocks or parameters are required. Table 38 describes each of the important blocks. Table 37 Block Name Usage Descriptions Export Block Name Usage Description Required - one or more Controls the NFS Export name and its parameters OVFS_WOS Required - one only Defines connection configuration to DDN's WOS backend storage. NFS_Worker_Param NFS Protocol Worker thread configuration worker NFS_Core_Param One instance CacheInode_client One instance Controls in-memory cache objects. An export defines the public name, access, protocols, and transport configuration for the NFS service (Table 38). Client Access The NFS client is a single host or network that establishes a connection to WOSAccess NFS requesting NFS file service. For example: Single host: a single machine identifiable by DNS name. Hosts by expressions: o wild cards: dell1*, matches will all the server name that starts with dell1 o range: dell-[0-9]-nfs, matches with dell-0-nfs through dell-9-nfs Networks: 10.40.30.0 describe all machines under the network for 10.40.30.* with netmask of 255.255.255.0 Combination: A combination of the above is possible to include multiple type access by using a comma as client separator o dell10, 192.168.1.0 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 116

Table 38 Export Parameter name Occurrence Description Path Required Path to export via NFS Tag Optional This option allows an alternative access for NFS v3 mounts. By using different Tag options, the same Path may be exported multiple times. Pseudo Optional This option specifies the position in the Pseudo FS this export occupies if this is an NFS v4 export. It must be unique. By using different Pseudo options, the same Path may be exported multiple times. Export_Id Required Unique positive integer for internal ID. Pseudo Required Clients Required Grant export entry access to host. That is, Clients="*"; grants access to all hosts. Squash Required List of hosts that have root access. Access_Type Default RW RO, RW, MDONLY, MDONLY_RO NFS_Protocols Optional Specifies the NFS versions supported: 3, 4, or 3,4 Transport_Protocols Optional TCP MaxOffsetRead Optional Max offset for read operation. No limit if not specified. MaxOffsetWrite Optional Max offset for write operation. No limit if not specified. MaxRead Optional Returned to NFSv3 client for read/write size negotiation MaxWrite Optional Same as above PrefRead Optional Same as above PrefWrite Optional Same as above FileSystem_id Required Unique per export, used by NFS client for internal metadata cache. Replicated system shall have identical filesystem_id SecType Optional SecurityType sys is the only supported one. 9.1.3 OVFS_WOS Table 39 describes the connection parameter for DDN WOS Storage and the metadata storage. Table 39 OVFS_WOS Block Parameter Name Occurrence Description OVFS_WOS_DB_HOST Required For Metadata storage, should be "localhost" DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 117

Parameter Name Occurrence Description OVFS_WOS_DB_PORT Required Default: 5432 OVFS_WOS_DB_NAME Required OVFS is the name OVFS_WOS_DB_USER Required OVFS - same as the DB_NAME OVFS_WOS_DB_PASSFILE Required Default: /root/.pgpass (generated during install) OVFS_WOS_CLUSTER Required Points to the primary DDN WOS CLUSTER IP. OVFS_WOS_POLICY Required WOS Policy name to store data into OVFS_WOS_USE_READ_STREAMS Required Default: "TRUE", recommended OVFS_WOS_CACHE_PATH Required Path to store temporary files during data ingestion OVFS_WOS_CACHE_EXP_TIME Required Default: 60 seconds. After expired time, data isready to flush to WOS OVFS_WOS_MAX_BATCH Required Default: 3000. Maximum number of files to process before going to pause OVFS_WOS_MAX_BATCH_THREAD Required Default: Same as NFS Worker Thread. OVFS_WOS_SLEEP_TIME Required Default: 20 sec. Sleeps for 20 seconds before next round of data flush OVFS_WOS_USER Required Default: guest - account to check for WOS capacity OVFS_WOS_PASSWORD Required Default: guest - password to check for WOS capacity DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 118

9.1.4 OVFS_BACKUP Parameters to control a WOSAccess Full backup are listed in Table 40. Table 40 OVFS_BACKUP Parameter Name Occurrence Description OVFS_BACKUP_SCHEDULE Required Schedule time - like cron job OVFS_WOSNAS_PATH Directory where wosaccess is installed OVFS_BACKUP_CONFIG_DIRS Configuration directory for WOSAccess Backup OVFS_BACKUP_LOG_DIRS Directory path for log files OVFS_BACKUP_UPLOAD_PATH Backup data files OVFS_BACKUP_STORE Defaults: yes OVFS_BACKUP_STORE_CMD Commands to send data to WOS OVFS_BACKUP_DB_FULL_HOURS Perform Full DB backup after specified hours OVFS_BACKUP_KEEP_DAYS Number of days to keep the backup OVFS_DB_ARCH_PATH pgr_man PGSQL archive log path OVFS_DB_DATA_PATH Original data path OVFS_DB_ARCH_KEEP_FILES Number of files to keep OVFS_DB_ARCH_KEEP_DAYS Number of days to keep. OVFS_BACKUP_MAIL_NOTIFICATION Send e-mail notification about backup activities OVFS_MAIL_SERVER Specify the mail server to use for sending mail OVFS_MAIL_ADDR_TO Recipients of the notification (list allow) OVFS_MAIL_USER_FROM Generic name for the FROM user OVFS_MAIL_SUBJ Subject prefix for the notification 9.1.5 CacheInode_Client Table 41 Cachelnode_Client Parameters Parameter Name LRU_NB_Call_Gc_Invalid LRU_Prealloc_PoolSize Entry_Prealloc_PoolSize DirData_Prealloc_PoolSize Description Number of request to handle before considering garbage collecting the LRU. Total number of pre-allocated LRU entry list in memory. Should be bigger than GC_Invalid. Pre allocated metadata entry for cache. Pre-allocated directory cache size. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 119

Parameter Name ParentData_Prealloc_PoolSize Attr_Expiration_Time Symlink_Expiration_Time Directory_Expiration_Time Use_Getattr_Directory_invalidati on Use_Test_Access Max_Fd Description Pre-allocated Parent directory cache size. Entry attributes expiration delay in second, before allowed for reclaim. Symbolic link content expiration delay in second, before allowed for reclaim. Directory content Expiration time. After dir are invalidated, use getattr. Validates access privilege before operation. Number of opened files. 9.1.6 NFS CORE Parameters Table 42 NFS Core Parameters Parameter Name Description NB_WORKER Number of worker threads NFS_PORT Default 2049 - port for NFS Service MNT_PORT Port for mount protocol (Default any available port) NFS_PROGRAM RPC Program for NFS (Default: 100003) MNT_PROGRAM RPC Program for mount protocol (default is 100005) Drop_IO_Errors Defines behavior of server with IO Error received form back-end storage. TRUE: request will be dropped. Client must retry. Drop_Serverfault_Errors Defines behavior of server when it loses database connectivity. In a fresh install this parameter is not present in the configuration file. It defaults to FALSE which means that the error is propagated to the client. To revert to the old behavior define the parameter in the configuration file and set it to TRUE. DupREQ_Expiration Lifetime of duplicate requests. Stats_file_path File for NFS statistic use wosacces_stat.pl to display Stats_update_delay Update frequency of the statistic in file NFS_protocols 3 9.1.7 Deprecated NFS Parameters The following table lists NFS configuration parameters from previous releases that are deprecated and no longer supported. Section Parameter Name Description CacheInode_Hash Index_Size Size of hash table: Prime number preferred. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 120

Section Parameter Name Description Alphabet_Length Prealloc_Node_Pool_Size Parameter for hashing algorithm less than 256. Number of pre-allocated entries in each hash table bucket. CacheInode_GC_Policy NB_Call_Before_GC Number of requests to process before starting garbage collection. This is different from the LRU GC. Runtime_Interval NBEntries_HighWater NBEntries_LowWater File_LifeTime Directory_LifeTime Time to sleep before running GC again (Per thread) Number of items over the mark (Per thread) If numbers falls below the watermark, stop the GC. Time in seconds before file is a candidate for GC. Time in seconds before directory entry is a candidate for GC. NFS_Worker_Param Pending_Job_Prealloc Number of pre-allocated entries for pending job. LRU_Pending_Job_Prealloc_ PoolSize NB_Before_GC NB_DUPREQ_PREALLOC LRU_DUPREQ_PREALLOC_ POOLSIZE Pending jobs LRU list Number of requests to process before considering GC. Number of pre-allocated entries for duplicate requests. Duplicate requests LRU Size. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 121

Section Parameter Name Description NB_DUPREQ_BEFORE_GC NB_IP_STATS_PREALLOC NB_CLIENT_ID_PREALLOC Number of requests to process before considering GC DUPREQUEST Queue. Number of pre-allocated entries for IP statistic tables NFSv4 client information cache entries. 9.2 Sync Configuration Configuration file located in /opt/ddn/nas/etc/wosaccess-cluster.conf. Table 43 WOSAccess Sync Configuration Parameters Parameter Name Description SYNC_BATCH_INTERVAL_SEC Entry lifetime before considering for sync SYNC_BATCH_TIMETOLIVE_DAY S Maximum number of days to keep sync entries in history SYNC_BATCH_SIZE Size of batches of sync entries to send to other nodes SYNC_DB_HOST Default localhost for the current node SYNC_DB_PORT Same as in wosaccess.nfsd.conf section OVFS_WOS SYNC_DB_NAME Same as in wosaccess.nfsd.conf section OVFS_WOS SYNC_DB_USER Same as in wosaccess.nfsd.conf section OVFS_WOS SYNC_DB_REPL_USER DB user name for replication/sync DDNPG_NODE_ID Starts with 1. All subsequent nodes MUST have a unique number. Must be less than 255. SYNC_LOG_FILE Location for sync log file SYNC_LOG_LEVEL Default INFO SYNC_MQ_EXCHANGE_NAME Message Exchange topic namespace SYNC_MQ_HOST Default localhost SYNC_MQ_USER guest SYNC_MQ_PORT Port for the message sender, default 5672 SYNC_MQ_PSWD guest SYNC_WOS_HOST IP address of WOS storage SYNC_WOS_POLICY Existing WOS policy SYNC_REDIS_HOST Default localhost SYNC_REDIS_PORT Port for the redis, default 6379 SYNC_PRODUCER_WORKER_THR Number of producer worker threads DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 122

EADS Parameter Name SYNC_CONSUMER_WORKER_TH READS SYNC_EXCEPTION_WORKER_TH READS SYNC_EXCEPTION_RETRY_TIME OUT_SEC SYNC_EXCEPTION_RETRY_COUN T SYNC_EXCEPTION_LOG_FILE Description Number of consumer worker thread (recommended to set it to value greater than number of producer threads at least in two times) Number of threads that work with exception log Time between SYNC_EXCEPTION_RETRY_COUNT retries Number of batch send retries for producer and batch apply retries before it moved to exception log. Exception log file path 9.3 SR Configuration WOS Access SR is an optional software package that uses Linux Corosync and pacemaker to provide WOS Access Services-Relocation services (Active/passive). The file is located in /opt/ddn/nas/etc/cluster.conf. The SR Configuration parameters are explained in Table 44. Table 44 SR Configuration Parameters Parameter name VIP VIP_INTERFACE VIP_MASK PRIMARY_NODENAME SECONDARY_NODENAME AIS_NETADDR AIS_REMOTE_NETADDR AIS_MCASTPORT Description Virtual IP to provide NFS Service. i.e.: eth0 or bond0 (for bonded NIC cards) Interface to provide the NFS Service over. (Must be the same interface for all of the participating member of the SR cluster) Network Mask for the VIP address Used for installation during bring up of services in the cluster. Name must be DNS resolvable or in /etc/hosts The second member of the SR pair NIC interface which is used by corosync for heartbeat check. NIC interface which is used by the second node for heartbeat check. Default=5450 (Port used by corosync for Unitcast) 9.4 CIFS Configuration Table 45 describes the configuration parameters for CIFS service. Table 45 WOSACCESS_CIFS Configuration Parameter Name Occurrence Description DB_HOST Required database host address DB_NAME Required database name DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 123

Parameter Name Occurrence Description DB_LOGIN Required database user DB_PORT Required database port LOG_FILENAME Required name of log file LOG_LEVEL Required log level for wos_fs library COMPONENT_FSAL Required log level for libwosposix WOS_CLUSTER Required WOS Storage address WOS_POLICY Required WOS policy WOS_USER Required WOS user account name WOS_PASSWORD Required WOS user account password FS_CACHEPATH Required path to disk cache folder in case of WOS disconnect CACHE_ENABLED Required Cache mode is enabled (TRUE) or disabled (FALSE) CACHE_SIZE Required Maximum size of cache folder specified in Megabytes or as a percentage of total local disk space use a value of -1 to use all available space CACHE_ALERT Required The storage limit of the cache folder at which email alerts will be generated, specified in Megabytes or as a percentage of total local disk space use a value of -1 to disable email alerts on CIFS cache size CACHE_THREADS Required cache threads pool size for WOS Upload feature DRW_USE_ROOT_TO_A DMIN_TRANSLATION Required use translation of permissions for root user (uid 0) and Administrator (uid 1800) for NFS share interoperability CIFS_ROOT_SHARE_PA TH WOS_ATTEMPTS_PER_ OPERATION Required Required the path to root share folder for CIFS shares the count of retries of I/O operations in case of error CIFS_MAIL_NOTIFY Required Email Notification is active (Y or y) or is switched off (n or N, by default) CIFS_MAIL_NOTIFY_AD Required Email notification receiver address DRESS CIFS_MAIL_SOURCE_A DDRESS Required Email notification sender address DATA_INTEGRITY_ENA BLED Required Data-integrity checksum calculation on write and check during file reading (TRUE if enabled) SHOW_NFS_STAGING_ Optional Enables showing of files written from DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 124

Parameter Name Occurrence Description FILES (FALSE if not present) NFS while they are zero in length in the database. Default value is FALSE, and if file is originally zero in length, it will be shown after writing to WOS. 9.5 quotactl command The quotactl utility is used to configure and manage quotas in WOS Access. The utility must be run as root on a WOS Access gateway server. All quotactl commands adhere to the following syntax: Parameters must be separated by spaces Unless otherwise indicated, parameters must be in the order specified Braces {} indicate mandatory parameters Brackets [] indicate optional parameters <> denotes a value place-holder 9.5.1 quotactl on/off NOTE: Unless otherwise indicated, the quotactl command only needs to be run on one node in a synchronization group. Quota settings will be synchronized with all nodes in the group. quotactl {on off} Use this command to enable or disable quotas. The following parameters are accepted: on Enable quotas on the current node. This will immediately cause quota limits to be enforced and usage to be monitored. off Disable quotas on the current node. Quotas will not be enforced and usage will not be accounted for. After enabling quotas, the quotactl check command should be run to update quota usage statistics. 9.5.2 quotactl status NOTE: This command only applies to the current node. It must be run separately on all nodes in a synchronization group. quotactl status Use this command to display whether quotas are enabled or disabled on the current node. NOTE: This command only applies to the current node. It must be run separately on all nodes in a synchronization group. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 125

9.5.3 quotactl cfg quotactl cfg [--grace=<secs>] Use this command to display or modify the current quota configuration. The following parameters are accepted: 9.5.4 quotactl set --grace Set the grace period, in seconds, that applies when a soft quota is exceeded. The default grace period is 7 days (604,800 seconds). quotactl set {-u -g} {-d -i <id>} {<softcap> <hardcap> <softinode> <hardinode>} Use this command to set user or group quotas. The following parameters are accepted: -u Modify a user quota -g Modify a group quota -d Modify the default user/group quota -i Modify the quota for a specific user/group. The <id> parameter must be a numeric UID or GID, identifying a user or group, respectively. User or group names are not accepted. <softcap> The soft capacity limit, in bytes <hardcap> The hard capacity limit, in bytes <softinode> The soft inode limit <hardinode> The hard inode limit All limit values must be specified. The limits can be specified directly or by using the following abbreviations: K (kilobytes), M (megabytes), G (gigabytes). A limit of 0 indicates that no quota will be enforced for that particular setting. The maximum limit value that can be specified is 4096 TB. 9.5.5 quotactl get quotactl get {-u -g} [-d -i <id>] [-h] Use this command to report user or group quotas and usage statistics. The following parameters are accepted: -u Report user quotas -g Report group quotas -d Report the default quota limits -i Report quota limits and usage data for a specific user or group. The <id> parameter must be a numeric UID or GID, identifying a user or group, respectively. User or group names are not accepted. -h Optionally display quota limits and usage data in human-readable format. If neither d or i is specified then a report for all users or groups will be generated. The following conventions are used in the quota report: DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 126

9.5.6 quotactl clear User or group ID values with a * are subject to the default quota limits. A quota limit of 0 means that no quotas are enforced for the user or group. NOTE: This command only applies to the current node. It must be run separately on all nodes in a synchronization group. quotactl clear {-u -g} {-d -i <id>} Use this command to clear user or group quota limits. The following parameters are accepted: -u Clear user limits -g Clear group limits -d Clear default quota limits -i Clear quota settings for the specific user/group. The <id> parameter must be a numeric UID or GID, identifying a user or group, respectively. Clearing a quota for a specific user or group will revert limits to the corresponding default quota or remove all limits if no default quota is defined. The grace period will be reset, if applicable. 9.5.7 quotactl check quotactl check Use this command to update storage usage statistics for users and groups. The statistics are used in checking quotas during I/O operations. This command should be run once after quotas are enabled. NFS and CIFS services should be disabled while this command is running to ensure accurate statistics. NOTE: This command only applies to the current node. It must be run separately on all nodes in a synchronization group. 10.0 Troubleshooting Problem: Suggestions, Diagnostics, Resolutions: 10.1 Troubleshooting Installation Installation has failed with "Spot check configuration files" Ensure you perform uninstall operation: $ sudo./uninstall.sh Perform install operation again. Installation has failed with "Please use the "-r" option to enable repository access." WOS Access requires some 3rd party packages. The install.sh will not attempt to access them unless you use the "-r" option. If you have used the "-r" option but you are getting this message, your machine may be having trouble accessing 3rd party DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 127

repositories and/or the internet. Contact your systems administrator. 10.2 Troubleshooting NFS Service 10.2.1 Server side errors Cannot start NFS server. Example: service wosnas-nfsd status wosnas.nfsd dead but subsys locked showmount -e rpc mount export: RPC: Unable to receive; errno = Connection refused Check for syntax errors in wosnas.nfsd.conf file. Check log file log/wosnas-nfs.log for more suggestions. OVFS_WOS_CLUSTER or OVFS_WOS_POLICY are not defined in wosnas.nfsd.conf configuration file. Metadata DB service is not running. Check it with following command: service postgresql status Check connection to WOS with appropriate WOS policy. The NFS service shows it has started, but status reports that service is stopped. Example: [root@dell41 nas]# service wosnas-nfsd start Starting wosnas.nfsd: [ OK ] [root@dell41 nas]# service wosnas-nfsd status wosnas.nfsd is stopped 1. Check the /opt/ddn/nas/log/wosnas-nfs.log. If the log file has the message, Bind_sockets :DISPATCH: FATAL ERROR: Cannot bind NFS tcp socket, error 98 (Address already in use). 2. Ensure that some other process or a defunct wosnas-nfsd from a previous run is not holding on to the port 2049. Use lsof. If some other process is holding on to the port, stop that process and start wosnas-nfsd again. 3. If there is a defunct wosnas-nfsd (Execute ps -ef grep wosnas to find defunct process) use kill 9 pid to clean it up. Like for any defunct daemon process a reboot of the machine may be needed to clear the defunct process entries. Once the defunct wosnas clears up, start the wosnas-nfsd again. [root@dell41 log]# lsof -i tcp:2049 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME nc 7716 root 3u IPv4 14638801 0t0 TCP *:nfs (LISTEN) Disk space is full 1. Check if /opt/ddn/nas/var/ovfs-cache contains too many files. Check if any error /opt/ddn/nas/log/wosnas-nfsd.log. 2. Check if WOS IP is disconnected or if WOS Policy no longer exists; use the command: # curl -v --request POST -H x-ddn-policy:"<wos- POLICY>" http://<wosip> /cmd/reserve 3. Fix the WOS Issue; free some disk space; and execute service wosnas- DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 128

10.2.2 Client side errors No such host. nfsd restart. 4. Check /var/log/messages for any crashes. If they exist, check the /var/spool/abrt directory. Move the crashes to another storage device to free up disk space. Execute service wosnas-nfsd restart. NFS or CIFS has unexpectedly shut down 1. The system disk space monitor may have shut down service to protect data integrity. Check the NFS or CIFS logs to determine the cause. 2. If you see a message matching "Resource limit matched Out of disk space.", then your system has run out of local disk space. See the sections "No free space left" and "Local backups occupy too much space". Files don't appear on NFS share for two minutes after restoring from TrashCan NFS uses cache for directory entries. The lifetime for cached directory entries is two minutes by default (Directory_Expiration_Time). Before the expiration of Directory_Expiration_Time, all information about the directory and its content will be extracted from cache. After the expiration of Directory_Expiration_Time, the NFS server will perform the direct query to the database and refresh cached directory entries. The restore of objects from TrashCan, which results in changes in database, is performed by trashcanutil utility. This utility is separate from the NFS server process. So these restoration changes will be visible to the NFS server only after direct access to the database - meaning after Directory_Expiration_Time. It is possible to reduce the Directory_Expiration_Time configuration value. But doing so will lead to a downgrade in performance, because direct query to the database occupies many more resources and time than the query to the cache. Name of the server is specified incorrectly. No such file or device. Either the local or remote file system is specified incorrectly. No such device. NFS is not configured into the client's kernel. NFS server is not responding. NFS server xxxx OK. The server is heavily loaded causing RPC timeouts, or the server has crashed. Stale file handle. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 129

The file is no longer available. This error can occur after restarting the NFS service or in the event of service relocation due to service or node failure on the primary node. This error will no longer be reported once the NFS service has been fully restarted or services relocation has completed. MOUNT_PROG not registered. rpc.mountd daemon never started up and registered. Too many levels of remote in path Attempting to mount a file system which is already an NFS mounted file system. Permission denied Accessing as root on the client and root is mapped to nobody. Or the user on the client does not have corresponding UID on the server. No space The server is out of space on the file system. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 130

Stale file handle. The file is no longer available. This error can occur after restarting or failing over of the NFS service. The error will disappear when the NFS service is ready to work: [DE12933] 1. on the secondary node in case of service relocation, 2. on the primary node after restart (it takes ~60 seconds for NFS to restart). 10.3 Troubleshooting Namespace Synchronization Service Too old or too new batch on local node exists. When these messages appear in logs: [2012/01/12 14:11:26] coldstart: BATCH_UUID_UNKNOWN message from node 1, switch to next active node [2012/01/12 14:11:26] coldstart: Local node contains unknown batch UUID. [2012/01/12 14:11:26] coldstart: Node has been down too long. Batch reached its TIME_TO_LIVE period and has been deleted. [2012/01/12 14:11:26] coldstart: Please re-join the cluster for full DB dump It may mean that not all of the batch that was sent from the local node had already been delivered to the remote node. 1. Check the state on the other nodes in the cluster. 2. Start them if their status is down. 3. Restart the local service: # service wosnas-replication restart Leave cluster process hangs Leave operation hangs if rabbitmq-server is stopped. 1. Start rabbitmq-server service: # service rabbitmq-server start 2. Leave cluster again: # nodectl l No free space left. 1. If rabbitmq-server and wosnas-replication services are not run for some reason, check whether disks are full. 2. If disks are full: a. Free some space. b. Restart rabbitmq-server service. c. Restart wosnas-replication service. Adding a new node to a sync group fails with this error if NFS or CIFS services on that node were started before it is joined to an existing sync group [root@dell41 ~]# nodectl -j dell40 Generating node ID Node ID: 2 ERROR: Database contains data, can't proceed. Error(-s) occurred during execution DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 131

1. Stop NFS and CIFS services on the node. 2. Drop the database on the node: (echo "DROP DATABASE ovfs" psql -U postgres) 3. Reinitialize the database: (/opt/ddn/nas/etc/ovfsdb_install_nfs.sh) 4. Restart the postgres database: (service postgresql restart) 5. Rejoin the node to the synchronization group. Successful rejoin example: [root@dell41 ~]# nodectl -j dell40 Generating node ID Node ID: 2 Configuring RabbitMQ cluster Declaring queues Installing SQL trigger and stored procedures INFO: Database is empty and needs to be synced with one of cluster nodes INFO: Applying database dump from host dell40 Sync process finished Starting wosnas-replication service 10.4 Troubleshooting RabbitMQ 1. Nodes are showing inactive or not running in WebGUI (as shown in red the following image) from RabbitMQ cluster This indicates that nodes were not removed properly from the RabbitMQ cluster in the first place. The following procedure is needed to remove the inactive nodes from the cluster. This involves setup of a new WOS Access node (referred to in the procedure as imposter and joining that to the cluster) DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 132

a. Create the imposter: /etc/init.d/rabbitmq-server stop vi /etc/rabbitmq/rabbitmq-env.conf and change the rabbitmq nodename to be the same as the dead node.. RABBITMQ_NODENAME=wosnfs4.. /etc/init.d/rabbitmq-server start b. Add the imposter into the existing cluster: rabbitmqctl stop_app rabbitmqctl reset rabbitmqctl force_cluster <rabbitmq_node_name@hostname_of_existing_cluster> <rabbitmq_dead_node_name@hostname_of_exsiting_cluster> DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 133

c. Remove the imposter from the cluster: rabbitmqctl start_app rabbitmqctl cluster_status rabbitmqctl stop_app rabbitmqctl reset rabbitmqctl start_app #### <- By this step, the node should be removed from the rabbitmq cluster 2. Renaming Host Names When a host name is renamed, it is important to perform the following steps to maintain data integrity using the new host name. Failing to do so will cause the error "Rabbit MQ did not start" from the WEB UI when starting the namespace synchronization group. After renaming a WOS Access host name, the following steps need to be done: a. Stop the RabbitMQ service and ensure no rabbitmq processes are running: service rabbitmq-server stop killall u rabbitmq b. Ensure that there are no erlang instances running: ps aux grep beam kill <pid of beam> c. Change hostname: echo newhostname > /etc/hostname echo 127.0.0.1 rabbit >> /etc/hosts hostname F /etc/hostname d. Remove existing mnesia directory: rm rf /opt/ddn/nas/lib64/mnesia/wosnfs* e. Start the RabbitMQ server: service rabbitmq-server start 10.5 Troubleshooting Administration Web UI Cannot Login User interface performs authentication via PAM. There is only one user supported at the moment: admin. So, if you have problems logging in, chances are that you can solve it by logging in using ssh: Check if admin user exists or not. It can be done using id admin command. If the user admin does not exist for some reason, it can be created using useradd admin command. You may want to change password for this user. Use passwd admin DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 134

under super user or sudo passwd admin under user with respective permissions. Also, the disk may be full and the system is refusing connections. About 1Kb is necessary to perform a successful login. Service status isn t changed after start/stop/restart was requested from the UI. Check the log of the appropriate service on the logs page. UI is not available Check your browser s address string: does it contain the IP address or hostname of the machine where UI was installed? Check if the machine you are trying to reach via http is available (via ssh, for example). Check if httpd is running (execute from superuser s shell: service status httpd). If httpd is not running, run it, execute as superuser service httpd start HA status is Failed, but supposed to be Active Check if HA nodes can access each other via ssh using keys for authentication. 10.6 Troubleshooting Disaster Recovery 10.6.1 Backup failure NOTE: Generally, there should be some useful debug information in the restore.txt file. Insufficient disk space Usually the system reports lack of space for operation explicitly. Clean up some space. If you are using local backup consider switching to WOS backup. Remove or move to safe place the old backups. See separate troubleshooting on reducing disk usage. Bad connection to WOS Failure of the storefio tool is reported. This can only happen if WOS storage of backup is enabled. You can check the wos connection: 1. Set the HOSTNAME and POLICY variables. 2. Issue this command: curl -v --data "testing from client $HOSTNAME - $RANDOM" -H x-ddn-policy:"$policy" http://$host/cmd/put Fix the WOS connection. Database configuration problem 1. This may result in stuck backup process. Check the database log in /var/lib/pgsql/data/pg_log/ for clues. DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 135

2. Repair database configuration. 3. Restart the database with the command, service postgresql restart. 4. Kill the runaway backup process. 5. Run the backup again. Disk space available is not enough for backup size 1. Check the pgsql_archive: # ls lt /var/lib/pgsql_achive 2. Delete the files created before the last successful backup to free up some disk space and reduce backup size. 3. Free up some disk space. 4. Execute backup. Manually run backup and got backup: cannot create backup: INFO: database backup start. ERROR: query failed: ERROR: WAL archiving is not active 1. Enable backup: /opt/ddn/nas/backup_ctrl e 2. Execute backup. 10.6.2 Backup uses up all the available disk space Local backups occupy too much space 1. Check the amount of space used by local backups: du -h <OVFS_BACKUP_UPLOAD_PATH> 2. Enable storing backups on WOS: OVFS_BACKUP_STORE= yes 3. Otherwise, reduce the number of files kept by reducing OVFS_BACKUP_KEEP_DAYS. Ineffective backup or cleanup schedule. 1. Check the directory specified with OVFS_DB_ARCH_PATH. The frequency of cleaning this directory is (roughly) determined by the frequency of full backups. Thus, if you run full backups very infrequently (say once in many days), this directory can become overpopulated. 2. Adjust the backup schedule. 10.6.3 No backup mail notification Mail notification not configured 1. Check the configuration in wosnas.nfsd.conf. 2. To receive email notification OVFS_BACKUP_MAIL_NOTIFICATION= yes DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 136

Mail server is unable to receive/relay the messages 1. Check the OVFS_MAIL_SERVER parameter in wosnas.nfsd.conf. 2. Check that you are able to connect to this server: telnet <server name> 25. Example if the server is functional: Connected to myserver.com. Escape character is '^]'. 220 myserver.. 3. Choose another mail server or reconfigure the existing one. Mail gets to spam folder 1. Check the spam folder. 2. Add your backup host to the whitelist on your mail service. 10.7 Troubleshooting SR Cluster Two interfaces of node located in one subnet Network manager, Selinux and iptables disabled but installation of SR failed with below-mentioned error: "ERROR: Timeout expired, check your firewall, selinux and /etc/hosts content" and file "/var/log/cluster/corosync.log" contains messages: "[TOTEM ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. " Corosync has problem with logic to bind Totem to the NIC when both interfaces of machine located in one subnet. The pseudo code used to determine if the interface should be bound to is: for all interfaces in system if bindnetaddr == iface.addr & iface.netmask bind to interface For example: If the local interface was 10.12.12.93 and the netmask was 255.255.255.0, Totem would execute the logical operation 10.12.12.93 & 255.255.255.0 and produce the value 10.12.12.0. This value would be compared against bindnetaddr and bind Totem to the NIC that matches. In this case, if interfaces configured in one subnet (Example: 10.12.12.92/24 and 10.12.12.93/24), then two interfaces matched to this pseudo code, corosync will bind Totem to the NIC to be first in the list of interfaces. Totem connection will be failed if bound interface not present in configuration file (for example, eth1 was specified in cluster.conf, but corosync bound Totem to eth0). Check which interface should be used for SR cluster: #ls -l --sort=time /var/lib/corosync/ringid_* head -1 grep -o -P "([\d]+.){3}[\d] Two-network interface setup on one host and only one fails If the broken interface was used in the cluster configuration (AIS_NETADDR or AIS_REMOTE_NETADDR parameter) on the active node, then services are relocated to the standby node. If network interface failed on the passive node, then this node will be offline. To solve this issue: 1. Run update command on the node with two network interfaces. 2. Set new parameter for AIS_NETADDR (network interface which DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 137

works): "/opt/ddn/nas/sr-cluster.sh" Corosync consumes 100% CPU Corosync traffic was blocked. Check iptables configuration on both hosts and make sure what required UDP ports are open. If required ports are opened, then traffic was blocked on the network appliance, which filters packets between nodes, and ports should be opened on that appliance. Service postgresql is not starting PostgreSQL database is not initialized. Before you can do anything, you must initialize a database storage area on disk. For example: # service postgresql initdb Initializing database: [ OK ] Fail over occurs At least one tracked service returns error for init-script status-command. To identify the failed service, run: # crm_mon -1 This will show all failed actions in field Failed actions. For example: # crm_mon -1 ============ ============ Online: [ MYHOST2.mycompany.com MYHOST1.mycompany.com ] failover-ip (ocf::heartbeat:ipaddr2): Started MYHOST1.mycompany.com failover-mon (lsb:wosaccess-mon): Started MYHOST1.mycompany.com Clone Set: cl_wosaccess_cluster [wosaccess_cluster] Started: [ MYHOST1.mycompany.com MYHOST2.mycompany.com ] Failed actions: failover-nfs-diskspace_monitor_30000 (node=myhost1.mycompany.com, call=30, rc=7, status=complete): not running A forced failover is required. To trigger a forced failover: # crm_resource --resource myresource --move --node altnode To emulate a failure of one of tracked services: # crm_resource --resource failover-dev --fail --node Node Upon failure of a WOS Access node, the state of files written to that node since the last successful checkpoint is undefined. When Services Relocation is configured, NAS Services will be relocated to a secondary node upon failure of the primary node, and any file data written to that failed node since the last checkpoint will need to be rewritten to the secondary node once the NAS Services have been relocated. 10.8 Troubleshooting CIFS Service Event Viewer errors in MS Management Console To prevent Event Viewer errors in MMC after joining the CIFS host in AD, check the registry settings on the AD Domain Controller: 1. Check that read permission to Local Service is set on winreg key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Sec urepipeservers\winreg DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 138

2. Verify that the ClientProtocols key exists under HKLM\Software\Microsoft\Rpc and contains the five default values listed in Table 46. 3. If the ClientProtocols key or any of the default values is missing, import the key from a known good server. Table 46 ClientProtocols key default values Name Type Data ncacn_http REG_SZ rpcrt4.dll ncacn_ip_tcp REG_SZ rpcrt4.dll ncacn_nb_tcp REG_SZ rpcrt4.dll ncacn_np REG_SZ rpcrt4.dll ncacn_ip_udp REG_SZ rpcrt4.dll Service is not available 1. Check the output of the command: sudo /opt/ddn/nas/bin/lwsm list. It will show status of all CIFS services. 2. If some service is stopped or dead, you can restart it by command: sudo /opt/ddn/nas/bin/lwsm start <servicename> where <servicename> is a name of corresponded service. NOTE: The service pvfs should be stopped and doesn t need to be restarted. Error BAD_NETWORK_PATH This error is caused by the wrong path in the SMB connection process. To check the right path: sudo /opt/ddn/nas/bin/lwnet share This command shows a list of available shared folders. The name of the share should be used as a path in the SMB connection. Error "RPC server is unavailable" on Windows client host The hostname of WOSACCESS CIFS server is configured improperly and should contain appropriate fully qualified domain name. Failed to map drive to WOS Access CIFS using new created Admin user DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 139

User may not be in Administrators group. To verify the new user is not in the Administrators group, use the command: # /opt/ddn/nas/bin/lw-lsa enum-members Administrators Ensure that Access Permission is configured for this new user when adding the cifs share to export list. From Client, error:no space left on device is observed when writing to a CIFS mount point The WOS Access CIFS service may need to be restarted because the WOS Access CIFS may have lost connection to the WOS. CRITICAL ERROR: WOS disconnection detected! may appear in the server log /opt/ddn/nas/log/wosposix.log. 1. Confirm that the policy and the WOS configuration is correct by manually checking the connection to the WOS with the curl command. 2. If curl is successful, restart WOS Access CIFS. 3. If curl is not successful, the WOS policy may be incorrect. curl example: # curl -v --request POST -H x-ddn-policy:"<wos-policy>" http://<wosip> /cmd/reserve [root@dell41 monitortools]# curl -v --request POST -H x-ddnpolicy:"cifs-test" http://10.40.30.65/cmd/reserve * About to connect() to 10.40.30.65 port 80 (#0) * Trying 10.40.30.65... connected * Connected to 10.40.30.65 (10.40.30.65) port 80 (#0) > POST /cmd/reserve HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > Host: 10.40.30.65 > Accept: */* > x-ddn-policy:cifs-test > < HTTP/1.1 200 OK < Content-Type: application/octet-stream < x-ddn-status: 202 UnknownPolicyName < Date: Mon, 10 Sep 2012 22:33:42 GMT < Server: WOS/0.1 < Content-Length: 0 DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 140

Stop error message on a computer that is running Windows Server 2008, Windows Server 2008 R2, Windows Vista, or Windows 7 system: 0x00000027 RDR_FILE_SYSTEM may occur. This error may be caused by incorrect operation of Mrxsmb20.sys driver in the Windows operating system. Please install these hotfixes for the appropriate operating system versions: Stop error message on a computer that is running Windows Server 2008 or Windows Vista: "0x00000027 RDR_FILE_SYSTEM" (http://support.microsoft.com/kb/974759). "0x00000027" Stop error occurs when you access shared network resources by using the SMB 2.0 protocol in Windows Server 2008 R2 or in Windows7 (http://support.microsoft.com/kb/2494418). "STATUS_OBJECT_NAME_NOT_FOUND" error message when you open a newly-created file in a shared folder in Windows 7 or in Windows Server 2008 R2 This error may occur because of a timing issue in the SMB driver on the Windows client. The following hotfix should be applied to fix this error: http://support.microsoft.com/kb/2628582/en DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 141

11.0 Support Contact DataDirect Networks Support at any time for assistance. Web: http://ddn.com/request-support Email: support@ddn.com North America: +1.888.634.2374 International: +1.818.718.8507 11.1 Available Online Documentation You may access both the latest version and archived versions of WOS Access documentation online. http://www.ddn.com/support/product-downloads-and-documentation 12.0 Appendix 12.1 List of Product Files Table 47 Notable product directories and files Description Home directory User tools Binaries Configuration CIFS Settings Log files Documentation Web UI Files Backup files Backup restore log file Stage files Postgres data Postgres archive Path /opt/ddn/nas /opt/ddn/nas/tools /opt/ddn/nas/bin /opt/ddn/nas/etc /opt/ddn/nas/share /opt/ddn/nas/log /opt/ddn/nas/doc /opt/ddn/nas/var/web /opt/ddn/nas/var/backup /opt/ddn/nas/var/ovfs-backup/restore.txt /opt/ddn/nas/var/ovfs-cache /var/lib/pgsql/data /var/lib/pgsql_archive For a complete file listing, see /opt/ddn/nas/doc/wosaccess-files.txt. 12.2 List of TCP/UDP ports used by WOS Access Table 48 WOS Access TCP/UDP Proto Port Program Description Part of module tcp 111 rpcbind NFS RPC port mapper NFS DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 142

Proto Port Program Description Part of module tcp/udp 875 rquotad NFS quotas NFS tcp 4369 epmd erlang port mapper NS tcp 22 sshd SSHd HA tcp 761 snmpd SNMPd/AgentX NFS udp 161 snmpd SNMPd NFS tcp 5432 postmaster Postgresql tcp 2049 wosnas.nfsd NFS proto NFS tcp 80 httpd httpd Web Admin UI tcp 443 httpd httpd Web Admin UI tcp 6379 redis Redis NS tcp 5672 beam.smp erlang NS udp 5449 corosync HA udp 5450 corosync HA udp 123 ntpd NTP NS tcp 445 lwsmd SMB protocol CIFS tcp 135 dcerpc Microsoft RPC service CIFS tcp/udp 2812 monit System Monitor tcp 80 wos_deleter WOS REST interface NFS/CIFS tcp 8085 WOS Access WOS communication All tcp 8088 WOS Access WOS REST management API All tcp 8090 WOS Access WOS communication All Abbreviations: NS - Namespace Synchronization HA - High Availability NOTE: Corosync port number depends on AIS_UCASTPORT in file /opt/ddn/nas/etc/cluster.conf. Table 49 Additional WOS Access TCP/UDP requirements for Active Directory Domain Proto Port Application tcp/udp 42 WINS tcp/udp 53 DNS tcp/udp 88 Kerberos tcp/udp 135 RPC tcp/udp 137 NetBIOS tcp/udp 139 NetBIOS SSN tcp/udp 1025-5000, AD: Replication, User and DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 143

Proto Port Application 49152-65535 Computer Authentication, Group Policy, Trusts tcp 389 LDAP tcp 636 LDAP/SSL udp 123 NTP udp 138 NetBIOS Data More information about Active Directory Domain services port requirements can be found at: http://technet.microsoft.com/en-us/library/dd772723(ws.10).aspx. [DE5797] DataDirect Networks WOS Access 1.4.0 Admin Guide Rev. B5 144

World Headquarters 9351 Deering Avenue Chatsworth, CA 91311 ddn.com Phone: +1.818.700.7600 Fax: +1.818.700.7601