Project Iris Server Cluster Support: High Availability through Microsoft Cluster Service DocBase Version 4.2 12/13/2000 Page 1 of 11
1.0 Overview There are many ways in which the Server may be configured to run on Microsoft Cluster Service. This document talks about how to set up two such configurations, active/passive cluster and active/active cluster, assuming the underlining DBMS resides somewhere else. The steps described here work for both NT 4.0 cluster and Windows 2000 Advanced Server cluster. In both configurations, two computers are connected together to form an MSCS cluster. These machines are called the first cluster node and the second cluster node. Resources such as virtual IP addresses, virtual network hostnames and shared disks are grouped together into one or more resource groups. At anytime, a resource group is owned by one node and is accessible only to that node. The DocBroker and the Server which utilize the resources are run as services or applications. They are also part of the resource group. When one node goes down, a resource group is automatically moved to the surviving node. As part of the resource group failover, the DocBroker and Server are restarted on the surviving node. With the active/passive configuration, the first node is the primary and the second node is the standby. The DocBase is stored on the shared disk. The DocBroker and the Server run on the primary. When the primary fails, the standby takes ownership of the resource group, and the DocBroker and the Server are restarted on the standby node. With the active/active configuration, each node is the standby of the other. Each node owns one resource group. Each resource group has its own virtual IP address, virtual network hostname, shared disk, DocBroker and Server. When one machine goes down, the resource group managed by this node is automatically moved to the other machine. Now the surviving machine manages two resource groups (until the failed node comes back up and the cluster administrator moves one group back). Note that the DocServers discussed in the active/active configuration are for different DocBases. As discussed in Cluster Support Functional Specification, Microsoft Cluster Service is a shared-nothing architecture. I.e., only one machine owns a shared disk (or other resources) at any time, and only that machine can access that disk. Therefore the MSCS active/active configuration does not support multiple machines for one DocBase (multiple machines per DocBase is supported on UNIX and NT or Windows 2000 systems as long as shared disks can be accessed by more than one machines). Page 2 of 11
1.1 Configuring active/passive cluster Client Client standby machine, no server primary server running primary machine server machine down standby machine docbase docbase shared disks shared disks Figure 1, MSCS active/passive cluster 1.1.1 Ensuring proper setup of Microsoft Cluster Service 1. Before Server is installed and configured, create a cluster resource group with the following through MSCS Cluster Administrator utility, A virtual IP address This is the IP address for Server to use. This IP address can be shared by other products as the DocBroker and the DocServer only listen on certain ports (more than one virtual IP addresses may be necessary if other products like DBMS also reside on the cluster and if it is desirable to use separate IP addresses). A virtual network hostname This is the virtual hostname for the IP address, ensure that there is a dependency to the virtual IP address. A shared disk partition This is where the DocBase resides. 2. Take the resource group on line and move the resource group back and forth a few times to ensure everything in the group can be failed over from one node to the other. Page 3 of 11
1.1.2 Installing Server on local drives Follow Installing Server for Microsoft Windows, Custom Installation, Copying files from Installation Media to install Server onto both nodes on identical local drives. Installing Server on identical disks eliminates many changes later on. 1.1.3 Configuring the DocBase on the first node 1. Ensure that the first node owns the cluster resource group created in section 1.1.1. 2. Follow Installing Server for Microsoft Windows NT, Chapter 3, Custom Installation, Configuring the Docbase to configure the DocBase on the first node, At step 5, choose Use Local DocBroker, and choose Manual. The DocBroker is usually started from MSCS Cluster Administrator utility. At step 9, choose Create SQL Server account and database if the DBMS is SQL Server, or choose Create Oracle account and tablespace if the DBMS is Oracle. At step 14, click on Manual to start the DocServer service manually when the computer restarts. The DocServer is usually started from MSCS Cluster Administrator utility. Create a directory \documentum\data\<docbasename> or make sure it exists on the shared drive where the DocBase will reside. At step 17, highlight Docbase Headstart and click Edit script, to edit the script to point the location object to the shared drive. For example, create dm_location object set object_name = storage_01 set path_type = directory set file_system_path = N:\documentum\data\<docbasename>\content_storage_01 set mount_point_name = go Here, N: is the shared drive. Click Next to run scripts that complete the configuration of the DocBase. 3. After the set up is complete, shut down both the DocServer and the DocBroker. 1.1.4 Configuring the DocBase on the second node 1. Use MSCS Cluster Administrator utility to move the resource group created in section 1.1.1 to the second node. 2. Follow Installing Server for Microsoft Windows NT, Chapter 3, Custom Installation, Configuring the Docbase to configure the DocBase on the second node, At step 5, choose Use Local DocBroker, and choose Manual. The DocBroker is usually started from MSCS Cluster Administrator utility. Page 4 of 11
At Step 7, type in the same Docbase Name and Docbase ID as the ones created in section 1.1.3. At Step 9, choose Use existing SQL Server account and database if the DBMS is SQL Server, or choose Use existing Oracle account and tablespace if the DBMS is Oracle. At step 14, click on Manual to start the DocServer manually when the computer restarts. The DocServer is usually started from MSCS Cluster Administrator utility. At step 17, click Cancel and exit Setup. The scripts have already been run during the configuration of the DocBase on the first node. 3. After the set up is complete, shut down both the DocServer and the DocBroker. 1.1.5 Configuring the DocBrokers to listen on the virtual IP address The following is performed on each node, 1. Execute Server Manager (Start->Programs->Docuemntum->EDM Server Manager), highlight Docbroker and click Edit Service. Modify Command to add host <virtual_host_name>, everything else remains the same. For example, if the virtual network hostname is dmcluster1, f:\documentum\product\4.2\bin\dmdocbroker.exe -host dmcluster1 -logfile f:\documentum\dba\log\docbroker.log -run_as_service 2. Edit server.ini to reflect the virtual network hostname, for example, [DOCBROKER_PROJECTION_TARGET] host = dmcluster1 3. Edit dmcl.ini and modify the host entry as follows, [DOCBROKER_PRIMARY] host = dmcluster1 1.1.6 Creating cluster resource for the DocBroker and the DocServer It is only necessary to perform the following steps on the node that owns the resource group. Assuming the resource group is called Documentum group 1, 1. Creating the DocBroker service resource Select Start->Programs->Administrative Tools->Cluster Administrator. Highlight Documentum group 1 and select File-> New->Resource. Page 5 of 11
Enter DocBroker as the Name, and enter Generic Service as the Resource Type. Click Next. At the Possible Owners dialog box, both nodes should be in the Possible Owners list box. Accept this default and click Next. DocBroker should be dependent on the virtual network hostname. Move the virtual network hostname from the Available Resources list box to the Resource Dependencies list box and click Next. Enter dmdocbroker as the service name, it is very important to click Use Network Name as computer name. Click Next, leaving startup parameters blank. Click Finish, then Ok. Bring the DocBroker generic service resource online by highlighting DocBroker and select File->bring online. 2. Creating the DocServer service resource Highlight Documentum group 1 and select File->New->Resource. Enter <DocBase Name> Docbase as the Name, and enter Generic Service as the Resource Type. Click Next. At the Possible Owners dialog box, both nodes should be in the Possible Owners list box. Accept this default and click Next. Move the Shared Disk Partition, the Virtual Network Name and DocBroker from the Available Resources list box to the Resource Dependencies list box and click Next (Note: The DocServer resource already depends on the DocBroker resource, which in turn depends on the Virtual Network Name resource. Therefore the DocServer resource is already indirectly dependent on the Virtual Network Name. However, we had to explicitly specify dependency on the Virtual Network Name on NT MSCS). Enter dmserver<docbase name> as the service name, it is very important to click Use Network Name as computer name. Click Next, leaving startup parameters blank. Click Finish, then Ok. Bring the DocServer generic service resource online by highlighting <DocBase Name> Docbase and select File->bring online. 1.1.7 Verifying the failover To verify that the failover works properly, attempt to connect to the DocBase from a client machine, make sure the dmcl.ini entries on the client machine refer to the virtual network hostname or virtual IP address. Use MSCS Cluster Administrator utility, move resource group Documentum group 1 created above from the cluster node it is running on to the other node. The client should be able to execute queries after the resource group has successfully moved to the other node and comes online. Page 6 of 11
1.2 Configuring active/active cluster Client Client Client Client server A machine A server B machine B machine A is down server A server B machine B docbase A with its shared disks docbase B with its shared disks docbase A with its shared disks docbase B with its shared disks Figure 2, MSCS active/active cluster Configuring an active/active cluster is very similar to that of active/passive cluster. The difference is that with active/active, each node has its own DocBase initially. At the beginning, two cluster resource groups are created and each node owns one resource group. In each of the cluster resource groups, there are one virtual IP address, one virtual network hostname, one shared disk drive for the DocBase, and one DocBroker service or application, and one DocServer. As only one DocBroker can be run as service, the DocBroker for the second resource group is created as a cluster application resource. 1.2.1 Installing Server and configuring the first resource group This is exactly the same as installing and configuring an Server active/passive cluster. Refer to steps in section 1.1, Configuring active/passive cluster to install Server on both machines, and configure the first DocBroker and the first DocBase for both nodes. 1.2.2 Configuring the second resource group This is very similar to configuring an Server active/passive cluster, except this time the Server is already installed, and the DocBroker is created as a cluster application resource. Notice that the step by step instruction is very similar to that of configuring Server for an active/passive cluster. Page 7 of 11
1.2.2.1 Ensuring proper setup of Microsoft Cluster Service 1. Create a new cluster resource group with following resources through MSCS cluster administrator utility, A virtual IP address This is the IP address for the second DocBase. A virtual network hostname This is the virtual hostname for the IP address, make sure there is a dependency to the virtual IP address. A shared disk partition This is where the second DocBase resides. 2. Take the resource group on line and move the resource group back and forth a few times to ensure everything in the group can be failed over from one node to the other. 1.2.2.2 Configuring the DocBase on the second node 1. Make sure the second node owns the cluster resource group created in section 1.2.2.1. 2. Follow Installing Server for Microsoft Windows NT, Chapter 3, Custom Installation, Configuring the Docbase to configure the second DocBase on the second node, At Step 5, choose Use Remote Docbroker, and put in the virtual network hostname created in section 1.2.1 as Host Name. Note that the virtual network hostname used here is from the first resource group. The DocBroker for this (the second) resource group is not yet created and is to be run as an application resource later. At step 9, choose Create SQL Server account and database if the DBMS is SQL Server, or choose Create Oracle account and tablespace if the DBMS is Oracle. At step 14, click on Manual to start the DocServer manually when the computer restarts. The DocServer is usually started from MSCS Cluster Administrator utility. Create directory \documentum\data\<docbasename> or make sure it exists on the shared drive where the second DocBase will reside. At step 17, highlight Docbase Headstart script and click Edit script, to edit the script to point the location object to the shared drive. For example, go create dm_location object set object_name = storage_01 set path_type = directory set file_system_path = M:\documentum\data\<docbasename>\content_storage_01 set mount_point_name = Here, M: is the shared drive for the second resource group. Page 8 of 11
Click Next to run scripts that complete the configuration of the DocBase. 3. After the set up is complete, shut down the DocServer for this DocBase. 1.2.2.3 Configuring the DocBase on the first node 1. Use MSCS Cluster Administrator utility to move the resource group created in section 1.2.2.1 to the first node. 2. Follow Installing Server for Microsoft Windows NT, Chapter 3, Custom Installation, Configuring the Docbase to configure the second DocBase on the second node, At Step 5, choose Use Remote Docbroker, and put in the virtual network hostname created in section 1.2.1 as Host Name. Note that the virtual network hostname used here is from the first resource group. The DocBroker for this (the second) resource group is not yet created and is to be run as an application resource later. At Step 7, type in the same Docbase Name and Docbase ID as the ones created in section 1.2.2.2. At Step 9, choose Use existing SQL Server account and database if the DBMS is SQL Server, or choose Use existing Oracle account and tablespace if the DBMS is Oracle. At step 14, click on Manual to start the DocBase manually when the computer restarts. The DocServer is usually started from MSCS Cluster Administrator utility. At step 17, click Cancel and exit Setup. The scripts have already been run during the configuration of the DocBase on the second node. 3. After the set up is complete, shut down the DocServer for this DocBase. 1.2.2.4 Editing server.ini and dmcl.ini Edit all the server.ini files (server.ini files for the first and the second DocBase) to reflect the virtual network hostname for this resource group. For example, assuming the virtual network hostname for the first resource group is dmcluster1 and the second resource group is dmcluster2, [DOCBROKER_PROJECTION_TARGET] host = dmcluster1 [DOCBROKER_PROJECTION_TARGET_1] host = dmcluster2 Edit dmcl.ini and modify the host entries as follows, [DOCBROKER_PRIMARY] host = dmcluster1 Page 9 of 11
[DOCBROKER_BACKUP_0] host = dmcluster2 1.2.2.5 Creating the DocBroker generic application resource The following steps are done on one node only, assuming the second resource group is called Documentum group 2, and is owned by this node, Select Start->Programs->Administrative Tools->Cluster Administrator. Highlight Documentum group 2 and select File-> New->Resource. Enter DocBroker2 as the Name, and enter Generic Application as the Resource Type. Click Next. At the Possible Owners dialog box, both nodes should be in the Possible Owners list box. Accept this default and click Next. DocBroker2 should be dependent on the virtual network hostname for this group. Move the virtual network hostname from the Available Resources list box to the Resource Dependencies list box and click Next. Enter DocBroker2 as the Application Name. If the virtual network hostname for this group is dmcluster2, and Server is on f:\documentum\product\4.2, fill in the following to Command Line f:\documentum\product\4.2\bin\dmdocbroker.exe -host dmcluster2 -logfile f:\documentum\dba\log\docbroker2.log Type in f:\documentum for Current Directory. This is very important to click Use Network Name as computer name. Click Next. Click Finish, then Ok. As DocBroker2 application resource is started from MSCS Cluster Administrator utility, ensure that the user who configured MSCS has read and write permission on the directory where the DocBroker log is residing for both nodes. In this case, f:\documentum\dba\log. MSCS does not give very informative message if this step is skipped. Bring the cluster resource DocBroker2 online by highlighting Docbroker2, and select File->bring online. 1.2.2.6 Creating the DocServer service resource The following steps are only done on the node that owns the second resource group, Highlight Documentum group 2 resource group and select File->New->Resource. Enter <DocBase Name> Docbase as the Name, and enter Generic Service as the Resource Type. Click Next. At the Possible Owners dialog box, both nodes should be in the Possible Owners list box. Accept this default and click Next. Page 10 of 11
Move the Shared Disk Partition, the Virtual Network Name and DocBroker2 from the Available Resources list box to the Resource Dependencies list box and click Next (Note : The DocServer resource already depends on the DocBroker resource, which in turn depends on the Virtual Network Name resource. Therefore the DocServer resource is already indirectly dependent on the Virtual Network Name. However, we had to explicitly specify dependency on the Virtual Network Name on NT MSCS). Enter dmserver<docbase name> as the service name. This is very important to click Use Network Name as computer name. Click Next, leaving startup parameters blank. Bring this DocServer service resource online by highlighting <DocBase Name> Docbase and select File->bring online. 1.2.3 Verifying the failover To verify the failover, attempt to connect to both DocBases from a client machine, make sure the dmcl.ini entries on the client machine refer to both virtual network hostnames (or virtual IP addresses). Use MSCS Cluster Administrator utility, move the two Documentum resource groups created above back and forth from each node. The client should be able to execute queries after the groups have successfully moved to the other node and come online. Page 11 of 11