White Paper EMC DOCUMENTUM XPLORE 1.2 AND XPLORE 1.3 HIGH AVAILABILITY IN AN ACTIVE-ACTIVE SETUP WITH LOAD BALANCER Abstract This white paper explains how to set up two xplore instances for load balancing on two clean host machines. This paper is organized in a set of steps that needs to be followed in sequence. November 2012
Copyright 2012 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Part Number h11213 2
Audience This white paper is intended for EMC customers, partners, internal developers and engineers. It is organized in a set of steps that are easy to follow. Whenever required, a reference to other published guide is provided in order to avoid the duplication of information. Active-Active setup Details As mentioned in the xplore Installation Guide, If you require xplore high availability, you will typically also require high availability for your Content Server. Therefore, you should coordinate your Content Server and xplore high availability deployments. Although many Content Server and xplore high availability configurations are possible, a single repository served by two Content Servers is typical and can be used as a building block for more complex deployments. See the EMC Documentum Content Server Enterprise Edition Installation Guide for more information about the different high-availability deployments and installation instructions for Content Server. HOST A Indexing Single Content Server or Content Servers with sharing Repository Querying HOST C Indexing HOST B Index Agent 1 Index Agent 2 Load-Balancer Index Server1 Index Server2 HOST A Indexing Single Content Server or Content Servers with sharing Repository IndexAgent1 Querying IndexAgent2 HOST C Indexing HOST B Load-Balancer Index Server1 Index Server 2 3
A. The following describes the steps to install and configure xplore High Availability in active-active mode as per the following diagram: Indexing Single Content Server HOST C Indexing HOST A Querying HOST B Index Agent 1 Index Agent 2 Load-Balancer Index Server1 Index Server2 The following steps must be strictly followed in the listed sequence: 1. Install Content Server D7 and configure one repository on host C. Please refer to EMC Documentum Content Server Enterprise Edition Installation Guide D7 for various other high-availability deployments and installation instructions for Content Server. 2. Install and configure Load Balancer. For the purpose of this white paper the testing is done with Cisco Load Balancer namely ACE 4710 Device Manager A3(2.0). 3. Install and configure the first xplore primary instance and index agent on host A. Please refer to EMC Documentum xplore Installation Guide. Stop the index agent if it is started. 4. On Content Server host C, execute the following dmbasic script located at %DM_HOME%\install\admin to update full-text index related objects in the repository: dmbasic -f create_fulltext_objects_ha.ebs -ehapreinstallstep -- repository superuser password Note: ehapreinstallstep is a single word and does not contain space. Also -- is a double dash and not. 5. Install and configure the second xplore primary instances and index agent on host B. And stop the index agent if it is started. 6. On Content Server host C, execute the following dmbasic script located at %DM_HOME%\install\admin to update full-text index related objects in the repository: Dmbasic -f create_fulltext_objects_ha.ebs -ehapostinstallstep -- repository superuser password 7. Restart Content Server. Execute following queries to validate dm_fulltext_index, dm_ftengine_config and dm_ftindex_agent_config objects are updated correctly. API>?,c,select r_object_id,index_name,ft_engine_id,is_standby from dm_fulltext_index 4
r_object_id index_name ft_engine_id is_standby ----------- ---------- ------------ ---------- 3b12d68780000100 repo3_ftindex_00 0812d68780000cd8 0 3b12d68780000500 repo3_ftindex_01 0812d68780000d01 1 (2 rows affected) API>?,c,select r_object_id,object_name from dm_ftengine_config r_object_id object_name ----------- ----------- 0812d68780000cd8 DSearch Fulltext Engine Configuration 00 0812d68780000d01D Search Fulltext Engine Configuration (2 rows affected) API>?,c,select r_object_id,object_name,index_name from dm_ftindex_agent_config r_object_id object_name index_name ----------- ----------- ---------- 0812d68780000cda cs4-lb-m1_9500_indexagent dm_fulltext_index_user 0812d68780000d9a cs4-lb-m2_9700_indexagent dm_fulltext_index_user (2 rows affected) 8. Start the indexing on both xplore instances. 9. Confirm the following URL will return True on both xplore instances if both instances are up: http://hostname:portnumber/dsearch/?action=checksearchable 10. Update dsearch_qrserver_host and dsearch_qrserver_port values of dm_ftengine_config object in the repository with the VIP (Virtual IP) and the port used by the load balancer for the xplore HA deployment. Both query plug-in point to the same load balancer IP address and port. API>?,c,select r_object_id,object_name from dm_ftengine_config r_object_id object_name ----------- ----------- 0812d68780000cd8 DSearch Fulltext Engine Configuration 00 0812d68780000d01 (2 rows affected) FOR 1.3 API>fetch,c, 0812d68780000cd8 DSearch Fulltext Engine Configuration API>get,c,l, param_name[3] dsearch_qrserver_port API>set,c,l, param_value[3] <port used by load balancer for xplore HA deployment> API>get,c,l, param_name[5] 5
dsearch_qrserver_host API>set,c,l,param_value[5] <vip used by load balancer for xplore HA deployment> API>save,c,l API>fetch,c, 0812d68780000d01 API>get,c,l, param_name[3] dsearch_qrserver_port API>set,c,l, param_value[3] <port used by load balancer for xplore HA deployment> API>get,c,l, param_name[5] dsearch_qrserver_host API>set,c,l,param_value[5] <vip used by load balancer for xplore HA deployment> API>save,c,l For 1.2 API>fetch,c, 0812d68780000cd8 API>get,c,l, param_name[3] dsearch_qrserver_port API>set,c,l, param_value[3] <port used by load balancer for xplore HA deployment> API>get,c,l, param_name[4] dsearch_qrserver_host API>set,c,l,param_value[4] <vip used by load balancer for xplore HA deployment> API>save,c,l API>fetch,c, 0812d68780000d01 API>get,c,l, param_name[3] dsearch_qrserver_port API>set,c,l, param_value[3] <port used by load balancer for xplore HA deployment> API>get,c,l, param_name[4] dsearch_qrserver_host API>set,c,l,param_value[4] <vip used by load balancer for xplore HA deployment> API>save,c,l 11. Following step is only for xplore 1.2: Add parameter dsearch_config_host and dsearch_config_port for both the dm_ftengine_config 6
API>fetch,c, 0812d68780000d01 iapi>append,c,l,param_name set>dsearch_config_host iapi>append,c,l,param_value set><index agent host ip> iapi>append,c,l,param_name set>dsearch_config_port iapi>append,c,l,param_value set><index agent port> save,c,l API>fetch,c, 0812d68780000cd8 iapi>append,c,l,param_name set>dsearch_config_host iapi>append,c,l,param_value set><index agent host ip> iapi>append,c,l,param_name set>dsearch_config_port iapi>append,c,l,param_value set><index agent port> save,c,l 12. The flag load_balancer_enabled need to be set up to true for both the dm_ftengine_config object. API>?,c,select r_object_id,object_name from dm_ftengine_config r_object_id object_name ----------- -------------- 0812d68780000cd8 DSearch Fulltext Engine Configuration 00 0812d68780000d01 DSearch Fulltext Engine Configuration API>fetch,c,0812d68780000cd8 API>append,c,l,param_name SET>load_balancer_enabled API>append,c,l,param_value SET>true API>save,c,l API>fetch,c, 0812d68780000d01 API>append,c,l,param_name SET>load_balancer_enabled API>append,c,l,param_value SET>true API>save,c,l 13. Restart the Content Servers to apply the new settings of the dm_ftengine_config objects. 7
14. Make sure that both the xplore instances are up and running. Testing the setup Following testing is performed on the single content server. Two types of testing can be performed in this setup. Load Balancer is set to balance the load in Round Robin without weights. Load Balancer is set to balance the load in Round Robin with the weights. A. Round Robin without weights Testing the Load Balance In this case the requests should be distributed to both the xplore Instances equally. The test setup had two xplore Instances configured on 10.8.79.198 and 10.8.79.197. We will test that load balancer has forwarded the request to xplore Instance equally by executing the following queries in sequence from iapi. If the first query was forwarded to first host, then second query should be forwarded to second Host. Following two queries are executed in succession. API>?,c,SELECT r_object_id FROM dm_sysobject SEARCH DOCUMENT CONTAINS '121212121212121' r_object_id ---------------- (0 rows affected) API> API>?,c,SELECT r_object_id FROM dm_sysobject SEARCH DOCUMENT CONTAINS '11414141414141411' r_object_id ---------------- (0 rows affected) 8
9
Observe that in the below screen first query is directed to xplore instance at 10.8.79.198. In the following screen observe that the second query is directed to xplore instance at 10.8.79.197. 10
Testing the failover 1. We will test the failover by stopping the each xplore instance by turn. Do not stop the instance from admin console as in that case it will not stop the appserver. Stop the first xplore Instance from services or from console. Make sure that xplore services have been stopped. 2. In this setup with the xplore instance is down at 10.8.79.197 and running at 10.8.79.198. xplore instance at 10.8.79.197 is named as PrimaryDsearch1 and xplore instance at 10.8.79.198 is named as PrimaryDsearch2. 3. Send a number of queries from the iapi at Content Server Host as shown in the following screen. 11
12
4. Check the dsearch log in the admin console to make sure that all the requests are being sent to the running server 10.8.79.198 ie PrimaryDsearch2. All the queries are redirected and executed at 10.8.79.198. The following section is part of the Dsearch log at 10.8.79.198 on which instance named PrimaryDsearch2 is running. a) 2012-09-17 19:52:51,640 INFO [http-0.0.0.0-9300-2] QueryID=PrimaryDsearch2$41c7dee2-e848-4a7c-b8fb- 1ee239cccce9,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.ind exserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_d omain,/dmftinternal/r_object_id'; for $i score $s in ((('1515151515151') with stemming))... b) 2012-09-17 19:53:13,999 INFO [http-0.0.0.0-9300-3] QueryID=PrimaryDsearch2$36c2e7c1-f040-4ca0-9943- 13
85962e63c258,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.ind exserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_d omain,/dmftinternal/r_object_id'; for $i score $s in ((('1616161616161') with stemming))... c) 2012-09-17 19:53:34,390 INFO [http-0.0.0.0-9300-3] QueryID=PrimaryDsearch2$fcec4cc1-4195-497b-9b06-356207a322d4,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.ind exserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_d omain,/dmftinternal/r_object_id'; for $i score $s in ((('171717177117171') with stemming))... B. Round Robin with weights Testing the Load Balance The weight for host 10.8.79.197 is set to 100 and for 10.8.79.198 is set to 1. All the requests are directed to xplore Instances at 10.8.79.197. Below is the screen shot showing the query executed. Starting from 1 to 12. Notice the all the queries executed in the section of log. 14
Total of 13 queries executed as shown in the preceding screen. Here is the sample output from the log. 1. 2012-09-27 17:43:26,751 INFO [http-0.0.0.0-9300-6] QueryID=PrimaryDsearch1$cf9b26f6-9771-453d-b35abeaccdb4c3ef,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('111111') with stemming))... 2. 2012-09-27 17:43:34,548 INFO [http-0.0.0.0-9300-6] QueryID=PrimaryDsearch1$e6d24559-8e57-4bad-bfbd- 15
f39cc4218428,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('222222') with stemming))... 3. 2012-09-27 17:43:39,923 INFO [http-0.0.0.0-9300-3] QueryID=PrimaryDsearch1$704a2cee-9fcd-437d-91fd- 9690781fd945,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('33333') with stemming))... 4. 2012-09-27 17:43:46,188 INFO [http-0.0.0.0-9300-3] QueryID=PrimaryDsearch1$5d670813-d80f-4421-b3bcb28658606b9c,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('4444444') with stemming))... 5. 2012-09-27 17:43:52,735 INFO [http-0.0.0.0-9300-6] QueryID=PrimaryDsearch1$fd1a4dca-f51e-4b64-87ef- 96c7c97cf15b,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in 16
((('55555555') with stemming))... 6. 2012-09-27 17:43:57,938 INFO [http-0.0.0.0-9300-6] QueryID=PrimaryDsearch1$d6960a52-04ce-4319-b661-202b4e58450a,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('66666666') with stemming))... 7. 2012-09-27 17:44:00,969 INFO [http-0.0.0.0-9300-6] QueryID=PrimaryDsearch1$9a975a77-f0b0-402c-95ae- 344e305f108d,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('66666666') with stemming))... 8. 2012-09-27 17:44:08,844 INFO [http-0.0.0.0-9300-3] QueryID=PrimaryDsearch1$30ae2856-9068-45ee-879dd8a3ddfe7c9c,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('7777777777') with stemming))... 9. 2012-09-27 17:44:14,532 INFO [http-0.0.0.0-9300-3] QueryID=PrimaryDsearch1$fccf0670-ec2c-4d0f-91c9- dad610f83956,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-paths- 17
values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('88888888') with stemming))... 10. 2012-09-27 17:44:19,798 INFO [http-0.0.0.0-9300-6] QueryID=PrimaryDsearch1$442287f0-b57d-4c4c-8a94-625e476d28a4,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive. IndexServerAnalyzer'; declare option xhive:ignore-emptyfulltext-clauses 'true'; declare option xhive:index-pathsvalues 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/ac l_domain,/dmftinternal/r_object_id'; for $i score $s in ((('99999999') with stemming))... Testing the Failover And so on... all the queries executed on PrimaryDsearch1. 10.8.79.197 is down by stopping the primarydsearch1 service from services. 18
Executed the queries as below: Check the logs at xplore Instance running at host 10.8.79.198. All the queries are executed at host at 10.8.79.198 as 10.8.79.197 was down. 1. 2012-09-27 18:05:59,296 INFO [http-0.0.0.0-9300-3] QueryID=PrimaryDsearch2$63f10df3-f898-46ff-8cc5- ac3b7696faad,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.in dexserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_ domain,/dmftinternal/r_object_id'; for $i score $s in ((('aaaaa1') with stemming)) 19
2. 2012-09-27 18:06:03,124 INFO [http-0.0.0.0-9300-8] QueryID=PrimaryDsearch2$0f84be91-a472-4b39-8b4a- 2971e67585d7,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.in dexserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_ domain,/dmftinternal/r_object_id'; for $i score $s in ((('aaaaa2') with stemming)) 3. 2012-09-27 18:06:06,640 INFO [http-0.0.0.0-9300-8] QueryID=PrimaryDsearch2$75952d28-3144-4d7c-bbe9-249272935a66,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.in dexserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_ domain,/dmftinternal/r_object_id'; for $i score $s in ((('aaaaa3') with stemming)) 20
Now 10.8.79.198 is down by stopping the primarydsearch2 service from services. 21
Execute the queries as below: Check the logs at xplore Instance running at host 10.8.79.197. All the queries are executed at host at 10.8.79.197 as 10.8.79.198 was down. 1. 2012-10-06 11:50:21,141 INFO [http-0.0.0.0-9300-2] QueryID=PrimaryDsearch1$712facc7-59b6-405c-970f- 84c971b6cd32,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.ind exserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_d omain,/dmftinternal/r_object_id'; for $i score $s in ((('XXXXXXXXXXXXXXXX1') with stemming)) 2. 2012-10-06 11:50:23,110 INFO [http-0.0.0.0-9300-2] QueryID=PrimaryDsearch1$e566d4be-fcb8-47bc-b32e- 174b238543f5,query-locale=en,query-string=declare option 22
'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.ind exserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_d omain,/dmftinternal/r_object_id'; for $i score $s in ((('XXXXXXXXXXXXXXXX2') with stemming))... 3. 2012-10-06 11:50:25,016 INFO [http-0.0.0.0-9300-2] QueryID=PrimaryDsearch1$f32d2aac-cce6-4a7f-9d0c- 17d9006d381b,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.ind exserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_d omain,/dmftinternal/r_object_id'; for $i score $s in ((('XXXXXXXXXXXXXXXX3') with stemming)) 4. 2012-10-06 11:50:27,579 INFO [http-0.0.0.0-9300-2] QueryID=PrimaryDsearch1$ef8c7d2e-033a-4f40-bf02-893e86285ade,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.ind exserveranalyzer'; declare option xhive:ignore-empty-fulltextclauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_d omain,/dmftinternal/r_object_id'; for $i score $s in ((('XXXXXXXXXXXXXXXX4') with stemming))... And so on. 23
B. The following describes the steps to install and configure xplore High Availability in active-active mode as per the following diagram: Indexing Single Content Server HOST C Indexing HOST A IndexAgent1 IndexAgent2 Querying HOST B Load-Balancer Index Server1 Index Server 2 1. Install Content Server D7 and configure one repository on host C. Please refer to EMC Documentum Content Server Enterprise Edition Installation Guide D7 for various other high-availability deployments and installation instructions for Content Server. 2. Install and configure Load Balancer. For the purpose of this white paper the testing is done with Cisco Load Balancer namely ACE 4710 Device Manager A3(2.0). 3. Install and configure the first xplore primary instance on HOST A. Install first primary index agent on the content server HOST C. Please refer to EMC Documentum xplore Version Installation Guide. Stop the index agent if it is started. 4. On Content Server host C, execute the following dmbasic script located at %DM_HOME%\install\admin to update full-text index related objects in the repository: dmbasic -f create_fulltext_objects_ha.ebs -ehapreinstallstep -- repository superuser password Note: ehapreinstallstep is a single word and does not contain space. Also -- is a double dash and not. 5. Install and configure the second xplore primary instances on HOST B. Install second primary index agent on the content server HOST C. Please refer to EMC Documentum xplore Version Installation Guide. Stop the index agent if it is started. 6. On Content Server host C, execute the following dmbasic script located at %DM_HOME%\install\admin to update full-text index related objects in the repository: Dmbasic -f create_fulltext_objects_ha.ebs -ehapostinstallstep -- repository superuser password 7. Restart Content Server. Execute following queries to validate dm_fulltext_index, dm_ftengine_config and dm_ftindex_agent_config objects are updated correctly. 24
API>?,c,select r_object_id,index_name,ft_engine_id,is_standby from dm_fulltext_index r_object_id index_name ft_engine_id is_standby ----------- ---------- ------------ ---------- 3b12d68780000100 repo3_ftindex_00 0812d68780000cd8 0 3b12d68780000500 repo3_ftindex_01 0812d68780000d01 1 (2 rows affected) API>?,c,select r_object_id,object_name from dm_ftengine_config r_object_id object_name ----------- ----------- 0812d68780000cd8 DSearch Fulltext Engine Configuration 00 0812d68780000d01D Search Fulltext Engine Configuration (2 rows affected) API>?,c,select r_object_id,object_name,index_name from dm_ftindex_agent_config r_object_id object_name index_name ----------- ----------- ---------- 0812d68780000cda cs4-lb-m1_9500_indexagent dm_fulltext_index_user 0812d68780000d9a cs4-lb-m2_9700_indexagent dm_fulltext_index_user (2 rows affected) 8. Start the indexing on both xplore instances. 9. Confirm the following URL will return True on both xplore instances if both instances are up: http://hostname:portnumber/dsearch/?action=checksearchable 10. Update dsearch_qrserver_host and dsearch_qrserver_port values of dm_ftengine_config object in the repository with the VIP (Virtual IP) and the port used by the load balancer for the xplore HA deployment. Both query plug-in point to the same load balancer IP address and port. API>?,c,select r_object_id,object_name from dm_ftengine_config r_object_id object_name ----------- ----------- 0812d68780000cd8 DSearch Fulltext Engine Configuration 00 0812d68780000d01 (2 rows affected) FOR 1.3 API>fetch,c, 0812d68780000cd8 DSearch Fulltext Engine Configuration API>get,c,l, param_name[3] dsearch_qrserver_host API>set,c,l, param_value[3] <vip used by load balancer for xplore HA deployment> 25
API>get,c,l, param_name[5] dsearch_qrserver_port API>set,c,l,param_value[5] <port used by load balancer for xplore HA deployment> API>save,c,l API>fetch,c, 0812d68780000d01 API>get,c,l, param_name[3] dsearch_qrserver_host API>set,c,l, param_value[3] <vip used by load balancer for xplore HA deployment> API>get,c,l, param_name[5] dsearch_qrserver_port API>set,c,l,param_value[5] <port used by load balancer for xplore HA deployment> API>save,c,l For 1.2 API>fetch,c, 0812d68780000cd8 API>get,c,l, param_name[3] dsearch_qrserver_port API>set,c,l, param_value[3] <port used by load balancer for xplore HA deployment> API>get,c,l, param_name[4] dsearch_qrserver_host API>set,c,l,param_value[4] <vip used by load balancer for xplore HA deployment> API>save,c,l API>fetch,c, 0812d68780000d01 API>get,c,l, param_name[3] dsearch_qrserver_port API>set,c,l, param_value[3] <port used by load balancer for xplore HA deployment> API>get,c,l, param_name[4] dsearch_qrserver_host API>set,c,l,param_value[4] <vip used by load balancer for xplore HA deployment> API>save,c,l 26
11. The following step is relevant only for xplore 1.2: Add paramenter dsearch_config_host and dsearch_config_port for both the dm_ftengine_config API>fetch,c, 0812d68780000d01 iapi>append,c,l,param_name set>dsearch_config_host iapi>append,c,l,param_value set><index agent host ip> iapi>append,c,l,param_name set>dsearch_config_port iapi>append,c,l,param_value set><index agent port> save,c,l API>fetch,c, 0812d68780000cd8 iapi>append,c,l,param_name set>dsearch_config_host iapi>append,c,l,param_value set><index agent host ip> iapi>append,c,l,param_name set>dsearch_config_port iapi>append,c,l,param_value set><index agent port> save,c,l 12. The flag load_balancer_enabled need to be set up to true for both the dm_ftengine_config object. API>?,c,select r_object_id,object_name from dm_ftengine_config r_object_id object_name ----------- -------------- 0812d68780000cd8 DSearch Fulltext Engine Configuration 00 0812d68780000d01 DSearch Fulltext Engine Configuration API>fetch,c,0812d68780000cd8 API>append,c,l,param_name SET>load_balancer_enabled API>append,c,l,param_value SET>true API>save,c,l API>fetch,c, 0812d68780000d01 API>append,c,l,param_name SET>load_balancer_enabled API>append,c,l,param_value 27
SET>true API>save,c,l 13. Restart the Content Servers to apply the new settings of the dm_ftengine_config objects. 14. Make sure that both the xplore instances are up and running. Testing the setup Following testing is performed on the single content server with index agents installed on Content Server machine. Testing the Load Balance 1) Start both the Index instances. In this test the index instances are installed at 10.8.79.197 and 10.8.79.198. They are called PrimaryDsearch1 and PrimaryDsearch2 respectively. The weights in the LB are set to 100 for 10.8.79.197 and 1 for 10.8.79.198. 28
29
2) Start both the index agents installed on the same host as Content Server. In this test it is 10.8.78.24. 30
31
3) Execute following queries using iapi at Content Server host. 4) The following section from the Dsearch log shows that all the preceding queries ran at PrimaryDsearch1 running at 10.8.79.197 when both the index instances ie PrimaryDsearch1 and PrimaryDsearch2 are up and running. That means all the requests are being directed to only one instance to which the weight is set to 100 in the load balancer. a) 2012-10-06 21:09:42,469 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch1$0f4ab398-1caa-4d3a-a63b-b65950611d42,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id'; for $i score $s in (dmftinternal/i_all_types = 32
'0312d68780000105') ) and ( ((('XXXXXXXXXXXXXa') with stemming))... b) 2012-10-06 21:09:49,173 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch1$c020c8b2-328c-429a-82d6-bd49ec0c0043,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id'; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXb') with stemming)) c) 2012-10-06 21:09:53,110 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch1$696efab6-4cc9-4d16-bcab-110cbc3a5e9e,query-locale=en,query-string=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id'; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXc') with stemming)) d) 2012-10-06 21:09:57,579 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch1$ae5e8485-693f-4dd4-99ea-3b739edea62e,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id'; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXd') with stemming)) Testing the failover 1) Stop the xplore instance running on 10.8.79.197 as shown in the following screen. 33
34
2) xplore instance running at 10.8.79.198 should be running as follows. 35
3) Make sure both index agents on the Content Server host are also running. 4) Execute the following queries using iapi at the Content Server host. Confirm the failover. Check the dsearch logs at the running xplore instances namely PrimaryDsearch2 at 10.8.79.198. All the queries must have run on this instance. a) 2012-10-06 22:10:04,968 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch2$7572aa68-2a81-410c-814f-c9685d2a2a39,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id '; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXP') with stemming)) 36
b) 2012-10-06 22:10:12,312 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch2$c4059e93-1101-4d8f-8295-fd617069874a,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id '; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXQ') with stemming)) c) 2012-10-06 22:10:16,468 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch2$56e7a6a6-450d-4ba4-99fd-81a997d3cf00,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id '; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXR') with stemming)) d) 2012-10-06 22:10:21,327 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch2$81d2f06f-4439-4d73-8354-7a8c4cc3634d,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id '; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXS') with stemming)) e) 2012-10-06 22:10:25,202 INFO [http-0.0.0.0-9300-1] QueryID=PrimaryDsearch2$7f39f36f-35ab-4a26-8167-d14f971cfdcc,query-locale=en,querystring=declare option 'com.emc.documentum.core.fulltext.indexserver.core.index.xhive.indexserveranalyzer'; declare option xhive:ignore-empty-fulltext-clauses 'true'; declare option xhive:index-paths-values 'dmftmetadata//owner_name,dmftsecurity/acl_name,dmftsecurity/acl_domain,/dmftinternal/r_object_id '; for $i score $s in (dmftinternal/i_all_types = '0312d68780000105') ) and ( ((('XXXXXXXXXXXXXT') with stemming)) Conclusion This concludes this White paper. Reference: xplore 1.2 and 1.3 Install guide. Disclaimer "All information included in this document is provided "as-is" and does not guarantee EMC's support or approval. Please consult the official product documentation or the Product Support team regarding any questions of supportability." 37