EMC DOCUMENTUM JAVA METHOD SERVER HIGH AVAILABLITY CONFIGURATION

EMC DOCUMENTUM JAVA METHOD SERVER HIGH AVAILABLITY CONFIGURATION JMS HA configuration for Fail over Support ABSTRACT This white paper explains how to configure the JMS HA/failover and configuring methods for failover. The white paper describes the steps to track the execution of methods from logs. November 2013 1

EMC WHITE PAPER TABLE OF CONTENTS ABSTRACT... 1 TABLE OF CONTENTS... 2 INTRODUCTION... 4 Java method server... 4 How does the method server work?... 4 It s a web application... 4 Java Method Server for High-Availability... 5 Failover... 5 Load balancing... 5 Audience... 5 JAVA METHOD SERVER FOR HIGH AVAILABILITY... 6 TYPE1: SINGLE HOST SETUP... 7 STEPS FOR CONFIGURING JMS HA 7.0... 8 1. Preparing for Installation/Prerequisite:... 8 Content Server in HA... 8 Packaging WebApps... 8 2. Creating second JMS Instance... 9 3. Verification of Instance... 15 dm_jms_config :... 15 4. Associating Content Servers with JMS Instance... 16 5. Refresh Content Server Cache... 17 6. DUMP_JMS_CONFIG_LIST... 18 Configuring HA parameters... 22 Enabling logs and Understanding logs... 22 Error Messages:... 23 Sample Logs... 23 Server.log... 23 Docbase.log... 24 System Methods details... 25 Options for method... 26 Verification steps... 26 Limitations... 27 Custom Method configurations... 27 TYPE2: TWO HOST HA SETUP... 28 Digital Signature Verification... 28 TYPE3: MULTIPLE HOSTS SETUP... 30 STEPS FOR CONFIGURING JMS HA 6.7 SP2... 31 2 TYPE1:SINGLE HOST SETUPFOR 6.7 SP2... 31 1. Preparing for Installation/Prerequisite:... 31 Content Server in HA... 31 Packaging WebApps... 31 2. Creating second JMS Instance... 31

3. Verification of Instance and Associating Content Server with JMS Instance... 32 TYPE 2: TWO OR MULTI HOST SETUP FOR 7.0... 32 STEPS FOR CONFIGURING JMS HA 7.1... 32 CONCLUSION... 33 References... 33 CONTACT US... 33 3

INTRODUCTION Java method server The Documentum 5 platform introduced the Java method server as a means of improving the performance and scalability of the Documentum method architecture. In the past couple of years, Documentum has focused on Java as a cornerstone for its technology. Documentum clients are being written in Java more and more frequently. The core Documentum libraries are implemented in Java. Running Java-based methods, however, has been the most painful of experiences. The Java method server aims to fix this. The method server runs as a process distinct from the Content Server and is responsible for the invocation of the individual methods. Each method invocation results in a call to the method server which manages the actual execution of the Java-based method implementation. As the method server is a continuously running process, there is no overhead involved in loading and tearing-down a Java virtual machine with each invocation. This yields an enormous benefit in overhead. As methods are typically executed over and over again, the classes involved in the implementation of the method only need be loaded once. Other overhead, such as the creation of docbase sessions may also be reduced as the method server may reuse sessions with connection pooling. The result is methods that execute in milliseconds instead of seconds and which use significantly fewer system resources. How does the method server work? It s a web application The method server itself is a Java-based web application. It communicates with the Content Server via HTTP calls. Each time a method is invoked, the Content Server makes an HTTP request passing the name of the Java class which implements the method along with any specified arguments to a servlet which knows how to execute the specified method. Methods in Documentum are defined via dm_method objects. To indicate that a Java method should execute via the Java method server, set the dm_method.method_type attribute tojava, the dm_method.use_method_server attribute to 1, and the dm_method.method_verb attribute to the classname of the method implemenation. When the Content Server sees a method matching this pattern, it opens a HTTP request to the Java method server requesting executing of the method with the specified class. This simple architecture solves the overhead problem and also introduces a convenient mechanism for dramatically improving the scalability of the Content Server. Although it is not configured out-of-the-box to support this, Method servers may be executed on a machine physically separate from the Content Server. It is even possible to run the method server on a platform different from the Content Server (ie. running the Content Server on Solaris and the method server on Windows). This offloads work from the Content Server which might enable the Content Server to handle more users. If the methods are expensive to execute, it is also possible to load-balance method servers using industry standard HTTP load balancers. There is nothing special or complicated about the method server. It simply requires a J2EE-compatible servlet container 4

Java Method Server for High-Availability Content Server and Java Method Server (JMS) each support high-availability (HA). Content Server supports two HA types: failover and load balancing. JMS supports failover only. Failover In a failover setup, if one of the Content Servers fails, the other Content Servers in the failover setup continues with the service. Load balancing Load balancing involves operating redundant Content Servers where the service load is balanced between Content Servers to maximize performance. In a standard Content Server load-balancing scenario, proximity values are used to determine which Content Server processes an item. In a cluster scenario, third-party load-balancers are used. Audience The information in this guide is for application and system administrators who use CS and JMS software to deploy and custom method. The intended audience is EMC support personnel, professional services, and customers. Terminology o o o o JMS: Java Method Server CS: Content Server CLI: Command-Line Interface GUI: Graphical User Interface 5

JAVA METHOD SERVER FOR HIGH AVAILABILITY JMS supports the following HA configurations: Content Server and JMS on a single host Content Server and JMS on two hosts Content Server and JMS on multiple hosts Content Server HA deployment involves two or more Content Servers. JMS HA involves adding JMSs to additional Content Servers such that each Content Server has a dedicated JMS. JMS HA is automatically enabled by associating each Content Server with their dedicated JMS. JMS HA flowchart Multiple content servers on different hosts serving single repository Multiple content servers on same host serving single repository To extend the system, you must add more content servers. Each content server has its own default local embedded JMS. Therefore, you can have as many JMSs as the number of CSs in the system. Limited to one JMS per content server, other CS s JMS are used as backup. JMS HA is not intended to address scaling issues. Under normal operation, Content server always communicates to its own default local embedded JMS. It will use failover JMS only if its default embedded JMS becomes unavailable. Content server will resume using its own default JMS when that it comes back online. 6

TYPE1: SINGLE HOST SETUP JMS HA on Single Host For a Single Host setup both the Primary and Secondary Content Server are installed on the same Host. The Primary JMS will be installed and configured during the Primary Content Server Installation. After the Content Servers are installed the second JMS is installed and configured. 7

STEPS FOR CONFIGURING JMS HA 7.0 1. Preparing for Installation/Prerequisite: Content Server in HA To Achieve JMS HA, a second (or another) Content Server associated with JMS. Single Content server does not support multiple JMS. To Install Content Server in HA refer to the Content Server Enterprise Edition 7.0 Installation Guide. Packaging WebApps The WebApps are deployed on the first JMS. Using the jmsconfig.exe, package WebApps to be deployed on the second JMS. ServerApps.ear - WebApp has to be deployed for the JMS server. In the webappstoinclude.txt, add the names of the WebApps to be packaged. This file is found under $Documentum\jmsTools\config\webappsToInclude.txt Run the jmsconfig.exe tool under $Documentum\jmsTools\bin. Select Package only option. And Click Next After the Packaging is complete the WebApps are located at $Documentum\jmsTools\webapps\jmsWebApps.jar. Copy the jmswebapps.jar file on the machine to install second JMS under $Documentum\jmsTools\webapps path. 8

2. Creating second JMS Instance Run the jmsconfig.exe under C:\Documentum\jmsTools\bin folder. Select Jmsn Config to create a new JMS instance and click Next. 9

On the Select Configuration mode page, select Create Instance and click Next. Provide an Instance Name for the JMS and Click Next. 10

Set the password for the Application Server - Admin User. Provide the port details to deploy JMS. JMS requires consecutive 20 ports. By default 9080 to 9099 ports will be selected for JMS. Port numbers can be changed, ensure consecutive 20 ports are available. 11

On Configure repository domain name, by default the fully qualified DNS is given. Verify and click next. Provide the Primary Instance Installation Owner and Primary Connection Broker details: o o o Installation Owner: Username and Password Primary Connection Broker: Connection Broker host: Machine Name/IP o Connection Broker port: Broker port (default port 1489) 12

13 Select the Repository and click next.

Select Yes in the below screen since the primary content server is installed on same host. For multi host setup - select No as the primary CS and secondary CS are on different host. Click on Done. Configuration process is complete. 14

3. Verification of Instance The new instance of JBoss application server is created under C:\Documentum\jboss5.1.0\server. Folder named DctmServer_instancename is created. Verify the packaged applications are deployed under C:\Documentum\jboss5.1.0\server\DctmServer_secondJMS\deploy. Start the new JMS instance. Run the startsecondjms.bat or from Services start Documentum Java Method Server instancename. Access the URL: http://secondhost:port/dmmethods/servlet/domethod (eg: http://localhost:9100/dmmethods/servlet/domethod ) from IE. Browser will display Documentum Java Method Server. Each JMS instance has a dm_jms_config object associated with it. dm_jms_config : This object represents an instance of Java method server. It contains information about servlet_names, base_uris, its associated content server(s), and respective jms to cs proximity value(s) Run Iapi query:?,c,select r_object_id,object_name from dm_jms_config An additional object will be created. Verify the object name displays JMS secondhost:port List of jms_config objects 15

4. Associating Content Servers with JMS Instance Associating the Java Method Server with Content Server Login to DA and go to Administration>Basic Configuration>Java Method Servers. Select first JMS and right click > Properties By Default the first JMS will be associated with primary/first Content Server. Set the Second Content Server as secondary Select second JMS and right click > Properties Set the Second Content Server as primary Set the First Content Server as secondary 16

5. Refresh Content Server Cache In a normal operational environment, refreshing content server s JMS config list cache is needed only when a new jms config object is added to, or an existing one is updated or deleted from the database. Content server cache update is handled by the server s change checker process/thread. That is, whenever there s a jms config object change in the docbase, the change checker will cause all content servers that are serving the same docbase to update their respective server cache automatically. The server cache might not be updated immediately due to change checker s frequency of checking changes. REFRESH_JMS_CONFIG_LIST RPC function is introduced to allow administrators to refresh the content server s cached JMS list immediately after a change is made to a jms_config object. However, issuing REFRESH_JMS_CONFIG_LIST will only update the connected server s cache. It does not update other non-connected servers caches. At run time, content server will figure out which JMS object defined in database is considered as default local embedded JMS, which one is for load balancing, and which one is for failover. In a multi-server-single-docbase environment, issuing this RPC call on one content server will only refresh that connected server s cache. It does not refresh other content servers caches. To force refresh a target content server s JMS config list kept in its cache, this RPC call must be issued from within a connected session (an established session to the target content server). 17

6. DUMP_JMS_CONFIG_LIST RPC function DUMP_JMS_CONFIG_LIST is introduced to allow administrators to see the current status of the JMS list kept in content server s cache (i.e. shared memory), in real time. The typed object returned from this RPC call contains information about the last time the list was refreshed, which jms are marked as dead in server cache (even though their is_disabled attribute in dm_jms_config object is set to F as not-disabled, in the database). If a JMS is marked as dead in server cache, DUMP_JMS_CONFIG_LIST will show when was the last failure that caused server to believe it s dead, and when will the server re-try to POST to this dead JMS again, how many times have we failed to send HTTP POST requests to that JMS. In a multi-server-single-docbase environment, running DUMP_JMS_CONFIG_LIST on different servers could result in different views/output. The same jms config object could be considered as default local embedded JMS for a primary content server, but it is also considered as a failover jms for secondary content server, but it would not have been considered a failover jms for 3rd content server if the 3rd is an RCS Connect to each content server, run REFRESH_JMS_CONFIG_LIST RPC to refresh content server cache, then run DUMP_JMS_CONFIG_LIST RPC to examine the JMS failover information stored in content server cache. Connect to CS1, then CS2, run these two RPCs one by one. API> connect,repository.csconfiguration,cs-administrator,password... s1 API> apply,c,null,refresh_jms_config_list... q0 API> close,c,q0... OK API> apply,c,null,dump_jms_config_list... q0 API> next,c,q0... OK API> dump,c,q0... USER ATTRIBUTES jms_list_last_refreshed : Fri Sep 13 09:18:46 2013 incr_wait_time_on_failure : 30 max_wait_time_on_failure : 3600 current_jms_index : -1 jms_config_id [0]: 0800009080000cc3 18

[1]: 0800009080000d77 jms_config_name [0]: JMS CS1DNS: Port for RepoJMS1.RepoJMS1 [1]: JMS CS2DNS: Port for RepoJMS1 server_config_id [0]: 3d00009080000102 [1]: 3d00009080000d2e server_config_name [0]: RepoJMS1 [1]: csjmsha_repojms1 jms_to_cs_proximity [0]: 1 [1]: 1 is_disabled_in_docbase [0]: F [1]: F is_marked_dead_in_cache [0]: F [1]: F intended_purpose [0]: DM_JMS_PURPOSE_DEFAULT_EMBEDDED_JMS [1]: DM_JMS_PURPOSE_FOR_FAIL_OVER last_failure_time [0]: N/A [1]: N/A next_retry_time [0]: N/A [1]: N/A failure_count [0]: 0 [1]: 0 SYSTEM ATTRIBUTES APPLICATION ATTRIBUTES INTERNAL ATTRIBUTES API> close,c,q0... OK 19

20 Content Server 1

21 Content Server 2

Configuring HA parameters Two new configuration parameters are introduced in server.ini for JMS failover. o o incremental_jms_wait_time_on_failure=30 jms_max_wait_time_on_failures=3600 These two config params are optional, if not present in server.ini, default values are 30 and 3600 seconds (3600 seconds is 1 hour) respectively. incremental_jms_wait_time_on_failure config value is used for content server to determine what time should it retry to POST to a previously failed Java method server again. By default, the incremental time interval is set to 30 seconds. jms_max_wait_time_on_failures config value is to put a cap on wait time so that we don t end up waiting for days or weeks for the next retry time. Default jms_max_wait_time_on_failures is 1 hr (i.e. 3600 seconds). Enabling logs and Understanding logs To turn on debug tracing on JMS s web.xml, This file is under C:\Documentum\jboss5.1.0\server\DctmServer_secondJMS\deploy\ServerApps.ear\DmMethods.war\WEB-INF for methods (this option requires app server restart) In the web.xml, set the trace parameter value to t <init-param> <param-name>trace</param-name> <param-value>t</param-value> </init-param> The trace details is logged in Server.log under $Documentum\jboss5.1.0\server\DctmServer_MethodServer\log To turn on trace_launch at content server level, run the below commands from IAPI For HTTP POSTs: apply,c,null,set_options,option,s,trace_http_post,value,b,t For Complete Launch: apply,c,null,set_options,option,s,trace_complete_launch,value,b,t The debug tracing details are logged in server.log under $Documentum\dba\log\RepositoryName.log To turn on trace_launch at method level, while running the method enable trace launch option. The debug tracing details are logged in server.log under $Documentum\dba\log\RepositoryName.log Installer log files JMS Configuration Program logs C:\Documentum\jmsTools\bin\ or $DOCUMENTUM_SHARED/jmsTools/bin/ install.log setuperror.log C:\Documentum\dba\config\docase\dm_jms_config_setup.out 22

Error Messages: DM_METHOD_E_HTTP_COMMUNICATION The app server which the Content Server tried to send POST request to, is unavailable/down DM_METHOD_E_HTTP_POST_APP_SERVER_NAME_NOTFOUND There s no matching app_server_name/servlet_name value found in any of the dm_server_config/dm_jms_config objects DM_METHOD_E_NO_JMS_AVAILABLE2 Content server believes all of the Java method servers are currently down and not accepting any POST requests. No one is available. DM_METHOD_E_HTTP_POST_FAILED Content server failed to send POST request to app server Sample Logs Server.log 03:09:00,212 INFO [stdout] (http--0.0.0.0-8080-2) <DCTM><DEBUG> HTTP request from 10.31.69.85, invoking com.emc.documentum.server.method.rename.userrename.execute(), ARGUMENTS [] 03:09:00,233 INFO [stdout] (http--0.0.0.0-8080-2) <DCTM><INFO> [http--0.0.0.0-8080-2:method name=com.emc.documentum.server.method.rename.userrename] Completed executing. Total Time Taken: 0 secs 03:09:28,186 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): signatureparams=method_verb, dm_docbase, dm_server_config 03:09:28,189 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): paramname=method_verb 03:09:28,192 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): paramvals[0]=com.emc.documentum.server.method.rename.userrename 03:09:28,194 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): encodedparamvalue=com.emc.documentum.server.method.rename.userrename 03:09:28,197 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): paramname= dm_docbase 03:09:28,199 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): paramvals[0]=repo1 03:09:28,200 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): encodedparamvalue=repo1 03:09:28,202 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): paramname= dm_server_config 03:09:28,204 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): paramvals[0]=csd70_repo12 03:09:28,206 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): encodedparamvalue=csd70_repo12 03:09:28,209 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructquerystring(): querystring=method_verb=com.emc.documentum.server.method.rename.userrename& dm_docbase =Repo1& dm_serve r_config =csd70_repo12 03:09:28,212 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validatedigitalsignature(): docbasename=repo1 03:09:28,213 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validatedigitalsignature(): signature=my6j6mgcbxac1xdy/b1al64cynyyh1e4uipb/yhohcq3uet43kvyvimyepgrbyasmjvft6wbhozws14qmlb6v+ihsfvi+3rn n9+dnrauh1ksy0l66qyr8fjbv3alcscoopajlpy6vsymcwav02iptkmw9c3er3p83b5ashjvrf0=, signature.length()=172 03:09:28,218 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validatedigitalsignature(): installownername=administrator 23

03:09:28,220 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validatedigitalsignature(): querystring=method_verb=com.emc.documentum.server.method.rename.userrename& dm_docbase =Repo1& dm_serve r_config =csd70_repo12, querystring.length()=117 03:09:28,224 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> verifydigitalsignature(): docbasename=repo1: installownername=administrator: inputdata=method_verb=com.emc.documentum.server.method.rename.userrename& dm_docbase =Repo1& dm_server_ config =csd70_repo12: encodeddigitalsignature=my6j6mgcbxac1xdy/b1al64cynyyh1e4uipb/yhohcq3uet43kvyvimyepgrbyasmjvft6wbhozws14qmlb 6V+ihSFVi+3rNn9+dnRAUH1ksY0L66qyr8FJbV3alcscOOpaJlPy6VsymCwAv02iptkmw9C3er3p83b5aShJVrF0=: encodeddigitalsignature.length()=172 03:09:28,233 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> verifydigitalsignature(): success 03:09:28,260 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:method name=com.emc.documentum.server.method.rename.userrename] Successfully located java method/module 03:09:28,264 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:method name=com.emc.documentum.server.method.rename.userrename] Is an instance of IDfMethod 03:09:28,267 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:method name=com.emc.documentum.server.method.rename.userrename] Preparing to execute. 03:09:28,271 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> HTTP request from 10.31.69.85, invoking com.emc.documentum.server.method.rename.userrename.execute(), ARGUMENTS [] 03:09:28,278 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:method name=com.emc.documentum.server.method.rename.userrename] Completed executing. Total Time Taken: 0 secs Docbase.log 013-10-22T03:08:58.072000 5596[7388] 0100007b800025a3 JMS DO_METHOD TRACE LAUNCH: user: Administrator, session id: 0100007b800025a3, JMS id: 0800007b80000cc7, method: dm_userrename_java, host:csd70.emcccsa.com, port:9080, path:/dmmethods/servlet/domethod, arguments:-method_verb com.emc.documentum.server.method.rename.userrename - dm_docbase Repo1 - dm_server_config csd70_repo12 2013-10-22T03:08:59.121000 5596[7388] 0100007b800025a3 JMS DO_METHOD TRACE LAUNCH: do_method launch: failed: user: Administrator, session id: 0100007b800025a3, JMS id: 0800007b80000cc7, method: dm_userrename_java, host:csd70.emcccsa.com, port:9080, path:/dmmethods/servlet/domethod 2013-10-22T03:08:59.127000 5596[7388] 0100007b800025a3 JMS TRACE LAUNCH: Failed to post to JMS 0800007b80000cc7 (JMS csd70.emcccsa.com:9080 for Repo1.Repo1): Error is "Failed to obtain socket for host:csd70.emcccsa.com, at port:9080", :Marking this JMS config as dead server cache. 2013-10-22T03:08:59.127000 5596[7388] 0100007b800025a3 JMS DO_METHOD TRACE LAUNCH: user: Administrator, session id: 0100007b800025a3, JMS id: 0800007b8000151e, method: dm_userrename_java, host:csd70.emcccsa.com, port:8080, path:/dmmethods/servlet/domethod, arguments:-method_verb com.emc.documentum.server.method.rename.userrename - dm_docbase Repo1 - dm_server_config csd70_repo12 2013-10-22T03:09:00.241000 5596[7388] 0100007b800025a3 JMS DO_METHOD TRACE LAUNCH: do_method launch: successful: user: Administrator, session id: 0100007b800025a3, JMS id: 0800007b8000151e, method: dm_userrename_java, host:csd70.emcccsa.com, port:8080, path:/dmmethods/servlet/domethod 2013-10-22T03:09:00.286000 5596[7388] 0100007b800025a3 DO_METHOD TRACE LAUNCH: method launch: successful, user: Administrator, session id: 0100007b800025a3, method: dm_userrename_java 24

System Methods details Below is the list of system methods and supportability for failover. Method Name dm_distoperations_dmcl40 dm_refreshreplicamethod dm_bp_transition_java dm_bp_schedule_java dm_bp_batch_java dm_bp_validate_java dmc_migratedbexprstojava dmc_migratedbexprstojavafortype dmc_setjavaexprenabled dmc_addcustomjavaexpr CTSAdminMethod dm_ldif_export dm_ldif_copy dm_ldif_import dm_ldif_status dm_ldapsynchronization LDAPSync LDAPSyncOnDemand dm_ldapcertdbautomation dm_dcfedit dm_run_dependent_jobs dm_getclientdocbrokers dm_webcache_publish dm_bpm_transition dm_bpm_timer dm_wfmstimer TCFActivityMethod TCFWorkitemActivityMethod dm_ftcreateevents dm_ftstateofindex dm_ftindexagentboot dm_ftoptimizeindex dm_event_template_sender dm_asynchronouswrite dm_precachecontent dm_noop_auto_method_java BPMCreateFolder JMS Location Constrain JMS_LOCATION=MAIN JMS_LOCATION=MAIN LOCAL_ONLY LOCAL_ONLY NA (does not use java method server) NA (does not use java method server) NA (does not use java method server) NA (does not use java method server) LOCAL_ONLY LOCAL_ONLY LOCAL_ONLY LOCAL_ONLY LOCAL_ONLY LOCAL_ONLY LOCAL_ONLY LOCAL_ONLY LOCAL_ONLY 25

BPMLifecycleAttach BPMLifecycleDemote BPMLifecyclePromote BPMLinkToFolder BPSFTIntegration BPSHTTPIntegration BPSJMSIntegration BPSMQJMSIntegration BPSSMTPIntegration BPSWebServiceIntegration BPSIntegration dm_bps_mail dm_qmcheckverificationthreshold dm_qmpriorityaging dm_qmprioritynotification dm_qmsetrequiredskill dm_qmthresholdnotification dm_wfcheckpoint dm_wfexportdata dm_wfreporting dm_wfreportpackagedata dm_wfsuspendtimer DmSampleActivityTemplate ECISMethod ObservationPoint PostEventtoParentWorkflow StartSub-Process XSLTransformation Options for method For a method to support JMS failover set the below parameters: o o Is_restartable to True (default is false) a_extended_properties[0] to or leave it blank (default value is blank) Verification steps Configure a Java Method to support JMS failover. Bring down first JMS. Run the Method from first Content Server. Method should be executed via second JMS. Verify the logs. Logs for second JMS would contain the method execution information. Bring down second JMS. Run the Method from second Content Server. Method should be executed via first JMS. Verify the logs. Logs for first JMS would contain the method execution information. 26

Limitations Not all java methods can be failover o Some java method must be run on primary content server, some must be run on same host as originating content server s host Default JMS for Content File Server/Remote Content Server (aka RCS) do not participate in JMS failover JMS node must be bound to a content server host o It cannot be installed on a stand-alone independent machine We do not support JMS load balancing. A single content server cannot be explicitly associated to multiple Java method servers. Limitations inherited from single JMS model o o JBoss cannot handle large number of HTTP POST requests If JMS is crashed during ASYNC method execution, content server cannot detect the crash, hence will not failover to another JMS Custom Method configurations If custom methods wish to take advantage of the JMS failover feature, make sure the method is failover aware. Set Is_restartable parameter to True (default is false If the method is failover aware, but method must be run on a JMS that s on the same host as originating content server host, then set a_extended_properties[0] = JMS_LOCATION=ORIGINAL If method can be run on any CS-bound JMS, then set a_extended_properties[0] = JMS_LOCATION=MAIN If method can be run on any JMS, then leave a_extended_properties attribute blank or assign value. If left blank, it is assume the method can be run on any JMS. Default value is blank. 27

TYPE2: TWO HOST HA SETUP JMS HA on two hosts Two Host HA Setup the primary and secondary Content Servers are installed on different hosts. On the Second Content Server, the second content server has to be installed and configured. For creating a second JMS, the same steps as single host JMS HA are followed. During the screen Is Primary Content Server, select the option NO. To Install Content Server in HA refer to Content_Server_Enterprise_Edition_7.0_Installation_Guide. Digital Signature Verification In a single JMS model, JMS and CS are always hosted on same machine. Before running the Java method, to prevent malicious cyber-attacks, to make sure the HTTP POST request is legit, JMS Do_Method servlet performs IP validation to make sure request is originated from the same host as JMS s host. IP validation would always succeed in single JMS model environment. In a JMS failover environment, IP validation would fail if content server tries to send method requests to a failover JMS on a different host. To address this IP security restriction, we ve introduced digital signature validation instead. 6.6 onwards, content Server generates digital signature, JMS verifies it Signature is signed by using the repository s private key, and is then verified using the repository s public key. 28

JMS uses the following parameters : $Documentum_shared\JBOSSx.x\server\DctmServer_MethodServer\deploy\ServerApps.ear\DmMethods.war\WEB-INF, make sure the init value for docbase and docbase_install_owner_name are present in web.xml. <init-param> <param-name>docbase_install_owner_name</param-name> <param-value>xxxxxx</param-value> </init-param> <init-param> <param-name>docbase</param-name> <param-value>xxxx</param-value> </init-param> 29

TYPE3: MULTIPLE HOSTS SETUP JMS HA on Mulitple hosts More than 2 Content servers can be clustered. Similarly the Java method server associated with each Content Server can be configured for High Availability. Same steps are followed for configuring the JMS HA as for 2 nodes. 30

STEPS FOR CONFIGURING JMS HA 6.7 SP2 TYPE1:SINGLE HOST SETUPFOR 6.7 SP2 1. Preparing for Installation/Prerequisite: Content Server in HA To Achieve JMS HA, a second (or another) Content Server associated with JMS. Single Content server does not support multiple JMS. To Install Content Server in HA refer to Content_Server_Enterprise_Edition_6.7_Installation_Guide. Packaging WebApps The WebApps are deployed on the first/primary JMS. Using the jmspackager.bat, package WebApps to be deployed on the second/additional JMS. ServerApps.ear - WebApp has to be deployed for the JMS server. In the webappstoexclude.txt, add the names of the WebApps to exclude from packaging. This file is found under $Documentum\jmsTools\config\webappsToExclude.txt. Run jmspackager.bat file under $Documentum\jmsTools\bin. After the Packaging is complete the WebApps are located at $Documentum\jmsTools\webapps\jmsWebApps.jar. Copy the jmswebapps.jar file on the machine to install second JMS under $Documentum\jmsTools\webapps path. 2. Creating second JMS Instance The steps are similar as for creating second JMS instance for 7.0. Run the jmsconfig.exe under $Documentum\jmsTools\bin folder. Select Create Instance and click Next. Provide JMS Instance Name Set the password for the Application Server - Admin User. Provide the port details to deploy JMS. JMS requires consecutive 20 ports. By default 9080 to 9099 ports will be selected for JMS. Port numbers can be changed, ensure consecutive 20 ports are available. 31

On the primary Installation owner screen : Provide the Installation Owner: Username and Password. Click on Next. The WebApps deploy and configuration process completes. Click on Done. 3. Verification of Instance and Associating Content Server with JMS Instance Verify the JMS Instance following the same steps as Single Host JMS HA for 7.0 Associate the Content Servers with JMS Instances, same steps as Single Host JMS HA for 7.0 is followed. TYPE 2: TWO OR MULTI HOST SETUP FOR 7.0 Two Host HA Setup the primary and secondary Content Servers are installed on different hosts. On the Second Content Server, the second content server has to be installed and configured. For creating a second JMS, the same steps as single host JMS HA for 6.7 Sp2 are followed. STEPS FOR CONFIGURING JMS HA 7.1 To Install Content Server in HA refer to Content_Server_Enterprise_Edition_7.1_Installation_Guide. For Single Host Setup, the same steps as for configuring JMS HA 7.0 are followed. For Two or more host setup, the Java Method Server Instance is created by the installer. Associate the Content Servers with JMS Instances, same steps as for configuring JMS HA 7.0 is followed. 32

CONCLUSION This whitepaper explains configuration of JMS High Availability with failover support setup. Describes the steps to enable methods for failover and track the execution of methods from logs. References Documentum Content Server 7.0 Installation Guide EMC Documentum Content Server Load Balance and FailOver EMC Documentum Content Server 7.0 Administration and Configuration Guide CONTACT US To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local representative or authorized reseller or visit us at www.emc.com. Copyright 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. 33 EMC2, EMC, the EMC logo, and the RSA logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. VMware a registered trademark of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Published in the USA. 08/12 White Paper H12673 www.emc.com