EQUELLA. Clustering Configuration Guide. Version 6.0

Transcription

1 EQUELLA Clustering Configuration Guide Version 6.0

2 Document History Document No. Reviewed Finalised Published 1 17/10/ /10/ /10/2012 October 2012 edition. Information in this document may change without notice. EQUELLA and its accompanying documentation are furnished under a non-disclosure, evaluation agreement or licence agreement. Copying, storing, transmitting, or otherwise reproducing the software or this document in any form without written permission from Pearson is strictly forbidden. All products, other than EQUELLA, named in this document are the property of their respective owners. Property of: Pearson Level 1, 11 Elizabeth Street Hobart, Australia, 7000 Copyright 2012 Pearson Contact documentation@equella.com for matters relating to documentation.

3 Table of Contents EQUELLA clustering overview... 4 Clustering environment... 4 Pre-clustering checklist... 6 Configure EQUELLA on the first host... 8 Edit the EQUELLA configuration files... 8 Edit the Tomcat configuration files... 8 Configure EQUELLA on subsequent hosts...10 Test the clustered environment...10 Display each server s unique ID...10 Enable Multicast...10 Test cluster node server configuration...10 Test server fail over...10 Cluster testing tool...11 Check Lucene indexes using Luke...16 Appendix A...17 Configure Apache as a load balancer...17 Contact Client Support...19

4 EQUELLA clustering overview The purpose of this guide is to provide system administrators with an understanding of the EQUELLA clustering process. EQUELLA clustering involves two or more EQUELLA instances sharing a common database and filestore behind a load balancer. The load balancer directs clients to one of the members (EQUELLA instances) of the cluster. A simple clustering scheme is shown in Figure 1 Example simple clustering scheme Each of these components will typically be on dedicated hardware. Clustering environment Prior to setting up clustering the following conditions must be met: EQUELLA must be installed on all cluster nodes, but not started. If you are using Apache as your load balancer and Apache is installed on the same hardware as one of your EQUELLA nodes, then you must not install bind EQUELLA s Tomcat webserver to port 80, because Apache will need that port. The network switch responsible for the LAN segments(s) on which the clusters are communicating must have multicast communication enabled. If it is not, the cluster nodes will not be able to communicate with each other. When clustered, EQUELLA requires a shared filestore to house the cluster file repository. This filestore must be ready and available to all hardware nodes in the cluster prior to operating EQUELLA. If each node has a unique device path to the common filestore, those device paths should be noted for ease of configuration. Once EQUELLA is installed on all hosts of the cluster, the network file share must be determined. Also ensure that the user who is designated to start EQUELLA services on each node is the same (in Windows, this must be a network user) and that the user has write permissions to the filestore (on each node).

5 EQUELLA must have access through any firewalls. It is recommended that you fill out the Pre-clustering checklist on page 6.

6 Pre-clustering checklist Prior to setting up an EQUELLA clustered install, it is recommended that you fill out the following checklist so that you know what goes where and on what port. The Variable Names in the table will be used as the variable values in the configuration file examples in this guide. Variable Name $filestore $fullserverurl Where the Variable Name will be added File: mandatoryconfig.properties Variable: filestore.root File: mandatoryconfig.properties Variable: admin.url Explanation and notes This is the network location of the shared file store, it is in the format of: //$FileStoreHost/$FileStoreShareName/ For example, if fileserver was sharing a file store with a sharename of equellafiles, then this would be: //fileserver/equellafiles/. (NOTE: The name is configured when you share the folder.) To test access to the filestore using Windows, type the path in the Run command using backslashes e.g.: \\fileserver\equellafiles This is the full URL of the cluster server. This name identifies the cluster and must be unique. $clustergroupname File: mandatory-configproperties Variable: cluster.group.name This is the Cluster Group Name. It is the unique name identifying the cluster (not to be confused with DNS name), e.g: CLUSTER_GROUP

7 Variable Name Where the Variable Name will be added Explanation and notes $currentclusternodename File: server.xml Variable: jvmroute The jvmroute variable should only be set when using Apache HTTPD with AJP as the load balancer. $currenthostipaddress File: optional-config.properties Variable: channelservice.bindaddress This is IP address of the host being currently configured. (It is the value in the Bind Addr field of the EQUELLA Cluster Messaging Tester dialog.) $multicastaddress File: optional-config.properties Variable: channelservice.multicastaddress This is the multicast address on which your clusters communicate. By default, this is (It is the value in the Multicast Addr field in the EQUELLA Cluster Messaging Tester dialog.) $multicastport File: optional-config.properties Variable: channelservice.multicastport This is the multicast port on which events are communicated between clusters. If not specified, is used. Ensure this port is free in your O/S. (It is the value in the Multicast Port field in the EQUELLA Cluster Messaging Tester dialog.)

8 Configure EQUELLA on the first host The following sections describe configuration modifications to various files suitable for Windows Server or Unix using two nodes and a software load balancer. Edit the EQUELLA configuration files Edit the following files, substituting the correct values for the variables. (All configuration files can be found in the <path-to-equella>\learningedge-config directory.) mandatory-config.properties In the <path-to-equella>\learningedge-config\mandatory-config.properties file edit the following variables: filestore.root = $filestore admin.url = $fullserverurl cluster.group.name = $clustergroupname optional-config.properties In the <path-to-equella>\learningedge-config\optional-config.properties file edit the following variables: channelservice.clusternodeid = $currentclusternodename channelservice.bindaddress = $currenthostipaddress If there is more than one EQUELLA installed on this host, one of the following variables will need to be changed. To change the Channel Service Multicast Address and/or Multicast Port from the default values: channelservice.multicastaddress = $multicastaddress (NOTE: If the Channel Service Multicast Address is not specified is used.) ChannelService.multicastPort = $multicastport (NOTE: If the Channel Service Multicast Port is not specified is used.) Edit the Tomcat configuration files Make the following modifications to the Apache Tomcat server.xml file (located in the <path-to-equella>\tomcat\conf directory): server.xml 1. Configure a connector port for either the HTTP Connector or AJP Connector section: HTTP Connector <Service name="catalina">

9 <!-- HTTP Connector --> <Connector port="$xxxx" protocol="org.apache.coyote.http11.http11nioprotocol" redirectport="8443" URIEncoding="UTF-8" usebodyencodingforuri="true" connectiontimeout="20000" /> Where $XXXX is the connector port number that the Tomcat service is run. (NOTE: This must be an open port on the server to allow the service to start properly.) AJP Connector If Apache HTTP Server is being used as a load balancer, we recommend using the AJP Connector. To do so, uncomment the following in the AJP Connector section: <Connector port="$xxxx" protocol="ajp/1.3" redirectport="8443" URIEncoding="UTF-8" usebodyencodingforuri="true" request.tomcatauthentication="false" tomcatauthentication="false" /> Use an appropriate connector port so that the Tomcat service can run. (NOTE: The two connectors can be used together, although this is not usually required.) Modify the following entry on each Tomcat server.xml file: <Engine name="catalina" defaulthost="localhost"> To become: <Engine name="catalina" defaulthost="localhost" jvmroute="$currentclusternodename"> (NOTE: There is no need to change the localhost entry.) The jvmroute identifier is appended to the generated session identifier for the purposes of sticky sessions when using the AJP connector. This will interfere with any load balancer s method of sticky session implementation. 2. Uncomment the following: <Cluster classname="org.apache.catalina.ha.tcp.simpletcpcluster"/>

10 Configure EQUELLA on subsequent hosts To configure EQUELLA on subsequent hosts, install EQUELLA and repeat the steps from the Configure EQUELLA on the first host section on page 8 on each node of your cluster. Ensure the variables used to configure the first host are updated where appropriate to refer to the second (or subsequent) hosts, to maintain uniqueness. Test the clustered environment The following tests ensure the clustered environment is working correctly by testing each server is being accessed, cluster tasks are being delegated correctly and that a single server failure is handled without error. Display each server s unique ID To display each server s unique identification in the bottom left-hand corner of the window: 1. In the <path-to-equella>\learningedge-config\optional-config.properties file, uncomment the following: channelservice.debug = true Enable Multicast Some switches disable multicast packets, so ensure your switch has multicast enabled. Test cluster node server configuration To test each server is running and being called correctly: 1. Log in to an EQUELLA instance and identify the unique cluster ID that is displayed in the bottom left-hand corner of the window, and then stop that instance. 2. Force a refresh after the service has stopped (Ctrl + R / Ctrl + F5). 3. The unique cluster ID should now identify a different machine in the cluster. 4. Repeat this process until all the cluster servers have been identified and are running correctly. Test server fail over This test stops a server part way through an resource contribution to simulate a server failure. 1. Ensure all the cluster web servers are running. 2. Create a test user (refer to the EQUELLA User Management Configuration Guide for further information).

11 3. Log in to EQUELLA using the new user and open a contribution wizard with multiple pages. 4. Enter some details on the first wizard page, and then navigate to another page in the wizard. (This commits the changes for temporary storage. Without this, the server wouldn t know that anything had been written.) 5. Leave the contribution wizard open and stop the server identified by the cluster ID. Once the server has stopped: 6. Return to the open contribution wizard. 7. Navigate to the wizard page containing the data entered in step 4. A correctly configured cluster will perform this series of tasks without error. Cluster testing tool The cluster network configuration can be tested using the EQUELLA Cluster Testing tool. This standalone tool can be downloaded from the Cluster Health page in the Institution Manager or from the EQUELLA Support Desk Download the testing tool from the Institution Manager To access the Institution Manager: 1. Open a browser and enter the EQUELLA address of the hosting server with /institutions.do?method=admin appended to the URL. (e.g. would become ). to display the Server administration Authentication page. 2. Enter the password used by the EQUELLA server administrator (this is set on the Server Settings page after installation, and can be changed in the System password section of the Settings page of the Institution Manager), and click. 3. On the resulting Server administration page, select Cluster health from the navigation menu. An example is shown in Figure 2. EQUELLA 6.0 Clustering Configuration Guide 11 P age

12 Figure 2 Server administration page Cluster health page The Cluster health page lists the IP addresses that have been recognised as part of this cluster. When not part of a cluster, only the current node is listed. 4. Below this information is a link to the EQUELLA Cluster Testing tool. Click this link to download the equella-cluster-tester.jar file. Download the testing tool from the Service Desk 1. Open a browser and navigate to the EQUELLA Service Desk at 2. Enter your authentication details and log in. 3. Select Downloads from the left hand menu pane. 4. Click the 6.0 link then the Tools directory. An example is shown in Figure 3. Figure 3 Support site Downloads page 5. Click the equella-cluster-tester.jar link to download the tool. Using the EQUELLA Cluster Testing tool This file can be run on any environment with the Java Runtime Environment (JRE) (or Java Development Kit (JDK)) installed. On Windows, it is possible to double-click on the EQUELLA 6.0 Clustering Configuration Guide 12 P age

13 file to open it. On Unix systems it may be necessary to define the file as executable before it will run via the double-click method. 1. Run the file to open the EQUELLA Cluster Messaging Tester window. 2. Enter the appropriate details to match that of the node you are trying to connect with. 3. Enter the Node ID for the appropriate host, and the Group ID of the cluster. The Group ID is particularly important, as this links the nodes. 4. Enter the Bind Addr, Multicast Addr, and Multicast Port as configured in the optional-config.properties file. Using the following details as an example to populate the testing tool as shown in Figure 4. Figure 4 Cluster testing tool configuration for first node 5. Click to enter the cluster so the tool can send and receive cluster messages. The display is updated with the connection string and information about its state, as shown in Figure 5. EQUELLA 6.0 Clustering Configuration Guide 13 P age

14 Figure 5 Cluster testing tool status after joining cluster with first node 6. Open and configure another Cluster Testing Tool with the details from the other node of the cluster. 7. Click to enter the cluster so both nodes are able to send and receive cluster messages. An example is shown in Figure 6. Figure 6 Cluster testing tool status after joining cluster with second node 8. Click to see how many machines in the cluster can receive the message, as shown in Figure 7. EQUELLA 6.0 Clustering Configuration Guide 14 P age

15 Figure 7 Cluster testing tool status on second node after sending message Each cluster node should receive its own message and a message from each of the other cluster members. An example is shown in Figure 8. Any machines that do not appear should have their network configuration checked and then be restarted. Figure 8 Cluster testing tool status on first node after receiving message To modify the connection string 1. Select the Advanced tab to display an editable version of the connection string. EQUELLA 6.0 Clustering Configuration Guide 15 P age

16 2. Enter changes then test on the cluster by clicking. To clear the messaging display 1. Click to remove the displayed text updates. Check Lucene indexes using Luke When clustering is working properly the Apache Lucene index on each node is checked automatically. Lucene will create the index files up to a certain size and then start on a new file. Once the number of files reaches a certain limit then all the files are optimised into one file and then that larger file size becomes the limit for each file. The number of items indexed on each server can be checked with Luke. Luke is a Lucene tool that can be pointed to the Lucene index on each application node. Comparing the results of each server will indicate if they are synchronising properly. Luke can be found at EQUELLA 6.0 Clustering Configuration Guide 16 P age

17 Appendix A Configure Apache as a load balancer Prior to configuring Apache as a load balancer, it is recommended that you fill out the following checklist so that you know what goes where and on what port. The Variable Names in the table will be used as the variable values in this appendix. Variable Name $cluster1nodename $cluster2nodename $clusterxnodename Explanation and notes Name of the first EQUELLA instance that is a member node of the cluster. This name must be unique. As defined by the jvmroute attribute in /tomcat/server.xml. Name of the second EQUELLA instance that is a member node of the cluster. This name must be unique. As defined by the jvmroute attribute in /tomcat/server.xml. Names of all subsequent EQUELLA instances that are member nodes of the cluster. These names must be unique. As defined by the jvmroute attribute in /tomcat/server.xml. For the third (and subsequent ) EQUELLA instances this will be $cluster3nodename, $cluster4nodename, etc. $hostname1 $hostname2 $hostnamex DNS name of the machine host on which the first node of the cluster is physically installed. Usually, the machine name. DNS name of the machine host on which the second node of the cluster is physically installed. Usually, the machine name. DNS name of the machine host on which all subsequent nodes of the cluster is physically installed. Usually, the machine name. For the third (and subsequent) EQUELLA instances this will be $hostname3, $hostname4, etc. $cluster1tomcatportno $cluster2tomcatportno AJP port that the first EQUELLA node s Tomcat server is listening on (set when installing EQUELLA). This must be same as the connector port number set in the server.xml file. AJP port that the second EQUELLA node s Tomcat server is listening on (set when installing EQUELLA). This must EQUELLA 6.0 Clustering Configuration Guide 17 P age

18 be the same as the connector port number set in the server.xml file. $clusterxtomcatportno AJP ports that all subsequent EQUELLA node s Tomcat servers are listening on (set when installing EQUELLA). For the third (and subsequent) EQUELLA instances this will be $cluster3tomcatportno, $cluster4tomcatportno, etc. Download and install Apache HTTP Server 2.2 Download Apache2.2 appropriate for your machine s architecture from 1. Install Apache2.2 with default options. Make a note of the Apache2 install directory. 2. Ensure that the Apache modules, mod_proxy, mod_proxy_ajp and mod_proxy_balancer, are enabled. 3. Start and test that Apache2 works. 4. Shut down Apache2. To configure Apache as a load balancer 1. Create a virtual host file for your site, where you can configure the load balancing proxy, or create one in Apache s httpd.conf file (or Linux equivalent). a. Create an entry for each member you wish to add to the cluster. For example: <Proxy balancer://$equellacluster/> balancermember ajp://$hostname1:$cluster1tomcatportno route=$cluster1nodename balancermember ajp://$hostname2:$cluster2tomcatportno route=$cluster2nodename balancermember ajp://$hostnamex:$clusterxtomcatportno route=$clusterxnodename ProxySet stickysession=jsessionid </Proxy> Where: route must be set to the value used in the jvmroute attribute of the <Engine> tag of the Tomcat server.xml file. $equellacluster is the load balancer referenced by its URL (any name can be chosen). b. Add a ProxyPass directive to your virtual host: <virtualhost *:80> ServerName $externalservername ProxyPass / balancer://$equellacluster/ nocanon </VirtualHost> Where: $externalservername must be either the hostname of an institution, or the hostname in mandatory-config.properties. $equellacluster is the load balancer referenced by its URL (any name can be chosen). EQUELLA 6.0 Clustering Configuration Guide 18 P age

19 nocanon ensures URLs are passed through to Tomcat without processing. An example directive is: <VirtualHost *:80> ServerName equella.institution.com ProxyPass / balancer://equella.institution.production.com/ nocanon </VirtualHost> Contact Client Support We are always happy to help. If your organisation has a support agreement with EQUELLA then help is available at EQUELLA 6.0 Clustering Configuration Guide 19 P age