Load Balancing and High Availability for BMC Remedy Action Request System Kelly Deaver, BMC Technical Marketing February 1, 2008
Agenda Why use server group and load balancing How server group enhances BMC Business Service Management Products covered Examples of possible environment Licensing Installing What to do and what not to do Configuring Managing BMC Remedy Knowledge Management and load balancing 2
Why Use Server Groups with Load Balancing Symptoms Screens taking too long to build/refresh Users not able to login (server busy or down) Server group as the solution Increased performance Administrative tasks can be assigned to a specific AR System Server User interactions are distributed between servers Redundancy for fail over If one AR System Server or AR System Mid-tier fails, others pick up the load and continue 3
BSM and AR System Powered Powered By By AR AR System System 4
Products Covered in this Presentation BMC Remedy Action Request System BMC Atrium CMDB BMC Remedy FTS (Full Text Search) BMC Remedy Flashboards BMC Remedy Asset Management BMC Remedy Change Management BMC Remedy Service Desk BMC Service Level Management BMC Knowledge Management 5
Configuration - Basic AR System Servers Administrative Operations Escalation, Reconciliation, etc. Clients Load Balancer Database Top server is primary for Administration and Escalation operations. Only one at a time can do this. The other two are for back up of Admin operations. All three are used for User interaction. 6
Configuration With Firewall and Load Balancer AR System Servers BMC1 (real server) 11.40.35.47:2020 BMCRemedy (virtual server) 11.40.34.94:3030 BMC2 11.40.35.49:2020 Clients Firewall Load Balancer Database Here the Client comes in through a firewall to port 3030. The load balance then acts as in the earlier example but the port for each server is 2020 BMC3 11.40.35.51:2020 7
Configuration Firewall, Web Servers & 1 Load Balancer As in the previous slide the client requests passes through the firewall and into the load balancer but this time it is a web client. The load balancer distributes the load to the 3 web servers. Each web server is paired with an AR System server by making an IP address entry in the hosts file of the web server BMCRemedy (virtual server) 11.40.34.94:3030 Web Servers Web 1 AR System Servers Web 2 Clients Firewall Load Balancer Database Create a hosts file entry for each AR System Server on the companion web server - BMCRemedy 11.40.35.51 Web 3 BMC3 11.40.35.51:2020 8
Configuration Firewall, Web Servers & 2 Load Balancers After Initial connection is made between the mid-tier and the AR Server it remains a connection until either server fails. When one server fails another pairing is made. Even if the other server comes back up, doesn t rebalance. Web Servers Only For Failover AR System Servers Clients Firewall Load Balancer Database Once a connection has been made through the 2nd load balancer, session affinity must be maintained. 9
Session Affinity Why User instance information is cached on the AR System server Large data transfers may involve chunking. All blocks of data need to follow the same path. If encryption is used, encryption is set between the two servers. How Determine the sticky affinity timeout period AR System default user session timeout is 60 minutes The Sticky Timeout must be longer than the user session timeout In AR.cfg / AR.conf Internal-User-Instance-Timeout (the session timeout) The default setting is 1 hour, the maximum is 2 hours, and the minimum is 30 minutes. On Your Load Balancer Set the sticky affinity timeout 10
Configuration DMZ, Firewall, Web Servers & 2 Load Balancers (Higher Security, WANs, etc) Q? A Clients 12 Firewall 5 16 4 1 13 14 3 2 15 Load Balancer Web Servers DMZ 6 AR System Servers LAN 11 7 10 8 9 Load Balancer Database 11
Configuration DMZ, 2 Firewalls, Web Servers & 2 Load Balancers (Better Performance) Load Balancer DMZ Clients Firewall 2 Network Pipes To Reduce Bottlenecks Web Servers Firewall AR System Servers LAN Load Balancer Database 12
Configuration - NOT Load balancing is for spreading the load, not condensing it. Plus you need to have a sticky connection between the db and the AR Server. AR System Servers Clients Load Balancer Load Balancer Database For Failover Use A Disk Array Instead 13
Purchasing and Licensing Purchases required per environment (dev, test, production) 1 copy of AR System per server running it 1 copy of each application (all servers in the group use the same application) License 6.x or 7.0.x on each server You must specify server group when ordering license keys You have to order through BMC Support-Licensing, not online key generation License is stored in license file on each server 1 AR Server Application (server group name must be consistent) Fixed User (*SITE*) Floating User (server group must be consistent) Licensing 7.1 is easier For in-depth info see the session BMC Remedy Customer Enforced Licensing Simplification Thursday 2PM 14
Licensing 7.1 Servers License is stored in shared database Create an entry for each AR System Server (with required key matching each host) Create one shared entry for each application Create one shared entry for Fixed Users (system or application) Create one shared entry for Floating Users (system or application) 15
Install Order Install on the initial server AR System, Approval Server, Assignment Engine, Email Engine Mid-Tier Server (same or separate physical server) Flashboards, CMDB Applications For subsequent AR System Servers All install paths must be consistent on each server in the group Install AR System Server Choose the same database when installing the servers in a server group Select the Share database option License the servers appropriately Install CMDB Specify Share Server Group only executables and config entries are created Install Flashboards Forms are only installed on the first install/primary server by default Applications Copy the application\bin sub-directories from the initial server Configure ar.cfg/.conf and armonitor.cfg/.conf Install subsequent Mid-Tier Servers 16
Configure Each Server Server group name (shared) Name shared by all servers in the group (BMCRemedy) Must match name of group applied with licenses Check Interval How often other servers check if it is online Full Text Search indexing Put index directory on a shared file system and point all servers there. Email Engine Port needs to match on all servers BMC Remedy Flashboards Port needs to match on all servers BMC Service Level Management Don t Don t configure through through the the load load balancer balancer Import the public key of the Data Collection Point security certificate from the initial server BMC Remedy Distributed Server Option (DSO) Specify any server in the group as the source Must specify explicitly a server for distributed returns, deletes and updates 17
Configure Each Mid-Tier Server Host file on each web server <Load Balancer Virtual Server Name> <Actual Server IP> BMCRemedy 11.40.35.51 18
Configure Each Server AR.cfg Server Name (unique) by default, the fully qualified name of the server bmc.mycompany.com You can specify a name in the ar.cfg file Server-Connect-Name : bmc1 Used by the other servers to talk to the server Copy from ar.cfg on initial server to all others Plugin <full path>omfobjiefilapi.dll" Plugin <full path>arfslasetup.dll" Plugin <full path>arfcbdata.dll" Plugin-Path <concatenation of paths> Alternate-Approval-Reg: T 19
Configure Each Server ARMonitor.cfg Copy from armonitor.cfg on initial server to all others <full path>/brsvcbrie.exe" -d <AR System path> -m <full path>/arrecond.exe" -i <AR System path> -m <full path>/arcmdbd.exe" -i <AR System path> -m 20
Configure Operation Ranking Create 1 entry for each server/operation combination. Business Rule Engine, Approval Server and CMDB must share ranking with Administration. They have to run on the same server. Servers are used based on ranking, lowest to highest. Null excludes the server from the operation The number of times the server does not respond before the next server for the operation takes over 21
Managing : Log - arsrvgrp Tracks starting/stopping operations evaluation of other servers recache requests Event time 22
Managing : Server Status Signaling You can review the status of your server group from the servgrp_board table in your database. 23
Configuring Knowledge Management Top server is primary for Administration and Escalation operations. Only one at a time can do this. The other two are for back up of Admin operations and primary for User interaction. KM Servers AR System Servers Application Server Hummingbird Search Server Clients Load Balancer Data Server Knowledge Solutions 24
Configuring Knowledge Management Servers You must copy and edit files on all servers Details can be found in the White Paper Configuring BMC Remedy Knowledge Management in a Load Balanced Environment 25