HP OO 10 OnBoarding Kit Community Assistance Team How to Set Up Disaster Recovery for HP OO Various global and sector-specific regulations, as well as standards such as COBIT DS4, ISO 2031 and others, require disaster recovery and business continuity plans. In order to certify and audit the level of readiness, organizations are required to periodically perform disaster recovery (DR) planning. This document describes an optional HP OO configuration, which addresses the common DR requirements for HP OO. HP Operations Orchestration HP Operations Orchestration (HP OO) is a next generation IT Process Automation solution that is designed from the ground up to increase automation adoption whether in a traditional data center or hybrid cloud environment. OO Community This guide is created for the OO community. To get started with HP OO 10 check out our community onboarding kit that contains four persona-based learning tracks for administrators, flow authors, end users, and integrators. Each track provides knowledge assets of the following types: Presentations Videos Tutorials Guides Best Practices
HP OO Disaster Recovery Overview DR scenario active passive with manual recovery Since database cluster solutions are limited and usually cannot address geographical distances between nodes, we recommend using database replication. This forces us (as is common in DR-related solutions) to take an active-passive approach. That means that the DR site is off and will be activated only once a disaster happens. The following diagram presents the solution:
Healthy status 1. There are no Central servers on the DR site. 2. The user enters the Central URL in his/her browser. 3. The global load balancer resolves the URL to the main site. 4. The local load balancer resolves the request to one of the main site Centrals. 5. Database replication is operational and any changes are synchronized to the DR node. When the main site goes down 1. In case of failure, the administrator will: a. Stop the Central server(s) in the main site. b. Check that the database in the DR site has been synchronized. c. Start the Central server(s) in the DR site. 2. The user enters the Central URL in his/her browser. 3. The global load balancer resolves the URL to the DR site. 4. The local load balancer resolves the request to one of the Centrals on the DR site. 5. The database has the latest information from the last replication interval and this is reflected in the Central UI. When the main site goes up again 1. Once the main site has been fixed, the administrator will: a. Stop the Central server(s) in the DR site. b. Check that the database in the main site has been synchronized. c. Start the Central server(s) in the main site. 2. The user enters the Central URL in his/her browser. 3. The global load balancer resolves the URL to the main site. 4. The local load balancer resolves the request to one of the Centrals on the main site. 5. The database has latest information from the last replication interval and this is reflected in the Central UI.
Note: The HP OO Central Topology tab reflects the worker changes as the main site goes up or down: Internal workers of the main/dr site are active/inactive once their corresponding sites are active/inactive too. External workers registered through local load balancer(s) are active/inactive once their corresponding sites are active/inactive too. External workers that are registered through global load balancer(s) remain as such and executions will continue accordingly once the Central server(s) on either the main site or the DR site are up and running. Tested environments The solution was tested on the following configuration: Central servers: Windows Server 2012 servers. DB servers: MS-SQL Server 2012 Enterprise Edition, configured with database mirroring. RAS servers: 2 RAS servers, on Windows Server 2012. Apache load balancer. Performed tests Database failover mimic on both sites. Network failover mimic. Central failover mimic. RAS failover mimic.
Configuration Details Notes: Make sure that the Central server(s) are configured with the same time zone. Otherwise, various features in the product are altered and inconsistencies may appear (for example, runs with inconsistent start and end times, based on the active site, different ROI value in the dashboard, and so on). The entire topology is visible in the Central UI, meaning that end users are exposed to the DR site/main site topology even if those are off. Make sure that workers are properly configured in worker groups, so that runs continue to work automatically as expected whenever the switch occurs between the DR and the main site. Use the same security keys on the DR and main site installation. Schedule jobs for the database backups. Schedule jobs for transaction log backups. Schedule the HP OO purging flows to run regularly. More about the purging flows can be found in the HP Solutions out-ofthe-box content pack. Prerequisites Both database servers (main and DR) use the same version. A network route exists between the main and DR sites. Sufficient free disk space exists on both database servers; thus, allowing database and transaction log backups. Facilitate database cross-site authentication. Install and configure the main and DR sites 1. Install Central(s) in the main site. 2. (Optional) Configure HP OO as needed: roles, users, LDAP, content packs, and so on. 3. Stop the main site Central(s). 4. Configure database mirroring. From this point, a database in the DR site will act as a replica. (Configuration details may differ between database vendors). 5. Make sure that the main site database status is as expected (Configuration details may differ between database vendors). 6. Start the main site Central(s) and check that everything is working properly. 7. Stop the main site Central(s). 8. Perform failover on the main site database; thus, swapping the roles between the main and DR site databases. 9. Copy the database.properties, encryption.properties and encryption_repository files from the main site Central(s) to the DR site.
10. Change the database.properties file to point to the database in the DR site. 11. Install the Central(s) in the DR site as a Central cluster. These Centrals should use the same security keys as the ones in the main site Central and the altered database.properties file; thus, pointing to the existing database in the DR site. 12. Start the DR site Central(s) and check that everything is working properly. 13. Revert to the main site Central(s) through a fail-back procedure: a. Stop the HP OO Central(s) in the running site. b. Perform failover on the database. c. Start the HP OO Central(s) in the other site. Install and configure RAS(es) 1. Install RAS(es) by pointing to the global or local load balancer(s). 2. Configure external workers of the newly installed RAS(es) to the relevant (desired) worker group. That s the end of guide. If you have any questions, please post them on the HP OO community forums: http://www.hp.com/go/oopractitionerforum Check the HP OO community front page for additional resources such as guides, tutorials, videos, content and more: https://hpln.hp.com/group/operations-orchestration