High Availability & Disaster Recovery Development Project Concepts, Design and Implementation
High Availability & Disaster Recovery Development Project CONCEPTS Who: Schmooze Com Inc, maintainers, core developers and stewards of the FreePBX project. Hours D a y s Minutes S e c o n d s No Solution Downtime in Terms of Days Offsite Backup Manual Restore Downtime in Terms of Hours Warm Spare Manual Failover and Failback Downtime in Terms of Minutes High Availability Auto Failover and Failback Downtime in Terms of Seconds WHAT: High Availability HA is the characteristic of a system to protect or recover from minor outages in a short time frame with largely automated means. Disaster Recovery is ensuring recoverability of your systems in the event of major system outages whether they are caused by hardware, software, connectivity, natural, human or other means. Each organization that runs a PBX inherently has their own pain tolerance for system downtimes. We are in the process of developing an easily implementable, replicable, supportable HA (High Availability) version of FreePBX for license by organizations that need a reliable HA solution. WHERE? FreePBX is the most prevalent open source PBX platform in use in the world today, with over 1 million active production systems, and over 20k new installations of the FreePBX Distro every month FreePBX is used in businesses ranging from SMB to fortune 1000 companies. WHEN? We estimate that the first private BETA will be released for testing during November 2013 WHY? We understand that your phone system is a major tool for your business, that s why keeping your business running with minimal downtime is our goal.
High Availability & Disaster Recovery Development Project BUSINESS CONTINUITY is the objective of HIGH AVAILABILITY SOLUTIONS, business continuity is the ability to do business under any circumstance and is an essential requirement of modern communications systems. 56% of enterprises in North America and 30% in Europe don't have a good disaster recovery plan, says a recent CA Technologies survey. Of the companies surveyed, small enterprises lost, on average, more than $55,000 in revenue due to IT failures each year, while midsize companies lost more than $91,000 and large companies lost more than $1,000,000. A data center outage by itself can cost an average of $5,600 per minute. How Downtime Impacts Business Continuity: Direct Cost: Cost directly attributable to the outage. device repair, shipping, external consultants, contract penalties for delays of delivery of a service. Additional Work Hours: not only additional IT hours to mitigate the problem, but additional hours across the company to cover downtime. Lost Work Hours: Sales and support calls that can t be made, 100 s of office workers/call center employees cannot work because of an outage. Lost Revenue: Toughest to measure, but likely the most important business consequence, if sales cannot be made, support not given on customer facing systems clients will go to competitors whose systems are working, outages can damage your reputation and result in a loss of clients. Regulatory Compliance and Risk Management: Certain industries and businesses must implement HA systems for legal reasons, or to mitigate risk management within their infrastructure. DESIGN Development Scope of Work- Rework FreePBX platform to integrate High Availability (HA) utilizing DRBD, Cluster Manager and Pacemaker. Mirroring- Enable live mirroring between FreePBX nodes. Failover- Utilizing Pacemaker, enable failover between FreePBX nodes. Failback- When nodes recover they failback to primary FreePBX nodes. Switchover- Enable an online management tool to switch between nodes within the FreePBX GUI. Managing Setup- The setup of the H/A will be done in the FreePBX GUI module and provide simple point and click setup for your H/A system. Monitoring and Alerting- You will be able to setup in FreePBX different levels of monitoring and alerts that you want to receive in the event of a problem. Documentation- Produce an administrator's guide outlining the installation and use of the
High Availability & Disaster Recovery Development Project FreePBX high availability service. Custom Development FreePBX (HA) high availability utilizing DRBD. FreePBX will integrate DRBD based HA clustering between FreePBX deployments/nodes. The FreePBX deployments will become separate nodes utilizing DRBD to maintain synchronization and pacemaker monitoring to determine when system outages occur triggering the failover between nodes. High Availability- Add the ability within FreePBX to enable node replication, clients (phones & trunks) will talk to an active node/freepbx deployment. The service including its IP address, can be migrated to other nodes/deployments in that cluster at any time, either due to a failure of the active node or as an administrative action from within the FreePBX GUI. The service will failover, or failback or automatically by and on command by an administrator. DRBD- DRBD can be understood as network based raid1, the bulk of the development cost will be to rework the FreePBX platform to integrate the replication and triggers to enable failover and failback when the system determines there is a problem with an active node. What Pacemaker does FILE SYSTEM FILE SYSTEM BUFFER CACHE BUFFER CACHE TCP/IP TCP/IP DISK SCHED DISK SCHED DISK DRIVER NIC DRIVER DISK NIC NIC DRIVER NIC DISK DRIVER DISK MACHINE IP IP MACHINE IP FAILBACK RAW DEVICE FAILOVER MACHINE IP IP RAW DEVICE MACHINE IP IP Mirroring- DRBD works on top of block devices, i.e., hard disk partitions or LVM s logical volumes. It mirrors each data block that it is written to disk to the peer node. Note: Data can only be accessed on the active node. Active nodes will be mirrored to inactive nodes. Failover- When a degraded service level is reached, this triggers a failover to a secondary (replicated node/frepbx deployment.) The secondary node s IP address will be replicated to the first and phones and trunks will reregister. Failback- When the service is restored back to the primary server, all services are migrated back to the primary server node. Switchover- Using the FreePBX GUI an administrator can manually trigger a switch between FreePBX deployments/nodes. (Useful for server maintenance windows, testing purposes or when manual control is needed).
High Availability & Disaster Recovery Development Project IMPLEMENTATION HOW? We estimate providing several product mixes based on this development project. Full turnkey solution, integration of all aspects of HA (SIP trunking, PBX hardware & phones and support) Turnkey HA software, PBX appliance & phones prebuilt HA FreePBX appliances and phones. HA software licensed FreePBX modules for configuring HA on your own HA certified hardware. Minimum Hardware Requirements 2 x servers with minimum of 250GB of disk space 1 network interface on each machine 2GB RAM Recommended Hardware 2 x servers with minimum of 500GB of disk space (same as above) 2 network interfaces on each machine, via different interface (eg, 1 onboard NICs, additional PCIe NIC) 8GB RAM IPMI/DRAC/ILO functionality on servers Configuration Select 'Install HA Build' on install of the FreePBX Distro All the rest is via FreePBX GUI How is a failover triggered? When the machine goes down or becomes unresponsive (Eg, machine B can't see machine A, machine B will take over the roles) When a service has failed a certain number of times (eg, asterisk unable to start, after repeated attempts) The services we will be monitoring are Asterisk, Apache and MySQL. Licensing The FreePBX H/A module that will manage and setup your H/A will be a commercial paid license. Each FreePBX server that is part of the node will need a licensed copy of the H/A module.
To purchase licenses of FreePBX High Availability, please log in to your Schmooze Portal account and click on the store tab. To create your FREE account visit http://schmoozecom.com/oss-signup.php (920) 886-8130 http://schmoozecom.com