1 High Availability & Disaster Recovery Development Project Concepts, Design and Implementation
2 High Availability & Disaster Recovery Development Project CONCEPTS Who: Schmooze Com Inc, maintainers, core developers and stewards of the FreePBX project. Hours D a y s Minutes S e c o n d s No Solution Downtime in Terms of Days Offsite Backup Manual Restore Downtime in Terms of Hours Warm Spare Manual Failover and Failback Downtime in Terms of Minutes High Availability Auto Failover and Failback Downtime in Terms of Seconds WHAT: High Availability HA is the characteristic of a system to protect or recover from minor outages in a short time frame with largely automated means. Disaster Recovery is ensuring recoverability of your systems in the event of major system outages whether they are caused by hardware, software, connectivity, natural, human or other means. Each organization that runs a PBX inherently has their own pain tolerance for system downtimes. We are in the process of developing an easily implementable, replicable, supportable HA (High Availability) version of FreePBX for license by organizations that need a reliable HA solution. WHERE? FreePBX is the most prevalent open source PBX platform in use in the world today, with over 1 million active production systems, and over 20k new installations of the FreePBX Distro every month FreePBX is used in businesses ranging from SMB to fortune 1000 companies. WHEN? We estimate that the first private BETA will be released for testing during November 2013 WHY? We understand that your phone system is a major tool for your business, that s why keeping your business running with minimal downtime is our goal.
3 High Availability & Disaster Recovery Development Project BUSINESS CONTINUITY is the objective of HIGH AVAILABILITY SOLUTIONS, business continuity is the ability to do business under any circumstance and is an essential requirement of modern communications systems. 56% of enterprises in North America and 30% in Europe don't have a good disaster recovery plan, says a recent CA Technologies survey. Of the companies surveyed, small enterprises lost, on average, more than $55,000 in revenue due to IT failures each year, while midsize companies lost more than $91,000 and large companies lost more than $1,000,000. A data center outage by itself can cost an average of $5,600 per minute. How Downtime Impacts Business Continuity: Direct Cost: Cost directly attributable to the outage. device repair, shipping, external consultants, contract penalties for delays of delivery of a service. Additional Work Hours: not only additional IT hours to mitigate the problem, but additional hours across the company to cover downtime. Lost Work Hours: Sales and support calls that can t be made, 100 s of office workers/call center employees cannot work because of an outage. Lost Revenue: Toughest to measure, but likely the most important business consequence, if sales cannot be made, support not given on customer facing systems clients will go to competitors whose systems are working, outages can damage your reputation and result in a loss of clients. Regulatory Compliance and Risk Management: Certain industries and businesses must implement HA systems for legal reasons, or to mitigate risk management within their infrastructure. DESIGN Development Scope of Work- Rework FreePBX platform to integrate High Availability (HA) utilizing DRBD, Cluster Manager and Pacemaker. Mirroring- Enable live mirroring between FreePBX nodes. Failover- Utilizing Pacemaker, enable failover between FreePBX nodes. Failback- When nodes recover they failback to primary FreePBX nodes. Switchover- Enable an online management tool to switch between nodes within the FreePBX GUI. Managing Setup- The setup of the H/A will be done in the FreePBX GUI module and provide simple point and click setup for your H/A system. Monitoring and Alerting- You will be able to setup in FreePBX different levels of monitoring and alerts that you want to receive in the event of a problem. Documentation- Produce an administrator's guide outlining the installation and use of the
4 High Availability & Disaster Recovery Development Project FreePBX high availability service. Custom Development FreePBX (HA) high availability utilizing DRBD. FreePBX will integrate DRBD based HA clustering between FreePBX deployments/nodes. The FreePBX deployments will become separate nodes utilizing DRBD to maintain synchronization and pacemaker monitoring to determine when system outages occur triggering the failover between nodes. High Availability- Add the ability within FreePBX to enable node replication, clients (phones & trunks) will talk to an active node/freepbx deployment. The service including its IP address, can be migrated to other nodes/deployments in that cluster at any time, either due to a failure of the active node or as an administrative action from within the FreePBX GUI. The service will failover, or failback or automatically by and on command by an administrator. DRBD- DRBD can be understood as network based raid1, the bulk of the development cost will be to rework the FreePBX platform to integrate the replication and triggers to enable failover and failback when the system determines there is a problem with an active node. What Pacemaker does FILE SYSTEM FILE SYSTEM BUFFER CACHE BUFFER CACHE TCP/IP TCP/IP DISK SCHED DISK SCHED DISK DRIVER NIC DRIVER DISK NIC NIC DRIVER NIC DISK DRIVER DISK MACHINE IP IP MACHINE IP FAILBACK RAW DEVICE FAILOVER MACHINE IP IP RAW DEVICE MACHINE IP IP Mirroring- DRBD works on top of block devices, i.e., hard disk partitions or LVM s logical volumes. It mirrors each data block that it is written to disk to the peer node. Note: Data can only be accessed on the active node. Active nodes will be mirrored to inactive nodes. Failover- When a degraded service level is reached, this triggers a failover to a secondary (replicated node/frepbx deployment.) The secondary node s IP address will be replicated to the first and phones and trunks will reregister. Failback- When the service is restored back to the primary server, all services are migrated back to the primary server node. Switchover- Using the FreePBX GUI an administrator can manually trigger a switch between FreePBX deployments/nodes. (Useful for server maintenance windows, testing purposes or when manual control is needed).
5 High Availability & Disaster Recovery Development Project IMPLEMENTATION HOW? We estimate providing several product mixes based on this development project. Full turnkey solution, integration of all aspects of HA (SIP trunking, PBX hardware & phones and support) Turnkey HA software, PBX appliance & phones prebuilt HA FreePBX appliances and phones. HA software licensed FreePBX modules for configuring HA on your own HA certified hardware. Minimum Hardware Requirements 2 x servers with minimum of 250GB of disk space 1 network interface on each machine 2GB RAM Recommended Hardware 2 x servers with minimum of 500GB of disk space (same as above) 2 network interfaces on each machine, via different interface (eg, 1 onboard NICs, additional PCIe NIC) 8GB RAM IPMI/DRAC/ILO functionality on servers Configuration Select 'Install HA Build' on install of the FreePBX Distro All the rest is via FreePBX GUI How is a failover triggered? When the machine goes down or becomes unresponsive (Eg, machine B can't see machine A, machine B will take over the roles) When a service has failed a certain number of times (eg, asterisk unable to start, after repeated attempts) The services we will be monitoring are Asterisk, Apache and MySQL. Licensing The FreePBX H/A module that will manage and setup your H/A will be a commercial paid license. Each FreePBX server that is part of the node will need a licensed copy of the H/A module.
6 To purchase licenses of FreePBX High Availability, please log in to your Schmooze Portal account and click on the store tab. To create your FREE account visit (920)
TECHNICAL WHITE PAPER: DATA AND SYSTEM PROTECTION Achieving High Availability with Symantec Enterprise Vault Chris Dooley January 3, 2007 Technical White Paper: Data and System Protection Achieving High
DeltaV Distributed Control System Whitepaper October 2014 DeltaV Virtualization High Availability and Disaster Recovery This document describes High Availiability and Disaster Recovery features supported
Technical Guide www.novell.com SAP Applications Made High Available on SUSE Linux Enterprise Server 10 prepared by: Markus Gürtler, Fabian Herschel, Matthias Schlarb, Alexander Krauth Table of Contents
WHITE PAPER Intelligent Disaster Recovery VERITAS Backup Exec 10 for Windows Servers Intelligent Disaster Recovery Option 1/17/2005 1 TABLE OF CONTENTS Executive Summary...3 Disaster Preparation Plan (DPP)...4
VirtuousIT Ltd 1 Table of Contents 1 Introduction 3 The Data Explosion 3 The Importance of Rapid System Recovery 3 2 The VirtuousIT Solution 4 3 RecoveryShield - Solutions 5 Large Business 5 Small Medium
Best Practices for the HP EVA Array using VMware vcenter Site Recovery Manager Table of contents Introduction... 2 HP StorageWorks Continuous Access EVA... 3 Data replication... 3 DR groups and copy sets...
Backup and Redundancy White Paper NEC s UC for Business Backup and Redundancy allow businesses to operate with confidence, providing security for themselves and their customers. When a server goes down
Veritas Storage Foundation and High Availability Solutions HA and Disaster Recovery Solutions Guide for Microsoft SQL 2008 and 2008 R2 Windows Server 2003, Windows Server 2008 5.1 Service Pack 2 Veritas
Best Practices to Ensure SAP Availability Abstract Ensuring the continuous availability of mission-critical systems is a high priority for corporate IT groups. This paper presents five best practices that
EMC NetWorker Version 8.2 SP1 Server Disaster Recovery and Availability Best Practices Guide 302-001-572 REV 01 Copyright 1990-2015 EMC Corporation. All rights reserved. Published in USA. Published January,
Best Practices for Deploying and Managing Linux with Red Hat Network Abstract This technical whitepaper provides a best practices overview for companies deploying and managing their open source environment
Best Practices Guide McAfee epolicy Orchestrator for use with epolicy Orchestrator versions 4.5.0 and 4.0.0 COPYRIGHT Copyright 2011 McAfee, Inc. All Rights Reserved. No part of this publication may be
1 Unitrends, Inc. Software and Hardware Support Handbook Overview This Customer Support Handbook details support services, contact information and best practices for contacting Unitrends Support to ensure
Microsoft Cross-Site Disaster Recovery Solutions End-to-End Solutions Enabled by Windows 2008 Failover Clustering, Hyper-V, and Partner Solutions for Data Replication Published: December 2009 Introduction:
WHITE PAPER Meeting Backup and Archive Challenges Today and Tomorrow Sponsored by: Fujitsu Nick Sundby November 2014 IDC OPINION IDC's end-user surveys show data integrity and availability remains a top
A MIMECAST SPONSORED WHITE PAPER Email As Part of a Business Continuity StrategY Why always-on-business requires always-on-email Few businesses could function without email, and email provides a critical
Symantec Disaster Recovery Orchestrator Deployment Guide Microsoft Azure 6.1 May 2014 Symantec Disaster Recovery Orchestrator Deployment Guide The software described in this book is furnished under a license
Document ID Syno_UsersGuide_DisasterRecovery_20111114 Table of Contents Chapter 1: Dual Server Disaster Recovery Plan Back up Primary Server's Configurations... 5 Synchronize Data between Primary and Backup
Introduction Small and medium businesses (SMBs) need to keep their businesses up and running just as much as larger companies. Yet they are challenged by several limitations when seeking out a solution
One Stop Data & Networking Solutions PREVENT DATA LOSS WITH REMOTE ONLINE BACKUP SERVICE Prevent Data Loss with Remote Online Backup Service The U.S. National Archives & Records Administration states that
WHY COX BUSINESS? SIP TRUNKING: BUSINESS CONTINUITY AND REDUNDANCY A White Paper 1 P a g e Table of Contents INTRODUCTION... 1 WHAT IS FAILURE?... 2 THE APPROACHES... 3 SINGLE SITE OPTIONS... 3 SEPARATE
Pervasive PSQL Meets Critical Business Requirements Pervasive PSQL White Paper May 2012 Table of Contents Introduction... 3 Data Backup... 3 Pervasive Backup Agent... 3 Pervasive PSQL VSS Writer... 5 Pervasive
The One Essential Guide to Disaster Recovery: How to Ensure IT and Business Continuity Start Here: Basic DR Only 6 percent of companies suffering from a catastrophic data loss survive, while 43 percent