VoIP Lifecycle Management. Best Practices and Business Value

Transcription

1 VoIP Lifecycle Management Best Practices and Business Value Viola Networks May 2006

2 VoIP Lifecycle Management VoIP Lifecycle Management Best Practices and Business Value When managing an enterprise VoIP environment or a multi-client infrastructure for delivering hosted services, there are several key requirements that must be addressed at each stage of its lifecycle. Viola Networks has defined a 4-stage lifecycle framework for managing the design and delivery of cost-effective business-class VoIP services. This framework can be used to evaluate and implement a VoIP lifecycle management system. A comprehensive VoIP management solution must address these requirements with tools that support best practices that create business value. If your organization is responsible for delivering a quality VoIP service and its economic benefits (such as higher revenue and lower expenses) then your ability to monitor service quality, detect and correct service problems and increase your operational efficiency are paramount. Viola s lifecycle management framework covers the full spectrum of management requirements, from planning through production, when building a large-scale VoIP services operation. During predeployment planning for VoIP services, the readiness assessment stage allows you to preview your VoIP service quality prior to deployment and determine the critical network engineering changes required to accommodate the special needs of VoIP. During service deployment, methodologies for streamlining the installation of equipment and handing-off the VoIP service to users and customers are described. Operational support refers to day-to-day monitoring and reporting of service quality combined with proactive management of service problems. Capacity planning is the on-going optimization of the infrastructure as you expand the service to cover more users, locations and higher call volume. Figure 1. Viola s model for Lifecycle Management provides a framework for applying best practices to VoIP management The Readiness Assessment Stage For enterprise IT organizations, migrating to VoIP from a legacy phone environment is a major commitment in terms of cost and user impact. For service providers, launching a hosted VoIP service into the market place is a business-critical undertaking. In both cases, it is imperative that any new VoIP service be previewed prior to deployment. VoIP assessment tools are used to take precision measurements of service quality using simulated VoIP traffic while at the same time evaluating the network for proper device configuration, bandwidth capacity and its network connectivity requirements. VoIP Lifecycle Management: Best Practices and Business Value -2-

3 Requirements Preview the VoIP service: Assessment tools must be able to replicate the behavior of real VoIP traffic on the target network infrastructure. This requires high volume simulation of RTP streams with packet payloads that reproduce all popular CODEC compression algorithms and the use of signaling standards. Prepare the network infrastructure: Integrated troubleshooting tools must quickly identify network problems such as duplex mismatches, QoS misconfiguration, or bandwidth constraints so they can be corrected during the predeployment phase of the project. Establish operational benchmarks: The readiness assessment stage should be used to establish the operational benchmarks and thresholds (such as MOS scores or maximum delay, loss and jitter impairments) for managing the postdeployment delivery of VoIP services. Best Practices Best practices for the readiness assessment stage are represented by the work flow diagram in figure 2. Readiness assessment begins with a discovery of the network inventory under evaluation. Then a configuration check is performed to validate critical settings such as full duplex operation and proper QoS operation. The troubleshooting and resolution of these problems should be completed prior to running a preliminary assessment. The preliminary assessment is a short test to methodically increase the VoIP traffic load on the network with simulated concurrent VoIP calls to determine the maximum number of calls that can deliver business-class voice services. The VoIP assessment system must provide integrated troubleshooting tools to detect and correct problems with delay, loss, or jitter. Figure 2. Readiness Assessment Workflow Once the maximum call volume is established, this peak traffic load is generated for a continuous test that spans a complete business cycle (a business day, multiple days or a full week). During busy hours, the traffic peaks that overrun the available network capacity will be caught and the network planner can build the appropriate headroom in the network design in order to avoid these congestion events once the service is operational. Business Value The business value created by these best practices is realized as expense reductions and productivity gains by an enterprise IT organization. Expense reductions come in the form of assessment fees that would otherwise be paid to an outside service provider or integrator charging $5k or more per site for assessments. VoIP Lifecycle Management: Best Practices and Business Value -3-

4 A VoIP assessment takes the guesswork out of bandwidth provisioning so you can optimize the cost and capacity of your network. If you over-build your network when mixing voice and data services, then you are negating the cost benefits of a converged infrastructure. In addition to these direct cost savings, productivity gains come from the ability of assessment tools (such as those from Viola Networks) to perform multiple concurrent assessments - allowing one engineer to do the work of many. Another important feature of VoIP assessment tools is remote operations that allow assessments to be conducted anywhere in the world from a central location, eliminating the time and travel expense associated with sending engineers to remote sites. Figure 3. Readiness Assessment Business Value Service providers will realize additional business value from incremental revenues generated by assessment services. Most IP telephony manufacturers now require assessments to be performed for all equipment installations. A cost-effective VoIP assessment tool can pay for itself after just a few assessments and the customer satisfaction resulting from a properly designed solution will pay additional dividends over time. The Service Deployment Stage During service deployment, it is critical that the installation process go smoothly and resources are utilized efficiently. Each installation should follow a procedure that verifies the service quality delivered to users, documents the installed equipment for each site and establishes the operational parameters for monitoring and supporting the service from a central NOC location. Streamlining the deployment process will enable a faster rate of service deployment - bringing more users online in a shorter time span. An accelerated rate of deployment will translate to more savings for enterprises and faster revenue growth for service providers. Requirements Verify service levels: During the time period between the site assessment and the installation of the service, conditions can change especially if the site location is not a greenfield installation. New data applications can be added, usage levels may increase, user populations can fluctuate, and equipment configurations might be modified. So it s important to re-verify the original assumptions and results produced by the VoIP assessment. VoIP Lifecycle Management: Best Practices and Business Value -4-

5 Identify and correct service level variances: If there are any variances between the assessment benchmarks and the newly installed VoIP service, they must be reconciled. The solution might require troubleshooting a performance problem, making a configuration change or adding additional bandwidth to the link. In any event, it s better to resolve the matter before the formal sign-off. Document the final installation: Once the site is ready for hand-off to the customer or user group, a final report should be produced to capture the inventory of equipment that has been put into service and the service levels that have been verified and activated for the users. Best Practices Best practices for the service deployment stage are illustrated in Figure 4. They begin with a verification of the VoIP infrastructure. This means ensuring the phones are properly registered with the IP-PBX and able to successfully dialout or accept incoming calls using its native signaling protocol. Any mismatch between phones and their IP-PBX registration information should be corrected at this point. After the infrastructure has been verified and the phone installation authenticated, a report should be generated to capture the relevant data for each end-point device including its address, extension, user name, and revision level along with a timestamp when the data was recorded. This will provide a snapshot summary that will be useful for future asset management activities. Figure 4. Service Deployment Workflow After the service hand-off is complete, the operational parameters are established. These parameters include thresholds for acceptable and nonacceptable performance, periodic service quality measurements and diagnostic testing procedures for analyzing the network pathways between connected sites. Your VoIP management system should be capable of automating these procedures to minimize the dependency on expert personnel to perform these functions manually. A final deployment report should be produced to capture all aspects of the installation including equipment, configuration parameters, service level thresholds and call quality measurements so there is a record for the central operations team and a deliverable to present to the customer or user group as a formal sign-off document. VoIP Lifecycle Management: Best Practices and Business Value -5-

6 Business Value A repeatable installation formula will accelerate the pace of service deployment. Rapid deployment of VoIP services will improve the business value of VoIP initiatives for both enterprise and service provider organizations. Greater consistency of service quality can be achieved by instituting a formal process for validating VoIP call quality and authenticating the installation of user devices with centralized IP-PBX systems. By comparing pre-deployment benchmarks with real user service levels at installation time, variances can be addressed by on-site personnel before they become larger problems that must be solved later by your support organization. Figure 5. Service Deployment Business Value Viola Networks believes this methodology will increase your speed of VoIP deployment by 50% or more. Faster deployment means a faster transition to low-cost VoIP services for enterprise IT organizations. It also means a faster revenue ramp for service providers. VoIP management tools can lower the cost of deployment by keeping your expert technical staff at home and off the road. Remote trouble-shooting tools are useful for diagnosing problems at a remote site with automated end-to-end route quality testing of network links and a centralized dashboard to view the results. With the right VoIP management tools and the best practices for using them, a small installation team can streamline the deployment process while improving the quality and consistency of the VoIP service deployment. The Operational Support Stage Once your VoIP service is in full production, having an efficient support operation can spell the difference between a technology nightmare and a service management dream come true. In the operational support stage of the VoIP management lifecycle, best practices feature the use of proactive management techniques to detect and correct problems before there is an impact on the user experience. Early detection of service degradation combined with rapid fault isolation is a simple concept that turns reactive fire-fighting into a proactive problem resolution process. However, putting a proactive management system in place requires sophisticated management tools and instrumentation on the appropriate locations in the network. VoIP Lifecycle Management: Best Practices and Business Value -6-

7 In addition to problem management, the operational support stage also includes best practices and management tools for service level monitoring and reporting. Your customers and user constituencies need to know that their service organization (the company s internal IT organization or an external service provider) is delivering voice and data services at an acceptable level of quality. Increasingly, SLA s are being established between the service delivery organization and their customers, sometimes formally and sometimes informally. Whether it s a formal SLA contract with cost penalties for violations or an informal agreement to provide service quality to a minimum performance standard, your organization needs the metrics, monitoring tools and reporting systems to track service levels and communicate the results to your customers. Requirements Monitor service levels: Operational support begins with monitoring the service levels for users, locations and business entities. The monitoring system may use active testing, passive monitoring or both. Metrics for VoIP call quality, including MOS scores and packet impairments (delay, loss and jitter), must be tracked continuously for real-time analysis or historical trending purposes. Detect service degradation: Early detection of service degradation is based on the proper setting of thresholds and the ability of the monitoring system to define thresholds intelligently (marginal vs. severe degradation, violations within a given timeframe, etc.). This will provide advance notification of service problems while avoiding the problem of false positives. Identify root cause: The task of identifying root cause can be simplified and accelerated with the use of automated end-to-end diagnostics of the call path. Whenever service problems are detected, the endpoints are identified and the network routes connecting them can be immediately analyzed while the problem is occurring. The results of an immediate route quality test can reveal the cause and location of the problem in minutes instead of hours or days. Best Practices The diagram in Figure 6 illustrates the workflow for the operational support stage of the lifecycle management framework. This workflow assumes a monitoring system for tracking call quality has been implemented and the operational parameters for service level thresholds have been defined in the system. Service problems can be detected in a number of ways. Alarms, triggered by threshold violations, are logged in system and appear on the service level dashboard. Problems can also be detected by user complaints. Ideally, the number of user complaints is small compared to the number of problems detected by the system. If there are too many user complaints, then thresholds should be set to a more sensitive value so that problems can be eliminated before the user is impacted by them. A third way that problems can be detected is by the technical support staff themselves. Their analysis of the trending reports might determine that traffic is spiking more frequently than normal and bandwidth is getting dangerously close to its upper limit. In this case corrective action can be taken proactively. The VoIP management system should employ active testing capabilities that perform end-to-end testing of the relevant network route(s) to examine route quality and to identify any problems in the routers along the path. Each router might be polled automatically to gather its health and performance data. Thresholds can be established for any of their critical device attributes. CPU, memory, and port utilization statistics provide visibility into how well the device is handling the offered load. Packet statistics provide information on traffic volume and error rates. Delay, loss and jitter statistics are especially relevant to VoIP quality and they too can be examined for each router. VoIP Lifecycle Management: Best Practices and Business Value -7-

8 In addition to active testing, the VoIP management system can also employ passive monitoring capabilities to gather information about the real user experience. This is accomplished by gathering call data records (CDR s) from the IP-PBX or gathering information directly from the IP phones. Thresholds can be applied to this information as another way to detect service problems and route quality testing can be invoked in response to these passive threshold violations. Figure 6. Operational Support Workflow CDR information is used to assist support personnel in their response to user complaints by providing a call diagnostics database that can be mined ondemand. If a user calls the help desk to complain about call quality, support personnel can have immediate access to call quality reports that detail the user s calling history and the diagnostic data relevant for each call. CDR data will help identify which calls were bad, when they were made and what source and destination phones where involved in the bad calls. Armed with this information, end-to-end troubleshooting tests can be launched to diagnose the infrastructure and pinpoint the origin of the problem. After the correction is made, problem resolution can be verified with a re-check of VoIP service quality for the affected users and by re-running the route quality tests over the affected links. Another important aspect of the operational support stage is service level monitoring and reporting for customers and user constituencies. Service level reports are used to verify that VoIP services are meeting SLA commitments or internal call quality standards. It should be possible to monitor call quality and aggregate the results separately for different user groups (based on IP-PBX or subnet assignments), network links, geographic locations or business affiliation (e.g., different customers served by a single service provider) and the reports should present the results according to the desired time interval (daily, weekly monthly) and compared with the service quality targets. Viola Networks has developed a Service Level Index (SLI) that provides a flexible metric for aggregating MOS scores into a single value that provides a ratio of bad calls to total calls. The SLI becomes the pass-fail metric used to evaluate the service levels delivered to each user group or customer location. VoIP Lifecycle Management: Best Practices and Business Value -8-

9 Business Value The operational support stage offers major opportunities for realizing business value from VoIP lifecycle management. By implementing the best practices described in this paper using a VoIP management solution, your organization will significantly reduce the cost of supporting VoIP services and service providers will be able to increase VoIP related revenues in a number of ways. Figure 7. Operational Support Business Value On the cost reduction side of the business value equation, your support team will see a reduced number of customer support calls when you implement a proactive management approach. Problem detection will be based on system generated alerts instead of user complaints. Diagnostics will be immediate and automated with testing focused on specific endpoints identified by the alerts. As a result, your existing technical staff can be utilized more efficiently and future headcount can be deferred or dedicated to new projects. Proactive management will also reduce downtime thereby improving end-user productivity and raising the level of customer satisfaction. On the revenue improvement side of the equation, service providers will reduce customer churn and thereby protect a very important recurring revenue stream. It goes without saying that the cost of acquiring a new customer far outweighs the cost of keeping an existing one. If best-in-class service and support is a key to retaining customers, then a proactive service quality management system is a critical element in building a world-class service and support capability. Reducing customer churn by just a few percentage points will probably pay for your VoIP management system in the first year alone. Other ways service providers can increase revenue is by offering premium services and tiered pricing options. Premium service offerings can take the form of gold or platinum level support programs that promise enhanced support (such as 24x7 coverage or accelerated response time). This premium service is enabled by an intelligent on-premise diagnostic appliance that monitors the network and triggers an escalated response from a centralized support center. Tiered pricing options can be introduced as well. The SLI metric described earlier could be used to match a higher grade of service quality to higher priced plan. For service providers, retaining customers while increasing the average revenue per customer is a formula for success. VoIP Lifecycle Management: Best Practices and Business Value -9-

10 The Capacity Planning Stage As you expand the VoIP environment to cover larger user populations and broader geographies, the network will need to scale in proportion to its usage levels and bandwidth requirements. Network traffic is constantly changing in response to user demand and the emergence of new applications or services. So, without the right tools and methodologies to understand capacity requirements, there is a strong risk of over-building or under-provisioning the network infrastructure. In the capacity planning stage of our lifecycle management framework, best practices revolve around the use of historical management data for trending analysis purposes. The comparison of key performance indicators, such as call volume, call quality and network utilization will tell us when, and how often, the demand for bandwidth is exceeding its supply. Operating a network at 20% or 30% utilization is costly and inefficient. Similarly, running at 80% or 90% utilization, even for brief periods of time, can be disastrous for both voice and data services. Your VoIP management system should provide the data-mining and reporting capabilities to help you to optimize the network for delivering maximum service quality at minimum cost. Requirements Analyze service level trends: The key performance indicators used for capacity planning (e.g., call volume, MOS scores and utilization rates) must be aggregated for various time intervals and network locations. Data should be archived for a reasonable period of time (several weeks or months of data) and used to compile periodic trending reports. Reports should be flexible in how they present the data - allowing them to be used for either internal analysis or external reporting. Anticipate the traffic peaks: Capacity planning reports must present the minimum, maximum and average values for key performance indicators. Minimum and maximum values are used to manage the peaks in traffic fluctuations. Average values provide the trending information used to anticipate the point in time when usage levels routinely over-run the available bandwidth. Optimize network capacity: Historical trends can be used to predict future growth and your VoIP management system can be used to model that growth with simulated voice and data traffic. Based on these projected bandwidth requirements, you can provision the network with the optimum level of increased capacity. Best Practices The best practices for capacity planning are illustrated by the workflow diagram in figure 8. They show the steps to be taken in building the right capacity planning reports to suit the needs of your organization. Those needs can vary depending on your business environment and the use of the information. A flexible reporting system will allow customization for multiple requirements. For internal purposes, capacity planning reports can be prepared in granular detail and with the errors, impairments and threshold violations that paint a complete picture of network behavior. For external purposes, capacity planning reports can be prepared in summary form, presenting only the relevant data that is appropriate for customers and user constituencies to see. Use these reports to monitor trends in usage and overlay this data with the changes in network utilization. Look for an impact on voice call quality as usage goes up. Initially the peaks in call volume will be reflected as occasional dips in minimum MOS scores. Eventually the average number of VoIP calls will increase to a point where the average MOS values decline. At this point the impact on service degradation will be felt by a significant number of users. Capacity planning reports can help you to identify the peaks and anticipate the need for more bandwidth before you reach the point of widespread user impact. VoIP Lifecycle Management: Best Practices and Business Value -10-

11 Figure 8. Capacity Planning Workflow A comparison of network utilization rates and VoIP call quality will tell you when the combined effect of voice and data traffic is beginning to impact end-user call quality and the extent of the impact. A comprehensive analysis of service quality, including the effect of packet delay, loss and jitter, will provide insights as to when it s more appropriate to reconfigure network or IP-PBX parameters for QoS or Call Admission Control as opposed increasing the capacity of the infrastructure. Business Value Optimizing the infrastructure is a major source of business value for any kind of service delivery organization, whether it s an internal IT department for a large enterprise or the engineering staff of a VoIP service provider. In the either case, the ability to lower infrastructure cost can produce major savings in both operating expenses and capital expenses. For service providers, there are also opportunities to generate more income from add-on revenues by selling more bandwidth to customers and by avoiding charge-backs from SLA violations. Figure 9. Capacity Planning Business Value VoIP Lifecycle Management: Best Practices and Business Value -11-

12 First let s look at lowering the cost of the infrastructure. Savings in operating expenses will come from more effective provisioning of backbone and access links for optimizing the network for convergence. Backbone connections have the potential for a high recurring cost for service levels required to meet the bandwidth and availability needs of mission critical voice and data applications. Access links are provisioned in high volume as they bring service to every remote office location so a small savings for each link will be multiplied across the organization and the savings will be realized on a monthly basis. Upgrading the routers and switches required to terminate all of those links is reflected in capital expense savings. If you are a service provider (or an IT organization that internally charges for services), the ability to bill customers at a higher rate for higher bandwidth is enabled by capacity planning reports. Concrete data showing increased usage, higher network utilization and the early signs of VoIP service degradation will provide the ammunition you need to charge at a higher rate. Managing the expansion of VoIP usage and network capacity proactively versus reactively will turn potential SLA violations into new revenue opportunities. Your VoIP management system should be the primary tool used to identify network efficiencies that will unlock costs savings while providing justification for charging appropriate fees to meet your customer s voice quality standards. Essential Ingredients for VoIP Management Systems The best practices presented in this paper represent concepts and methodologies for managing the four stages of VoIP lifecycle management. When implemented with the right VoIP management system, they translate into real business value for your organization while enhancing your ability to deliver high quality VoIP services. When selecting a VoIP management solution, there are several key ingredients important for the successful implementation of these best practices. They are described below. Multi-vendor VoIP management Vendor-specific tools restrict your ability to implement best practices in a manner that is independent of the equipment in your infrastructure. If you change vendors or add new ones, your ability to realize the same business benefits will be compromised. For example, when collecting IP-PBX data for monitoring the user experience, data structures and protocols can vary from one manufacturer to another. Supporting a single vendor can lock in the value for just one type of equipment. Be sure to choose multi-vendor VoIP management tools to avoid these problems. Integrated lifecycle management applications Avoid stand-alone point products for VoIP assessment, monitoring, troubleshooting and reporting. Your ability to leverage management information across the stages of the VoIP management lifecycle will be limited. You will want to use the results of your pre-deployment assessment as the benchmarks for monitoring service levels once the service is in production. The troubleshooting tools you learn to use during the assessment stage can be re-used during the operational support stage for resolving service problems. Real-time monitoring data can be archived and aggregated for analysis by the capacity planning tools. These and many other examples illustrate the value associated with integrated lifecycle management applications. VoIP Lifecycle Management: Best Practices and Business Value -12-

13 Remote management and concurrent operations Remote management is the ability of the VoIP management system to perform assessments, conduct diagnostic testing and gather passive data when managing remote user locations all under the control of a centralized system. Remote management eliminates the need for technical personnel to travel on-site when engaged in these activities. Concurrent operation refers to the ability of the VoIP management system to perform assessments for multiple sites or customers at the same time. It also refers to the ability of the system to be your pre-deployment assessment tool, real-time monitoring tool and capacity planning tool at the same time. This allows you to leverage expert personnel letting one do the work of many. Active testing and passive monitoring Active testing refers to the use of distributed agents to simulate VoIP calls when performing assessments test the call quality between sites on a deterministic basis and initiate end-to-end diagnostic tests when threshold limits are crossed. Passive monitoring refers to SNMP-polling of network devices for gathering health and performance data as well as passively polling IP-PBX systems and phones for call quality data. Both capabilities are required to provide a holistic view of VoIP service levels and the infrastructure used to deliver them. Combined voice and data management On a converged infrastructure, voice and data services do not operate in a vacuum. It is often the interaction of these two worlds that creates an undesired impact on one class of service or the other. Whether its pre-deployment assessment, postdeployment problem solving, or on-going network optimization and capacity planning, it s important to have some level of data service management integrated with the VoIP management system. The VoIP management system should be able to simulate voice and data traffic, test and troubleshoot converged network pathways and analyze service level trends for voice and data applications whenever they use a common infrastructure. Modular, scalable and easy-to-use software Finally, your VoIP management system must scale with your VoIP service delivery environment. That means starting small as you design and deliver your first pilot system. It also means scaling up to support global deployments with unlimited user populations. The software used by the system should be modular, allowing a pay as you grow strategy for expanding the system. It should use a distributed architecture, so there are no performance bottlenecks that restrict the ability of the system to perform multiple management tasks or to aggregate large volumes of management information. It must also be easy to use so that your technical support staff can readily transfer their knowledge of supporting data applications to the challenge of operating a VoIP service efficiently and cost-effectively. The Viola Networks Lifecycle Management System The capabilities described above represent essential ingredients for a comprehensive VoIP lifecycle management system. Viola Networks has developed a solution called NetAlly Lifecycle Manager for VoIP service level management that incorporates all of these ingredients. It is designed for enterprise and service provider environments. NetAlly provides business value at the earliest pre-deployment stage and continues to offer additional benefits throughout the complete VoIP lifecycle management process. VoIP Lifecycle Management: Best Practices and Business Value -13-