Best Practices: NetIQ Analysis Center for VoIP

Transcription

1 Best Practices: NetIQ Analysis Center for VoIP A White Paper for VoIP Quality July 19, 2005 Contents Overview: How Analysis Center works... 1 Getting started with the console... 2 Recommended VoIP Quality reports Conclusion: Report planning and execution Appendix A: Context tab functions The VoIP Quality reporting package enhances the report creation and analysis capabilities of Analysis Center 2.0 SP 1 by providing flexible access to voice over IP (VoIP) data. With the reader-friendly formats of the Operational and Service Levels reports, you can capture and distribute vital call quality information, such as MOS, R-value, jitter, data loss, and delay. This paper discusses the benefits inherent in reporting with Analysis Center, and highlights the steps you should take to get the most from Analysis Center in your VoIP environment. The VoIP Quality reports described inside can be added to Analysis Center by checking in the.xml files packaged in the Web downloadable file. These reports use data collected by the VoIP Quality module in AppManager. Note: This paper is intended to supplement, not replace, the Analysis Center User Guide.

2 THIS DOCUMENT AND THE SOFTWARE DESCRIBED IN THIS DOCUMENT ARE FURNISHED UNDER AND ARE SUBJECT TO THE TERMS OF A LICENSE AGREEMENT OR A NON-DISCLOSURE AGREEMENT. EXCEPT AS EXPRESSLY SET FORTH IN SUCH LICENSE AGREEMENT OR NON-DISCLOSURE AGREEMENT, NETIQ CORPORATION PROVIDES THIS DOCUMENT AND THE SOFTWARE DESCRIBED IN THIS DOCUMENT AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. SOME STATES DO NOT ALLOW DISCLAIMERS OF EXPRESS OR IMPLIED WARRANTIES IN CERTAIN TRANSACTIONS; THEREFORE, THIS STATEMENT MAY NOT APPLY TO YOU. This document and the software described in this document may not be lent, sold, or given away without the prior written permission of NetIQ Corporation, except as otherwise permitted by law. Except as expressly set forth in such license agreement or non-disclosure agreement, no part of this document or the software described in this document may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, or otherwise, without the prior written consent of NetIQ Corporation. Some companies, names, and data in this document are used for illustration purposes and may not represent real companies, individuals, or data. This document could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein. These changes may be incorporated in new editions of this document. NetIQ Corporation may make improvements in or changes to the software described in this document at any time NetIQ Corporation, all rights reserved. U.S. Government Restricted Rights: If the software and documentation are being acquired by or on behalf of the U.S. Government or by a U.S. Government prime contractor or subcontractor (at any tier), in accordance with 48 C.F.R (for Department of Defense (DOD) acquisitions) and 48 C.F.R and (for non-dod acquisitions), the government s rights in the software and documentation, including its rights to use, modify, reproduce, release, perform, display or disclose the software or documentation, will be subject in all respects to the commercial license rights and restrictions provided in the license agreement. Check Point, FireWall-1, Provider-1, SiteManager-1, and VPN-1 are trademarks or registered trademarks of Check Point Software Technologies Ltd. ActiveAgent, ActiveAnalytics, ActiveAudit, ActiveReporting, ADcheck, AppAnalyzer, Application Scanner, AppManager, AuditTrack, Chariot, ClusterTrends, CommerceTrends, Configuration Assessor, ConfigurationManager, the cube logo design, DBTrends, DiagnosticManager, Directory and Resource Administrator, Directory Security Administrator, Domain Migration Administrator, End2End, Exchange Administrator, Extended Management Pack, FastTrends, File Security Administrator, Firewall Appliance Analyzer, Firewall Reporting Center, Firewall Suite, Ganymede, the Ganymede logo, Ganymede Software, Group Policy Administrator, immarshal, Intergreat, Knowledge Scripts, MailMarshal, Marshal, Migrate.Monitor.Manage, Mission Critical Software, Mission Critical Software for E-Business, the Mission Critical Software logo, MP3check, NetIQ, the NetIQ logo, the NetIQ Partner Network design, NetWare Migrator, OnePoint, the OnePoint logo, Operations Manager, PentaSafe, PSAudit, PSDetect, PSPasswordManager, PSSecure, Qcheck, RecoveryManager, Security Analyzer, Security Manager, Security Reporting Center, Server Consolidator, SQLcheck, VigilEnt, Visitor Mean Business, Vivinet, W logo, WebMarshal, WebTrends, WebTrends Analysis Suite, WebTrends for Content Management Systems, WebTrends Intelligence Suite, WebTrends Live, WebTrends Log Analyzer, WebTrends Network, WebTrends OLAP Manager, WebTrends Report Designer, WebTrends Reporting Center, WebTrends Warehouse, Work Smarter, WWWorld, and XMP are trademarks or registered trademarks of NetIQ Corporation or its subsidiaries in the United States and other jurisdictions. All other company and product names mentioned are used only for identification purposes and may be trademarks or registered trademarks of their respective companies.

3 Overview: How Analysis Center works Analysis Center is designed to import raw data from multiple AppManager repositories, transform that data into useful information about the computing infrastructure that supports your business, and publish that information in the form of reports. In general terms, the Analysis Center architecture is a structure whereby data is copied from AppManager repositories to Data Marts, and from the Data Marts to the Analysis Center Data Warehouse. Queries of the warehouse are made from the Analysis Center Console. Data Mart. There is one Data Mart for each AppManager repository you identify as a source of raw data. A Data Mart serves several purposes: Data copied from the AppManager repository is stored for processing. Stored procedures process the copied data. Processed data is stored in fact and dimension tables. The fact tables function as a component of the Data Warehouse. Data Warehouse. The Data Warehouse is made up of a relational database and a multidimensional database. The relational database (AC_Warehouse) serves as the immediate source of data for the multidimensional database. The relational database is managed by SQL Server 2000, and contains, among other things, the dimensional data copied from the Data Marts and views linked to the fact tables in the Data Marts. The multidimensional database (AC_OLAP) is managed by SQL Server 2000 Analysis Services, Values are stored in cubes designed to facilitate the rapid query of large amounts of data. A cube is a structure that organizes data in hierarchical dimensions. Dimensions describe data from a particular point of view (for example, the computer and application from which it was generated, or the date and time during which it was generated). The hierarchical structure of dimensions lets you select data from any point in the hierarchy (for example, data for all machines, data for certain Server groups, or data for individual computers). You use the Context tab of the Analysis Center Console to control the dimensions that will filter the data that you want to include in the report. We ll discuss the use of the Context tab in greater detail throughout the rest of this paper. SQL Server functions. The SQL Server Agent on the Data Mart server reads from the AppManager repository and writes to the Data Mart and Data Warehouse relational databases. This process is facilitated by DTS packages that run as SQL jobs on the Data Mart server. The SQL Server Agent on the Data Warehouse server creates and reads from the Data Mart databases, reads from and writes to the Data Warehouse relational databases, and processes the cube in the Data Warehouse multidimensional database. These activities are facilitated by DTS packages that run as SQL jobs on the Data Warehouse server. Best Practices: Analysis Center for VoIP 1

4 Getting started with the console The following topics provide brief how-tos for working with Analysis Center for VoIP Quality. Customizing an original VoIP Quality report on page 2 Understanding folders on page 8 Deploying reports to Reporting Services on page 8 Making the most of dashboards on page 9 Customizing an original VoIP Quality report Analysis Center includes a number of application-specific reports that you can use as-is or alter to suit your needs. These original reports are scattered throughout various folders within the Reports > Application Specific > VoIP Quality folder in the Navigation pane. Key to understanding and troubleshooting VoIP quality is to know which network segments are involved in the trouble spots. VoIP Quality metrics are collected for each stream, which is defined by a talker and listener pair of endpoints. In the following examples, we ll walk through the steps involved in creating a good-acceptable-poor (GAP) report and a maximum jitter report. Before creating a GAP report, make sure that you ve run the AppManager VoIPQuality Knowledge Script for the codec(s) that you re monitoring. The scripts supply the data that your GAP report will use. MOS GAP by Stream This report presents GAP for each selected stream (talker/listener pair). In this example are three locations, with an agent at each location: Raleigh, San Jose, and Houston. For each location, a VoIPQuality_CallPerf_G711u test is running to the other two locations. 1. In the Navigation pane, navigate to the Reports > Application Specific > VoIP Quality > Operational folder and select VoIP Quality Good-Acceptable-Poor Metric by Stream. Notice that once you ve selected a report, the Context, Tasks, and Properties tabs appear at the bottom of the Tasks pane, which is on the right side of the Console. The tabs don t appear until after you ve selected a report in the Navigation pane. Note: To view a detailed description of the report, right-click on the report title and select Configuration Card. The Overview section contains a description of the report s functions and frequently offers suggestions for customizing the report. The rest of the Configuration Card is a graphical representation of the configured contexts and parameters that make up the report. 2 White Paper

5 2. If you re running VoIP tests for multiple codecs or if you re collecting data with both CallPerf and CiscoSAA, then use the Knowledge Script context to select the Knowledge Script(s) that are generating the data you want included in the report. By selecting scripts, you filter the data for only the codec that you want to represent in your report. If you do not want to filter data based on whether CallPerf or CiscoSAA gathers it, or if you want your report to represent all codecs, then make no changes to the Knowledge Script context. 3. Click on the Context tab, select the Metric context and then scroll to the Mean Opinion Score of VoIP Call Performance metric. Select each stream (talker/listener pair). Make sure to deselect any other metric that is selected by default. You use the Metric context options to define the data for your report. The selections that appear are related to AppManager Knowledge Script data stream legends. 4. Click the Properties tab. In the Parameters section, change the Good Operator to > (greater than) and the Good Standard to Change the Poor Operator to < (less than) and the Poor Standard to 3.6. It s important to change the parameters because their values default to those for delay. To create a report for any metric other than delay, you ll have to change the parameters. Note: The Configuration Card provides the recommended standards for each metric. 5. Click Execute Report. The customized report appears in the Results pane. Best Practices: Analysis Center for VoIP 3

6 Each column in the chart represents the good-acceptable-poor MOS for a selected stream. To sort the columns by poorest MOS, click on the % Poor column in the table so that the sorting indicator points down. Sorting indicator After the report is sorted, it s clear that the problem areas are the network links between Houston and San Jose. Once you save it, this report will always sort from poorest to best. Streams with poor MOS 6. Once you ve finished customizing the report, click Report > Save Report As to save with a new title under Reports > Application Specific > VoIP Quality> Operational. Make the new title as explicit as possible it should tell you exactly what purpose this report has. You can t save a customized original report without giving it a new title. Once you ve saved the report, it appears in the Navigation pane. New report title appears in the Navigation pane. 4 White Paper

7 The new title also appears on the customized report in the Report pane. New title appears on report Contents of Description field from Properties tab Tip: To make the report s function obvious to users, make good use of the Description field on the Properties tab. Accurately describe what s shown in the report and include important parameters such as the date range the report covers, the metric being used, and the machines being included. Although these parameters are apparent in the Analysis Center Console, they re not visible to users once the report is deployed to SQL Server 2000 Report Services. Maximum Jitter by Hour This report identifies the maximum jitter for tests running between Houston and San Jose over the last seven days. In the previous example, we identified the San Jose and Houston streams as having the worst MOS. In this report, we ll pinpoint the times of day during which jitter is the worst over the San Jose/Houston network links. In addition, we ll compare the Houston/San Jose jitter with that of the jitter between Raleigh and San Jose/Houston over the last seven days. For each location, a VoIPQuality_CallPerf_G711u test is running to the other two locations. 1. In the Navigation pane, navigate to the Reports > Application Specific > VoIP Quality > Operational folder and select VoIP Quality Performance Data Metrics by Hour. Best Practices: Analysis Center for VoIP 5

8 2. If you re running VoIP tests for multiple codecs or if you re collecting data with both CallPerf and CiscoSAA, then use the Knowledge Script context to select the Knowledge Script(s) that are generating the data you want included in the report. By selecting scripts, you filter the data for only the codec that you want to represent in your report. If you do not want to filter data based on whether CallPerf or CiscoSAA gathers it, or if you want your report to represent all codecs, then make no changes to the Knowledge Script context. 3. Click on the Context tab, select the Metric context, and then scroll to the Jitter of VoIP Call Performance metric. Notice that we ve selected four streams: Houston to Raleigh, Houston to San Jose, San Jose to Houston, and San Jose to Raleigh. Make sure to deselect any other metric that is selected by default. 4. Select the Measures context. Deselect Average and select Maximum. 5. Select the Time context and change the Date Range to Last 7 Days. 6 White Paper

9 6. Click on the Properties tab. You use the Properties tab to set miscellaneous report properties such as NumericFormat, which controls the number of digits that are displayed after a decimal point. Because the Analysis Center jitter results are reflected in whole numbers, you ll want to change the NumericFormat property from F2 to F0. You can also use the Properties tab to format the look of your report. For example, you can change the chart to use columns instead of lines. 7. Click Execute Report. The customized report appears in the Results pane. The x-axis reflects the hours of the day in military time. The y-axis reflects the maximum amount of jitter (in milliseconds) that occurred during each hour. Jitter above 60 ms is considered poor. From this report, you can easily see that the Houston/San Jose network links experienced poor jitter from 7:30 AM to 6:30 PM. The Raleigh network links never experienced poor jitter. Notice the breakdown of jitter by stream Once you ve finished customizing the report, save it with a new title. Don t forget to change the report s description on the Properties tab. Best Practices: Analysis Center for VoIP 7

10 Understanding folders In the Navigation pane, folders are containers in which you can keep the reports that you create. You can use the folders to sort reports by report function (such as Service Levels), by the application that you re monitoring, by the recipient of the report (such as a specific administrator or business unit), or by myriad other factors that suit your VoIP needs. We recommend that you create a folder called Temporary, In-progress, Staging, or some other title that indicates that the folder is a container for reports that are not yet ready for prime-time. You can leave an in-progress report in the temporary folder as long you need, until you re done making all of the context changes necessary to produce a report that s exactly what you want. With a temporary folder, you can save a report that s not quite finished, come back to it later, and not have to wonder where you saved it. When you ve finished the report, you can save it in the permanent folder in which it belongs. Details about creating, moving, renaming, and deleting folders are described in the Analysis Center User Guide. Deploying reports to Reporting Services Analysis Center employs SQL Server 2000 Reporting Services to deliver reports that provide up-tothe-minute analyses of your VoIP environment. Once you ve configured a report to provide the information in which you are interested, you can then deploy that report to SQL Server 2000 Reporting Services to have it run periodically on the schedule you choose, and have each iteration of the report delivered via subscription (by or written to a file share) to one or more users. If you don t want to deliver a report by subscription methods, you can allow users to view the report using Web browser access to SQL Server 2000 Reporting Services. The deployed version of the GAP report you generated looks like this: For detailed instructions on deploying reports, refer to the Analysis Center User Guide. 8 White Paper

11 Making the most of dashboards Dashboards allow you to group a number of reports together on the same page, which makes it easy to see related data at a glance. For your convenience, Analysis Center contains a Service Levels report called VoIP Quality Service Levels Overview, which you ll find in the Service Levels folder in the Navigation pane. This dashboard report displays four separate Service Levels reports: VoIP Quality MOS Good-Acceptable-Poor VoIP Quality Jitter Good-Acceptable-Poor VoIP Quality Delay Good-Acceptable-Poor VoIP Quality Lost Data Good-Acceptable-Poor The dashboard presents a thumbnail version of each member report. Click on the title of a member report to view the full report. If you change a report in a dashboard, and then redeploy the report, you also need to redeploy the dashboard report. You are not limited to this single preconfigured dashboard report. Create as many as you find useful for yourself and other users. For detailed instructions on creating a dashboard report, refer to the Analysis Center User Guide. Best Practices: Analysis Center for VoIP 9

12 Recommended VoIP Quality reports In your VoIP environment there are certain metrics that you must monitor in order to keep your users happy with call quality and telephone service, which is undoubtedly the most mission-critical application running on your company s network. We recommend that you create, run, and deploy reports in the following categories. You ll find original reports in these categories within the Reports > Application Specific > VoIP Quality folders in the Navigation pane. These original reports have been configured to filter for VoIP Quality data, so you can use them pretty much right out of the box. Service Levels on page 10 Operational on page 11 Service Levels The reporting capability of Analysis Center enables organizations to demonstrate the value of IT and how well IT is aligned with business objectives. To these ends, run the following Service Level management reports to reflect VoIP availability and call quality. VoIP Quality Delay Good-Acceptable-Poor. Use this report to see a pie chart of good, acceptable, and poor values of delay. By default, this report uses the Delay of VoIP Call Performance data that is collected by the AppManager VoIPQuality_CallPerf_(codec) Knowledge Script. Good delay values are those below 150 ms.; poor delay values are those above 400 ms. To change these values, use the Parameters section of the Properties tab. Use the other contexts to filter data. For example, to show results for a particular codec type, use the Knowledge Script context to select the Knowledge Script running tests for that codec. VoIP Quality Jitter Good-Acceptable-Poor. Use this report to see a pie chart of good, acceptable, and poor values of jitter. By default, this report uses the Jitter of VoIP Call Performance data that is collected by the AppManager VoIPQuality_CallPerf_(codec) Knowledge Script. Good jitter values are those below 40 ms.; poor jitter values are those above 60 ms. To change these standards, use the Parameters section of the Properties tab. Use the other contexts to filter data. For example, to show results for a particular codec type, use the Knowledge Script context to select the Knowledge Script running tests for that codec. VoIP Quality Lost Data Good-Acceptable-Poor. Use this report to see a pie chart of good, acceptable, and poor values of lost data. By default, this report uses the Percent Lost Data of VoIP Call Performance that is collected by the AppManager VoIPQuality_CallPerf_(codec) Knowledge Script. The Good Standard is data loss below 0.50 percent; the Poor Standard is data loss above 1.00 percent. To change these standards, use the Parameters section of the Properties tab. Use the other contexts to filter data. For example, to show results for a particular codec type, use the Knowledge Script context to select the Knowledge Script running tests for that codec. 10 White Paper

13 VoIP Quality MOS Good-Acceptable-Poor. Use this report to see a pie chart of good, acceptable, and poor MOS (mean opinion score) values. By default, this report uses the MOS score of VoIP Call Performance data that is collected by the AppManager VoIPQuality_Call_Perf_(codec) Knowledge Script. Good MOS scores are those above 4.03; poor MOS scores are those below 3.6. To change this report to show R-value instead, use the Metric context to select R-Value of VoIP Call Performance, and then change the Good and Poor Standards defined in the Parameters section of the Properties tab. If you re using R-value, set the Good Standard to a score above 80 and the Poor Standard to a score below 70. Use the other contexts to filter data. For example, to show results for a particular codec type, use the Knowledge Script context to select the Knowledge Script used for that codec. VoIP Quality Service Levels Overview. This dashboard report provides an overview of underlying reports that reflect key VoIP quality metrics. The member reports show the good-acceptable-poor levels of MOS, jitter, delay, and lost data. Click on the title of any member report to see the full view of that report. When deploying this report, be sure to deploy each member report first. Operational The operational side of your organization may not be the flashiest aspect, but it s probably one of the most vital in terms of VoIP functionality. Operational reports provide the details behind the Service Level management reports. Creating reports helps you isolate the servers that are experiencing problems. Analysis Center offers several original Operational reports that allow you to quickly create what you need. VoIP Quality Availability. Use this report to see the availability of the various VoIP Quality or call setup tests that are being run by AppManager. Use the Metric context to select the type(s) of tests you are interested in; expand the Metric folder to select the specific instances (streams) for which you want to see the availability. This report presents the availability of your VoIP Quality Knowledge Script test results as a percentage. A test is considered unavailable if the Knowledge Script cannot run for any reason, such as network connectivity. VoIP Quality Good-Acceptable-Poor Metric by Stream. Use this report to see the VoIP Quality metrics between each pair of endpoints (in other words a listener/talker pair). Use the Metric context to expand the individual metric you want to display and then select each individual stream instance. By default, thresholds are set for the Delay metric. Use the Properties section of the Parameters tab to adjust these thresholds for the metric you are including in the report. We recommend using the following thresholds. For MOS, good is greater than 4.03; poor is less than 3.60 For R-Value, good is greater than 80; poor is less than 70 For jitter, good is less than 40 ms.; poor is greater than 60 ms For delay, good is less than 150 ms.; poor is greater than 400 ms For percent lost data, good is less than 0.50 percent; poor is greater than 1.0 percent For percent jitter buffer loss, good is less than 0.50 percent; poor is greater than 1.0 percent Best Practices: Analysis Center for VoIP 11

14 VoIP Quality Performance Data Metrics by Date and Time. Use this report to easily compare multiple metrics by date and time. For example, you can compare the percent of lost data vs. the percent of loss due to the jitter buffer. Or compare multiple instances of the same metric, such as the average jitter between each of your VoIP Quality endpoints. Use the Metric context to select the metrics to include in the report. To select specific instances of a metric, expand the metric description and then select the instances that you want to include in the report. Use the Time context to set the time range and interval (for example, Last 28 Days by Day). The interval you select determines the time aggregation (for example, if you select Day, there is one value for each date; if you select Hour, there are 24 values for each date). Use the other context controls as data filters. For example, use the Group context to select which computers or groups to include in the report. VoIP Quality Performance Data Metrics by Hour. Use this report to examine individual or multiple metrics by hour of the day. For example, use this report to look at the average or maximum delay being measured between the VoIP Quality endpoints by the hour of the day. This type of report can give you insight into whether quality is being affected during specific times of the day. Use the Metric context to select one or more metrics to include in the report. To select specific instances of a metric, expand the metric description and then select the appropriate instances that you want to include in the report. Use the other context controls as data filters. Conclusion: Report planning and execution Now that you re a bit more familiar with the reports that are available, keep in mind the following suggestions before you begin executing and deploying reports. Before you do anything else, think about why you want to create a report and who ll be using it. 1. Decide what purpose the report will have. What information do you want the report to show? 2. Look through the original reports. Does one exist that s similar to what you want? Consider not only the reports within the VoIP Quality folder, but also the other Analysis Center original reports. 3. Look at the Configuration Card to see the report properties. Is this report still a good match? 4. If so, execute the report to see whether it actually does what you want. 5. If the report does what you want, save it under a new, content-specific title in an appropriate folder. 6. If necessary, deploy the report to SQL Server 2000 Report Services for access by other users. 7. If none of the original reports are suitable, look through the Templates folder to find a report that s structured the way you want. Save it under a new title in an appropriate folder, customize the report to present the data that you need, and then execute the report to check its function. Finally, deploy the report. 12 White Paper

15 Appendix A: Context tab functions This appendix provides a brief summary of each icon on the Context tab of the Tasks pane, and a short description of how to select objects (nodes) on the Context tab. The Context tab appears only when you have selected a report in the Navigation pane. The context of a report refers to the computers, data streams, and time frame that are the subject of the report, and may also include measures of the data, such as average or maximum. The multi-state tree control of the Context tab allows you to make a variety of different selections in order to create different orientations of the data. The same tree control is used for the Group, Data Source, Knowledge Script, and Metric contexts. Variations on the control icon, a blue and white square, represent the five different states of selection that can apply to a node in the tree. The following table defines and provides instructions for using the five variations of the tree control icon. Icon Description The parent node is not selected, and none of its children is selected. With this selection option, none of these objects will be included in your report. The parent node is explicitly selected; none of its children (if it has any) is selected. With this selection option, your report will display data only for the parent object. Click once to select only the parent node. The parent node is not selected, but one or more of its children is selected. With this selection option, your report will display data only for the child object(s) that you have selected. Ensure that the parent node is not selected, and then click each child node once. Best Practices: Analysis Center for VoIP 13

16 Icon Description The child node is selected by rule; none of its children is selected. To select a node by rule means to select the child node by double-clicking on the parent node. When you select a child node in this manner, you cannot change the child s selection or that of its children by clicking directly on their nodes. You must click again on the parent node to change the child s selection. With this selection option, your report will display data only for the child objects that are selected by rule. Click twice on the parent node to select the child node(s) by rule. Note This selection option applies only to the Group context. The parent node is explicitly selected, and one or more of its children is selected (by any method). With this selection option, your report will display data for the parent object AND any of the child objects that you have selected. Select the parent node, and then select one or more child nodes. 14 White Paper