John Inverso Product Report 29 September 2003 NetIQ Chariot and Application Scanner for Application Performance Management Summary NetIQ Chariot tests network equipment by generating application-layer traffic and measuring performance. Application Scanner enhances it with custom scripts for evaluating specific network elements. Table of Contents Overview Analysis Pricing Competitors Strengths Limitations Insight List Of Tables Table 1: Overview: NetIQ Chariot and Application Scanner Table 2: s and s: NetIQ Chariot Table 3: s and s: NetIQ Application Scanner Gartner Reproduction of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice.
Corporate Headquarters NetIQ Corporation 3553 N. First Street San Jose, CA 95134, U.S.A. Tel: +1 408 856 3000 Fax: +1 408 273 0578 Toll free: +1 888 323 6768 Internet: www.netiq.com Overview Table 1: Overview: NetIQ Chariot and Application Scanner Product Type Application testing Product Performance and application monitoring and simulated transaction generation Operating Systems Unix, IBM Advanced Interactive Executive (AIX), OS/2, Multiple Virtual Storage Supported (MVS), Novell NetWare, HP-UX, Linux, Microsoft Windows (3.1, 95, 98, Me, NT, 2000, XP), Silicon Graphics (SGI) IRIX, Sun Solaris, Ixia, Compaq Tru64 Unix, FreeBSD, SCO UnixWare Platforms Supported Windows, 2000, NT, XP Base Price $4,000 Target Markets Large-scale enterprises Table 2: s and s: NetIQ Chariot Performance Monitoring Application Monitoring and Testing Evaluates the performance of networked applications and network segments by emulating transaction traffic and measuring how the network and applications respond. Uses software agents (Performance Endpoints) distributed on computers throughout the network to test specific network segments by imitating real-world traffic and gathering associated statistics. Endpoints measure the response time, throughput and other performance metrics and send the test results back to Chariot Console. Chariot also measures jitter, delay and lost data for streaming applications. Using the Voice over IP (VoIP) Test Module, the user can also measure overall call quality. Chariot Console sends instructions to the Endpoints, directing them how to emulate a designated application. The instruction scripts specify the type and amount of data the Endpoints should send and receive as well as when to connect and disconnect to the application and what delays they should use to mimic actual end-user traffic. Running the application scripts, the Endpoints drive the network protocol stacks and generate traffic like an end user would. 29 September 2003 2
Table 2: s and s: NetIQ Chariot VoIP Testing Service-Level Management Reporting Network Planning The VoIP Test Module for Chariot (VoIP Test Module) enables users to test VoIPenabled network equipment and to troubleshoot and tune network performance in preparation for Voice over IP. Network managers can customize application scripts to measure the service levels of specific applications to ensure that quality-of-service demands are met. Provides statistics on the short- and long-term performance of designated applications and network segments that help network managers identify performance problems and network faults. A VoIP Test Module measures an overall Mean Opinion Score (MOS) and the network statistics behind the score, which provides information for determining the ideal network architecture to handle VoIP before or after deployment. Provides baseline tests on overall network and application performance and predicts how the network and applications will respond to new users and other changes. Table 3: s and s: NetIQ Application Scanner Network Planning Application Management Develops custom scripts to project application responses before deploying new systems and users. Can develop scripts that tell the Endpoints how to test application traffic by tracing actual transactions. Analysis NetIQ s Chariot delivers performance management by conducting ongoing tests of how network elements respond to simulated transactions. Chariot approaches network management from the end-user perspective by emulating real-life transactions from the actual network applications, devices and supporting systems. Chariot mirrors application traffic as it generates bi-directional traffic that can vary along the transmission, just like real traffic would, as opposed to simulated transmissions that provide a constant, unwavering data stream. Chariot interacts with the protocol stack instead of just generating packets. These capabilities enable network managers to test sophisticated technologies that vary from traditional transactions, such as a complex SAP R/3 transaction, to streaming multimedia transmissions. Because these transactions are both resource-intensive and mission-critical, they rank as a top priority to senior management as well as to the IT department. Unfortunately, they are also prone to reliability and quality problems, making them even more difficult to effectively manage. Chariot can measure the jitter, delay and lost data that often accompanies these technologies as well as overall network performance. NetIQ offers a range of options that extend Chariot s usefulness in evolving networks. For example, the VoIP Test Module enables network managers to test VoIP-enabled network equipment to ensure it can effectively handle the anticipated traffic demands. In addition, NetIQ s Application Scanner enhances Chariot s ability to handle specific environments without requiring the programmers to perform extensive customization before it can be implemented. Network managers can trace applications to develop custom scripts, and they can even produce scripts from available line traces. To troubleshoot problems, Chariot can run on-demand trace routes on all leading Windows, Unix and Linux operating systems. 29 September 2003 3
These capabilities enable network managers to not only test network performance, but also to tune network systems, tweaking components where necessary and reallocating resources to handle peak demands. Network managers can potentially use the Chariot/Application Scanner combination to spot the elements contributing to performance degradation and upgrade them before performance falls below acceptable thresholds. The reverse should also occur, as network managers will not rush to replace components in hopes of boosting network performance, but rather discern faulty from overburdened elements and act accordingly, reallocating resources and adding new ones when necessary. Pricing Chariot Console licenses start at $4,000. Endpoints are free. Volume discounts are available. GSA Pricing Yes. Competitors NetIQ offers point solutions that compete directly and indirectly with a host of network testing and troubleshooting products. The testing tools include hardware solutions from companies such as Spirent, Ixia and Agilent. Chariot competes less directly with diagnostic tools, such as analyzers from Network Associates, Fluke and Finisar. Strengths Scalability Chariot can emulate the traffic of large enterprise networks, testing up to 10,000 connections on a userdefined basis. These connections represent the traffic generated by hundreds of thousands of end users, with a mix of applications and protocols. Chariot also pulls from a library that exceeds 125 application scripts to emulate actual application performance. The scripts include applications such as Web browsers, FTP, Exchange, Citrix, Active Directory, Baan, PeopleSoft and SAP. The Application Scanner expands Chariot s scalability by enabling the network manager to create custom scripts for emulating additional applications and tailoring testing procedures. Proactive Monitoring Chariot promotes proactive planning by simulating multiple types of network traffic over standard enterprise transmissions to cutting-edge VoIP and streaming multimedia transmissions. Chariot s application scripts enable network managers to proactively test how new applications and additional users will affect network traffic. Because they are based on the network s actual usage and performance, they provide a customized evaluation of the new element s potential impact. Flexibility Chariot and Application Scanner feature flexible and easy-to-use interfaces. Chariot s Visual Test Designer lets managers add hundreds of endpoint pairs. Application Scanner offers a wizard to help users define a new application script and develop custom scripts. The Script Editor extends this flexibility by allowing users to modify script variables to handle specific testing and monitoring tasks. Limitations Lacks Extensive Report-Writing Capabilities 29 September 2003 4
Chariot collects, analyzes and presents the information needed to ensure effective network performance and to identify trouble spots where resources may need to be reallocated. Although Chariot delivers excellent short-term trending statistics for optimizing the network, it lacks the comprehensive reporting capabilities needed for simulating network activity over the long term. Chariot would have to integrate with a supplemental reporting tool to provide this capability. Limited Interoperability Application Scanner currently supports only NetIQ Chariot and End2End Performance Monitor. The technology behind the Application Scanner could substantially ease the implementation process with its automated scripting capability for other NetIQ products as well as complementary third-party systems. Insight NetIQ Chariot has monitored and predicted network performance at large-scale enterprises for years. Its performance management technology can handle traditional transmission technologies and hardware systems as well as adapt to newer technologies, such as VoIP and streaming video. Application Scanner leverages Chariot s management potential by automatically creating scripts. These scripts can be useful for network managers, as they do not require tedious and lengthy script writing to monitor network elements. Chariot and Application Scanner can provide an affordable alternative to a standard management framework or resource-intensive homegrown solution. 29 September 2003 5