SmarteLoad Competitive Comparison

Transcription

1 SmarteLoad Competitive Comparison Subject SmarteLoad LoadRunner Performance Methodology Technology Foundation Test Suite Optimization General Features Execution Scheduler tool (per Group - ramp up, Ramp down, duration) Synopsis/Scenario Execution Run-Time Settings Automated Transaction Measurements Enter Transaction Names During recording Record Init, Actions & End Components Pacing and Iteration Configurations Monitor Users Progress - Record and Playback well-organized user-friendly grid script with server requests and responses in easy to read format - Automated script conversion from existing SmarteScript scripts - Automated conversion from other vendors functional tools - No coding required Conversion and migration use consistent flow which was implemented in the functional test but enhanced for performance testing usage. - Record and Playback Tree and script views - No conversion or migration - Coding required for most tools Users have to re-record scripts in a Load testing tool even if they have existing functional scripts - Re-inventing the wheel

2 Monitor Synopsis Status & Execution Progresses Real-Time Analysis Graphs Real-Time Analysis Summary Display Statistics upon Graph Selection Real-Time Statistics Host Generator Limitation Does Not Required Script Coding Script Conversion From Functional/Regression tool Data Generation Tool Automated Remote Agent Installations Dynamic Data Correlation Methods Complex Dynamic Correlation (boundary change) Complex Sequential Dynamic Correlations with Numerous Embedded Correlations NO Use as many Host Generator Agents as needed Remote Host Agent installation is performed from remote through the Host Management facility Manual and fully automated process with undo option for all or specific correlation Fully automated process with advanced Regular Expression pattern matching approach Fully automated process with advanced Regular Expression pattern matching approach NO Several vendors charge more for additional Load Generator agent installations Script enhancements require coding Except LoadRunner User must manually install or schedule system admin to install; this causes delays in test environment readiness Manual process; automation is limited (doesn t find many correlations) which requires programming Controller Automated Transaction Automated transactions in the Controller; Manual Inserts of Start and End Transaction commands into

3 no need to go back to the script the script Implementing SyncPoints (multiuser intersect points for Stress Testing) Multi-Protocol Oracle RCS Protocol + HTTP/S Automated SyncPoints within the Controller; no need to go back to the script Includes latest Oracle RCS protocol (New Educational Protocol) Manually insert rendezvous commands into the script, going back from controller to script Multi-Protocol WinSock + HTTP/S Included.NET Protocol Recording Java Protocol Libraries and Customization (with Java) Analysis & Statistics Multiple Real-Time Graphs View Included Easily integrates with java libraries for additional scustom protocols using Java (Included) Require additional costs for extension and Virtual Users for.net protocol Maximum Number of Graphs in Real-Time 8 8 Some have fewer RT graphs Monitors Analysis View Analysis Summary Report Analysis Graphs Add and Merge Reporting Reports View Included in the tool s price no hidden costs Automatic merge and scale Additional costs per monitor Some have but not all features

4 Export Report to various Formats Also export to PowerPoint Some have but not all formats Transaction Summary Report Report Filtering Export Reports Script Generation Ease-of-Use, Readability & Consistency Grid Visualization system allows user-friendly interface with easy to read scripts and flow. Grid script displays server requests and responses in a clear page by page format. This allows consistent script generation for all users. Relies on engineers recording and updating the scripts. The process is inconsistent and subjective to the users. Each engineer may record and program in his/her own format. Script Customization (Uses JavaScript) Data Input Methods: Does not require programming, uses pre and post conditions JavaScript commands for functional and data validation as well as script enhancements Random & Sequential and Unique Data Testing Import & Export Test Data Both Import and Export Import, most tools Investment Virtual Users limitation All Inclusive Price Resources Skill Level All virtual user pricing is the same for all protocols No hidden or additional costs for additional protocols Out of the box price includes all protocols, monitors and extensions No hidden or additional costs for additional components Basic QA knowledge is sufficient to operate. Testers, Developers and even Business Analysts can operate within hours. Minimal training, significantly short ramp up period to create and run a Synopsis, a suite of multi-user integrated The cost for virtual users depends on the type of protocol required for testing Requires additional costs for Protocols, Monitors and Extensions Requires lengthy training, ramp up period, advanced testers and programmers to create functions and customize. At least 3 months ramp up time just to learn how to script and use the tool properly.

5 performance/load scripts. Budget/Costs Performance/Load Test QA Infrastructure ROI Overall Testing Results Test Development Cycle Effectiveness & Efficiency Overall low costs per virtual user, project and time-to-test cycles. Example: SmarteLoad list price is a fraction of the LoadRunner price. Long-term solution with consistent performance testing methodology. Every tester/developer uses the same methodology and process Significantly high; increases over time due to testers increase in productivity Highly predictable script development throughout the performance test cycles. Scripts can be automatically generated from another tool. Overall high costs per virtual user, test project and time-totest cycles If testers leave the environment, performance testing has to be recreated (subjective to testers). Not a long term solution. Low due to high costs and ramp up time; tedious, timeconsuming maintenance Difficult to measure and predict script development and performance test cycles Subjectivity One unified methodology, high consistency, no subjectivity Highly subjective to QA engineer s own scripting methods Technical Support Technical Support On-site Available Technical Support Over the Phone Technical Support Over the Web