Windows with Microsoft SQL Server Performance Load Testing on Clarity 7.5.1 Technical White Paper Version: 2.0 October 20, 2005
Table of Contents EXECUTIVE SUMMARY... 3 TEST ENVIRONMENT... 4 TEST RESULT SUMMARIES... 6 APPENDIX A: DETAILED LOADRUNNER TEST SCRIPTS... 7 APPENDIX B: USER RIGHTS... 8 APPENDIX C: MEDIUM USER BASE TEST DETAILED RESULTS... 15 APPENDIX D: SMALL USER BASE TEST DETAILED RESULTS... 17 Windows with Microsoft SQL Server Performance Load Testing on Clarity 7.5.1
Executive Summary Introduction In order to establish and publish performance benchmarks, Computer Associates Corporation conducted load tests of Clarity 7.5.1 on medium and small scale Windows environments. The objectives of these tests were to: Demonstrate Clarity s ability to serve small and medium user bases (1,000 to 2,000 users) on Intel based hardware using a Microsoft SQL database and Orion application servers, and to Establish performance benchmarks under load using a core set of application functionality on Intel based hardware and Windows environment. The load tests used a mixture of read and write actions to simulate user interactions. A concurrency rate of 20% was used, with an average wait time of 60 seconds between page requests; for the purposes of our load testing, the average wait time was accelerated to 15 seconds. The performance was measured for 30 minutes, excluding a ramp-up time of 15 minutes. The user-base tests used a single database server with two load balanced application servers. The actual experience of customers will depend on how their hardware is set up and how many Clarity modules they deploy. Results and Conclusion Clarity 7.5.1 can adequately support small user bases and medium user bases on dual (load balanced) application server Windows configurations. See the Test Environment section of this document for more information about test set-up. See Appendices C and D for detailed test results. Response times for the ninety percent of user activity typically measure less than 0.8 seconds. Measurement Small Load Windows Medium Load Windows Average transaction response time 0.366 seconds 0.4 seconds 90th percentile response time 0.481 seconds 0.531 seconds Average Throughput 67 Kbs 124 Kbs Average Database server CPU utilization 2.46 % 4.79 % Average Application server CPU utilization 7.58 % 15.41 % Disclaimer This white paper is not meant as a sizing guide or as a recommendation guide but instead portrays possible user loads given a particular Clarity configuration. We understand that customer requirements and environments differ. The Clarity functionality exercised during these performance load tests may or may not match the functionality that individual companies deploy. Computer Associates does not recommend that customers acquire hardware based on this white paper but instead seek the sizing expertise of our technical pre-sales and professional services personnel. The test environments outlined in this document do not consider reporting, process engine, and background service or failover requirements. Windows with Microsoft SQL Server Performance Load Testing on Clarity 7.5.1 Page 3 of 18
Test Environment Test Overview All tests were written using Mercury s LoadRunner tool set. Appendix A contains the detailed test scenario descriptions. Identical LoadRunner test scripts were run for all load tests (in the same ratios; see below). One Virtual User represents 20 real users (i.e. uses a 20% concurrency and an accelerated think time of 15 seconds, representing a normalized real world think time of 60 seconds). All runs were 30 minutes long, excluding ramp up/down time (approx 15 minutes). Test Scenarios The test scenario reflects a realistic mixture of User activities on the system. We use a combination of 5 navigation scripts and a timesheet creation script in the following ratios: Scenario General navigation. 30% Timesheet navigation. 25% navigation. 20% navigation 20% Timesheet creation 5% Weighting User Simulation Small this corresponds to the average Customer deployment size. Medium this corresponds to the average Customer deployment size. Size Platform Number of VUsers Registered Users Small Windows/MSSQL 50 1,000 Medium Windows/MSSQL 100 2,000 Test Data Set The data used for these tests was generated by Computer Associates; customer data was not used. The data was loaded by XOG and through the User Interface via Mercury WinRunner scripts. Because we used specific users for each test (and avoided using global rights), no users could see all the data elements. The data used for these tests is as follows: Element Quantity Notes s 5366 s 579 Timesheets 20,472 Timesheet reporting periods 53 Only 4 were open Portfolios 5 Programs 8 Ideas 9 Windows with Microsoft SQL Server Performance Load Testing on Clarity 7.5.1 Page 4 of 18
User Base Configuration The user-base test runs simulated the load that can be expected for customers with small and medium systems. The hardware to support this consists of two load-balanced application servers each running two application server instances. The table in the following diagram shows the details for the Windows environment. No SSL or LDAP services were running. Performance Environment Hardware Configuration Load Balancer 1 Foundry Networks ServerIron XL 24 Port Load Balancing Switch Switch Catalyst 2900 Windows AppServers SuperMicro, Win 2003, Dual 2.8Ghz, 4Gb RAM. Application and background services. MS SQL 2000 SuperMicro, Win 2003, Dual Database Server 2.8Ghz, 4Gb RAM. DB service. LoadRunner Controller LoadRunner Agents SuperMicro, Win 2003, Dual 2.8Ghz, 2Gb RAM Win 2K, 663 MHz, 260K RAM Windows with Microsoft SQL Server Performance Load Testing on Clarity 7.5.1 Page 5 of 18
Test Result Summaries Medium User Base Results Two SuperMicro application servers, both running two Clarity 7.5.1 FP02 application instances (1.5 GB JVMs), can support a simulated load of 2,000 registered users (100 virtual users). Average transaction response time: 0.4 seconds 90th percentile response time: 0.531 seconds Average Throughput: 124 KB per second Average Database server utilization: 4.79 % Average Application server utilization: 15.41 % See Appendix C for more detailed results. Small User Base Results One SuperMicro application server running a Clarity 7.5.1 FP02 application instance (1.5 GB JVM) can support a simulated load of 1,000 registered users (50 virtual users). Average transaction response time: 0.366 seconds 90th percentile response time: 0.481 seconds Average Throughput: 67 KB per second Average Database server utilization: 2.46 % Average Application server utilization: 7.58 % See Appendix D for more detailed results. Windows with Microsoft SQL Server Performance Load Testing on Clarity 7.5.1 Page 6 of 18
Appendix A: Detailed LoadRunner Test Scripts To minimize excessive log in and log out operations, virtual user scripts will iterate the login steps from the main script 2-3 times after log in. After the iterations complete, the virtual user will log out and the next virtual user will start. In other words, there is an inner loop between log in and log out. Navigation all links in the left pane: Log in as the link_viewer user. Click everything on the left navigation frame (Overview, Organizers, Report and Jobs, Account Settings, Portfolios, Programs,, Applications, Products, Idea). Log out. Timesheet Navigations: Login as perfuser01-59. User click timesheet link on personal view. Filtering for open timesheets (page down/up). Click first open timesheet. Click task. Click cancel. Log out. s navigation: Login as project_view. Click the projects link on the navigation frame. Click sort ascending and descending on project name column. Click sort ascending and descending on project ID column. Click to open the first project from the list. Click the project properties, team, tasks and process tabs. Log out. s navigations: Log in as resource_viewer. Click on the resource link. Click sort ascending and descending on resource name column. Click sort ascending and descending on resource ID column. Click to open the first resource from the list (resource_properties). Click tab. Click Calendar tab. Log out. Timesheet entry: Log in as perfuser01-50. Open the first time sheet. Enter times for 1 task. Save. Submit for approval. Open the submitted time sheet Click return timesheet button. Log out. Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 7 of 18
Appendix B: User Rights The test users performing the test scenarios described in Appendix A above were given the following rights: Link_View User Instance Rights: Application - Approve Application Allows approval of an Application Application - Edit Application Allows editing an application, includes delete Application - Edit Application Allows editing of Application access rights page Access Application - View Application Allows viewing an application Asset - Approve Asset Allows approval of an Asset Asset - Edit Asset Allows editing an asset, includes delete Asset - Edit Access Asset Allows editing of Asset access rights page Asset - View Asset Allows viewing an asset Job - Run Job/Report Run a specific job. Allows editing of the scheduled job properties and viewing of the output of the job. This right is dependent on Jobs - Access being granted. Other Investment - Other Investment Allows approval of an Other Investment Approve Other Investment - Other Investment Allows editing an other investment, includes delete Edit Other Investment - Other Investment Allows editing of Other Investment access rights page Edit Access Other Investment - Other Investment Allows viewing an other investment View Portfolio - Edit Portfolio Allows editing of a portfolio, includes delete Portfolio - Edit Access Portfolio Allows editing of Portfolio access rights page Portfolio - Scenario Portfolio Allows creation of Scenarios for a Portfolio Create Portfolio - View Portfolio Allows viewing of a portfolio Portfolio - View All Portfolio Allows viewing of all scenarios for a Portfolio Scenarios Product - Approve Product Allows approval of a product Product - Edit Product - Edit Access Product Allows editing of Product access rights page Product - View Product Allows viewing a product - Approve Allows approval of a - Attach Requisition Entry Allows resource to attach resources to requisition entries. This right includes the right to navigate to the requisition pages. s - Create - sub_project Allows resource to create sub_project sub objects within a specific master object. - Create - Allows resource to create sub_sub_project sub objects within a sub_sub_project - Create/Edit Requisitions specific master object. Allows resource to create and edit requisition properties. This right includes the right to navigate to the requisition pages, but not to create or edit the entries within the requisitions. - Delete Allows resource to delete a project or program. - Edit Allows resource to edit all parts of a project except for Document Management, Calendar, Action Items, Discussions and Custom Defined Fields. - Edit Access Rights Allows resource to manage access rights for a project. This right is dependent on - Edit Management being granted for the project. - Edit All - Allows resource to edit all sub_project sub objects within a specific Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 8 of 18
sub_project master object. - Edit All - sub_sub_project Allows resource to edit all sub_sub_project sub objects within a specific master object. - Edit Assigned Edit Assigned Tasks on a Tasks - Edit Financial Allows resource to edit the general and management properties, roster and key tasks. The resource can also add sub-projects to the project and the project can be viewed and edited in Microsoft or Open Workbench. - Edit Management Allows resource to edit the general and management properties, staff and tasks for the selected project if it has been enabled for management. This includes the ability to add sub-projects to it as - Edit Opportunity - Edit Plan - Edit Requisition Entries - Financial Planning - Financial Planning - Budget - Financial Planning - Forecast - Financial Planning Approval - Financial Planning Approval - Budget - Financial Planning Approval- Forecast - Modify Baseline - Risk, Issue, Change Request - Create/Edit - Risk, Issue, Change Request - Delete - Risk, Issue, Change Request - View well as edit it in Microsoft and Open Workbench. Allows resource to edit opportunity and general properties of the project. This right implies that the opportunity module for the project has been enabled. Allows resource to add unplanned tasks to the selected project when completing his timesheet if he is a team member on that project. Allows resource to create and edit requisition entries. This right includes the right to navigate to the requisition pages. Allows resource to view and edit the financial plan for the projects. This allows the resource to enter budgets and forecasts and view the general project properties. Allows resource to view and edit the financial budget plan for the projects. This allows the resource to enter budgets and view the general project properties Allows resource to view and edit the financial forecast plan for the projects. This allows the resource to enter forecasts and view the general project properties. Allows resource to approve and submit the financial plan for the project. This right allows resource to enter budgets and forecasts and view the general project properties. Allows resource to approve and submit the financial budget plan for the project. This right allows resource to enter budgets and view the general project properties. Allows resource to approve and submit the financial forecast plan for the project. This right allows resource to enter forecasts and view the general project properties. Allows resource to baseline a project if it has been enabled for Management. The rights also allows resource to view the general and management properties and processes. Allows resource to create and edit risks, issues, and changes for a project or program instance on which they are a staffed. Allows resource to delete risks, issues, and changes for a project or program instance on which they are a staffed. Allows resource to view risks, issues, and changes for a project or program instance on which they are staffed. - View Allows resource to view the general, management, financial and opportunity properties, custom defined fields, roster, tasks, processes and sub-projects for the project. - View Access Rights Allows resource to view access rights for a project. From within the application, this right implies that the resource has - View right to the project also. For a resource trying to view the access rights within the Administration section, this right is dependent on the having - Edit Administration granted. Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 9 of 18
- View All - sub_project Allows resource to view all sub_project sub objects within a specific master object. - View All - sub_sub_project Allows resource to view all sub_sub_project sub objects within a specific master object. - View CDF Allows resource to view the general properties and custom defined fields for a project. - View Documents Allows resource to view documents or folders even if resource is not a project participant, as long as the documents or folders are marked open to non-participants. This right includes the - View Base right. - View Allows resource to view the general and financial properties for the Financial - View Management project. Allows resource to view management properties, roster and key tasks of the project. This right also allows resource view access to the project in Microsoft and Niku Workbench. - View Opportunity Allows resource to view the general and opportunity properties for the project. - View Requisition Entries Allows resource to view requisition entries. This right includes the right to navigate to the requisition pages. - View Requisitions Allows resource to view a requisition. This right is dependent on - External Access being granted. - View Tasks Allows resource to view tasks for a project. This right is dependent on - View Base being granted. Report - Run Job/Report Allows resource to run a specific report. The right also allows editing of the scheduled report properties and viewing of the output of the report. This right is dependent on Reports - Access being granted. - Approve Ideas - Approve Time Allows approval of ideas for individual resources Allows the resource to approve and reject timesheets for a resource. The right does not imply - Enter Time. The Timesheets link will appear on the personal page. - Edit Allows the resource to view and edit all properties for a resource. - Edit Access Rights - Edit Administration This right is dependent on - Navigate being granted. Allows resource to grant or revoke access rights for a resource. This right is dependent on - View or - View - All being granted to access the page from within the Application. To access the page from within Administration section, this right is dependent on - Edit Administration being granted. Allows resource to update authentication and authorization information for a resource. This right is dependent on - Authorization Access Screens being granted. - Edit Calendar Allows resource to edit their own calendar. This right is dependent on - Navigate being granted. - Edit Ideas Allows editing of ideas for individual resources, includes delete - Enter Time Allows resource to complete and submit and reject timesheets for a resource. The Timesheets link will appear on the personal page. - Hard Book Allows resource to soft- or hard book a resource to a project to which he has the - View Management right. - Self Rights automatically granted to a resource when created. Includes - Edit Ideas. - Soft Book Allows resource to soft-book a resource to a project to which he has the - View Management right. The right also allows the resource to edit any soft-booking for the specified resource (or to remove it from the project). - Update Allows resource to update the skills profile for a resource as well as Skills to modify the general resource properties. - View Allows resource to view general information, contact information, Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 10 of 18
financial information, custom fields, skills, project list and working calendar for a resource. - View Access Rights Allows resource to view access rights for a resource. From within the Application this right is dependent on - View being granted. From within the Administration section this right is dependent on - Edit Administration being granted. - View Book Allows resource to view bookings for a resource. - View Allows resource to view financial information for a resource. Financial - View Ideas Allows viewing of ideas for individual resources Link_View User Global Rights: Application - Navigate Access to the application management pages Asset - Navigate Access to the asset management pages Ideas - Navigate Access to the ideas management pages Jobs - Access Allows resource access to the jobs pages. This right is dependent on either the Jobs - Run - All right or the resource being granted instance level rights Management - Programs such as Job - Run, Job - View Output or Job - Edit Properties. Allowed resource access to the Program Management features for programs and projects the resource has access to. This right is dependent on the resource having rights to programs and projects at either the instance level or OBS level. Other Investment - Access to the other investment management pages Navigate Portfolio - Navigate Access to the portfolio management pages Product - Navigate Access to the product management pages Reports - Access Allows resource access to the reports pages. This right is dependent on either the Reports - Run - All right or the resource being granted instance level rights such as Report - Run, Report - View Output or Report - Edit Properties. - Navigate Timesheets - Navigate _View User Instance Rights: Allows resource access to the list, Finder, and Requisitions pages. The resource will only be able to view resource information for resources to which the resource has access. Allows resource to access the timesheets list page. The resource will only be able to see timesheets the resource has edit or approve timesheets rights over. - Approve Allows approval of a - Attach Requisition Entry Allows resource to attach resources to requisition entries. This right includes the right to navigate to the requisition pages. s - Create - sub_project Allows resource to create sub_project sub objects within a specific master object. - Create - Allows resource to create sub_sub_project sub objects within a specific sub_sub_project - Create/Edit Requisitions master object. Allows resource to create and edit requisition properties. This right includes the right to navigate to the requisition pages, but not to create or edit the entries within the requisitions. - Delete Allows resource to delete a project or program. - Edit Allows resource to edit all parts of a project except for Document Management, Calendar, Action Items, Discussions and Custom Defined Fields. - Edit Access Allows resource to manage access rights for a project. This right is Rights dependent on - Edit Management being granted for the project. - Edit All - Allows resource to edit all sub_project sub objects within a specific Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 11 of 18
sub_project - Edit All - sub_sub_project - Edit Assigned Tasks - Edit Financial - Edit Management - Edit Opportunity - Edit Plan - Edit Requisition Entries - Financial Planning CA Confidential No Disclosure Without NDA master object. Allows resource to edit all sub_sub_project sub objects within a specific master object. Edit Assigned Tasks on a Allows resource to edit the general and management properties, roster and key tasks. The resource can also add sub-projects to the project and the project can be viewed and edited in Microsoft or Open Workbench. Allows resource to edit the general and management properties, staff and tasks for the selected project if it has been enabled for management. This includes the ability to add sub-projects to it as well as edit it in Microsoft and Open Workbench. Allows resource to edit opportunity and general properties of the project. This right implies that the opportunity module for the project has been enabled. Allows resource to add unplanned tasks to the selected project when completing his timesheet if he is a team member on that project. Allows resource to create and edit requisition entries. This right includes the right to navigate to the requisition pages. Allows resource to view and edit the financial plan for the projects. This allows the resource to enter budgets and forecasts and view the general project properties. - Financial Planning - Budget - Financial Planning - Forecast - Financial Planning Approval - Financial Planning Approval - Budget - Financial Planning Approval- Forecast - Modify Baseline - Risk, Issue, Change Request - Create/Edit - Risk, Issue, Change Request - Delete - Risk, Issue, Change Request - View Allows resource to view and edit the financial budget plan for the projects. This allows the resource to enter budgets and view the general project properties. Allows resource to view and edit the financial forecast plan for the projects. This allows the resource to enter forecasts and view the general project properties. Allows resource to approve and submit the financial plan for the project. This right allows resource to enter budgets and forecasts and view the general project properties. Allows resource to approve and submit the financial budget plan for the project. This right allows resource to enter budgets and view the general project properties. Allows resource to approve and submit the financial forecast plan for the project. This right allows resource to enter forecasts and view the general project properties. Allows resource to baseline a project if it has been enabled for Management. The rights also allows resource to view the general and management properties and processes. Allows resource to create and edit risks, issues, and changes for a project or program instance on which they are a staffed. Allows resource to delete risks, issues, and changes for a project or program instance on which they are a staffed. Allows resource to view risks, issues, and changes for a project or program instance on which they are staffed. - View Allows resource to view the general, management, financial and opportunity properties, custom defined fields, roster, tasks, processes and sub-projects for the project. - View Access Rights Allows resource to view access rights for a project. From within the application, this right implies that the resource has - View right to the project also. For a resource trying to view the access rights within the Administration section, this right is dependent on the having Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 12 of 18
- Edit Administration granted. - View All - sub_project Allows resource to view all sub_project sub objects within a specific master object. - View All - sub_sub_project Allows resource to view all sub_sub_project sub objects within a specific master object. - View CDF Allows resource to view the general properties and custom defined fields for a project. - View Documents Allows resource to view documents or folders even if resource is not a project participant, as long as the documents or folders are marked open to non-participants. This right includes the - View Base right. - View Allows resource to view the general and financial properties for the Financial - View Management project. Allows resource to view management properties, roster and key tasks of the project. This right also allows resource view access to the project in Microsoft and Niku Workbench. - View Opportunity Allows resource to view the general and opportunity properties for the project. - View Requisition Entries Allows resource to view requisition entries. This right includes the right to navigate to the requisition pages. - View Requisitions Allows resource to view a requisition. This right is dependent on - External Access being granted. - View Tasks Allows resource to view tasks for a project. This right is dependent on - View Base being granted. - Enter Time Allows resource to complete and submit and reject timesheets for a resource. The Timesheets link will appear on the personal page. - Self Rights automatically granted to a resource when created. Includes - Edit Ideas. _Viewer Instance Rights: - Approve Allows approval of ideas for individual resources Ideas - Approve Time Allows the resource to approve and reject timesheets for a resource. The right does not imply - Enter Time. The Timesheets link will appear on the personal page. - Edit Allows the resource to view and edit all properties for a resource. This - Edit Access Rights - Edit Administration right is dependent on - Navigate being granted. Allows resource to grant or revoke access rights for a resource. This right is dependent on - View or - View - All being granted to access the page from within the Application. To access the page from within Administration section, this right is dependent on - Edit Administration being granted. Allows resource to update authentication and authorization information for a resource. This right is dependent on - Authorization Access Screens being granted. - Edit Calendar Allows resource to edit their own calendar. This right is dependent on - Navigate being granted. - Edit Ideas Allows editing of ideas for individual resources, includes delete - Enter Time Allows resource to complete and submit and reject timesheets for a resource. The Timesheets link will appear on the personal page. - Hard Book Allows resource to soft- or hard book a resource to a project to which he has the - View Management right. - Self Rights automatically granted to a resource when created. Includes - Edit Ideas. - Soft Book Allows resource to soft-book a resource to a project to which he has the - View Management right. The right also allows the resource to edit any soft-booking for the specified resource (or to remove it from the Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 13 of 18
project). - Update Skills Allows resource to update the skills profile for a resource as well as to modify the general resource properties. - View Allows resource to view general information, contact information, financial information, custom fields, skills, project list and working calendar for a resource. - View Access Rights - View Book - View Financial - View Ideas _Viewer Global Right: Allows resource to view access rights for a resource. From within the Application this right is dependent on - View being granted. From within the Administration section this right is dependent on - Edit Administration being granted. Allows resource to view bookings for a resource. Allows resource to view financial information for a resource. Allows viewing of ideas for individual resources - Navigate Allows resource access to the list, Finder, and Requisitions pages. The resource will only be able to view resource information for resources to which the resource has access. Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 14 of 18
Appendix C: Medium User Base Test Detailed Results Below are the detailed Mercury LoadRunner results for the Medium User Base test. Action Description Average Response Time 90th Percentile Number of Tests Run TRS_ENTERTIME_00_login Time enter user login 0.797 1.008 59 TRS_ENTERTIME_00_logout Time enter user log out 0.277 0.384 59 TRS_ENTERTIME_00_startup Time enter script start session 0.071 0.123 5 TRS_ENTERTIME_01_timeperiod_ascending Sorting time periods ascend 0.192 0.414 59 TRS_ENTERTIME_01_timesheetlink Click time sheet link 0.183 0.273 59 TRS_ENTERTIME_02_opentimesheet Open a time sheet 0.265 0.404 59 TRS_ENTERTIME_02_savetimesheet Save time sheet 0.41 0.514 59 TRS_ENTERTIME_03_submitTS Submit Time Sheet 0.854 1.008 59 TRS_ENTERTIME_04_listsubmittedTS List submitted time sheet 0.118 0.153 59 TRS_ENTERTIME_04_opensubmittedTS Open submitted Time sheet 0.203 0.324 59 TRS_ENTERTIME_04_returnTS Return Time Sheet 0.348 0.494 59 TRS_NAVIGENERAL_00_login General navigation user login 0.771 0.847 30 TRS_NAVIGENERAL_00_logout General navigation user logout 0.232 0.253 30 TRS_NAVIGENERAL_00_startup General navigation session startup 0.062 0.073 30 TRS_NAVIGENERAL_01_AcctSettings Account settings link 0.085 0.123 276 TRS_NAVIGENERAL_01_Organizer Organizer link 0.188 0.253 276 TRS_NAVIGENERAL_01_Overview Overview page 0.361 0.484 276 TRS_NAVIGENERAL_01_ReportsJobs Reports and Jobs link 0.186 0.263 276 TRS_NAVIGENERAL_02_Applications Applications link 0.319 0.444 276 TRS_NAVIGENERAL_02_Assests Assests link 0.386 0.524 276 TRS_NAVIGENERAL_02_Ideas Ideas link 0.221 0.303 276 TRS_NAVIGENERAL_02_OtherInvests Other investments link 0.343 0.474 276 TRS_NAVIGENERAL_02_Portfolios Portfolios link 0.341 0.454 276 TRS_NAVIGENERAL_02_Products Products link 0.322 0.444 276 TRS_NAVIGENERAL_02_Programs Programs link 0.458 0.595 276 TRS_NAVIGENERAL_02_s s link 0.554 0.756 276 TRS_NAVITIMESHEET_00_login Timesheet navigation user login 0.774 0.907 330 TRS_NAVITIMESHEET_00_logout Timesheet navigation user logout 0.264 0.374 330 TRS_NAVITIMESHEET_00_startup Timesheet navigation session startup 0.059 0.063 25 TRS_NAVITIMESHEET_01_timeperiod_ascending Time period sorting: ascending 0.176 0.253 330 TRS_NAVITIMESHEET_01_timeperiod_descending Time period sorting: descending 0.168 0.233 330 TRS_NAVITIMESHEET_01_timesheetlink Timesheet link 0.178 0.243 330 TRS_NAVITIMESHEET_02_cancel_tmshtproject Timesheet project view cancel 0.231 0.324 330 TRS_NAVITIMESHEET_02_opentimesheet Open a time sheet 0.277 0.374 330 TRS_NAVITIMESHEET_02_opentimesheetproj Time sheet project link 1.11 1.352 330 TRS_NAVPRJ_00_login navigation user login 0.667 0.766 20 TRS_NAVPRJ_00_logout navigation user logout 0.265 0.344 20 TRS_NAVPRJ_00_startup navigation startup 0.056 0.063 20 TRS_NAVPRJ_01_OpenProjList List 0.551 0.766 180 TRS_NAVPRJ_02_sort_proj_name_acs Sort by name: ascending 0.565 0.776 180 TRS_NAVPRJ_02_sort_proj_name_desc Sort by name: descending 0.571 0.766 180 TRS_NAVPRJ_03_sort_proj_id_acs Sort by ID: ascending 0.603 0.867 180 TRS_NAVPRJ_03_sort_proj_id_desc Sort by ID: descending 0.591 0.816 180 TRS_NAVPRJ_04_open_proj_overview navigation user overview 1 1.281 180 TRS_NAVPRJ_04_open_proj_properties properties tab 0.972 1.241 180 TRS_NAVRES_00_login navigation user login 0.711 0.877 20 TRS_NAVRES_00_logout navigation user logout 0.23 0.263 20 TRS_NAVRES_00_startup navigation session startup 0.06 0.063 20 Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 15 of 18
Action CA Confidential No Disclosure Without NDA Description Average Response Time 90th Percentile Number of Tests Run TRS_NAVRES_01_firstpage list first page 0.523 0.756 193 TRS_NAVRES_01_lastpage list last page 0.397 0.575 193 TRS_NAVRES_01_nextpage list next page 0.521 0.726 193 TRS_NAVRES_01_resasc list sorting: ascending 0.513 0.695 193 TRS_NAVRES_01_resdesc list sorting: descending 0.517 0.726 193 TRS_NAVRES_01_resfilter list filter for opened 0.676 0.917 193 TRS_NAVRES_01_reslink s link 0.484 0.655 193 TRS_NAVRES_02_calendertab calendar tab 0.191 0.273 193 TRS_NAVRES_02_openres open 0.485 0.625 193 TRS_NAVRES_02_projtab project tab 0.47 0.665 193 TRS_NAVRES_02_skillstab skills tab 0.189 0.293 193 Averages 0.4 0.531 Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 16 of 18
Appendix D: Small User Base Test Detailed Results Below are the detailed Mercury LoadRunner results for the Small User Base test. Action Description Average Response Time 90th Percentile TRS_ENTERTIME_00_login Time enter user login 0.76 0.867 42 TRS_ENTERTIME_00_logout Time enter user log out 0.305 0.514 42 TRS_ENTERTIME_00_startup Time enter script start session 0.076 0.103 4 TRS_ENTERTIME_01_timeperiod_ascending Sorting time periods ascend 0.212 0.444 42 TRS_ENTERTIME_01_timesheetlink Click time sheet link 0.2 0.514 42 TRS_ENTERTIME_02_opentimesheet Open a time sheet 0.205 0.253 42 TRS_ENTERTIME_02_savetimesheet Save time sheet 0.374 0.404 42 TRS_ENTERTIME_03_submitTS Submit Time Sheet 0.758 0.867 42 TRS_ENTERTIME_04_listsubmittedTS List submitted time sheet 0.197 0.434 42 TRS_ENTERTIME_04_opensubmittedTS Open submitted Time sheet 0.184 0.193 42 TRS_ENTERTIME_04_returnTS Return Time Sheet 0.331 0.454 42 TRS_NAVIGENERAL_00_login General navigation user login 0.719 0.776 15 TRS_NAVIGENERAL_00_logout General navigation user logout 0.258 0.243 15 TRS_NAVIGENERAL_00_startup General navigation session startup 0.061 0.073 15 Number of Tests Run TRS_NAVIGENERAL_01_AcctSettings Account settings link 0.075 0.103 129 TRS_NAVIGENERAL_01_Organizer Organizer link 0.164 0.203 129 TRS_NAVIGENERAL_01_Overview Overview page 0.31 0.384 129 TRS_NAVIGENERAL_01_ReportsJobs Reports and Jobs link 0.161 0.213 129 TRS_NAVIGENERAL_02_Applications Applications link 0.277 0.354 129 TRS_NAVIGENERAL_02_Assests Assests link 0.321 0.394 129 TRS_NAVIGENERAL_02_Ideas Ideas link 0.2 0.253 129 TRS_NAVIGENERAL_02_OtherInvests Other investments link 0.315 0.414 129 TRS_NAVIGENERAL_02_Portfolios Portfolios link 0.294 0.364 129 TRS_NAVIGENERAL_02_Products Products link 0.276 0.354 129 TRS_NAVIGENERAL_02_Programs Programs link 0.405 0.514 129 TRS_NAVIGENERAL_02_s s link 0.474 0.595 129 TRS_NAVITIMESHEET_00_login Timesheet navigation user login 0.706 0.806 159 TRS_NAVITIMESHEET_00_logout Timesheet navigation user logout 0.305 0.615 159 TRS_NAVITIMESHEET_00_startup Timesheet navigation session startup 0.077 0.073 13 TRS_NAVITIMESHEET_01_timeperiod_ascending Time period sorting: ascending 0.178 0.253 159 TRS_NAVITIMESHEET_01_timeperiod_descending Time period sorting: descending 0.195 0.364 159 TRS_NAVITIMESHEET_01_timesheetlink Timesheet link 0.169 0.223 159 TRS_NAVITIMESHEET_02_cancel_tmshtproject Timesheet project view cancel 0.229 0.303 159 TRS_NAVITIMESHEET_02_opentimesheet Open a time sheet 0.242 0.303 159 TRS_NAVITIMESHEET_02_opentimesheetproj Time sheet project link 0.996 1.19 159 TRS_NAVPRJ_00_login navigation user login 0.669 0.968 9 TRS_NAVPRJ_00_logout navigation user logout 0.25 0.534 9 TRS_NAVPRJ_00_startup navigation startup 0.053 0.063 9 TRS_NAVPRJ_01_OpenProjList List 0.463 0.555 76 TRS_NAVPRJ_02_sort_proj_name_acs Sort by name: ascending 0.498 0.565 76 TRS_NAVPRJ_02_sort_proj_name_desc Sort by name: descending 0.528 0.655 76 TRS_NAVPRJ_03_sort_proj_id_acs Sort by ID: ascending 0.556 0.897 76 TRS_NAVPRJ_03_sort_proj_id_desc Sort by ID: descending 0.505 0.635 76 TRS_NAVPRJ_04_open_proj_overview navigation user overview 0.911 1.18 76 TRS_NAVPRJ_04_open_proj_properties properties tab 0.853 1.038 76 TRS_NAVRES_00_login navigation user login 0.677 0.756 9 TRS_NAVRES_00_logout navigation user logout 0.242 0.414 9 TRS_NAVRES_00_startup navigation session startup 0.055 0.063 9 TRS_NAVRES_01_firstpage list first page 0.458 0.645 83 Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 17 of 18
Action Description Average Response Time 90th Percentile TRS_NAVRES_01_lastpage list last page 0.357 0.454 83 TRS_NAVRES_01_nextpage list next page 0.456 0.625 83 TRS_NAVRES_01_resasc list sorting: ascending 0.44 0.544 83 TRS_NAVRES_01_resdesc list sorting: descending 0.435 0.534 83 TRS_NAVRES_01_resfilter list filter for opened 0.552 0.645 83 TRS_NAVRES_01_reslink s link 0.388 0.484 83 TRS_NAVRES_02_calendertab calendar tab 0.17 0.253 83 TRS_NAVRES_02_openres open 0.424 0.514 83 TRS_NAVRES_02_projtab project tab 0.417 0.534 83 TRS_NAVRES_02_skillstab skills tab 0.211 0.404 83 Averages 0.366 0.481 Number of Tests Run Windows SQL Server Performance Load Testing on Clarity 7.5.1 Page 18 of 18