USmax methodology to Conduct Commercial off-the-shelf (COTS) Product Evaluation

Transcription

1 USmax methodology to Conduct Commercial off-the-shelf (COTS) Product Evaluation Text Block WHITE PAPER Copyright 2010 USmax Corporation. All rights reserved. This document is provided for the intended recipient s informational purposes only and shall not be duplicated, otherwise used, or disclosed to a third party in whole or in part without the express written consent of USmax Corporation.

2 2 A USmax Whitepaper: USmax Methodology to conduct Overview USmax utilizes proven best practices that are consistently implemented across the industry for COTS product evaluation. The methodology developed by USmax and outlined in this document is Capability Maturity Model Integrated (CMMI) Level II compliant and utilizes an approach that focuses on customer goals and objectives. This method was created to use a uniform and repeatable (iterative) evaluation process that takes customer requirements into account. The approach is designed for full lifecycle COTS product evaluation and is implemented in four (4) phases. The objective and outcome is to select the right COTS product in alignment with the customer requirements for sound and effective future COTS product adoption. USmax COTS Product Evaluation Methodology Our methodology is divided into four (4) distinct phases as indicated in the figure 1 below. Phase I Business Process Definition and Requirements Development Phase II Evaluation Criteria Development and Market Research Phase III COTS Product Evaluation and Demonstration Phase IV Validation of Vendor References and Past Performance Figure 1: USmax Corporation methodology for COTS product evaluation. USmax Corporion 382 Gambrills Road, Gambrills MD 21054

3 3 A USmax Whitepaper: USmax methodology to conduct Phase I Business Process Definition and Requirements Development Phase I of our methodology is all about documenting current business processes and requirements specification. We commence the evaluation process by conducting a comprehensive review of the customer documentation. We conduct interviews with the customer staff and also meet with several internal and external stakeholders to develop and verify business requirements. As the requirements are developed and validated, we confirm the existing requirements, as well as assumptions, and document any process or system changes that may have been implemented since the requirements were developed. Also, based on our extensive experience, we believe it is of critical importance to identify and review any upcoming Information Technology (IT) initiatives, in order to ensure the COTS product we choose remain compatible with the future IT vision of the customer. USmax utilizes the high level steps described below in Table 1 to document business process and develop and validate the system s technical, functional, and managerial requirements. Requirements Development and Validation Steps Identify key stakeholders for interviews Establish interview schedule for key stakeholders Approve the updated requirements Research stakeholder roles and responsibilities Interview key stakeholders to document and update requirements Document data collection processes and procedures Identify the gap between old and new requirements Present stakeholders with updated requirements for review Table 1: Phase I Requirements Development and Validation Steps Phase II Evaluation Criteria Development and Market Research In Phase II, USmax develops a set of evaluation criteria based on the customer requirements and technical environment to determine the best fit COTS product. In addition to the technical ability to meet the requirements, USmax also considers the fit of the product within customer standards and Enterprise Architecture (EA) for the ability to integrate with other applications, ease of training, implementation, security requirements and overall application architecture. During the initial evaluation USmax, along with the customer stakeholders review the COTS USmax Corporion 382 Gambrills Road, Gambrills MD 21054

4 4 A USmax Whitepaper: USmax Methodology to conduct products based on the core requirements identified to establish a set of four products that will be selected for vendor demonstrations and full system evaluation. An example of an initial product review of a workflow based COTS product can be conducted on factors such as: Full lifecycle procurement management Integrated workflow and electronic forms Conformance with Federal mandates Open architectural standards Integration capabilities, platform, and database Web based graphical user interface Phase III COTS Product Evaluation and Demonstration In Phase III, the four products that best fit the customer requirements are scheduled for review and vendor demonstration to the customer stakeholders. The stakeholders are composed of members from various groups and divisions within the organization. At least one stakeholder from each division or group is required to attend every review and demonstration. The four products are reviewed by the stakeholders and scored based on a set of evaluation criteria. Based on the evaluation of the stakeholders and USmax s analysis, the final two products are determined. From the two final products, ranked as primary and secondary, USmax creates the recommended solution document for the customer. The lists of available alternatives are further reduced through the following criteria: Ease of use Implementation experience Integration experience Training experience Recommendations USmax Corporion 382 Gambrills Road, Gambrills MD 21054

5 5 A USmax Whitepaper: USmax methodology to conduct Prior to the product demonstration, the vendors are provided with the core requirements list to demonstrate capability as necessary to meet the needs of the customer. The evaluation process rules and requirements criteria are provided to the customer stakeholders for the product demonstration. The stakeholders evaluate the products against the evaluation criteria based on a scoring scale for each requirement. The score for each requirement is averaged across each division to create an overall division score. This allows each division to have equal input into the final score regardless of the actual number of people who attend each session from a particular division. The total score is then based on the weighting of each business area rather than the total number of points. The scoring of each requirement is based on a scale of 0 3 for the following justification: 0 not shown at all 1 Product has the capability but it requires customization 2 The requirement is demonstrated and would meet the need 3 The requirement is exceeded Phase IV Validation of Vendor References and Past Performance After the completion of the product evaluations by the stakeholders, USmax requests customer references and past performance reviews from the final two vendors. Phase IV includes customer reference checks, demonstrations of products, implementation reviews, integration assessments with other legacy systems at federal agencies/commercial clients, customer service, and usability. USmax also analyzes the COTS product with various internal and external systems. This additional analysis helps to provide the best solution for the customer. USmax s recommendation is determined based on some of the following criteria: Stakeholder evaluation Technical ability to meet requirements Customer mission, vision, standards, and objectives USmax Corpoaion 382 Gambrills Road, Gambrills MD 21054

6 6 A USmax Whitepaper and Case Study: USmax Methodology to conduct Integration points for other systems Training and implementation considerations Security Requirements Deliverables Below is the list of few deliverables developed as part of USmax COTS product evaluation methodology. 1. Documented Business Processes and Requirements Specifications. 2. COTS Product Alternatives Analysis Report. 3. Recommended COTS Product Solution Report. To learn more and discover how USmax delivers value, simply call or send us an e mail today. USmax Corporation 382 Gambrills Road Gambrills, MD Federal Group Contact Information. Phone: (301) Fax: (410) gov@usmax.com Commercial Group Contact Information. Phone: (410) Fax: (410) corpsales@usmax.com USmax Corporion 382 Gambrills Road, Gambrills MD 21054