Evaluation of SOA Vendors A Technical White Paper from Firstservis

Transcription

1 Evaluation of SOA Vendors A Technical White Paper from Firstservis Author: Alexander Roussekov, Senior SOA Architect, Firstservis Pty Ltd. Abstract This paper describes a SOA Vendors Evaluation Methodology which when applied to analytical sources such as Gartner Group or Forrester reports shortlists appropriate vendors in an objective way. The objectivity of the vendor s evaluation is ensured by vendor ranking and scoring against a set of evaluation criteria but within the corresponding evaluation models. A final short list is produced based on the correlation between short lists obtained from each model. Using more than one analytical evaluation model ensures objectivity and synergy in respect to the evaluation criteria. Introduction The objective of the Evaluation Methodology is to shortlist SOA vendors providing adequate SOA technology stacks suitable for SOA implementations. The Vendors Evaluation is considered to be only the initial step of the evaluation process aimed at filtering vendors which will then be evaluated further in respect to appropriate capabilities applicable to the entire Enterprise Landscape. The approach is based on analysis of the results from studies of Gartner Group and Forrester identifying the Best-of-Breed vendors available on the market. The procedure followed in the evaluation process consists of the following steps: Selection of appropriate sources from Gartner and Forrester Ranking vendors based on the results in the Gartner and Forrester sources Short-listing of vendors Correlation of the short lists Review of the Evaluation Methodology Evaluation based on Gartner Group Reports Three major reports published by the Gartner Group in 2007 have been included in the current SOA vendors study as follows: Magic Quadrant for Application Infrastructure for Back-End Application Integration Projects, 2Q07 Magic Quadrant for Application Infrastructure for Composite-Application Projects, 2Q07 Magic Quadrant for Application Infrastructure for New Service-Oriented Business Application Projects, 2Q07 The Back-End Integration analysis puts the focus on the integration with EIS and legacy systems relevant in the existing IT landscape. The selection of vendor(s) and an appropriate integration framework is considered critical. In this context the predominant types of software employed for the back-end application integration usage scenario are Integration Suites and ESBs. Composite-Application Development is perceived to be the next technological step on the enterprise s SOA roadmap and involves building new style SOA applications based on full end-to-end business process automation and reuse of exiting business processes and services. Gartner has identified a composite applications infrastructure market as one that offers integrated suites of application infrastructure components coming with end-to-end set of functionalities of user interface management, composition and orchestration, integration and adapters, interoperability and messaging protocols, container technology, etc. Development of new Service-Oriented Business applications is a further step in the SOA implementation and addresses building of SOA applications from scratch by employing the new SOA methodology and infrastructure. Gartner Group characterises this market segment by key focus on service orchestration and less Page 1 of 10

2 emphasis on integration. The major product categories expected to constitute the market are application servers, horizontal portals, design, development, modelling and governance tools, SOA infrastructure tools and metadata repositories, business process management (BPM) and orchestration tools, various user-interaction tools and high-end performance enhancement tools. Gartner has identified new markets for application infrastructure in all these areas. The Gartner Group methodology is based on so called magic quadrants positioning and categorises vendors in respect to: Ability to Execute, and Completeness of Vision The Gartner s three major categories of SOA Market Research and Vendors Evaluation are adequate to the SOA Implementation Strategy and SOA Roadmap. Each category is assumed to have an equal weight in the overall evaluation and the same critical impact for the future state of integration and application services. Conceptually, this reflects the connections between the Past, Current and Future which corresponds to the evolution path from Back-End Application Integration projects, to Composite-Application Projects and subsequently to New Service-Oriented Business Application Projects. The importance of these categories is addressed by the Enterprise SOA Maturity Reference Model [1], which also describes the Business Requirements, Key Challenges, Exit Criteria and Success Factor pertinent to each category. The ranking methodology employed by this study is governed by the following rules: Only vendors scoring and represented in each of the Gartner s three categories will be considered. Only the first 6 vendors identified in the Leaders, Challengers or Niche Players magical quadrants are considered for further ranking and evaluation. Vendors are ranked by their positioning in the Gartner s magic quadrants in both Ability to Execute and Completeness of Vision dimensions. A Ranking score is assigned to each vendor in each dimension which is determined by the rank position. The score of the vendors coming from the Challengers magical quadrant is increased by one, and the score of the vendors coming from the Niche Players magical quadrant is increased by two. Total score is assigned for each of the three categories calculated as the sum of the scores in Ability to Execute and Completeness of Vision dimensions. Final ranking is produced by gradation of grand totals of the totals of each vendor across each investigated category Vendors not scoring in all categories are dropped from the evaluation. Vendors with lighter scores are considered superior. The Gartner s reports results are represented graphically in Figure 1, 2 and 3 below. Table 1 shows the vendors ranking carried out in accordance with the evaluation rules defined previously. Table 2 summarises the results of Table 1 and displays the scores of each vendor subject to evaluation. Page 2 of 10

3 Figure 1 - Magic Quadrant for Application Infrastructure for Back-End Application Integration Projects, 2Q07 Figure 2 - Magic Quadrant for Application Infrastructure for Composite-Application Projects, 2Q07 Page 3 of 10

4 Figure 3 - Magic Quadrant for New Service-Oriented Business Application Projects, 2Q07 Table 1 - Vendor Ranking based on Gartner Reports for 2007 SOA Vendors Ranking Using Gartner s Evaluation Reports Gartner Model Evaluation Ability To Execute Ranking Completeness Vision of Combined Ranking Magic Quadrant for Application Infrastructure for Back- End Application Integration Projects, 2Q07 1 Microsoft 2. Tibco 3. Oracle 4. BEA 5. IBM 6. Sun 1. Tibco 2.Sun 3. IBM 4. BEA 5. Oracle 6. Microsoft 1. Tibco 2. Microsoft 3. IBM 3. Oracle 3. BEA 3. Sun Magic Quadrant for Application Infrastructure for Composite-Application Projects, 2Q07 1. IBM 2. Oracle 3. InterSystems 4. SAP 5. BEA* 6. Microsoft* 1 InterSystems 2. SAP 3. IBM 4. Oracle 5. BEA* 6. Microsoft* 1. InterSystems 1. IBM 2. SAP 2. Oracle 3. BEA* 4. Microsoft* Page 4 of 10

5 SOA Vendors Ranking Using Gartner s Evaluation Reports Gartner Model Evaluation Ability To Execute Completeness of Combined Ranking Ranking Vision Magic Quadrant for Application Infrastructure for New Service-Oriented Business Application Projects, 2Q07 1 IBM 2. BEA 3. Oracle 4. Fujitsu 5. Microsoft* 6. Hitachi ** 1 Oracle 2. IBM 3. BEA 4. Fujitsu 5. Microsoft* 6. NEC** 1 IBM 2. Oracle 3. BEA 4. Fujitsu 5. Microsoft* 6. Hitachi ** 6. NEC** *Vendor is positioned in the Challengers magic quadrant **Vendor is positioned in the Niche Players magic quadrant The Combined Ranking column in Table 1 is produced by adding the ranks of each vendor in respect to a given category. The lighter the score, the higher is the rank in the classification ladder. For example, Microsoft is ranked first in respect to Ability to Execute criteria and sixth in respect to Completeness of Vision criteria of the Gartner s Back-End Application Integration Projects category. The overall score for Microsoft is 7 which positions the vendor second in the Combined ranking. Please note that vendor s scores are increased by 1 or 2 when the vendors are coming from the Challengers and Niche Players magic quadrants respectively. Table 2 - Summary of Vendors Ranking based on Table 1. SOA Vendors Ranking Using Gartner s Evaluation Reports Vendor Total Sore of Ability To Execute Ranking Total Sore of Completeness of Vision Grand Total Score 1. IBM Oracle BEA Microsoft Table 2 shows the combined scores of vendors across all of the analysed categories. Vendors which failed to score in all of the required categories were not considered in the final evaluation. Page 5 of 10

6 Evaluation based on Forrester Group Forrester has published a report that analyses the positioning of Integration-Centric Business Process Management Suites (IC-BPMS). Integration-Centric Business Process Management Suites is a concept that covers the link between SOA, legacy integration and BPM therefore this linkage is reflected in the ranking of the products evaluated. The Forrester Wave Model [2] comes with three dimensions (evaluation criteria): Current offering - vendors capabilities in the areas of connection, BPM, and SOA. Strategy - vendor vision and product road map features, as well as solution cost and strategic alliances. Market presence - vendors installed customer base, new customers in the past quarter and year, and the level of global sales and implementation support. The evaluation of the IC-BPMS market as defined by Forrester is recommended to be a starting point of evaluation and vendor analysts are encouraged to adapt the criteria weightings to fit their individual needs through the Forrester Wave Excel-based vendor comparison tool. This study accepts the results of Forrester presented in the report [2] and applies the evaluation methodology based on the assumption that all the dimensions (criteria) of the model have equal weightings. The ranking methodology employed by this study is governed by the following rules: Vendors are ranked by their positioning in respect to the Forrester Wave Model evaluation dimensions: Current Offering, Strategy and Market Presence. Only the first 6 high-ranked vendors are considered for further ranking and evaluation. Total score is assigned to each vendor as a sum of the three categories rank numbers. Final ranking is produced by gradation of grand totals of the totals of each vendor across each investigated category The Forrester Wave Model results are represented graphically in Figure 4. Table 3 shows the vendors ranking carried out in accordance with the evaluation rules defined before. Table 4 summarises the results of Table 3 and displays the scores of each vendor subject to evaluation. Page 6 of 10

7 Figure 4 - Forrester Wave : Integration-Centric Business Process Management Suites, Q4 06 Table 3 - Vendor Ranking Based on Forrester IC-BPMS Market Evaluation SOA Vendors Ranking Using Forrester Wave Evaluation Model Current Offering Strategy Market Presence 1 webmethods 2. Tibco 3. IBM 3. BEA 4. Software AG 5. SAP 6. Sun 6. Oracle 6. Vitria 1 webmethods 1. Tibco 2. IBM 3. Oracle 4. BEA 5. Sun 6. Software AG 1. IBM 1. BEA 1. Oracle 1. Microsoft 2. Tibco 2. webmethods 2. Sun 3. Software AG 3. Magic Software 3.iWay 4.Vitria 4. Cordys Page 7 of 10

8 Table 4 - Summary of Vendor Ranking based on Table 3 SOA Vendors Ranking Using Forrester Wave Evaluation Model Vendor Current Offering Rank Strategy Rank Market Presence Rank Total Score 1. webmethods IBM Tibco BEA Oracle Sun Software AG Correlation of the Short Lists SOA Vendors Short List Comparison Table 5 - SOA Vendors Short List Comparison Ranking based on Gartner Sources Ranking based on Forrester Sources Vendor Score Vendor Score 1. IBM webmethods 4 2. Oracle IBM 5 3. BEA Tibco 5 4. Microsoft BEA 8 5. Oracle Sun Software AG 13 Table 5 gives a summary of the results of evaluations carried out in the previous two sections. Only vendors which are represented on both short lists are considered in order to ensure a high level of synergy between the Gartner and Forrester Evaluation Models. Further ranking of the short-listed vendors is achieved by ordering the sums of the scores listed in Table 6. Page 8 of 10

9 Table 6 - SOA Vendors Short List SOA Vendors Short List Vendor Total Score 1. IBM 20 2.Oracle 28 3.BEA 33 Review of the Evaluation Methodology Four evaluation reports three from Gartner Group and one from Forrester were used in the SOA vendors evaluation. SOA Vendors Evaluation Sources Source Magic Quadrant for Application Infrastructure for Back-End Application Integration Projects, Magic Quadrant for Application Infrastructure for Composite-Application Projects Magic Quadrant for Application Infrastructure for New Service-Oriented Business Application Projects The Forrester Wave : Integration-Centric Business Process Management Suites Time period 2Q07 2Q07 2Q07 4Q06 These are the most representative evaluation studies based on the analysis of Gartner Group and Forrester both highly-regarded in the IT industry In this context the evaluation methodology employed in the process of selection of viable SOA vendors combines the selection criteria (dimensions) of both Gartner and Forrester as described in Table 7. Summary of SOA Vendors Evaluation Criteria Table 7 - SOA Vendors Evaluation Criteria Source Dimension (Criteria) Description Gartner Ability to execute Product/Service Overall Viability (Business Unit, Financial, Strategy, Organization) Sales Execution/Pricing Market Responsiveness and Track Record Marketing Execution Customer Experience Operations Page 9 of 10

10 Summary of SOA Vendors Evaluation Criteria Gartner Completeness of Vision Market Understanding Marketing Strategy Sales Strategy Offering (Product) Strategy Business Model Vertical/Industry Strategy Innovation Geographic Strategy Forrester Current offering. Vendors capabilities in the areas of connection, BPM, and SOA. The connection area includes the overall architecture, development environment, support for protocols and adapters, transformation features, and security. The BPM section looks at event management, industry accelerators, human-centric features, monitoring, partner management, process modelling, and business rules management. The SOA section includes a review of ESB capability, governance, composite applications capability, registry/repository support, and support for Web services. Forrester Strategy Vendor vision and product road map features, as well as solution cost and strategic alliances. Forrester Market Presence Vendors installed customer base, new customers in the past quarter and year, and the level of global sales and implementation support. The objectivity of the vendor s evaluation is ensured by vendor ranking and scoring against each of the evaluation criteria but within the corresponding evaluation model (Gartner or Forrester). The final short list is based on correlation between the short lists produced by each model and reflects the synergy in respect to the evaluation criteria. References [1] SOA Practitioners Guide Part 2: SOA Reference Architecture, 2006 [2] The Forrester Wave : Integration-Centric Business Process Management Suites, Q Page 10 of 10