CMS Review Report. Version 2.0. A presentation to The Department of Finance and Deregulation (AGIMO)

Transcription

1 CMS Review Report Version 2.0 A presentation to The Department of Finance and Deregulation (AGIMO) June

2 1. Project Overview 2. Methodology 3. Research Results 4. Discussion / Review / Adjustments? 5. Further Considerations 6. Moving Forward Contents 7. Appendix 2

3 Project Overview 3

4 Project Overview and Goals The Department of Finance and Deregulation (Finance) has decided to replace their Web Content Management System (WCMS) due to concerns around extendibility, usability, difficulties in content editing across all sections of the site and the cost of support. The objective was to review and select an appropriate Open Source WCMS that meets Finance s needs. An evaluation methodology was applied to the three shortlisted CMS's to present a ranking. This report will present an outline of the processes, Finance's specific requirement priorities, research results, scoring and future considerations. 4

5 Methodology 5

6 CMS Review Framework The framework based on the QSOS (open source) method was used. It provides an iterative approach with varying levels of granularity. Key processes include: Definition Requirements Evaluation Scoring and Weighting Criteria Qualification Research and Evaluation Selection 3 CMS s for Evaluation 6

7 Requirements Definition Requirement definition is a key part of understanding your business needs and how these translate into CMS requirements. Your requirements criteria Categorised Requirements Requirements workshop CMS Evaluation Criteria (by category) Normalisation (functional & non-functional) 7

8 Functional Requirement Categories Requirements are grouped into the following functional categories and sub-categories. Web Content Management - Content Authoring & Collaboration - Content Presentation - Navigation - Publishing - Workflow & Lifecycle Management Content Personalisation - Content Personalisation - Content Targeting e-commerce Support User Management - Admin & Content Editor Management - Public User Management Search Analytics & Reporting Digital Asset Management Modules 8

9 Non-Functional Requirement Categories These are the non-functional categories and sub-categories. Company & Product - Company - Product / Developer Community System Requirements Security Performance - Performance - Availability - Scalability Integration Interoperability Management CMS Usability 9

10 Evaluation Approach Requirement Prioritisation Each requirement is evaluated as a priority and these priorities are weighted. Description Priority Weighting Mandatory 3 Desirable 2 Optional 1 Not required 0 10

11 Evaluation Approach Requirement Scoring For each CMS to be reviewed, every requirement is evaluated and scored (out of 3): Score Description 3 The CMS meets the requirement out of the box. 2 The CMS requires slight customisation and/or configuration to meet the requirement. (e.g. integration of an existing plug-in or module). 1 The CMS requires considerable customisation and/or configuration to meet the requirement. (e.g. development effort required / no plug-in available). 0 The CMS does not meet the requirement. 11

12 Scoring Research and scoring is based on: Eminent reports (e.g. Gartner, Forrester) / CMS documentation / desktop research / product support / sizing / history and roadmap Subject Matter Experts input and review / Previous CMS selection project results Speaking to the vendors Requirements were scored twice independently then crosschecked to obtain an agreed score. 12

13 Evaluation Approach Requirement Scoring The scores are then multiplied by the priority that was agreed upon during the requirements gathering stage. This provides us with the Weighted Score. Raw Score (out of 3) Priority (out of 3) Weighted Score 1 (out of 9) 13

14 Scoring The Weighted Scores are added together to find the Category Score, weighted against its scaled value. Weighted Score 1 (out of 9) Weighted Score 2, 3, 4, 5 (out of 9) Weighted Category Score (each category has a scaled value) 14

15 Scoring The Weighted Category Scores are added together to calculate the Final Score for each CMS. Functional Requirements Non-Functional Requirements Weighted Category Score Weighted Category Score Weighted Category Score Weighted Category Score Weighted Category Score Weighted Category Score Weighted Community Score Total Score for CMS 1 (out of 100) 15

16 Research Results Requirements 16

17 Normalised Requirements We normalised the Functional and Non-Functional Requirements from various inputs, resulting in 401 requirements to be evaluated. CMS Review Methodology 401 Requirements 266 Functional Finance Draft Business Requirements (DBR s) & Requirement Workshops 103 Non-Functional 32 Community/Company + 13 Exclusions 17

18 Research Results Scoring and Weightings 18

19 Requirements Categories and Rankings The requirements were categorised and the importance of each requirement category was evaluated and expressed as a value percentage. Functional Requirements (50%) Category Breakdown % Web Content Management 30% User Management 15% Search 15% Content Personalisation 10% Analytics and Reporting 10% Modules 10% Digital Asset Management 5% E-Commerce 5% Total 100% *Community/Product/Company was considered a non-functional requirement category but we agreed to give it a separate scaling system due to its importance to Finance s needs. Non-Functional Requirements (30%) Category Breakdown % Performance 15% Performance: Availability 15% Performance: Scalability 15% CMS Usability 15% Integration 10% Management 10% Security 10% System Requirements 5% Interoperability 5% Total 100% Company/Product/Community (20%)* 19

20 Research Results Shortlisted CMS s 20

21 Shortlisting We began with a screening process and ended with a detailed scoring of requirements. Number of CMS possibilities reduces Development of long list Project management plan, Project schedule, Project budget tracking, issues and risk register, status report High-level research and shortlisting Level of detail in research increases Final scoring 21

22 Shortlisting Two key requirements helped to quickly narrow down the field further: 1. The WCMS must be truly Open Source. 2. The WCMS must not be a.net or Ruby based solution. This resulted in a long list of 18 CMS: Alfresco Daisy DotClear Drupal Expression Engine ez Publish Liferay Joomla! Magnolia Mambo OpenCMS Hippo CMS PHP-Fusion PHP-Nuke Pixie RavenNuke CMS Squiz WordPress 22

23 Shortlisting In order to narrow down the long list, each CMS was ranked based on the below criteria: A high-level view of our requirements criteria. The CMS s presence and reputation in the market. Eminent reports such as the Gartner Magic Quadrant for Web Content Management and Forrester WCMS material. Previous WCMS selection project results and Subject Matter Experts input. 23

24 Shortlisting High-level research results: Points High-Level Scoring Results CMS s 24

25 Shortlisting These were the final three Open Source CMS s that were shortlisted for evaluation. 25

26 The Results 26

27 Results The CMS s ranked as follows: Magnolia 75% Liferay 73.5% Drupal 70% 27

28 Results (without portal requirements) Liferay dropped from 73.5 to 72 points. Magnolia 75% Liferay 72% Drupal 70% * All following scores include portal requirements 28

29 Results Weighted score overview. Points Total Functional Non-Functional Community Magnolia Liferay Drupal CMS s 29

30 Functional Requirements - Breakdown CMS Web Content Mgt Content Personalisation User Mgt e-commerce Support Search Digital Asset Mgt Analytics & Reporting Modules Total Magnolia Liferay Drupal Max Score Web Content Management Content Personalisation 50 User Management 40 e-commerce Support 30 Search 20 Digital Asset Management 10 Analytics & Reporting 0 Magnolia Liferay Drupal CMS A CMS B CMS C Modules 30

31 Non-Functional Requirements Breakdown CMS System Requirements Security Performance Integration CMS Usability Management Interoperability Total Magnolia Liferay Drupal Max Score Magnolia Liferay Drupal CMS A CMS B CMS C System Requirements Security Performance Inegration Management CMS Usability Interoperability 31

32 Company & Product Requirements Breakdown CMS Company & Product / Developer Community Magnolia 172 Liferay 171 Drupal 188 Max Score Magnolia Liferay Drupal 32

33 Additional Breakdowns Metadata The following are some additional breakdowns that were requested by Finance. Requirements for metadata appeared 38 times, representing 9.5% of the unweighted evaluation criteria. CMS Metadata Requirements Magnolia 212 Liferay 205 Drupal 179 Max Score Metadata Requirements Magnolia Liferay Drupal 33

34 Magnolia Liferay Drupal Magnolia Liferay Drupal Additional Breakdowns Forms Requirements for forms appeared 3 times, representing 0.7% of the unweighted evaluation criteria. Raw Score Weighted Score Req. ID Category Requirement Priority FW.FW.052 Web Content Management The solution must provide the ability to manage user feedback generated through the submission of forms. Optional FW.CA.035 Web Content Management The solution must provide the ability to create and modify forms within the CMS, and the subsequent validation, capture and management of this form data (e.g. sending s on form submission, exporting to / from a database in various formats). Desirable NF.MA.011 Management The solution must provide the ability to manage user-generated content. Desirable Total Score (out of 15) - Forms

35 Magnolia Liferay Drupal Magnolia Liferay Drupal Additional Breakdowns Multiple Sites Requirements for multiple sites appeared 3 times, representing 0.7% of the unweighted evaluation criteria. Raw Score Weighted Score Req. ID Category Requirement Priority FW.PB.003 Web Content Management The solution must republish dependant and associated content automatically to maintain synchronisation, automatically updating appropriate content used by multiple pages or sites so that it remains consistent. Mandatory FW.CP.006 Web Content Management The solution must provide the ability for content (including pages and single page objects) to be authored once, but published in multiple locations on the site, and on other websites. Mandatory FW.PB.002 Web Content Management The solution must provide the ability to publish/depublish content items to one or many locations. Mandatory Total Score (out of 27) Multiple Sites

36 Additional Breakdowns Modules Requirements for modules appeared 15 times, representing 3.7% of the unweighted evaluation criteria. Raw Scores Weighted Scores Req. ID Category Module Priority Magnolia Liferay Drupal Magnolia Liferay Drupal FM.MO.001 Modules Blogs Optional FM.MO.002 Modules Ratings / Tagging Optional FM.MO.003 Modules Chat Desirable FM.MO.004 Modules Classifieds / Advertising Desirable FM.MO.005 Modules Forums Optional FM.MO.006 Modules Document Management Optional FM.MO.007 Modules Events Calendar Desirable FM.MO.008 Modules FAQ Management Mandatory FM.MO.009 Modules Graphing and Charting Tools Optional FM.MO.011 Modules Error Reporting Mandatory FM.MO.012 Modules Link Management Mandatory FM.MO.013 Modules Newsletter Management Optional FM.MO.014 Modules Polls Desirable FM.MO.015 Modules Site Maps Mandatory FM.MO.016 Modules Surveys Desirable Total Score (out of 84) Modules

37 Magnolia Liferay Drupal Magnolia Liferay Drupal Additional Breakdowns CMS Extendibility Requirements for CMS extendibility appeared 6 times, representing 1.5% of the unweighted evaluation criteria. Raw Scores Weighted Scores Req. ID Category Requirement Priority NF.IG.003 Integration The solution must provide an API extensibility. Mandatory NF.IG.004 Integration The solution must provide Web Services integration to support custom widgets. NF.IG.011 Integration The solution must provide the ability to develop modules that may be incorporated in the CMS to modify functions of the CMS. NF.IG.012 Integration The solution must provide the ability to develop modules that may be incorporated in the CMS to add functions to the CMS. NF.IG.013 Integration The solution must ensure that the process to develop to modules is well supported and understandable. NF.IO.001 Interoperability The solution must provide an extensible, fullydocumented Application Programming Interface (API) and architecture to connect and interact with external systems. Mandatory Mandatory Mandatory Desirable Mandatory Total Score (out of 51) CMS Extendibility

38 Scoring Overview Key findings: If implemented, all three shortlisted CMS s would require customisation to meet Finance s needs completely. Each CMS has good support for module extendibility (page 37) i.e. the ability to develop or plugin modules for specific functionality. This is separate from the Modules category (page 36) evaluated which covers specific functional requirements around blogging, chat, forums etc. While the final scores are close, some categories resulted in large ranges across the three CMS s (e.g. User Management / CMS Usability). By excluding portal requirements, the results show no significant difference. 38

39 Further Considerations 39

40 Portal vs. CMS Considerations In our research we also looked at the influence of portals on the CMS. How much can we take portal requirements into consideration? Our findings show that the portal requirements did not affect the end ranking. The industry trend is to use one product to cover both. All of the shortlisted CMS s are dynamic environments that render in real time. 40

41 Implementation Partners Cost of implementation is usually greater than license cost so the right implementation partner is crucial for project success: Providing a complete technical solution including necessary customisation. Non-technical build activity: User experience Creative design Project management Ability to provide extendibility and maintenance. 41

42 Is Open Source the only solution? There are some points to consider before committing to a CMS. Licensing vs development costs? Core functionality vs customisation (and upgrades)? Product support (SLA) advantages vs limitations? Community growth (not just relevant to Open Source)? The end-to-end user experience? 42

43 Moving Forward 43

44 Next Steps The following actions are expected following this presentation. Finalising the CMS Selection (Prototype / POC) Selection of an implementation partner Rebuilding australia.gov.au 44

45 Appendix 45

46 About Magnolia Release version reviewed: Magnolia CMS 4.5 Enterprise Edition (released Mar 2012) Technology: Java Open Source solution. JSR-170 (and JSR-168). Spring Framework and Struts integration, RSS feed support, standard templates, data caching and backup, also supports: JSR-168, CMIS, JCR 2.0, HTML5 and Java 6.0 Company information: Founded in Basel, Switzerland, first release Community channels: Magnolia Conference, Documentation, Forums, Forge, Mailing List, Blogs, Social Networks and Training and Events (typically outside Australia). Reference sites (3): US Navy, Travel.com.au, Digital UK Australian SI s: Certified (2): Ventiv (VIC) HotHouse (NSW) Other (2): Hyro (NSW) Permeance (VIC, WA) Freelancers on LinkedIn: 2+ 46

47 Magnolia: Scoring overview Magnolia presents itself as a robust and reliable CMS, with strong potential moving forward. It outperformed in: Web Content Management, Search, Digital Asset Management, System Requirements, Performance, Integration, CMS Usability and Interoperability. Although its community presence is more active than Liferay, it is still a way off that of Drupal. Strengths: Strong usability Enterprise support Weaknesses: Steep learning curve Smallest SI presence in Australia Strong core based on open standards 47

48 About Liferay Release version reviewed: Liferay Portal 6.1 Enterprise Edition (released Jan 2012) Technology: Java Open Source solution based on JSR-286. Platform independent. Also complies with JSR-168 and JSR-170. Company information: Headquarters in LA, California, USA. Founded in Community channels: Community, Documentation, Forums, Blogs, Wiki, User Groups, Marketplace, Liferay Projects, Community Projects, Training and Events Reference sites (3): Barcelona Turistic Bus, Terrassa City Hall, Allianz Insurance Australian SI s: Certified (3): Permeance (VIC, WA) Editure (VIC, SA) Mexuz (NSW) Other (1): NSW Technology (NSW, VIC) Freelancers on LinkedIn: 5+ 48

49 Liferay: Scoring overview Liferay s WCMS is strong in its core capabilities but these aren t as extensive as the other CMS s. It is highly extensible however and has the advantage of being a highly regarded Portal. Its biggest downfall is that its current community support is the poorest of the CMS s evaluated. Strengths: Solid core functionality Robust open standards Weaknesses: Learning curve Limited community Potential for customisation of UX Building momentum Enterprise support 49

50 Drupal: Scoring overview Release version reviewed: Drupal 7 (7.14 released May 2012) Technology: PHP solution which runs any platform that supports both a web server capable of running PHP and a server. Requires PHP or higher. Company information: Founded in the Netherlands in Community channels: Community, Documentation, Support, Marketplace, Forums, Chat, Mailing Lists, Member Directory, Training and Events. Reference sites (3): Australian Law Reform Commission, data.gov.uk, whitehouse.gov Australian SI s: Official community: ACT based SI s (2): Agileware (ACT) OPC IT (ACT) Other Australian SI s: 10+ in Victoria, 3+ in New South Wales Freelancers on LinkedIn:

51 Drupal: Scoring overview Drupal is the clear winner when it comes to community support. Arguably its strong modularity makes up for its lack of core functionality compared to Magnolia and Liferay. However this can make upgrades and support more difficult. Detailed user documentation is difficult to produce although this is recognised within the community and moves to improve this are underway. Strengths: Proven capability Extensive community Weaknesses: Dependency on modules User experience Strong modularity (including a large selection of pre-developed modules) Third party support 51