Central Hosting Case Study October, 2014
Central Hosting Case Study Background An Chéim was originally set up to procure, distribute and implement key MIS systems (hardware and software) for the Institutes of Technology (IoT) in Ireland. This significant undertaking involved a public procurement process, contract negotiations, widespread distribution of equipment, requirements definition and implementation of databases, software, operating systems and applications adapted to a single common specification across all Institutes. The initial implementation model consisted of delivery of a multi-tiered architectural solution to each individual IoT, requiring Unix, Oracle and Windows installations for the 4 key systems being implemented (a total of 56 individual projects). Support was to be provided locally for the infrastructure with application functional support from respective vendors and an Chéim. Driver Over time, the sector recognised that advancements in the technological landscape, particularly around the area of network provision combined with the challenges of support requirements may allow for an alternative, more sustainable and efficient delivery model. On this basis the sector commissioned a report to analyse the model of delivery. The report, produced by IBM in January 2003, put forward a strong case for pooling resources and set out the advantages of a centralised approach. Analysis revealed material differences in favour of the centralised model for three key areas: support, enhancements, and cost. The report found that a centralised approach would; Protect the common design. Provide consistency in support services. Allow access to specialist expertise, which would otherwise be out of reach of individual institutes. Free personnel within the Institutes to focus on the activities they were originally intended to carry out. Provide significant infrastructural and organisational cost savings for the Institutes.
Proof of Concept In order to determine the technical feasibility of the proposed new model, a proof of concept exercise was undertaken in two of the Institutes, Carlow and Dundalk. During this exercise, the Banner Student system was fully hosted, supported and managed using resources outside the individual Institutes. The success of the pilot implementations demonstrated that it was possible to provide a viable service from a centralised location. In January 2004 the Consortium Board met and agreed that An Chéim, acting on behalf of the Institutes, should proceed with central hosting and managed service procurement. An Chéim secured commitment from the Institutes to enter into a collective procurement process on behalf of all the Institutes, allowing progression to a detailed procurement project. Procurement A considerable public procurement exercise was undertaken and managed as a project by an Chéim; team members brought procurement, business, technical and legal expertise to the process. A highlevel overview of the process is provided below: PIN Nov 2003 Due Diligence Dec 2004- Mar2005 Contract Award April 2005 Contract Notice Published May 2004 Preferred Bidder Selection Nov 2004 Shortlisting evaluation of PQQ June 2004 Invitation to Tender (4 parties) July2004 Figure 1: Procurement Process Overview
Selection criteria included; 1) Service delivery and support 2) Ability to address a knowledge gap in the Banner student application 3) Software development competence 4) Design of the overall infrastructure architecture 5) Professional services competence 6) Completeness of service offering 7) Overall quality of the proposed data centre 8) Ability to understand and meet the requirements of the contracting authority 9) Transition implementation plan 10) References and existing customer service satisfaction levels Implementation: Following completion of the public procurement process, an Chéim undertook a programme to implement the centralised model. A phased approach was adopted during which all four systems for 14 Institutes (56 implementations in total) were migrated into the managed services environment. Implementation: Phase 1 Design and Build Although the Invitation to Tender phase included proposals around the infrastructure to meet the 99.9% availability SLA, a significant body of work remained around the detailed technical design. In order to ensure High Availability, the solution implemented by An Chéim included provision of resilient power, network, firewalls, servers, storage, databases and applications architecture. In addition to dual provisions at the production site, an off-site Disaster Recovery facility was also implemented with a 15 minute recovery point objective and a 24 hour recovery time objective. The solution had to be designed to handle average usage, whilst allowing some margin for additional traffic at peak times. The technical design and build phase took place over a four month period and involved expertise from a wide array of specialist areas (O/S, Database, Networks, Application, Security and Systems Architecture).
Implementation: Phase 2 - Application Rollout In parallel with the technical design and build phase, planning for the application implementation began. There were a large number of stakeholders to consider during this phase, from crossfunctional end users of the systems to the vendors supporting the functionality of the systems. This complex environment needed to cater for the autonomous nature of the Institutes whilst maintaining a baseline level of commonality to ensure support could be sustained. Planning questionnaires were circulated to the Institutes to ensure that all pre-requisites were in place prior to implementation and that any localisations implemented were identified and addressed during the rollout. The rollout for each application followed the blueprint of Pilot and Validation sites, followed by roll-out in waves of three, four and five Institutes. There was engagement with the Institutes in the form of wave specific workshops and user acceptance testing. An overview of the timelines involved in the project is provided below: Apr 05 Jun 05 Aug 05 Oct 05 Dec 05 Feb 06 Apr 06 Technical Design and Build Apps Implementation Planning App 1: Student Management System 14 sites App 2: Finance Management System 14 sites App 3: HR/Payroll Management System 14 sites App 4: Library Management System 14 sites Figure 2: Implementation and Rollout Overview
Governance and Standards Applied An Chéim recognised that whilst the implementation and rollout programme was an exercise of substantial magnitude that required meticulous planning and communications, it was necessary to build a sustainable offering that would require ongoing governance and monitoring to ensure service excellence. Policies and processes were put into place to allow management and continued service delivery; these included: The ITIL framework (the industry standard) was followed to ensure excellence in service delivery. This ensured that processes were in place to cover incident management, change management, problem management, release management, availability management etc. There was a high level of security built into the solution. Data would be kept in compliance with the Data Protection Act 1988 and EU directive 95/46EC. A common state of practice was agreed between the Institutes, An Chéim and the managed services supplier. Technical security was built into the overall architecture, with penetration testing to be carried out by third party experts. A service level agreement was drawn up to ensure optimal service delivery. It contained a system of credits and bonuses to drive performance. A penalty was applied for missing the SLA target and a credit awarded for exceeding it. This arrangement not only served to enforce the SLA, but also incentivised performance above target levels. A defined reporting regime was set up, with monthly performance checked against the SLA, root cause analysis for priority 1 and 2 calls, and access to call status via the web. Reporting requirements were defined in the contract and were used to analyse performance and report to management and the An Chéim board on actual performance against contract. Institute obligations, included nominating a local service coordinator to liaise with the managed services agency and An Chéim, supplying names of personnel authorised to log calls, and appointing individuals to the escalation chain. The Institutes were also required to make available necessary staff to cooperate with the managed services agency in the event of a fault requiring further investigation. A communications strategy was developed for the new managed service, which included devising a user guide and escalation paths. An Chéim dedicated a service management office to oversee the delivery of the services as defined in the contract and to initiate service improvement programmes as required.
Conclusion All systems were centralised by mid 2006. Each year, approximately 1 billion has been managed in financial transactions and 500 million in payroll payments using the systems implemented by An Chéim and managed in the central hosting environment. There are currently more than 50 terabytes of data and over 150 oracle databases under management for the 13 i Institutes of Technology and their 100,000 full and part-time students and 7,000 staff. All this has happened in the context of rigorous contract and SLA management, driving optimal service delivery. In terms of performance statistics, between 2009 and 2013 97% of calls were resolved within the target resolution time, average service availability was 99.9%, and the average customer satisfaction rating was 4.3 out of 5 ii. These performance statistics highlight the success of the central hosting model implemented by An Chéim. i The number of Institutes changed from 14 to 13 with the merger of Tipperary Institute and Limerick Institute of Technology ii Source: HP service management statistics 2009-2013