Appendix 1c. DIRECTORATE OF AUDIT, RISK AND ASSURANCE Internal Audit Service to the GLA REVIEW OF INCIDENT AND PROBLEM MANAGEMENT

Transcription

1 Appendix 1c DIRECTORATE OF AUDIT, RISK AND ASSURANCE Internal Audit Service to the GLA REVIEW OF INCIDENT AND PROBLEM MANAGEMENT

2 DISTRIBUTION LIST Audit Team David Esling, Head of Audit Assurance, Risk Management Steven Snaith, Baker Tilly Business Services Ltd Tim Moynihan, Baker Tilly Business Services Ltd Report Distribution List David Munn, Head of Information Technology Jawaid Bhatti, Technology Operations Manager Tom Middleton, Head of Governance and Resilience

3 CONTENTS EXECUTIVE SUMMARY Page Background 1 Audit Assurance 1 Areas of Effective Control 1 FINDINGS and RECOMMENDATIONS Review Objectives 2 Scope of Review 2 Reporting IT Incidents to the Service Desk 3 Roles and responsibilities 3 Process framework 3 Incident Management 4 Track-It! logical security 5 Performance Management 6 TG Communication 7 User satisfaction 7 Problem management 8 Risk and Audit Assurance Statement Definitions 9 March 2014 Incident and Problem Management

4 EXECUTIVE SUMMARY 1. Background 1.1 This review of the GLA s IT Incident and Problem management control framework was carried out as part of our 2013/14 plan. 1.2 IT Incidents and Problems are managed by the GLA Service Desk (SD) team and support engineers in the Live Team, under the Operations Manager in the Technology Group (TG). Approximately 1000 IT users are supported and these are a mix of corporate and political staff. Most corporate staff are based in City Hall. There are also a number of remote staff e.g. based in Brussels or London Boroughs. Political staff includes the Mayor, Assembly members and their support staff. 1.3 The IT Service Desk is available during work hours from 8am until 6pm from Monday to Friday. In addition, the Mayoralty and high profile GLA staff (VIP) have access to an IT Emergency Out Of Hours (OOH) support service primarily responsible for managing and dealing with access management incidents. 1.4 The Track-IT! Service Desk application has been implemented and is used to manage incident Work Orders. It is also used to manage some other TG Work Order types for example installations or miscellaneous used primarily for software licence requests. 1.5 Users can raise incidents and review the status of submitted requests by accessing the On-line Self Service Centre through the Intranet. 2. Audit Assurance Substantial Assurance There is a sound framework of control operating effectively to mitigate key risks, which is contributing to the achievement of business objectives. 3. Areas of Effective Control 3.1 The IT Incident and Problem management policies and procedures in place adequately set out the range of operational processes. In particular, a comprehensive SD process and protocol document has been designed for receiving and resolving incidents facilitating a controlled approach to incident management. 3.2 Roles and responsibilities are formally defined and communicated, providing an adequate framework to support the consistent management of any IT incidents. 3.3 Adequate monitoring controls are in place to provide TG visibility that SD handling and outcomes of incidents is being actively managed and the Service Level Agreements are being realised. 3.4 Effective problem management processes are in place to help ensure wider problems and underlying systems weaknesses that require escalation are identified and actions initiated, within an overall action planning framework. March 2014 Review of the London Plan and Implementation 1

5 5. Review Objectives 5.1 We reviewed the adequacy and effectiveness of the control framework in place designed to manage IT incidents and problems. In particular, we were looking to provide assurance that: 6. Scope IT Incident and Problem management policies and procedures are in place that set out the processes to follow to address an IT issue arising. Operational IT Incident and Problem management operational procedures and defined roles and responsibilities are in place to address and action an IT issue. IT Incident and Problem management monitoring controls are in place that capture response performance and provide management information to detect any recurring IT issues trends. IT Incident and Problem management action planning processes are in place, designed to ensure improvement in IT operations where applicable. 6.1 The review included an assessment of IT Incident and Problem management policies and procedures, operational controls, related roles and responsibilities, monitoring processes, reporting and action planning. 6.2 The review did not cover other Technology Group activities that use the Track-It! application such as installations, moves and changes or miscellaneous i.e. new software requests. March 2014 Review of the London Plan and Implementation 2

6 7. Reporting IT Incidents to the Service Desk 7.1 We found adequate communication and escalation processes are in place for users to access the Service Desk (SD) and report an IT issue/ incident. The induction material and the Intranet site provide details of the methods to be used by users to contact the Service Desk. In addition, controls are in place whereby users can raise issues and view the progress of their requests via the Online Service Desk webpage on the Intranet. The Online Self Service Centre provides the ability to view the progress and status of all requests that have been made by the user. The following supporting control processes are in place: The TG Emergency Guide has been circulated to staff. This provides a self-help toolkit for users to fix common problems themselves. The Online Self Service Centre also enables users to search for solutions to common issues and incidents. GLA Induction material identifies a number of priorities for incidents with definitions of Urgent, Important and Normal, providing an adequate framework from controlled escalation. A performance criteria is in place that sets out service Desk target to resolve all faults (incidents) within 5 working days; and priority requests for help within one day. A Service Desk Priority Matrix has been documented that determines the priority of an incident and the Service Level Agreement (SLA) is calculated by referencing the urgency of resolution against the incident s impact on the business. Track-It! has been configured with 9 Work Order priorities 7.2 Our testing verified that expected incident resolution / closure timeframes have been configured within the system based upon incident creation times that align with the incident SLA priorities. 7.3 Overall, SD processes established to enable users to raise and formally record incidents reduce the risk that corrective action is not initiated that may affect the achievement of business objectives. 8. Roles and responsibilities 8.1 The Senior Systems Engineer Business Applications has designated overall responsibility for the SD and the Live Team and reports to the Head of Technology. A SD team leader is established with a team of 4 support officers. Responsibilities are described using RACI techniques (Responsible, Accountable, Consulted, and Informed). A spread-sheet is maintained that identifies that Track-It! is a Business Application and the service owner is the TG Security Technician. 8.2 The assignment of responsibilities is effective to reduce the risk that incidents and problems are not being efficiently identified or effectively and consistently handled. 9. Process framework 9.1 Process and procedure manuals have been documented by TG, known locally as Assured Quality Action Procedures (AQAP).These include detailed instructions and are available to SD staff and are controlled by an index. AQAP are subject to formal approval and regular review and update. March 2014 Review of the London Plan and Implementation 3

7 9.2 The process and protocol documents were found to be adequately designed and reduce risk that incidents are not being efficiently identified or effectively and consistently handled. 10. Incident Management 10.1 Procedures are provided to SD and Live Team staff relating to recording and resolving incidents received via phone, and intranet. The overall process is summarised in Incident Management (Process model). This is supported by a narrative in Logging calls on the Service Desk. The process lifecycle has stages defined i.e. taking phone calls, incident identification, logging, categorisation, initial diagnostics, escalation, investigate & diagnosis, resolution & recovery and incident closure. Receiving incidents 10.2 GLA has implemented an Automatic Call Distribution System (ACD) to route phone calls made to the SD (extension 4333) to the available Service Desk support officers. If no support officer is available a recorded message is played until one is available. In the event of all officers being engaged on calls the ACD system will alert the Desk when a customer is waiting and an additional alert when more than one is waiting. A support team member is required to follow the Service Desk Call Handling script and verify the caller s identity/ username, contact details and affected asset and incident details are required to be logged directly into the Track-It! System Mail to the TGServiceDesk mailbox is not automatically processed, instead the designated meet and greet support team member on the SD rota will monitor and manage the mailbox. Users will be called back to confirm the issue and verify the assigned Work Order (WO) type e.g. 18 Issue , and agree a priority. Details from the may be cut and pasted into Track-It! Processed s are moved from the Inbox to the sub folder s dealt with. There is a defined process for Accepting, processing and managing requests via paper forms e.g. internal post. A similar process is used for walk-ins. The following supporting control arrangements are in place. After SD staff have completed logging incident details received by phone and , an is automatically sent to the user that identifies the assigned TG staff member dealing with the incident and summarises the incident details recorded in Track-It! As mentioned earlier, users can log directly into the Online Self Service Centre to access Track-It! using a personal login created as a part of the user account creation/starter process. The incident is automatically linked to their individual network login. As part of the Service Desk Call Handling script the SD will assign one of the 64 incident types that have been configured in the system. The Track-It! system is set up with Service Level Agreements for which the analyst will pick the one appropriate to the issue being reported. March 2014 Review of the London Plan and Implementation 4

8 Incident investigation, escalation and resolution 10.4 Documents have been designed to assist SD staff take a structured approach to incident investigation. These include the Initial Diagnostic Procedure and the Investigate & Diagnose Incident checklist. Where incidents cannot be fixed first time by the SD technician, incidents may be reassigned to the Live Team, or even to third party suppliers. After incident resolution, technicians are required to close down the call by changing the TrackIT Work Order status from Open to Closed Due to the size of the organisation and the SD traffic volumes, Track-It! does not record third party service suppliers. The SD incident owner will own the contact but can escalate to the service owners if required. Only the Incident Progression Team contacts third parties in the event that third party expertise is required to resolve an incident. Any s from support companies arrive in the TG SD mailbox and are used to update calls/users Documentation for receiving and resolving incidents is clear and sufficient to evidence each stage of contact with the customer. The Service Desk takes ownership for the incident control process and act as the single point of contact for receiving all requests/incidents and progress on incidents regardless of origin, this is done to avoid confusion to users. This reduces the risk the IT incidents are not being efficiently identified or effectively and consistently handled. 11. Track-It! logical security 11.1 Track-It! operates two classes of accounts, namely Users and Technicians. GLA staff and VIPS are listed in the User group to support self-service. The Online Service Desk webpage on the Intranet can be used to access the Online self-service centre for Track-It! Although SD staff can also log in using this route they will typically use client software installed on the TD workstations that does additional functionality. There are 18 Technician accounts that provide console access. Three security groups are in use: Group 2 admin = administrators. We reviewed membership and confirmed there is a valid business case for Track-It! access for each of the 4 accounts. Group 67 Helpdesk team. 10 technician accounts. Group 102 Asset Team =asset management (out of scope) 11.2 No rules are built into the application, but Technicians have 6 character passwords (they include at least one of each 1 upper, 1 lower case, 1 number and nonalphanumeric required). No passwords are required for self-service users. MS Active Directory can be enabled for authentication on an account by account basis, but not enabled. We verified that only administrators can set and change account passwords Controls over Track-It! privileged accounts and application security were found to be accurate and reduce the risk of accidental changes to the system configuration or overall loss to systems security. March 2014 Review of the London Plan and Implementation 5

9 12. Performance Management 12.1 The ACD system provides visibility of the SD activity on a wall-mounted large screen TV within TG. The top window is a graphical view illustrating the handsets on the TG SD showing the position of each handset is as it is on the helpdesk. The second window illustrates real-time available call statistics for the SD extension. Traffic light colours are used as follows: Red Urgent requires attention; Yellow Calls waiting in queue or/and not enough agents logged in; Green All OK The Track-It! system is set up with Service Level Agreements for which the analyst will determine the one appropriate to the issue being reported. 64 issue types have been configured in Track-It to provide granularity for management information and problem management. Review of the Track-It! Work orders for incidents and the Dashboard Report produced for the weekly Operations Team meeting confirmed categories are used GLA Induction material states that periodic reports will be published on the performance of the TG SD. The TG Online Service Desk web page contains links to historic performance reports. At the time of our fieldwork the most recent uploaded report was dated October SD reports are generated and sent to the Service Review Team on a monthly basis. We were advised that the Service Review Team meet quarterly to discuss SD performance and we were provided a copy of the January 2014 report The Track-It! system includes a number of locally designed reports that can be identified as they use GLA in title to discriminate from the built-in system view reports. This is supported by daily procedures e.g. Service Desk Morning Check List. Daily checks are made of open Works Orders (WO) including incidents that includes GLA - Open calls - Detailed Aged Outstanding Call Report by Engineer. This can be used to monitor SD staff performance including identifying any unassigned work incidents. GLA Completed Calls - By those within target. This is a monthly report that provides metrics on completed incidents by priority and calculates achievement of service levels. GLA Snapshot of open calls. Any calls that fail the SLA are highlighted in red Main Track-It! user interface, allows filtering of WO records. This is used for generating monthly report used for Service Review Team stats. Instructions are documented in the Closed Late Calls Stats procedure. Data fields include Assigned Technician, Priority, Date Entered, Summary, Requestor, Expected Completion Date and WO Type The SD statistics for January 2014 identify that on average calls to extension 4333 are answered with minimal waiting time and few are abandoned by users, providing assurance regarding the effectiveness of the process. March 2014 Review of the London Plan and Implementation 6

10 Telephone calls to the service desk during January 2014 Calls made to SD 1485 Calls taken 1385 Calls abandoned 0.7% Average time to answer 0.16 seconds 12.7 In addition, analysis of reports and of Track-It! data exported during the audit confirmed that SLAs are being achieved. Work Orders completed by the Service Desk during January 2014 Priority Number Percentage Critical under 1 hour 26 issues, 7 installs & 1 misc. 100 Urgent up to 2 hours 45 issues, 14 installs & 1 misc. 100 Important up to 1 day 292 issues, 83 installs & 18 misc. 96 Standard up to 5 63 issues, 133 installs & 18 misc. 100 days Minor up to 15 days 37 issues, 27 installs & 7 misc Monitoring controls provide TG visibility that SD handling and outcomes of incidents is being actively managed and the Service Level Agreements are being realised. This reduces the risk that the quality of the service provided by the Service Desk may be impacted, leading to incidents not being effectively addressed affecting GLA operational performance. 13. TG Communication 13.1 A formal process has been established for the ICT Management Team, Project Consultants and the Service Desk Leader to pass important time-critical information to the Service Desk. SD staff are advised to share solutions in the Daily Brief, Service Desk Meeting, Operations Meeting, Project Meetings and the TG daily afternoon meeting. We verified that daily briefings are held and are operating as intended Overall, our testing was able to verify that the TG communication processes support the effective handling of user IT incidents. 14. User satisfaction 14.1 A number of related performance management controls are in place, in particular: After an incident is logged an is automatically sent to the user that advises that comments or suggestions please contact the TG Service Desk Manager and provides an extension number. Feedback is requested from customers when an issue has been resolved. Compliment slips have been designed that SD staff leave during support visits to staff. An online questionnaire Let us know how well we are doing Survey has been designed based upon the compliments slip and is planned for launch on 17th March. It will be integrated into the call closure process. March 2014 Review of the London Plan and Implementation 7

11 A register of feedback is maintained spreadsheet that lists responses. The Monday TG Operations meeting discusses any user feedback. There is a formal procedure which sets out the complaint handling process. This includes a number of stages: record, discuss with originator, investigate and propose resolution and escalation. Any complaints that are received by the SD are referred to a team leader. There is a defined escalation route. We found that no complaints have been received during the last year User feedback and complaint processes were found to be adequate to help ensure TG is aware of user satisfaction of the delivered service and provides opportunities to resolve and improve the service. 15. Problem management 15.1 Processes are in place designed to ensure, where applicable, that IT Incidents identified through the Service Desk reflect wider problems and underlying systems weaknesses that require escalation (for example through GLA-wide communications, or within the GLA risk register) or address the root cause in existing systems. Processes for the management of user problems are documented. Stages include recording, classification, investigation and diagnosis. In some instances, the Service Desk may identify areas where investment is required in the IT infrastructure to address underlying weaknesses. These actions may result in a TG change control Moreover, the following supporting controls are in place: On the TG shared drive, a Problem Management folder is in place that is organised to record investigations and meetings. Service Outage Analysis processes can be initiated to require SD staff to capture extra data for Live Team investigation as part of incident handling actions. Management spreadsheets are generated to record the data and support diagnosis. Monthly SD reports produced for the Service Review Team identify trends in issues raised and compare trends for achieving SLA We found that Problems management processes were adequately designed to provide effective and efficient IT support functions to support business operations. March 2014 Review of the London Plan and Implementation 8

12 ASSURANCE STATEMENT DEFINITIONS RISK AND AUDIT ASSURANCE STATEMENT DEFINITIONS Overall Rating Substantial Adequate Limited No Assurance Criteria There is a sound framework of control operating effectively to mitigate key risks, which is contributing to the achievement of business objectives. The control framework is adequate and controls to mitigate key risks are generally operating effectively, although a number of controls need to improve to ensure business objectives are met. The control framework is not operating effectively to mitigate key risks. A number of key controls are absent or are not being applied to meet business objectives. A control framework is not in place to mitigate key risks. The business area is open to abuse, significant error or loss and/or misappropriation. Impact There is particularly effective management of key risks contributing to the achievement of business objectives. Key risks are being managed effectively, however, a number of controls need to be improved to ensure business objectives are met. Some improvement is required to address key risks before business objectives can be met. Significant improvement is required to address key risks before business objectives can be achieved. March 2014 Review of the London Plan and Implementation 9