INS Problem Management Manual

Transcription

1 INS Problem Management Manual Process, Policies & Procedures Version 0.4 Draft Ver 0.4 INS Problem Management Manual.doc Page 1 of 47

2 Document Control Document Versions Date Version Person Changing Brief Summary of Change 08/01/ Gary Bullard, IT Consultant 14/01/ Gary Bullard, IT Consultant 11/06/ Anna Pegg, CTI Project Manager 17/07/ Anna Pegg, CTI Project Manager First Draft for Review by Process Owner Updated with changes from Process Owner: Bruce Scott. For review by reference group on 18 Jan 2010 Updated with changes from Project Manager SDT Phase 2: Anna Pegg. Based on updates required to reflect approach taken when implementing Problem Management. Updated Document approval in order to allow document to be published online. Document Location The document is located at: \\staff.ad.griffith.edu.au\groups\ins\its\its-projects\ins Projects_Closed\INS Projects_Closed 2011\INS473 Service Desk replacement (Servicenow.com)\ITIL Processes\ITIL Problem Management\Current Document Approval* *The Problem Management Roles will be reviewed after the 2013 New INS changes have come into effect. The detailed Problem Process is still current; however reporting structures can be confirmed with your manager. Authorising Officer Name Signature/s Date Process Owner Problem Manager Document Owner Bruce Scott To be determined Bruce Scott Draft Ver 0.4 INS Problem Management Manual.doc Page 2 of 47

3 Table of Contents Document Control...2 Table of Contents...3 Problem Management Process...4 Overview...4 Benefits of Problem Management...4 Related Processes...5 Ownership & Responsibility...5 Document Location:...5 Contact Person...5 Inputs & Outputs of Problem Management...6 Problem Management Business Rules...7 Problem Prioritisation...9 Problem Categorisation...10 Problem Status Codes...11 Problem Resolution Codes...12 Problem Management Procedures...13 High Level Process Flow...13 Defined Problem Process Flow Problem Prioritisation and Categorisation Procedure Problem Investigation & Diagnosis Procedure Workarounds & Known Errors Problem Resolution Problem Closure Communication Proactive Problem Analysis...33 Problem Management Roles...36 Problem Governance Team...36 Process Owner...37 Problem Manager...38 Service Desk (Service Desk and EIS Assist)...39 IT Support Specialist...40 Client...41 Process Metrics...42 Problem Management Reporting...46 Reports required by IT Support Team Leads...46 Reports required by Manager Planning (Account Management role)...47 Reports required by Problem Manager...47 Draft Ver 0.4 INS Problem Management Manual.doc Page 3 of 47

4 Problem Management Process Overview Problem Management is the process responsible for managing the lifecycle of all problems and the primary objectives of Problem Management are to prevent problems and resultant incidents from happening, to eliminate recurring incidents and to minimise the impact of incidents that cannot be prevented. Prevention of problems requires INS to take a proactive stance which requires direction and leadership within INS support teams. This is covered more fully in the Procedures section of this Manual (refer 8.0 Proactive Problem Analysis). By implementing Problem Management in Information Services (INS) we are seeking to achieve best practice processes using the IT Infrastructure Library (ITIL) process framework as a guide. A problem is defined within ITIL as the unknown cause of one or more incidents, however this definition can be extended to include matters which have been identified as having the potential to trigger incidents such as matters which have not been fully resolved at system implementation, and matters which have been identified through proactive monitoring of the IT infrastructure and incident trends. Problem Management focuses on analysing problems with a view to identifying the root cause and then ensuring that satisfactory resolution of the problem is achieved. Problem resolution will often involve changes being to be made to one of more IT infrastructure items (e.g. programs, hardware etc.) and the process disciplines in supporting processes such as Change Management and Release Management will be relied on to ensure problems are successfully resolved. Problem Management will also maintain information about problems and the associated workarounds and/or permanent fixes that are implemented. This will be achieved using the Knowledge Management functionality of the support system and more particularly data repositories such as the Knowledge Base. The availability and proper use of this information will greatly assist in incident management and contribute to more efficient problem resolution, ensuring that client service availability is maximised. This document covers the both the strategic and operational Problem Management practices within INS and details the process flow, business rules, procedures and metrics. It also identifies reporting requirements and the roles and responsibilities of process participants. It is critical that Problem Management links effectively with other INS ITIL oriented processes such as Incident, Change and Knowledge Base Management. These linkages are reflected in the High Level Process Flow which is detailed in the Problem Management Procedures section of this Manual. Benefits of Problem Management Disciplined management of problems will deliver INS and business clients some real and measurable benefits over time; in particular: Higher availability of IT services Higher productivity of business and IT staff Reduced expenditure on workarounds or fixes that do not work Reduction in cost of effort related to fire-fighting and/or resolving repeat incidents Draft Ver 0.4 INS Problem Management Manual.doc Page 4 of 47

5 Better quality management information on the profile and status of problems Related Processes For more detail on interfacing processes refer to the following links: Incident Management Process Handbook Version 1.5 Change Management Process Manual Version 4.0 Knowledge Management Process document Ownership & Responsibility The Manager of the Office of the Director ICTS is the Process Owner The *** To be Determined **** is the Problem Manager, in lieu of a Problem Manager the Service Owner related to the problem will take on the Problem Manager responsibilities. * Service Desk staff & IT Support Specialists are responsible for complying with the documented Problem Management processes and procedures The Manager of the Office of the Director ICTS is the Document Owner *In this document the term Service Desk Analyst is used to represent the first level support staff in the Library & IT Help and EIS Assist teams, and the term Support Specialist is used to represent staff providing higher levels of support (2 nd level, 3 rd level etc. for instance the Server Support Services and the Messaging & Collaboration teams). Document Location: The latest version of this document is able to be accessed on the INS intranet. Refer the following link: \\staff.ad.griffith.edu.au\groups\ins\its\its-projects\ins Projects_Active\INS473 Service Desk replacement (Service-now.com)\ITIL Processes\ITIL Problem Management\Current\Draft Ver 0.3 INS Problem Management Manual.doc Contact Person Any queries or suggestions for improvement to the scope and operation of the Problem Management procedures should be discussed first with the team leader and, as necessary, directed through to the document owner for discussion. Document owner contact details as below: Phone: [email protected] Draft Ver 0.4 INS Problem Management Manual.doc Page 5 of 47

6 Inputs & Outputs of Problem Management Scope This section lists all the relevant inputs and outputs to the process. Inputs Process Inputs Unresolved Incidents from Incident Management Problems identified from formal reviews of Priority 1 incidents. Problems identified from trend analysis and other proactive problem management initiatives Problems identified from the review of status messages and alerts generated from systems, network and application management platforms. Problems identified through system testing e.g. shortfalls in meeting business requirements which require root cause analysis and problem resolution Problems identified post system implementation; e.g. failed changes Notification of CI faults direct from manufacturer Outputs Output Objective Resolved Problems Known Errors Requests for Change (RFC) Management Information Knowledge Articles Description The removal of the underlying cause of problems through reactive and proactive PM activities Once the underlying cause of the problem is identified a known error will ensue. Underlying causes of problems may require an infrastructure change in order to remove them. An RFC must be raised to support the change in line with the established Change Management Process. Regular reports on the performance of the Problem Management process. Formal review reports on major outages and other Priority 1 rated incidents. A known error with a workaround becomes a knowledge article. INS will use the support system Knowledge Base functionality to store Known Errors and any associated workarounds and root cause fixes. These Known Error records will be able to be searched to facilitate resolution of open incidents. Draft Ver 0.4 INS Problem Management Manual.doc Page 6 of 47

7 Problem Management Business Rules This section describes all the process rules that are fundamental to the effective and efficient operation of the INS Problem Management Process. The detailed procedures which are detailed later in this document embody these rules. Business Rules Problem Management 1. Strategy and Communications Business Rule 1.1 The focus of all problem management effort is on detecting and correcting the root cause of problems. 1.2 A single Problem Management process is to be utilised by all INS teams as defined within the Problem Management Manual. 1.3 Support Team Leaders will balance their problem related work efforts between reactive problem management (i.e. acting on advised incidents) and proactive problem management (i.e. identifying trends and targeting preventative actions before problems occur). 1.4 All changes required to resolve problem root causes must be channelled through, and follow the prescribed procedures within the INS Change Management and Release management processes. 1.5 Every problem must have a designated Problem Owner who is responsible for the lifecycle of the problem 1.6 The IT Support Team Leader must ensure that any open problems which are owned by a person who is leaving INS are moved to other team members to action and resolve. 1.7 Every action in relation to a problem is to be documented within the problem record, and the problem status is to be maintained to reflect the true disposition of the problem at a point in time. 1.8 Problem records may only be viewed by INS personnel. Clients will have no access to Problem Records and must direct any problem related queries to either Service Desk or the Problem Owner 1.9 All client queries and communications in regard to problems under management by INS are to be directed through either Service Desk or the nominated problem owner. Activity history will not be available to clients by direct access to the support system Problems are to be managed by team members with the best levels of skills and knowledge and may be reassigned to the most appropriate group during the lifecycle of a problem Process metrics will be developed and regularly reported at both team and senior management levels. Benefit Resolution of problem root cause prevents future related incidents from occurring and improves productivity of support teams. Standard and consistent approach which ensures that all problems have visibility and are managed effectively and efficiently. Increased integration between IT service areas. Reduced service outages and improved SLA performances on system availability Reduction in the number of incidents. Changes implemented have higher likelihood of success following comprehensive testing and approvals. Establishes clear accountability for actions associated with a particular problem. Facilitates reporting by support areas. Ensures that problems are kept under notice and progress to a resolution as soon as practicable. Assists with communication with clients and other support staff. Assists with resolution of future incidents and problems Facilitates accurate reporting. Clients receive the best quality information on the status and nature of the problem. Support staff productivity is improved with fewer distractions to the core activities associated with problem root cause analysis and resolution. Consistent client communications and feedback Reduced risk of activity information being misinterpreted Problems are resolved quicker with more robust client solutions are delivered. Support team knowledge is shared and overall team collaboration is improved. Process improvement areas can be more readily identified and actioned Draft Ver 0.4 INS Problem Management Manual.doc Page 7 of 47

8 2. Recording and Classification Business Rule 2.1 Every problem identified from all channels, including advices from external providers, must be accurately recorded, categorised and prioritised. 2.2 All logged problems are matched against all other current problems. 3. Problem Investigation and Diagnosis Business Rule 3.1 Problem queues and statuses must be regularly reviewed by Support teams. 3.2 The Problem Manager, INS senior management and the relevant Service Manager are automatically advised when Priority 1 incidents occur 3.3 Problems where the cause is known should have their status changed to known errors. 3.4 All workarounds must be discussed and agreed with the client and recorded in the Knowledge Base. 4. Major Problem review (Priority 1 incidents) Business Rule 4.1 The Problem Manager and the relevant Service Line Manager are to be notified of a Priority 1 incident. 4.2 A formal review is to be convened and completed by the Problem Manager for all Priority 1 incidents and a problem record raised to incorporate the report. Any additional problems identified for root cause analysis must be linked back to the parent problem. 4.3 Priority 1 Incident review reports must be tabled for discussion at each fortnightly ICTS Management Team meeting. 5. Problem Resolution and Closure Business Rule 5.1 The root cause of a problem must be confirmed with the relevant Service Line Managers before the problem status is updated to a Known Error. Benefit Facilitates reporting of new problems per channel. Allows for incident trends to be monitored. Work effort can be better directed to addressing problems which are rated as more significant. High urgency/impact problems can be more readily determined and reported. Problem trends can be detected. Problem management staff can work together resolving problem causes. Benefit Any undue delays or bottlenecks in the problem management cycle are identified as early as possible. Provides early advice on any critical service outages so that analysis and recovery efforts can be expedited to reduce system downtime. Ensures the solution is promptly available for support staff. Improved quality of service for clients. Minimise business downtime. Benefit Communicates critical outages to relevant areas A well structured and comprehensive review will identify additional problems and reduce the risk of the same or similar high impact service failures. Review document is readily available to all staff and shares learnings, outcomes and improvement strategies and initiatives. Review findings and any associated recommendations can be reviewed and confirmed by a management group independent of the process operation. Benefit The Service Managers is better informed as to any open problems relating to the service/s under their management. Known errors are accurately defined for future reference. Allows for trend analysis. Future review can lead to known errors being found. Service to clients can be restored. Provides communication details for the client. Knowledge can be shared 5.2 A record is kept of all closed but unresolved problems with documented reasons for not resolving. 5.3 When a problem status is changed to known error, all incident managers with incidents related to the problem are to be informed and the Knowledge Base will be updated. 5.4 Problem resolutions are to be applied only after any Reduces the risk of changes failing on implementation changes have been approved and scheduled for release Minimises rework of changes and increases through the INS Change/Release Management process. productivity of support teams 5.5 The Knowledge Base is updated when a solution is Ensures other support staff have the information they determined need to resolve problems 5.6 Problems can only be closed by the Problem Manager Provides independent assurance to management that the process has been properly followed and the information recorded is of the highest quality (including resolution codes and known error records). Draft Ver 0.4 INS Problem Management Manual.doc Page 8 of 47

9 Urge ncy Enhances the integrity of reporting and provides a more accurate base of information for management decision making on forward problem related strategies and initiatives. Problem Prioritisation All problems logged within the support system will be given a suitable priority. Like incidents, problems will be prioritised using a combination of impact and urgency to the business. The following matrix will be used to determine Problem Priority: The priority of a problem is determined by: 1. Impact: Impact of the problem on the business. The number of clients or importance of system affected. The hierarchical position of the client is included in this variable. 2. Urgency: How severely the client s work process is affected. The Impact/Urgency matrix, shown below, determines the priority of the problem. Impact Low Medium High Low Medium High The assessment methodology for the impact and the severity is explained in more detail below. Note: SLA requirements and policies for Problem Management are not a part of ITIL requirements as such there are no SLA escalations associated with Problem prioritisation. Impact Problems will be placed into High, Medium and Low impact categories. The key factor in measuring impact is the impact the problem has on the business and the following criteria guides the impact assessment. Impact High Medium Low Description Whole organisation affected; Site or multiple sites affected; Multiple groups of clients affected; Critical business process interrupted; or System-wide outages to Learning@Griffith, Staff portal, or Group of clients, a Pro Vice Chancellor (PVC), or a member of the Vice Chancellor s (VC s) Office staff affected; Non-critical business process interrupted. One client affected (other than VC s Office or PVCs) Draft Ver 0.4 INS Problem Management Manual.doc Page 9 of 47

10 Urgency Problems will be placed into High, Medium and Low urgency categories. The key factor in measuring urgency is how severely the client s work process is affected and the following criteria guides the urgency assessment. Urgency High Medium Low Description Process stopped; client(s) cannot work Process affected; client(s) cannot use certain functions Process not affected; change request, new/extra/optimised function Problem Categorisation All problems opened in Service Desk Tool will be categorised by Service Line and associated Service. This categorisation information is presented in drop down lists and will be maintained to align with the INS Service Catalogue information that is incorporated in the INS to Griffith University Service Level Agreement 2010/2011. The Service Line and Service fields are mandatory when creating a Problem record. Categorising problems in this way enables the Service Level Manager to profile open problems by Service Lines and Services and better positions INS for communications with business on service related matters. The following link displays the current INS Service Catalogue incorporating Service Lines, associated Services and detailed service descriptions. **** Insert hyperlink to INS Services Catalogue ***** Draft Ver 0.4 INS Problem Management Manual.doc Page 10 of 47

11 Problem Status Codes Scope The status of a problem reflects the current position in its lifecycle, sometimes known as its 'workflow position'. All problems logged in the support system will have the relevant status applied at each stage of its progression toward closure. The Status is a mandatory field. Status Codes The following status codes will be used for problems logged by INS: Problem Status Codes Open Work in Progress Known Error Awaiting External Response Awaiting Change Request Completion Resolved Closed No QA issues Explanation This is the system default when a problem record is first created and indicates that a problem is logged but no action has yet been taken. Indicates that problem investigation and diagnosis has commenced and/or actions are underway to develop the solution or workaround. Indicates that INS has raised a known error record and communicated the scope and nature of the problem out to all stakeholders. INS has determined that widespread knowledge of the problem would be beneficial. A response from an external vendor is required to inform the problem resolution. Indicates that a Request For Change (RFC) has been raised to resolve the problem and work on the RFC is still in progress. Note: The problem record will be linked to the RFC item (if the RFC is a standard change) or referenced to the RFC item (if the RFC is a nonstandard change within the SQISS system). This status indicates that EITHER a permanent solution to the problem has been found, applied and tested successfully to meet business and/or INS requirements. OR it has been determined, in agreement with business, that a permanent solution will not be implemented due to technical, financial or other business reasons. Indicates formal closure of the problem and also that the quality assurance review performed by the Problem Manager detected no issues requiring discussion and resolution prior to closure. Closed QA Issues Indicates formal closure of the problem and also that the quality assurance review performed by the Problem Manager detected some issues requiring discussion and resolution prior to closure. Work Around: A work around indicates that a client agreed work around has been implemented. Work is still proceeding to establish a permanent fix to the problem and as such the Work in Progress status code should be used. Workarounds can be improved and updated at any stage of the process, refer to section Advise Workarounds Draft Ver 0.4 INS Problem Management Manual.doc Page 11 of 47

12 Problem Resolution Codes Scope In addition to the categorisation of problems, which can in itself be a valuable source of information as to the types of problems being experienced, resolution codes should be used to provide additional information about how the problem was resolved. A resolution code should be assigned to the problem, indicating the type of resolution action that was eventually taken. All problems logged in the support system will have the relevant resolution code applied Resolution Codes The following resolution codes will be used by INS. Problem Resolution Code Fix with changes/s required Fix Provided No Fix Available Workaround Provided NFA Unknown Cancelled Explanation Used when raising a Change is necessary to solve the root cause of the problem. Used when an explanation of how to fix the problem is supplied to the client and the fix does not require a change to any configuration item. Used when there is no fix available to a particular problem and a decision has been made by INS to live with the problem and associated incidents. Used where an effective workaround to a problem has been developed and a permanent fix has been decided against. Used when the problem cause goes away with no action having been taken. This status is applied when it has become apparent that the record raised is not appropriate (e.g. duplicate record or as a result of a team leader review). Draft Ver 0.4 INS Problem Management Manual.doc Page 12 of 47

13 Problem Management Procedures The following section documents the workflow and specific procedures to be followed during the lifecycle of a problem under INS management. High Level Process Flow INS Problem Management: High Level Process Flow Client Service Desk Problem Manager INS Support Team (Technical & Applications) Source: Receive Third Party Problem Advices; Perform Proactive Problem Monitoring; Identify Problem/s at System Implementation; Perform Proactive Problem Monitoring; Service Desk and Incident Management Process Gover nance 1.0 Problem Identification and Recording 2.0 Categorise and Prioritise Problem 3.0 Investigate & Diagnose Problem Yes No 4.0 Workarounds & Known Errors Assessment Investigate Further? Workaround Solution? Root Cause Known? No Yes Fix Identified? No No Yes Change Needed? Known Error Knowledge Database Yes Change Management Process Change Successful? Yes Yes 5.0 Resolve Problem 6.0 Close Problem 7.0 Communication End No No Draft Ver 0.4 INS Problem Management Manual.doc Page 13 of 47

14 Defined Problem Process Flow INS Problem Management: Defined Process Flow Problem Manager Governance Perform Proactive Problem Monitoring A B Conduct Review Additional Problems Identified? Yes No A Review Problems/ Reports Issue Report 6.0 Close Problem End INS Support Team (Technical & Applications) Start Perform Proactive Problem Monitoring Receive Third Party Problem Advices Identify Problem/s at System Implementation A 1.2 Raise Problem Record Yes 1.1 Problem Exists? Known Error Knowledge Database 2.0 Categorise and Prioritise Problem Update/Post Knowledge 3.0 Investigate & Diagnose Problem Yes C 4.0 Workarounds & Known Errors Assessment Investigate Further? No Workaround? Root Cause Known? No Yes Yes Communicate Workaround/Fix Record Known Error C No Workaround solution? Fix Identified? No Yes Yes C No Change Needed? Yes Document Fix C C Raise RFC Change Management Process Yes Change Successful? No E 5.0 Resolve Problem No No Client Service Desk Incident Management Process 1.1 Problem Identified? Yes Problem Record exists? No Yes No Link Incident Escalate Incident Priority 1 Incident? Yes Manage Major Incident A B Communicate Problem Status Query Problem Status Receive Advice of Workaround /Fix Draft Ver 0.4 INS Problem Management Manual.doc Page 14 of 47

15 1.0 Problem Identification and Recording Scope These procedures deal with the initial identification and recording of problems. Within INS this will normally be performed by the IT Systems Support. The Problem Manager will raise a problem record for every Priority 1 incident that occurs. These incidents demand a stringent review due to the criticality of impact. All problems will be recorded in the support system so that they can be tracked, monitored, and updated throughout their life cycle. This information can then be utilised for proactive problem management, reporting, process optimisation, and planning purposes. Activity 1.1 Identification Sub Activity Description In order for Problems to be managed, they first have to be identified. There are a number of triggers for a problem record to be raised; i.e. when: matching the reported incident to existing problems and known errors is not successful during the initial incident support and classification stage and a solution is not immediately evident to the support specialist analysis of incident data reveals recurrent incidents analysis of incident data reveals incidents that are not yet matched to existing problems or known errors analysis of the IT infrastructure indicates a problem that could potentially lead to incidents notification is received from a supplier that a problem exists that needs to be resolved testing associated with systems approved for production implementation discloses issues where user requirements are not being fully met a change fails and an infrastructure related problem is identified a review of a Priority 1 incident discloses issue/s which require structural solution/s to be found Once a Problem has been identified go to 1.2 Recording 1.2 Recording Basic Facts All identified problems will be formally recorded and categorised within the support system so that problem statuses can be readily determined and reported on as necessary. A problem record will be raised with an initial status of open and will be automatically be assigned a problem record number. A problem record can be raised from within a Service Request record or alternatively by selecting the Create Problem Record from the main menu. Throughout the problem life cycle, the problem will pass through Draft Ver 0.4 INS Problem Management Manual.doc Page 15 of 47

16 Activity Sub Activity Description a number of different states before finally being closed. The status field within the support system is used to quickly identify a problem's current state and facilitate any communications to clients. These status values are explained in more detail at Section headed Problem Status Codes. This status field will default on initial recording of the problem to open. It is important that the status field is kept up-to-date; so that all IT staff can easily determine the current state of the problem. Each problem record will be given a unique reference ID automatically by the support system. All users that have an incident appended to the problem will receive an auto through the support system to inform them that a problem exists related to their incident and provide them with the problem ID number. This ID will be used to easily locate the correct record if the user contacts Service Desk again or uses the web portal to update information or check progress. Go to Is the problem related to an incident escalated from Library and IT Help? Recording problems associated with an escalated incident If yes go to otherwise go to When the problem is identified from diagnosis of a referred incident create a new problem record using the create problem record tab from within the incident record. The user information will auto populate the problem record from the service request (incident) record and each problem record that is raised from within a service request record will be automatically linked to the service request record. Descriptions should be succinct but thorough, containing all relevant details such as: WHAT is the failure? WHERE is the failure located? WHEN did the failure occur? WHY did the failure occur? HOW did the failure occur? The standard is that any other support specialist should be able to Draft Ver 0.4 INS Problem Management Manual.doc Page 16 of 47

17 Activity Sub Activity Description read the description, understand it and be able to progress the problem at any time during its life cycle. It is also important that any Service Desk analyst can understand the description so they can keep the user informed. Service Desk is responsible for initially obtaining all the required information from the user when recording the now appended incident. However, there may be instances when some clarification or additional information is required. In these instances, the support specialist should contact the user to obtain the information. Go to 2.0 Problem Prioritisation and Categorisation Procedure Recording problems not associated with a referred incident A system menu item termed Create Problem Record should be used to create the problem record when the problem has been identified from an event such as a failed change or as a result of a detailed review of a Type 1 incident. Refer to for Problem Logging Form and data field descriptions. If available, the Configuration Item (CI) identifier must be recorded in the problem record. System drop down lists will facilitate identifying CIs. Recording this information assists future analysis and reporting of where problems are occurring in the IT infrastructure. (Note: As at January 2010 the INS Configuration Management Data Base has only been partly developed). Where the initial incident is reported because of an event or an auto alert from a monitoring system, the event or alert reference number should be included within the problem record. This allows support specialists investigating the problem to identify and view the original event or alert. The IT Support Analyst must append the incident/s auto generated by the alert that initiated the problem investigation. Any further incidents reported relating to the CI problem will be appended by Service Desk. The support specialist needs to be aware of the number of incidents being appended. If the number becomes too high the problem priority may need to change. Go to 2.0 Problem Prioritisation and Categorisation Procedure Draft Ver 0.4 INS Problem Management Manual.doc Page 17 of 47

18 2.0 Problem Prioritisation and Categorisation Procedure Scope Problems must be appropriately prioritised and categorised so they can be handled as effectively as possible. This procedure covers: Defining the priority of the problem. Specifying the related business service Specifying the primary Configuration Item (CI) affected. Identify the appropriate 3 rd party vendor for escalation (if appropriate). Activity 2.1 Problem Classification Sub Activity Complete Priority Field Complete Category Fields Description This is a mandatory field and is to be completed for every problem record created. There is no support system default for this field. Each support specialist is to complete the field according to the INS approved Priority Matrix. To facilitate completion refer to section of the document titled Problem Prioritisation. The support system provides a drop down list to select Once the impact and urgency fields have been completed within the support system, the priority field will auto populate with the relevant priority. Go to Before any further action can be taken the relevant category fields must be completed. The Business Service fields are to be completed using the approved list of INS Service Lines and associated Services. Refer to section of document titled Problem Categorisation. Go to 3.0 Problem Investigation & Diagnosis 3.0 Problem Investigation & Diagnosis Procedure Scope These procedures deal with the investigation of the problem and the diagnosis of the root cause. This data can then be used to help the support group assess the resources and skills required to resolve the cause of the problem. Investigation activities should include the provision of workarounds as soon as possible for the incidents related to the Problem so the end user can continue with business priorities. Draft Ver 0.4 INS Problem Management Manual.doc Page 18 of 47

19 Activity Sub Activity Description Is the problem a priority 1? If yes go to 3.1.1, if no go to Investigation & Diagnosis Major Incident Procedure Major Incidents are those for which the degree of impact on the user community is significant and at INS all typically rated as Priority 1. The Service Desk makes the initial incident rating which in turn is confirmed by the Problem Manager who is required to raise a new problem record associated with the Parent Priority 1 incident record. The relevant Support Group Team Leader will be responsible for coordinating recovery action and keeping Service Desk Team Leader informed of progress. Regular communication both within IT and with the affected users is paramount under these circumstances Service Desk Team Leader will be responsible for communication and liaison with affected users or business units. The normal investigation and diagnosis action will commence immediately with the major problem given priority over all other problems being investigated. Go to Investigate & Diagnose Investigation will commence to try to diagnose the root cause of the problem. Once investigation commences change the problem status to Work in Progress The support specialist will then undertake a more in-depth search of the knowledge base, known error database and any other technical resource material to analyse the problem further and attempt to find a solution. If the search is unsuccessful there are a variety of sources that can be used to assist in the investigation of problems. The following are often valuable resources for this data: External vendors. Manufacturers can provide valuable information about components and infrastructure items that they are producing. They commonly produce information on known errors that either they or other organisations using the products have experienced. Suppliers can provide information about upgraded products that can be used to resolve known errors. Users of the service. Users of the service can provide additional information about how the service is affected or important Draft Ver 0.4 INS Problem Management Manual.doc Page 19 of 47

20 Activity Sub Activity Description background information concerning the operational requirements of the services that are affected. Forums and user groups. User groups can be an excellent source of information for solutions to problems and can save valuable and costly investigation and diagnostic time. Log files. Event logs can be analysed to provide a history of events and activities taking place at the point of failure. Development cycle. Development staff know of known errors identified during other development projects, and this information should be made available for inclusion in the knowledge base linked to the known error database. Internet. The Internet is a valuable repository of information and contains many useful sites, user communities, supplier information pages, FAQs, and a host of other sources of information, which can be used for analysis. If the root cause cannot be determined by using the above resources then the next step should be to investigate which similar parts in a similar environment are functioning properly. With this, an answer can be formulated to the question of which parts could be showing the same problem but are not. It should then be possible to search effectively for relevant differences in both situations. Furthermore, past changes, which could be the cause of these differences, should be identified. The list of differences and changes thus generated will most likely contain the cause of the problem. Attempts should be made to extract the possible causes from this list. Each possible cause should be assessed to determine whether it could be the cause of the problem's symptoms. In this way, some of the possible causes can be eliminated. The remaining possible causes should be checked to see whether they are the source of the problem. Support specialists should first address the possible causes that can be verified quickly and simply. During problem investigation, it can be beneficial to sort the collected evidence into some sort of order or timeline. What at first might seem to be a collection of unconnected events or error messages may reveal a pattern when placed in timeline order. Similarly, a group of apparently random failures occurring over a number of servers may, once sorted by server address, reveal that the problem is limited to servers on a particular network segment. The use of sorting to reveal patterns within problem evidence can be an invaluable technique during problem analysis. Progress should be regularly reviewed. In instances where Draft Ver 0.4 INS Problem Management Manual.doc Page 20 of 47

21 Activity Sub Activity Description sufficient progress is not occurring, the support specialist should initiate management escalation by notifying the team leader. The team leader should discuss the problem and confirm the priority, resources, and time that should be allocated to it. If at any stage the support specialist feels that the problem should now be designated a "major problem", they should contact the team leader, who then becomes responsible for making this decision. If the root cause cannot be found go to If the root cause is found and the problem is diagnosed go to rd Party Support Where the root cause of a problem cannot be diagnosed, the problem needs to be referred to a third party for further investigation, change the status to Awaiting External Response and monitor against the Underpinning Contract with the third party supplier. Once a resolution or workaround is provided by the third party supplier go to 4.0 Workarounds & Known Errors Should the third party supplier be able to assist with resolution a decision the disposition of the problem needs to be discussed with the Problem Owner. Go to 5.0 Problem Resolution Draft Ver 0.4 INS Problem Management Manual.doc Page 21 of 47

22 4.0 Workarounds & Known Errors Scope Finding workarounds and creating known errors are the elements within problem management that allow the problem to progress to final resolution. It covers the procedures involved in developing workarounds and creating known errors. The objective is to change IT components, systems or procedures to remove problems affecting the IT infrastructure and thus prevent any recurrence of incidents. Workarounds and known errors directly interface with and operate alongside the change management, release& deployment management & knowledge management processes. Activity Sub Activity Description 4.1 Assessment Workaround Once problem investigation and diagnosis has commenced the IT Support Specialist will change the problem status to work in progress and begin the task of finding a workaround. If a workaround is possible and subsequently developed, test the workaround to ensure suitability for use (do not just assume it works!). Even when a workaround has been found, it is still important that work on a permanent resolution continues (where this is justified) to ensure the Incidents do not continue to occur. In some instances a permanent fix may be identified that can be implemented just as quickly as developing a workaround. Go to If, after assessment, a workaround cannot be developed update the problem record with the relevant details and go to 4.2 Request For Change Known Error A known error is a Problem for which the root cause is known and ideally a temporary workaround or a permanent fix has been identified. At this stage the problem will move to a Known Error status and a record created in the Knowledge Base using the Post to Knowledge tab that is available within the problem record. The problem record will be linked automatically to the known error record. The Known Error record should be raised as soon as it is deemed useful to do so and certainly must be raised when diagnosis is complete and a workaround has been found (even though it may not yet be a permanent resolution). Draft Ver 0.4 INS Problem Management Manual.doc Page 22 of 47

23 Activity Sub Activity Description Update the known error record with a detailed description of the workaround or the permanent fix. The support system will link this to the knowledge base and automatically inform Service Desk so they can inform the affected users. In cases where a workaround is found, it is therefore important that the problem record remains open and details of the workaround are always documented within the Problem Record A workaround should contain enough detail to ensure that the workaround can be implemented successfully. It is only necessary to complete this tab field in the case where a workaround has been identified; otherwise this field should remain blank. Go to 4.2. Draft Ver 0.4 INS Problem Management Manual.doc Page 23 of 47

24 5.0 Problem Resolution Scope These procedures cover the steps required to resolve Problems. Problem resolution details are to be fully recorded in the support system. It is vital to save data on the configuration items (CI s), symptoms, and resolution or circumvention actions relating to all problems so as to build up the knowledge base. This data is then available for incident matching, providing guidance during further investigations on resolving and circumventing incidents, and for providing management information. Activity Sub Activity Description 5.1 Solution Does the problem have a permanent solution? If no, go to If yes, go to No Permanent Solution No Fix Available: If there is no workaround available and a decision has been made to permanently live with the problem then the problem can be resolved and the support specialist should complete the following steps: Select Resolved from the problem status code drop down list (Note: Library & IT Help will be notified automatically by the support system of the resolved status Select No Fix Available from the Resolution Code drop down list and type in text that further explains how the resolution was reached. Ring the client/s and personally explain the situation giving the reason that the problem cannot proceed further. Save the resolution and exit the problem record. Draft Ver 0.4 INS Problem Management Manual.doc Page 24 of 47

25 Activity Sub Activity Description Workaround No Fix Available: If a workaround has been developed and is considered to provide the best permanent solution to the problem without any configuration changes required, the support specialist should complete the following steps: Ensure all related knowledge articles including the workaround details are completed and then posted to the Knowledge Base using the Post to Knowledge functionality. Select Resolve Related Service Requests from the Problem form drop down action list. This will update the related Service Requests status to resolve and send the automated resolution to the client. Any knowledge articles will be automatically attached. Select Resolved from the problem status code drop down list (Note: Library & IT Help will be notified automatically by the support system of the resolved status. Select Workaround NFA from the Resolution Code drop down list and type in text that further explains the resolution that was reached. Communicate the workaround to the client using the Communicate Workaround button. This has the effect of automatically notifying all clients associated with incidents which are linked to the problem record. Ring the client/s and personally explain the situation giving the reason that the problem cannot proceed further. Save the resolution and exit the problem record. Refer to 7.0 Communications Go to 5.3 Perform pre close check Is a change required? If no, go to If yes, go to Draft Ver 0.4 INS Problem Management Manual.doc Page 25 of 47

26 Activity Sub Activity Description Permanent Solution - No changes required Fix Provided : A problem may be able to be permanently solved by implementing a fix which does not require a change to any configuration items. In these cases the support specialist should take the following steps: Select Resolved from the problem status code drop down list (Note: Library & IT Help will be notified automatically by the support system of the resolved status). Select Fix Provided from the Resolution Code drop down list and type in text that further explains how the resolution was reached. Document the fix provided in the Work Notes section of the problem record, making sure that the explanation of the fix is clear and understandable. Click on the Post Knowledge tab and a Knowledge article will be submitted for inclusion in the Knowledge Base. Save the resolution and exit the problem record. Refer to 7.0 Communications Go to 5.3 Perform pre close check Raise RFC In the most likely case, when a non standard change is required, then an RFC should be raised in the INS Change Management system (i.e. outside of the support system). The problem status must then be updated to Awaiting Change Request Completion. If the change is approved by the relevant Product Service Manager go to If change not approved go to Build Change & Release Once the change is approved it is built and released according to the relevant procedures outlined in the INS Change Management Process. If the problem is resolved go to If the problem is not resolved after implementing the change go to Draft Ver 0.4 INS Problem Management Manual.doc Page 26 of 47

27 Activity Sub Activity Description Permanent Solution - Changes required Fix with change/s required: In most cases it will be necessary to raise a Change to manage the implementation of a permanent solution. If this is the case then the support specialist should take the following steps: Raise a Request For Change (RFC) and forward the RFC through to the Change Management process for review and approval to proceed. (Note: the RFCs will typically be non-standard changes which are not in scope for the first implementation of the support system). Once the change has been tested and proven, select Resolved from the problem status code drop down list (Note: Library & IT Help will be notified automatically by the support system of the resolved status) Select Fix with change/s required from the Resolution Code drop down list and type in text that further explains how the resolution was reached. Manually reference the change control documentation in RFC Link in the Problem Linking area of the Problem record so that it is able to be linked back to details of change/s, testing etc. Click on the Post Knowledge tab and a Knowledge article will be submitted for inclusion in the Knowledge Base. Save the resolution and exit the problem record. Refer to 7.0 Communications Go to 5.3 Perform pre close check Draft Ver 0.4 INS Problem Management Manual.doc Page 27 of 47

28 Activity Sub Activity Description Live with problem A decision may be made by the Product Service Manager not to fix a problem by implementing a change. Some typical reasons for not fixing a problem through a change might include: Too costly a solution or costs outweighed by the delivered benefits. Only causing minimal "pain" at the current time, so resources better used elsewhere. Insufficient budget left during the current financial year. Lack of skills available to resolve. The recommended solution carries inherent business risk. Very effective workaround currently established. Planned technology refresh will remove this problem in the future. Infrastructure requirements are due to change as a new business direction is being approved. Application/service nearing the end of its life cycle. In these cases the problem remains a known error and can move to resolution. Refer to 7.0 Communications Go to 5.3 Perform pre close check 5.3 Perform pre close check Review problem information Before readying the problem record for final closure by the Problem Manager locate problem in the support system and check to ensure that all the following information is recorded and is accurate: All client and problem details Configuration Item information Workaround or resolution is captured in the Knowledge Base Known error/problem information is captured Resolution code is recorded and is accurate Related records have been resolved Go to 6.0 Problem Closure Draft Ver 0.4 INS Problem Management Manual.doc Page 28 of 47

29 6.0 Problem Closure Scope These procedures cover the final phase in the problem lifecycle and provide for a quality assurance check by the Problem Manager prior to final closure of the problem. This check will confirm the accuracy of information, including resolution codes and also confirm that all necessary process steps have been followed and all required documentation attached. All High Impact Incidents (Priority 1) must have a Problem record raised by the Problem Manager, who in turn will conduct a comprehensive review of the circumstances of these Priority 1 incidents. This formal review may disclose additional problems and result in additional problem records being raised. Activity Sub Activity Description 6.1 Determine problem priority Does the closure relate to a Priority 1 Incident? If no go to If yes, go to Pre closure Quality Assurance (QA) Check Major Problem Review Prior to formal closure of the problem record the Problem manager must work through a checklist of items to confirm that the proper steps have been followed and the information contained in the problem record and associated data bases such as the Knowledge Base is comprehensive and accurate. Should there be issues raised then these issues must be successfully resolved to the satisfaction of the Problem Manager prior to the problem record being closed. Once this has been achieved the Problem Status must then be maintained to Closed QA issues. Alternatively, if the QA check discloses no issues then the Problem Status must then be maintained to Closed No QA issues. There must be a clear trail of what issues were raised and responded to captured in the Knowledge base so that the metric related to closure can be better informed (Refer Process Metric 1.2 in latter part of this Manual. High impact problems warrant careful and comprehensive review to minimise the potential for similar system outages to occur in the future. In this regard then, on completion of the resolution of a Priority 1 incidents and in addition to confirming that the QA check items at have been successfully addressed, it is important that an independent review is completed by the Problem Manager. Draft Ver 0.4 INS Problem Management Manual.doc Page 29 of 47

30 Activity Sub Activity Description The review must examine and confirm the following aspects: The background and scope of the problem was well described The priority level was appropriate to the circumstances of the service outage All issues arising from the service outage were identified and where appropriate, additional problem records were created The lessons learned from the service outage have been clearly detailed and communicated to the problem governance committee (i.e. the ICTS Management Team) What additional preventative measures need to be taken What additional pro active measures could be taken to forewarn on the service outage A formal report was prepared by the IT Team leading the problem resolution Draft Ver 0.4 INS Problem Management Manual.doc Page 30 of 47

31 7.0 Communication Scope This section covers the various communications that occur during the problem lifecycle to ensure that the clear accountability for actions is assigned and to ensure that all relevant stakeholders are kept fully briefed on the progress that is being made to resolve a problem. Activity 7.1 Problem related advices 7.2 Communications Sub Activity Automated Notices Advise Workarounds Major Problems Description The support system provides automated notifications when the following events occur: A problem is re-assigned to a support specialist within a particular IT Support Group during the course of the problem being investigated, diagnosed and resolved (the new assignee receives the automated advice) Problem ownership is transferred to a new IT Support group (the Team Lead of the group to which the problem has been redirected receives the automated advice). A comment is appended to a problem record by a person other than the assignee (the assignee receives the automated advice which details the id of the person adding comments and the text of the comments) The support system provides functionality to facilitate advising clients of workarounds and other information relating to the problem under management. To advise a client of a developed work around the support specialist must enter workaround details in the Work Area section of the problem record and then click on the Communicate workaround (refer 4.1.2). On the rare occasion when a Problem Stakeholder has been added into the Stakeholder group after a work around has gone out then the workaround detail must be cut and pasted and sent out to the stakeholder in a separate from the IT Support Specialist. Any other general communications can be sent through to stakeholders using the Client Communications. In the case of Priority 1 Problems (Major Problems), the IT Team Leader responsible for the applicable support group is required to keep both the PVC (information Services) and the Director ICTS fully briefed on the status of the problem. Draft Ver 0.4 INS Problem Management Manual.doc Page 31 of 47

32 Activity Sub Activity Description Refer to the Incident Management Handbook Section 2.6 for full details. In addition a Post Implementation Review will be conducted for each Major Problem. Refer to Major Problem Review for full details Advanced Warning At any time should there be an expectation of a failure or loss of a service due to a problem, a communication will be forwarded to clients through Service Desk, providing reasonable forewarning (if possible) Progress Reports The Incident Management Handbook (Section 7 major Outage Procedure) covers the communication protocols for those Priority 1 problems which are determined to be major outages; i.e. an incident that results in significant destruction to GU staff and students. More generally clients may approach the Service Desk to be informed on the status of any problem under management by INS. Draft Ver 0.4 INS Problem Management Manual.doc Page 32 of 47

33 8.0 Proactive Problem Analysis Scope Proactive analysis activities are concerned with identifying and resolving problems and known errors before incidents occur, thus minimising the adverse impact on the service and business as a whole. The main activities are trend analysis and the targeting of preventive action. Problem prevention ranges from prevention of individual problems, such as repeated difficulties with a particular feature of a system, to strategic decisions to fundamentally change some elements or the entire infrastructure. Activity 8.1 Proactive Analysis Sub Activity Description Problem management is responsible for analysing incident, problem, and known error data to produce management information and identify underlying problems. The objective is to be proactive by identifying areas where there is an increasing trend for a particular type of incident or problem. These trends should then be investigated and their root cause determined. The root cause should then be logged as a problem record and a resolution found, so as to counter any trends before they become major issues for the University. 8.2 Trend Analysis Each IT Support Group team leader and the Service Desk team leader will allocate one staff member for half a day per month to trend analysis. This will be on a rotational basis amongst their staff. The support specialist or Service Desk analyst rostered should seek to identify trends within historical data by considering these questions: Is the number of incidents of a particular type (category or sub category) increasing? Is the number of incidents assigned a particular closure category increasing? Is the number of incidents within a particular site or part of the infrastructure increasing? Trends in these areas may indicate underlying infrastructure problems, increasing support requirements, the need for improved documentation, or specific client training needs. Decreasing trends may indicate changing support Draft Ver 0.4 INS Problem Management Manual.doc Page 33 of 47

34 Activity Sub Activity Description requirements, dissatisfaction with the support service, or shortcutting of the incident management process. Not all trends indicate underlying problems, but IT support teams should attempt to understand why trends are occurring. Other trends that should also be questioned include: Is the number of unresolved incidents for a particular category or sub category increasing? Is the overall number of unresolved incidents increasing? Is the number of unresolved incidents assigned to a particular support group or individual increasing? Is the number of unresolved calls increasing within any particular tier of the support structure? Is the number of incidents being resolved within particular resolution groups or tiers changing? Trends in these areas may indicate changing support workloads and skills or resource shortages within resolution groups. Some of these trends can also indicate how well the incident management process and particularly the functional escalation elements are working. Another useful area to analyse is: The number of incidents being logged at each priority level. The number of incidents being identified as "major incidents." The number of incidents reaching high priority at some stage during their life cycle. Trends in this area can reflect performance in proactively identifying and resolving issues before they become high priority or major incidents. Draft Ver 0.4 INS Problem Management Manual.doc Page 34 of 47

35 Activity 8.3 Preventive Action Sub Activity Description Analysis of problem and known error data can also make an important contribution to the IT strategy and can indicate where preventive action is required. As such, trends should be reported as soon as possible. If high impact problems or known errors exist with regard to a particular type of configuration item (CI), the replacement or upgrading of the CIs should be considered for inclusion in the strategy. A large number of lower impact problems on a particular CI type may mean that the future of the CI type needs to be evaluated. Any identified trends will be reported at the next weekly team leader s meeting. Draft Ver 0.4 INS Problem Management Manual.doc Page 35 of 47

36 Problem Management Roles Scope Following are detailed descriptions of the key roles within the Problem Management process. The key roles in INS Problem Management process are: Problem Governance Team Process Owner Problem Manager IT Support Specialists Service Desk (Service Desk & EIS Assist) Client The authorities, tasks and responsibilities of each of these roles follow: Problem Governance Team Profile The existing ICTS Management Team will operate as the governance body for INS Problem Management. The Process Owner has a seat in this governance body which meets fortnightly. The main role of the Problem Governance Team is that the Problem Management processes and accompanying procedures continue to produce the required service delivery outcomes. Authorities The Problem Governance Team is authorised to: - Provide high level direction to the Process Owner on the current and future use of the Problem Management process. - Provide final approval on all change and budget initiatives proposed by the Process Owner. - Provide final resolution on any escalated issues that are related to the operational running of the Problem Management process. - Provide final approval on the Problem Management metrics and reports. Tasks/Responsibilities The Problem Governance Team will: Convene fortnightly to consider Problem Management as a standing agenda item with a strong focus on the implementation of proactive problem management processes and procedures across all of INS. Review and provide feedback on monthly process metric reports forwarded through from the Process Manager each month Draft Ver 0.4 INS Problem Management Manual.doc Page 36 of 47

37 Process Owner Profile The person fulfilling this role has the overall responsibility for the integrity of the end-to-end Problem Management Process and the way in which it functions and develops. The main role of the Process Owner is to ensure that the design of the processes and support systems are efficient, effective, and fit-for-purpose. The Process Owner ensures the successful integration of Problem Management processes with other related processes such as Incident Management, Change Management, Knowledge Base Management and Service Level Management. Authorities The Problem Management Process Owner is authorised to: - take all necessary actions to ensure that all of INS use the Problem Management as directed by documented policy and procedures - Approve any changes to the Problem Management process. - Provide budget for training IT staff. - Decide the priorities if there is a conflict between ITIL processes. Tasks/Responsibilities The Process Owner will: Oversee the design and implementation of the suite of problem related reports Oversee the implementation of process Key Performance measures and metrics, and set service improvement targets. Analyse and provide feedback to the Problem Manager on the configured reports. Review and approve proposed changes to the Problem Management policy and procedures. Review integration issues between the various processes and make decisions on priorities to resolve these issues. Communicate changes to the Problem Management process to all stakeholders. Establish training budgets and review and approve training requests Attend senior management meetings to assess the impact of organisational decisions on the Problem Management environment. Recruit Problem Management staff where needed. Draft Ver 0.4 INS Problem Management Manual.doc Page 37 of 47

38 Problem Manager Profile The Problem Manager reports to the Process Owner and performs the day-to-day operational and managerial tasks demanded by the process flows. This role quality assures the problem record prior to closure. The Problem Manager has delegated responsibility to identify opportunities for improvement and for auditing the use of the process on an operational level to ensure compliance to the process by all IT Support Team Leaders and Analysts. The Problem Manager is responsible for reporting to other ITIL Processes (where required) and ensuring the output of the process in terms of agreed KPIs. The Problem Manager is responsible for ensuring that processes are used correctly, in as much, that he is the joint custodian of quality for this specific discipline. Authorities The Problem Manager is authorised to: - Formally close all problem records. - Propose any changes to the Problem Management process. Tasks/Responsibilities The Problem Manager will: Promote the Problem Management process and perform periodic audits to confirm that the specified process is being correctly followed. Provide IT management and other processes with steering information. Co-ordinate and develop process metric related and any other reporting to meet the requirements of all process participants, in particular the Process Governance Committee (ICTS Management Team) and IT Support Team Leaders. Ensure that the Problem Management process operates effectively and efficiently and interfaces seamlessly with other supporting processes such as Incident Management, Change Management and Knowledge Management. Escalate any issues disclosed with process operation or design to the Process Owner. Ensure that staff are trained, competent and empowered in their jobs in regards to Problem Management and provide coaching as required. Monitor the Problem Management process, using reports against Key Performance Indicators. When applicable, function as a point of escalation. Identify opportunities for process improvement and make relevant recommendations to the Process Owner. Perform quality assurance activities at problem closure and at various times in the problem management lifecycle so as to ensure the highest quality of captured information. Draft Ver 0.4 INS Problem Management Manual.doc Page 38 of 47

39 Ensure knowledge information is updated in a timely manner and that solutions are easy to understand and able to be explained by Service Desk to affected users in terms that they can understand. Maintain all associated all process associated documentation (work instructions, tables, checklists) and circulate to all appropriate staff. Service Desk (Service Desk and EIS Assist) Profile All IT related incidents are referred in the first instance to either one of the two Service Desks in INS; i.e. EIS Assist (for incidents relating to specific enterprise wide applications) or Library and IT Help (for all other incidents). Both of these Service Desks operate as first tier support and depending on their levels of skill and knowledge will attempt to resolve the incident. Both these Service Desks are focussed on a restore to service for the client and may escalate the incident to relevant IT Support Specialists for further analysis and diagnosis. These Service Desks provide a standard point of contact for any client queries regarding open problems. Authorities The Service Desk is authorised to: Escalate incidents to the appropriate second or third tier support team. Propose any changes to the Problem Management process. Provide advice to clients on the problem status (after consultation with the relevant support team) Tasks/Responsibilities The Service Desk function will: Ensure that client requests in regard to problem status are responded to in a timely manner Facilitate communications between IT Support Specialists and staff and the client as and when required to enable the progress of the Problem. Resolve and close all incidents where a client restore to service outcome has been achieved. Draft Ver 0.4 INS Problem Management Manual.doc Page 39 of 47

40 IT Support Specialist Profile Support Specialist is a generic name, which refers to any INS staff member who is part of a support group (be that second level, third level or beyond). At times the INS support groups may refer problems out to an external vendor for advice and/or resolution, however the management and ownership of the problem remains with the relevant INS support group. The role of the Support Specialist is to attempt to resolve all assigned problems. Authorities The IT Support Specialist is authorised to: Detect problems (through reactive or proactive means) Create problem records in the Service Desk Tool system Prioritise the problem and categorise the problem against the appropriate service line and service category Reassign problems to other support groups when appropriate. Record detailed work arounds and/or permanent fixes to each problem in the Knowledge Base Liaise with clients on problem status Indicate a need for more training and/or information where identified as being required. Make suggestions through the Team Lead as to how the Problem Management process could be improved Tasks/Responsibilities Reactive Problem Management Identify Problems (by analysing incidents escalated from Service Desk). Investigate Problems, according to priority, through to resolution or error identification. Update Problem records as required and as the status changes. Raise Requests For Change (RFC s) to clear errors. Monitor progress on the resolution of problem records against SLA metrics. Advise Service Desk staff on the best available workarounds for Incidents related to unresolved Problems/Known Errors. Assist with the handling of Major Problems and in identifying the root cause. Obtain the technical and organisational knowledge required to perform these activities. If applicable, escalate problems to the Problem Management Problem Manager (or other appropriate manager where circumstances dictate). Pro-active Problem Management Identify trends and potential Problem sources (by reviewing Incident and Problem records). Draft Ver 0.4 INS Problem Management Manual.doc Page 40 of 47

41 Monitor IT systems to highlight potential problems. Raise RFC s to prevent recurring incidents or problems. Identify opportunities for improvement to the process or the infrastructure. Pro-actively keep informed of current and past, problems and known errors, e.g. if there is a priority one problem, all staff are to be aware. Client Tasks/Responsibilities The Client has the responsibility for: Receiving information and updates about the progress of Problems that relate to Incidents affecting them. Providing information via the Service Desk as may be required from time to time to help progress Problems. Draft Ver 0.4 INS Problem Management Manual.doc Page 41 of 47

42 Process Metrics There are four major goals associated with the implementation of a Problem Management process within INS; i.e. To improve process compliance To improve service quality To minimise the impact of problems To reduce the cost to users of problems A preliminary set of measures and metrics which serve to measure achievement of these goals is detailed in the tables that follow. It should be noted that there will be no service levels agreed with clients in regard to problem response or resolution, however INS will routinely measure and report on process metrics to ensure process performance targets are achieved. The configured metrics will be reviewed by the ICTS Management Team at least annually and maintained to align with prevailing initiatives and priorities. For clarification it is useful to understand the fundamental differences between measures and metrics and how they combine with each other. Measure A measure provides single-point-in-time raw data views of specific, discrete factors. Measures are typically generated by counting and over a period of time these discrete measures can be combined to form a metric. For instance a monthly count of all problems raised makes up a measure and over a period of time a trend line can be established to represent the average number of problems being raised across INS during a particular time period. Metric A metric is derived by comparing to a predetermined baseline two or more measurements taken over time. Good metrics are those that are SMART, i.e. specific, measurable, attainable, repeatable and time-dependent. Truly useful metrics indicate the degree to which goals and objectives are being met. They also drive actions taken to improve service to clients and achieve cost and performance improvements. In the first six months of the implementation of the Problem Management process INS will be establishing baseline performance levels, after which time specific improvement targets will be discussed and agreed. Draft Ver 0.4 INS Problem Management Manual.doc Page 42 of 47

43 Goal 1: To improve process compliance Metric Rationale Measure / Data Source Target 1.1 Number of problems raised per support team There should be problems raised by all IT Support teams as a result of activities associated with both reactive and pro-active problem management. By reporting on problems raised by IT Support teams, anomalies are more easily identified. Monthly count of all problem records created. Measure sourced from Service Now. A representative contribution of problem records from all IT Support Units. 1.2 The percentage of problem records that are closed with no issues. The Problem Manager quality assures all problems prior to closure. Any irregularities in process conformance, information recorded etc. will be brought to the attention of IT Support teams. This operates to educate support team personnel and lever process and service improvement. Monthly count of all closed problems which passed QA. Monthly count of all closed problems which did not pass QA. Measure sourced from Service Desk Tool using the Problem Statuses Closed No QA issues and Closed QA issues Gradual increase in the percentage of problems closed with no QA issues raised. Goal 2: To improve service quality Metric Rationale Measure / Data Source Target 2.1 Ratio of the total number of problem records raised to the total number of incidents recorded. Effective problem management should result in a reduction of repeat incidents and problems. By measuring total incidents and problems on a monthly basis a comparison by month can be produced. Monthly count of incidents raised and problems Measure sourced from Service Now. Gradual reduction in the ratio over time as process matures. Draft Ver 0.4 INS Problem Management Manual.doc Page 43 of 47

44 Goal 3: To minimise the impact of problems Metric Rationale Measure / Data Source Target 3.1 Average time to resolve Problems The average time to resolve the various priority groupings of problems and various problem root causes should reduce as the knowledge database matures Measured each month and calculating the period of time (days) from when a problem is opened until it is resolved. Gradual reduction over time as process matures. Measure sourced from Service Desk Tool using Problem status codes open and resolved. 3.2 Average time to diagnose problems and identify the root cause. Shorter problem diagnosis times tend to indicate that IT Support personnel are becoming more skilled in problem diagnosis and/or have better quality Knowledge Base information available. Measured each month and calculating the period of time (days) from when a problem is opened until the root cause is identified. Gradual reduction over time as process matures. Measure sourced from Service Desk Tool using Problem status codes open and known error. Draft Ver 0.4 INS Problem Management Manual.doc Page 44 of 47

45 3.3 Average number of undiagnosed problems. Higher problem diagnosis rates lead to improved client productivity rates as there is less potential for problems to reemerge. Measured each month and totalling the number of problems closed with root cause not found. Gradual reduction over time as process matures. Measure sourced from Service Desk Tool using Problem resolution code codes unknown and no fix available. Goal 4: To reduce the cost of problems to clients Metric Rationale Measure / Data Source Target 4.1 Average time to diagnose problems and provide a workaround Client able to regain productivity and save downtime costs. Measured each month and calculating time the period of time (days) from when a problem is opened until a work around is created Gradual reduction over time as process matures. Measure sourced from Service Desk Tool using Problem status codes open and work around. Draft Ver 0.4 INS Problem Management Manual.doc Page 45 of 47

46 4.2 The number of problems escalated to third party Escalation incurs further delays and increases costs. Measured at month end totalling the number of problems with a status of awaiting external response. Gradual reduction over time as process matures. Measure sourced from Service Now. Problem Management Reporting Scope This section describes the main reporting requirements of Problem Management. These reports will be used as the basis for management of the process. There will be no automatic escalation of problems but rather there will be reporting and analysis of the lapse times to resolve problems and various exception reports which indicate problems that have not had status changed for a prolonged period of time. Process Metric reports will be considered at the regular fortnightly meetings of the ICTS Management Team. The initial set of reports required to meet the requirements of IT Support Teams, the Manager Planning (SLA Account Manager) and the Problem Manager follows: Reports required by IT Support Team Leads These reports provide a convenient snapshot of the performance of a support group (or individual) or overall performance of the process in a specified category or priority. They also provide information to analyse the performance individuals within a team. Summary of Problems Allocated: provides a list of problems allocated to specific support specialists in each support group. Summary of Active Problems by Status and Priority: provides a list of all active problems, their priority and current status. Summary of Resolved/Closed Problems by Status: provides a list of all problems that have been either resolved or closed in a particular month together with the associated resolution status. Summary of Problems by Configuration Item: provides a list of problems by individual Configuration Item (CI). Summary of Knowledge Base Items Posted provides a list of all KB items posted related to either temporary/permanent workarounds and/or permanent system fixes Draft Ver 0.4 INS Problem Management Manual.doc Page 46 of 47

47 Exception Report Potentially stalled problems: provides a listing of all problems that have not progressed from their current status in the last 60 days Reports required by Manager Planning (Account Management role) The INS Account Manager requires pertinent information relating to both active and closed problems relating to particular Service Lines. The INS Account Manager liaises on a regular basis with Service Line Managers and Business Managers. Summary of Problems by Category: provides a list of problems by category type ( i.e. by Service Line and Service). Reports required by Problem Manager These reports inform the Problem Management process metrics that are detailed in the previous section. The Problem Manager is responsible for compiling the metric reports each month for presentation and discussion at the ICTS Management Team meeting. Summary of Major Problems: provides a list of all Priority 1 (major impact) incidents that have occurred during a particular month. Summary of All Problems Raised across all Support Units: provides a count and listing of all problems records raised in a particular month (informs Metric 1.1 and Metric 2.1) Summary of All Problem Records Closed: provides a count and listing of all problem record closures which passed QA (no issues) and which failed QA (issues noted) (informs Metric 1.2) Summary of All Incidents raised: provides a count and listing of all incidents raised in a particular month (informs Metric 2.1) Summary of All Problem Records Closed: provides a count and listing of all problem record closures which passed QA (no issues) and which failed QA (issues noted) (informs Metric 1.2) Status Duration Open to Resolved : this report summarises all problems reaching a resolved status in the month and provides an average duration time for the problem to progress from an Open Status to a Resolved status (informs Metric 3.1) Status Duration Open to Known Error : this report summarises all problems reaching a known error status in the month and provides an average duration time for the problem to progress from an Open Status to a Known Error status (informs Metric 3.2) Status Duration Open to Work Around : this report summarises all problems reaching a work around status in the month and provides an average duration time for the problem to progress from an Open Status to a Work Around status (informs Metric 4.1) Unidentified Problem Causes -: provides a count and listing of all problem record closures where the root cause was not found (informs Metric 3.3) Exception Report Problems Awaiting Third Party: provides a count and listing of all problems that are awaiting information from a third party as at close of month and also provides the average time that the problems have remained in the status awaiting external response (informs Metric 4.2) End Process Draft Ver 0.4 INS Problem Management Manual.doc Page 47 of 47