Afundamental requirement for effective auditing is to

Transcription

1 FEATURE Information Technology Auditing and Facilitated Control Self-assurance By Ken Doughty, CISA, CBCP, and John O Driscoll, CISA, CIA Afundamental requirement for effective auditing is to provide an opinion to the executive team and the board audit committee on the adequacy of the internal control framework operating within the organisation s information technology and telecommunications (IT&T) environment. This requirement, while ongoing, may have specific meaning at some point, e.g., financial year-end when management is required to sign off on the end of year accounts. IT auditors have used a range of audit methodologies and techniques to support their audit opinions. This paper will outline an approach recently utilised within a financial services organisation to provide an annual assessment of the IT&T internal control framework. The approach used is based on a set of internationally recognised IT service delivery and support process models called ITIL (Information Technology Infrastructure Library) and relies extensively on the use of control self-assurance (CSA) workshops facilitated by IT audit staff. This approach can be applied with equal success to internal and outsourced IT&T environments. CSA is a risk management program where risks and controls are examined and assessed to provide reasonable assurance to management that business objectives will be met. IT management and staff involved in the delivery of services and products to an organisation participate in all phases of the process. For CSA to be effective it must have support from IT&T management and staff. The strengths of this approach are that it: Engages stakeholders in the review process Provides an end-to-end process perspective Increases participants knowledge and understanding of their processes and potential risk exposure Provides process participants with an opportunity to air common concerns and participate in process improvement Preparation Training and ITIL Certification A certified ITIL trainer presents a three-day in-house training program on ITIL. The training participants include all internal IT auditors and representatives from the organisation s external auditors, IT department and IT&T service providers. The training program also incorporates a number of teambuilding activities. At the end of the three-day training program, all the attendees have completed the ITIL Foundation Level Certification examination. The main reason for inviting representatives outside of the organisation s IT audit department is to establish and maintain an open and trusted relationship with all stakeholders. Further, it helps gain acceptance of the new approach being adopted. Senior Buy-in A number of presentations are made to senior management within the organisation and the IT&T service providers to explain the purpose and approach and gain their acceptance. A brochure to aid the communication on the purpose and context of the process is prepared and distributed to all key stakeholders. ITIL Processes The IT&T environment covers a large range of processes that deliver services and products to the organisation. To ensure there is effective coverage of the IT&T environment in using FCSA (facilitated control self-assurance), the auditor utilises a best practice IT framework with defined supporting processes. One such framework with well-defined supporting processes is ITIL ( a series of best practice guidelines in IT management. Developed in the late 1980s when IT professionals gathered to develop a comprehensive life cycle for all areas of IT service management, it created a common set of terminology for the IT team to put services into business terms and align them with key objectives. This nonproprietary framework consists of a number of interrelated processes that provide an end-to-end perspective to IT&T service provision, with the aim to drive continuous improvement. Today, many industry leaders recognise ITIL as the de facto world standard in IT service management. management is defined by ITIL as the process of maintaining and gradually improving business-aligned IT service quality, through the constant cycle of agreeing, monitoring, reporting and reviewing IT service achievements and through instigating actions to eradicate unacceptable levels of service. Since the mid-1990s, ITIL has had a proven track record with corporations and governments worldwide, such as Microsoft, Procter & Gamble, AXA and ABN AMRO. The benefits of using ITIL include: IT services meeting business requirements, i.e., the IT effort is focused on assisting the organisation achieve its strategic objectives Improved efficiency and quality of service delivery Clear understanding of service delivery and support priorities

2 Improved relationship among customers, IT and vendors Improved lines of communication between IT and its customers Monitoring of service delivery which facilitates the identification of areas of process weakness ITIL is particularly effective in an outsourcing environment, where a business relationship between IT&T supplier(s) and customers aims to provide an optimal balance between highquality services and controlled costs. Figure 1 indicates the interrelationships among service delivery, service support, application management, infrastructure management and the business perspective. Delivery The Business Perspective Figure 1 The ITIL Jigsaw Applications Support Infrastructure The ITIL service delivery process model is set out in figure 2 and further description of each component is included in table 1. Figure 2 ITIL Delivery Process Model Table 1 ITIL Component Goal level To maintain and gradually improve businessmanagement aligned IT service quality through a constant cycle of agreeing, monitoring, reporting and reviewing IT service achievements and through instigating actions to eradicate unacceptable levels of service Availability To optimise the capability of IT infrastructure management and supporting organisation to deliver a costeffective and sustained level of availability that enables the business to satisfy its objectives Capacity To understand the future business management requirements (the required service delivery), the organisation s operation (the current serviced delivery), the IT infrastructure (the means of service delivery), and ensure that all current and future capacity and performance aspects of the business requirements are provided cost-effectively Financial To provide cost-effective stewardship of the management IT assets and the financial resources used in for IT services providing IT services IT service To support the overall business continuity continuity management process by ensuring that the required IT technical and services facilities can be recovered within required and agreed business time-scales Figure 3 describes the ITIL service support process model. Each component is broken down in table 2. Figure 3 ITIL Support Process Model The Organisation, Customers and Users The Organisation, Customers and Users Level Availability Capacity Incident Problem Change Financial IT Continuity Release Configuration

3 Table 2 Figure 4 ITIL Component Goal Incident To restore normal service operation as management quickly as possible with minimum disruption to the business, thus ensuring that the best achievable levels of availability and service are maintained Problem To minimise the adverse effect on the business management of incidents and problems caused by errors in the infrastructure, and to proactively prevent the occurrence of incidents, problems and errors Change To ensure that standardised methods and management procedures are used for efficient and prompt handling of all changes, to minimise the impact of any related incidents upon service Release To take a holistic view of change to an IT management service and ensure that all aspects of a release, both technical and nontechnical, are considered together Configuration To provide a logical model of the IT management infrastructure by identifying, controlling, maintaining and verifying the versions of all configuration items in existence There are two other critical supporting processes to the ITIL service management model that the auditor needs to be aware of when using ITIL as the best practice IT management framework: service desk and security (table 3). ITIL Component desk Security Table 3 Goal To act as the central point of contact between the user and IT service management; to handle incidents and requests and provide an interface for other activities such as change, problem, configuration, release service level, and IT service continuity management To manage the defined level of security on information and IT services FCSA s Facilitated CSA workshops are conducted by the auditor to assist IT&T management and staff in identifying risks, assessing the control environment and developing a service improvement plan (SIP) to mitigate the risks. Structure To facilitate workshop participation and outcomes, the participants should receive a handout describing the FCSA process (figure 4) including the tools being used in the WORKSHOP PARTICIPANTS RESPONSE TO STATEMENTS IDENTIFY CONTROL RISKS Overview of FCSA Process DEVELOP PROCESS CONTROL STATEMENTS PROCESS RISK AND CONTROL PROFILE DEVELOP SERVICE IMPROVEMENT PLAN SELF- ASSURANCE CONFIRM CONTROL EFFECTIVENESS REPORT RESULTS workshop. This information is important, as it will assist in alleviating any misconception the workshop participants may have about the workshops and its outcomes. It is important to remember that the workshops should be structured to include: An introduction (overview of workshop process, tools and a clearly defined purpose and context) A description of the IT&T process to be covered A description of the process (e.g., problem management) to capture responses from participants (i.e., information) feedback (i.e., outcomes) The way forward Experience shows that: The workshop duration should not be more than two hours, as it requires a large amount of concentration by participants. The number of workshop participants should be between five and nine. statements should be limited to a maximum of 25. The workshop should be held in a closed room to minimise disruptions. Participants It is important that the appropriate participants are selected to attend all workshops. The participants should include IT&T management and staff, including the IT&T department managers and senior/experienced staff members who are involved in the delivery of services and products to the organisation. It is critical that they know their processes and are empowered to implement the appropriate controls. The IT&T department managers are selected, as they are the process owners. They have the accountability for managing the IT&T risks on a day-to-day basis. They must take ownership of the information technology risk and control processes and proactively manage these over time.

4 Statements and Assessment Criteria One of the critical parts of the FCSA is developing the statements to be asked of the workshop participants for response. The statements are developed using information technology best practice standards for service delivery and support. Up to 25 statements per process form the basis of the FCSA workshops. Individual workshop participants are asked to respond to each statement using the following seven-point scale: Strongly disagree Slightly disagree Neither agree or disagree Slightly agree Strongly agree In particular, the answers from participants should elicit discussion based upon their knowledge of and experience with the processes. An example of a statement in relation to problem management is: There are clear criteria for prioritising a problem. Tool (Optional) In this approach, an anonymous voting tool is used during each workshop. Each participant uses a separate keypad to vote on each statement. A bar chart analysis of all responses is displayed after each statement. This promotes open discussion and drives suggestions for process improvement. The outcome of the discussion, i.e., the process risk and control issues, are clearly identified and recorded. Facilitator It is essential that the workshop facilitator is an experienced IT audit practitioner with strong communication skills. The quality of information collected in the workshops is dependent upon the facilitator gaining the participants confidence in their ability to manage the workshop. Further, the facilitator will be required to clarify and elicit additional information as needed, based upon the participants responses to the statements. Deliverables Documentation Each workshop requires a resource to document the risk and control issues from the discussions based upon the participants responses to the statements. It is important that the resource also is an experienced IT audit practitioner as this person will assist and support the workshop facilitator seeking clarification of responses that may otherwise have been either misunderstood or not clearly enunciated. From the responses collected for each workshop, a detailed worksheet is prepared. The worksheet details the following information: participants names, job titles and contact details (this will facilitate follow-up) Responses to each statement (option finder) Risk and control issues discussed Analysis of responses (detailed information of process control issues) Figure 5 Response Strongly 2 2 Slightly 1 Neither nor Slightly Control Issues Miscategorisation of severity level by help desk staff Severity levels definitions are too broad Severity levels definitions are too loosely applied Customers insist on making all issues a severity level Strongly 0 Details from participants in support of their responses Improvement Help desk staff to receive greater training to meet operational requirements Existing severity level definitions to be reviewed and revised where appropriate Guidelines for the application of severity levels to be developed, documented and training given to staff on application Communications plan to be developed to educate users on severity levels Total No. of Responses 8 Recommendations to address issues

5 Figure 5 is an example of a workshop worksheet with a participant s response to a specific statement in relation to problem management. The statement used for the example is There are clear criteria for prioritising a problem. Audit Testing If evidence is required to support the workshop outcomes, then audit testing may be performed, particularly where the radar map (refer to the section of this article) indicates that the IT&T service provider exceeds the benchmark. The extent and type of testing performed, i.e., compliance and/or substantive, are dependent upon the level of comfort required to support the workshop outcomes. Experience has shown that if the IT&T service provider has failed to meet or exceed the benchmark, then additional evidence to support the workshop outcome is not required. Action Plans Action plans are developed based on the outcome of each workshop. It is essential that these plans are agreed upon and accepted by the appropriate parties. Figure 6 indicates the process model utilised in the FSCA. Audit IT Provider Figure 6 Audit IT Provider Figure 7 Improvement Plan Process Problem Control/ Issue 1. Miscategorisation of severity level by help desk staff 2. Severity level definitions are too broad 3. Severity level definitions are too loosely applied 4. Customers insist on making all issues a severity level 1 Impact H High M Medium L - Low H M H L Improvement Action 1. Help desk staff are to receive additional training to meet operational requirements. 2. Existing severity level definitions are to be reviewed and revised where appropriate. 3. Guidelines for the application of severity levels are to be developed, documented and training given to staff on application. 4. A communications plan is to be developed to educate users on severity levels. Target Date 30 June 1 May 30 June 1 May FCSA & SIP Documentation of control issues Determine if tesing required Prepare recommendations for improvement Review Docs for accuracy & completeness. Response to recommendations Review response, & prepare Improvement Plan (SIP) & Audit Report & SIP Review & approve for issue & SIP Executive & IT Provider Improvement Plan A service improvement plan (SIP) provides the necessary details upon which the IT&T service provider can act. IT provides details of the control issue and the agreed-upon action to be taken to address the issue and names the designate who has responsibility for implementing the agreed action and the target date for implementation (see figure 7). After working through the control issues and responses with the IT&T service provider, a report is prepared. A radar map is used in the executive summary to assist executive management to gain an understanding of the issues without having to read all the detail that supports the auditor s opinion. A radar map provides a high-level overview assessment of the results of FCSA. Figure 8 Audit Rating Scale Rating Description 5 Unsatisfactory The auditable activity was not in compliance with policies, systems and procedures. Major risks were identified that have adversely impacted the auditable activity s contribution to the organisation s strategies. 4 The auditable activity was not always in compliance with Needs improvement policies, systems, and procedures. Major risks were identified which adversely impact or are likely to adversely impact the auditable activity s contribution to the organisation s core strategies. 3 The auditable activity was generally in compliance with Average policies, systems, and procedures. Some control deficiencies were identified that, if not promptly corrected, may lead to major risks adversely impacting the auditable activity s contribution to the organisation s core strategies. 2 The auditable activity was in compliance with Good policies, systems, and procedures. Some control deficiencies were identified but these did not/are not expected to lead to major risks or adversely impact the auditable activities contribution to the organisation s core strategies. 1 The auditable activity has achieved its goals and Commendable objectives. The auditable activity was in compliance with policies, systems, and procedures. No control deficiencies that would hinder the attainment of goals and objectives of the organisation were identified.

6 The workshop results are converted into a standard fivepoint audit scale to provide consistency for all audit reports within the organisation (see figure 8). The following is a hypothetical example of what can be expected from this type of review. An arbitrary audit rating benchmark is set, say 2.5. The radar map indicates where the IT&T service provider s performance is for each ITIL process. Two colours are used to indicate performance. Green means it meets or exceeds benchmark, and red means it failed to meet or achieve the benchmark. Figure 9 indicates that two processes exceeded the benchmark and identifies a number of processes that require further remediation or improvement to achieve or exceed the benchmark. Continuity Security Figure 9 Internal Control Framework Assessment Incident Problem Change Support Processes Everyone involved in the review has been very positive about the approach used, and the value of the outcomes. This approach should be considered as a key strategy in the delivery of IT audit services to an organisation. References IT IT Forum Ltd ISBN Ken Doughty, CISA, CBCP is executive audit manager, IT&T Commonwealth Bank Group. He has more than 20 years IS audit experience with more than 10 years business continuity experience in the public and private sectors. He speaks regularly at seminars and conferences in Australia and internationally. He also has published papers on IS auditing and business continuity in Australia and the US. He can be reached at doughtke@cba.com.au. John O Driscoll, CISA, CIA is executive audit manager, IT&T Commonwealth Bank Group. He has more than 15 years IS audit experience in the public and private sectors. He regularly speaks at security and IS audit seminars and conferences and presents training courses on behalf of ISACA and IIA Sydney chapters. He can be reached at john.odriscoll@cba.com.au. Financial Delivery Processes Release Copyright K. Doughty, J. O Driscoll 2002 Capacity Configuration Level Availability Areas of Risk Exceeded Benchmark Conclusion The use of FCSA workshops with an industry-recognised IT best practice process model provides a valid assessment of the internal control framework of the IT&T environment which meets the demands of the organisation s senior executives. The experience of using FCSA for IT auditing that was documented in this article has resulted in a saving of more than 35 percent in resources and time for this type of audit compared to a traditional approach. While this is impressive, the biggest benefit resulted from the enthusiastic acceptance of the approach by the organisation s IT&T service providers and key stakeholders. Commitment to resolving control deficiencies and implementing process improvement has been outstanding at the workshop participant and senior management level. Some of the comments the organisation has received regarding the use of this approach include: The processes and tools used encouraged transparency, collaboration and contribution from all parts of the organisation involved. The report and related documents are of excellent quality and easily provide a basis for our organisation to progress with improving critical areas and there were many areas identified that span the end-to-end service delivery process.