How Do You Audit an Elephant? (Learning Management and Student Information Systems)

Transcription

1 How Do You Audit an Elephant? (Learning Management and Student Information Systems) DAVID MAGGARD & KIM RUTLEDGE Texas A&M University System Internal Audit

2 Very simple answer.

3 Texas A&M University System TEXAS A&M UNIVERSITY SYSTEM

4 System Size 11 universities, 7 state agencies, 2 service units, and a comprehensive health science center Approximately 131,000 students Nearly 28,000 faculty and staff Physical presence in 250 of the state s 254 counties Reaches 22 million people through service each year

5 Objectives Scoping the application review Identifying testing methods Role of general IT controls

6 The largest living land animal

7 Evolution Student Information Systems A software application for education establishments to manage student data. Equivalent to an ERP for corporate customers. Began as systems used only by administrators Students then began to register for classes Now they are used for financial aid, loan management, self degree audits, grades, etc Learning Management Systems A software application for the administration, documentation, tracking, reporting and delivery of e-learning education courses. Not used widespread until early 2000 s at TAMU, later for the regional campuses Used for true online courses and to enhance the traditional classroom Used for exams, calculating grades, communicating with students, etc.

8 Changing Environment Large complex mission critical systems. Difficulty in scheduling maintenance windows due to the reliance on these systems Due to their size and newness, determining how to audit them is a challenge. The topics we discuss can be applied to any application audit (financial, student, LMS, document imaging, personnel/payroll etc.)

9 Where do you start? Don t be afraid Don t wait on someone else to develop the perfect audit plan. If possible, obtain training (not a requirement) Don t focus on the whole thing, think of it as pieces. The pieces are the business processes and general IT controls.

10 Scoping the review Methodology Less about security more about availability and accuracy of information contained within the systems Focus is on business processes not specific systems Includes all environments (owner, custodians, and users) Identify key business processes that use a particular system (i.e. student information system or learning management system)

11 How to eat that elephant Interview the business folks Interview the custodians/administrators of the system Interview users Interview someone from any governing bodies Think of the processes in the traditional inputprocessing-output framework

12 Next bite Obtain documentation, user manuals, vendor manuals, anything you can get your hands on to determine the following: Identify the applications involved Identify the process owners Identify the key screens Identify what risk exists with each process Rank each process for likelihood and impact (L=5, M=10, H=15) Select sample of processes for further review

13 Things you want to know Interfaces Feeds Reports

14 Business Process Overview Input Accurate Complete Authorized Correct Data Processed as intended in acceptable time period Accurate Complete Output Accurate Complete A record is maintained that tracks the process of data input, storage and output.

15 Flowcharts

16 Our Approach to Planning Perform a risk assessment of processes Identify the scope Determine depth/range of coverage Develop and communicate plan

17 Process Identification Process Applications Involved Process Owner Key Screens Risk High School Transcripts EDI.Smart, Banner Registrar n/a EDI Application and Transcripts EDI.Smart, Banner Registrar SZRETPG, SAAEPAPS, SZR189U, SWPCLAD, SAAEAPS, SARETMT, GOAMTCH Transcript data is not entered accurately into Banner Application and transcript data is not loaded accurately or completely into Banner

18 Testwork Process Owner Process Control Identified Admissions High School Transcripts Access to SOAHSCH and SOATEST are appropriately restricted. Admissions High School Transcripts Transcript entries and updates in Banner are recorded in an audit trail and reviewed periodically. Testwork Obtained a list of users with access to the SOAHSCH and SOATEST forms in Banner and reviewed for appropriateness. Inquired of business owner and IT personnel if an audit trail for the transcript entry/updates exist.

19 Testing Methods Manual processes Inquiry, observation, inspection or reperformance Uses sampling to determine if effective Risk is inconsistency of control result due to human involvement Automated processes Edit checks or error reports Testing alternatives A mix of inquiry, observation, inspection or reperformance Negative assurance testing Testing to confirm that invalid input is detected Invalid dates, text in numeric field, override controls

20 General IT Controls Availability Data Integrity, Accuracy and Security IT Management

21 General IT Controls IT governance Systems Development IT Operations Physical Security and IT Continuity Information Security

22 General IT Controls Why do we care? When should we perform this assessment? How is it reported?

23 Student Information System Processes Input processes Data loads Manual vs. automated Output processes Reports Views Audit trails General Control issues Governance User accounts and security Infrastructure, system, and application maintenance Security physical and logical Hosted, in-house, mixed

24 Learning Management Processes Input processes Setting up courses Assigning faculty to the courses (manual, batch or realtime from the Student Information System) Populating the students in the courses Output processes Grades Interfaces with other applications such as Turnitin, Respondus, media, etc. General Control issues Setting up student accounts Setting up faculty accounts Setting up user access directly into the application Security/Password settings

25 LMS Governance Academic not IT Advisory committees Surveys Interview, Interview, Interview (faculty and students if possible.) Software as a service (The Cloud!) Things to look for in the contract

26 Conclusion: Scoping the application review Identify testing methods Role of general IT controls

27 Summary Remember one bite at a time. Identify the scope. Identify resources and key business processes. Open communication. No need to be the expert.

28 Questions? David- Kim