Assessing the Effectiveness of an Online Simulation Exercise in Homeland Security Education

Transcription

1 Assessing the Effectiveness of an Online Simulation Exercise in Homeland Security Education Ouanessa Boubsil Interim Executive Director, Security Studies Laboratory University of Maryland University College

2 Outline Background University of Maryland University College Homeland Security and Information Assurance Education Simulation Exercise The village of San Luis Rey (SLR) Study Setting Methodology Study Tools Results SLR Survey results Course Evaluations results Areas for further exploration

3 The University of Maryland University College Founded in 1947 to provide education for military personnel and working adults Fully accredited institution of higher education headquartered in Adelphi, Maryland, USA Globally focused with more than 3,300 faculty members and 90,000 students worldwide 29 bachelor s degrees, 19 masters degrees, a doctoral program and more than 80 certificates, most of them offered both online and face-to-face.

4 Homeland Security & Information Assurance Programs Undergraduate Majors Homeland Security Information Assurance Specializations within different Master of Science degrees Information Assurance Homeland Security Management Undergraduate and Graduate Certificates in Information Assurance Homeland Security Management Doctor of Management program Core course requirement Enterprise Continuity and Information Assurance

5 Graduate Enrollment Graduate enrollment in Information Assurance (IA) and Homeland Security (HS) courses Enrollement FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY

6 Undergraduate Enrollment Undergraduate enrollm ent in Inform ation Assurance (IA) and Homeland Security (HS) courses FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY

7 The Simulation Exercise: The Village of San Luis Rey (SLR) Multimedia scenario-based simulation, enhanced with news releases, online discussions, photo and schematic imagery, and audio elements to create a realistic experience A large database of sector reference material including maps, news articles, a library etc. Instructor controlled staged information releases

8 Setting An introductory graduate course in Homeland Securityincluding infrastructure protection and issues in technical areas such as interconnectivity and interoperability A scenario-based simulation exercise -The Village of San Luis Rey (SLR)- by Teleologic Learners apply their learning of homeland security and critical infrastructure protection to try to prevent a multi-prong terrorist attack on the critical infrastructure sectors.

9 Project Class is divided into Homeland Security Strategy Teams (HSSTs) 4 or 5 students Each HSST is assigned to the protection of one of six critical infrastructure sectors (Banking, Electric Power, Water, Healthcare, Transportation and Communication) HSSTs inventory, assess, analyze and submit a written report to the mayor of the village of SLR detailing their team s findings and recommendations on how to protect the infrastructure.

10 Methodology 2 modules of the same course, taught by the same instructor, using the same textbooks and other instructional materials 2 consecutive semesters (Fall 2005 and Spring 2006) Fall 2005 the control module used a group paper Spring 2006 used the SLR simulation exercise

11 Study Tools Student Survey specifically designed for the use of SLR The generic end-of-semester course evaluations used for all courses taught at UMUC Course statistics measuring students participation and interactions Students comments

12 San Luis Rey (SLR) Survey Subjects 2 online sections of ITSM 620 Concepts in Homeland Security 56 students in total Survey 30 questions 41 completed surveys (77% response rate) 21 students provided comments

13 Areas Surveyed Personal information Technology SLR exercise content Students learning Students satisfaction Team dynamics Exercise organization and workload requirements Students comments and suggestions

14 Survey Results: Strengths Login was easy Links worked well Technology functioned well Sector information was mostly helpful SLR helped in gaining knowledge

15 SLR helped in gaining knowledge The SLR exercise was helpful in gaining understanding of critical infrastructure concepts 60% 50% 40% Percentage 30% 20% 10% 0% Opinion 1: Strongly Disagree 2: Disagree 3: Neutral 4: Agree 5: Strongly Agree

16 SLR was enjoyable I enjoyed the exercise Percentage 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Opinion 1: Strongly Disagree 2: Disagree 3: Neutral 4: Agree 5: Strongly Agree

17 SLR would attract students I would take another course that uses the SLR exercise 40% 35% 30% Percentage 25% 20% 15% 10% 5% 0% Opinion 1: Strongly Disagree 2: Disagree 3: Neutral 4: Agree 5: Strongly Agree

18 Improvement areas Library information Maps Staged scenarios Time period too short Some team members didn t work well together

19 Comments Common Themes Some felt it added depth to the course Some wanted to be in more than one sector Directions were too broad Time to complete the exercise was too short Too much information

20 Student Course Evaluation Evaluation Item/Module on a scale of 1 to (without simulation) Course Overall (with the simulation) Interaction Pedagogy Course Objectives

21 Students Interactions Item/Module Conference postings (Number of words) 0509(without simulation) (with the simulation) Study Group postings (number of words) Teacher

22 Students Comments 0509 (without the SLR) Strongest features of the course Reading the material (2/6) Discussion and Interaction (2/6) Recommendations to improve the course Less paper writing assignments Adapt course to include response to natural disasters and illnesses

23 Students Comments 0602 (with the SLR) Strongest features of the course Teacher developed a really cool project which helped us learn a lot Strongest feature was class discussion(2) Most invigorating course I have taken SLR was the best Enjoyed interaction with other class members although we don t always see eye to eye

24 Positive responses when prompted for areas for improvement Recommendations to improve the course I can t think of any at the moment - I am very pleased It is a solid built course Should be taught face to face (2) Needs a sequel as for this course it was excellent. There are no further recommendations. Although the topic for the group project was a good idea, I would not change the scenario the week the project is due.

25 Areas for further exploration Correlation between satisfaction and select variables from the survey such as sector assigned, grade, etc. Focus on the cognitive gains by comparing samples of the written assignments Group dynamics and effects on student performance and satisfaction ( size and homogeneity) Explore the Return On Investment (ROI) factors--impact on enrollment and retention

26 Thank you for your attention!