STUDENT LEARNING ASSESSMENT REPORT MS Cybersecurity

Transcription

1 STUDENT LEARNING ASSESSMENT REPORT MS Cybersecurity SUBMITTED BY: DIANE MURPHY DATE: OCTOBER 15, 2014 BRIEFLY DESCRIBE WHERE AND HOW ARE DATA AND DOCUMENTS USED TO GENERATE THIS REPORT BEING STORED: BLACKBOARD COURSES FOR FILING CABINET IN BALLSTON ROOM 407 List all of the program's learning outcomes EXECUTIVE SUMMARY Learning Outcome 1. Identify and solve cybersecurity issues in business and society by managing cybersecurity operations using available tools and techniques 2. Review and understand the legal, regulatory, policy, and ethical issues related to securing cyberspace and ensuring the privacy of personally identifiable information (PII) 3. Communicate effectively with others, including technologists and managers in the cybersecurity, IT, and users and managers in the business context 4. Use specialized knowledge and skills to obtain skills and, if applicable, certifications in the cybersecurity field 5. Optimize the effectiveness of cybersecurity in an organization by performing vulnerability assessments, risk mitigation, auditing, certification and accreditation of information systems 6. Work effectively as a member or as a leader of a cross-disciplinary team in the cybersecurity field where teamwork is essential to the success of a time-critical project 7. Develop the knowledge and skills required to pursue life-long learning, in areas relating to cybersecurity and to adapt to an ever-changing, global technological and business environment through information literacy activities relevant to a fast-changing discipline Year of Last Assessment Year of Next Planned Assessment Marymount s MS in Cybersecurity Program is designed to prepare working professionals for high-level positions in computer and information security by developing the necessary knowledge, skills, and abilities in the technology and management of cybersecurity. Graduates will work in the protection of the digital world for the federal government, state and local governments, nonprofits, and industry. 1

2 The program is still in its beginning stages (officially beginning in Fall 2013) and so there are no data from students exiting the program. The outcomes are from the initial courses in the program only. Because of the university's proximity to federal agencies, including the Department of Homeland Security, the Department of Defense, the National Security Agency, and the National Science Foundation, faculty members in the program are involved in and aware of current federal government initiatives and requirements. Further, program leaders are able to call on practitioners in the field as guest speakers. This 36-credit program is offered primarily online. Courses are rigorous, including readings, collaboration, and practical assignments using inquiry-based learning techniques with online multimedia presentations, online tools, and online simulations and labs. The program is designed primarily for working professionals to prepare them for promotion to a cybersecurity leadership position, and to meet the certification requirements currently imposed on the profession, particularly by the Department of Defense on its employees and contractors. While all other coursework can be completed online, the culminating course is completed in residency in order to take advantage of the cybersecurity resources of the Washington, DC, area. Students will gain hands-on practice with equipment and tools, interacting with cybersecurity experts in government. The program is committed to exhibiting the highest professional and ethical standards addressing the needs of working individuals, full-time students, and business and government organizations. Describe how the program s outcomes support Marymount s Mission, Strategic Plan, and relevant school plan: The MS in Cybersecurity program is fully compliant with the graduate education mission of Marymount University and the outcomes include both the opportunity to acquire a high level of competency in the cybersecurity domain (Identify and solve cybersecurity issues in business and society) and to gain experience in the application of advanced knowledge and skills (including management, leadership, policy development, compliance, and technical competence). It recognizes the changing nature of the IT field and hence the need for life-long learning based on skills in information literacy and the ability to keep current with changes in the field. Two of the learning outcomes to be assessed in this initial assessment relate to the technical and management knowledge gained during the early part of the program, while the third concern itself with a necessary soft skill, communication. As stated above, this is the first year of the program and mode detailed outcomes assessment will occur in subsequent academic years. In the school year the President articulated a new "vision" for the university. The items in the vision statement that apply specifically to the graduate cybersecurity program include: 2

3 Academic Excellence Emphasize inquiry learning at all levels and provide students and faculty with opportunities for research collaboration.. Offer programs that enable graduate students to develop the knowledge, skills, and values needed for professional success. Ensure a personalized education through small classes and faculty/student collaboration. Integrate an emphasis on ethics throughout the curriculum. Encourage cross-disciplinary collaboration. Inquiry learning is a key in the program and all professors (full-time and part-time) are encouraged to use activities and labs, inclassroom or as homework assignments, to reinforce the subject-matter learning in the classroom. Students are required to hoist a lab environment on their personal computers to carry out the labs and exercises at their work or home locations. These activities my occur through individual and group assignments. Cybersecurity graduate classes average around 11 students, large enough to have a variety of opinions and experiences on the discussion board, but small enough to allow for individual attention and extensive faculty/student collaboration for the lab work and other activities. Note: some of these courses are also taken by IT graduate students who are specializing in the cybersecurity domain. Ethics are an integral part of each course but particularly emphasized in the required course IT570, Cybersecurity: Law, Policy, Ethics, and Compliance. Cross-disciplinary collaboration occurs in the early courses such as IT530, Computer Security, s this includes students from other programs including MSIT, and the MS HCM/IT and MMBA/IT dual degree programs. Community Engagement Use DC-area resources and new technologies to enhance the global perspective of the Marymount community. Marymount is surrounded by operational cybersecurity locations, such as the Department of Homeland Security and the CIA We make also use of the DC area as a center of cybersecurity expertise adjuncts, for frequent guest speakers and for members of our and the Cybersecurity Roundtable. Student Profile Pursue opportunities for growth in key graduate programs. The program has had a very successful start and we are currently exploring a DSc program in Cybersecurity. For the academic year, there were 10 students in Fall semester, 20 in Spring 2014, and 12 in Summer

4 Provide a brief description of the assessment process used including strengths, challenges and planned improvements: This is the first year of the MS in Cybersecurity program (MBA/IT and HCM/IT). In Fall 2014, the Director of the MS in IT program (with input from graduate faculty members) analyzed the results of the previous academic year s learning outcomes, writes the Outcomes Assessment Report and designed the initial Outcomes Assessment Plan for the next academic year. The assessment process could not be spread across the entire curriculum because this was the first year. Focus has been on outcomes for the initial courses in the program. It does, however, include technical, management, and 21 st century skills outcomes. The first outcome measures the ability of the students to successfully manage cybersecurity as a function of experiences with best practices. Both measures were met as part of the first two courses in the program. The second learning outcome measures the ability of graduates to manage enterprise-wide information systems to ensure that an organization is competitive in today s global and high performance environment following strong ethical principles. The third learning outcome to be measured is the ability to communicate effectively with others, including technologists and managers in the IT field and users and managers in the business context. Describe how the program implemented its planned improvements from last year: This is the first outcomes assessment report and so there are no planned improvements to make. Provide a response to last year s University Assessment Committee review of the program s learning assessment report: There was no outcomes assessment report from last year. 4

5 Outcome and Past Assessment Learning Outcome 1: Identify and solve cybersecurity issues in business and society by managing cybersecurity operations using available tools and techniques Is this outcome being reexamined? No Assessment Activity Outcome Measures Performance Standard Data Collection Analysis. Direct: Use of cybersecurity best practices to respond to a realistic cyberattack in IT530, Computer Security Direct: Development of an effective policy document as part of Assignment 3 in IT570, Cybersecurity: Law, Policy and Compliance. 70% of students are able to successfully analyze the scenario in Assignment 2 and provide an adequate response (4 or more) on a scale of 0 (no response) to 5 (excellent). 70% of all students generate an effective policy in response to a class assignment (a grade of 18 or more on the rubric) The assignments were collected from Blackboard for the fall 2013 semester (10 of 11 registered students) and for the Spring 2014 semester (14 of the 16) students. The assignments were downloaded before they were graded by the professor. The assignments were collected from Blackboard for the spring 2014 semester (9 of 10 registered students). The assignments were downloaded before they were graded by the professor. The responses were reviewed by a panel of three professors: the Chair, one full-time faculty, and one adjunct faculty, all experts in the field of cybersecurity. Each panel member was each given an opportunity to rate the solutions for their adherence to best practices in the field (0 through 5). These responses were averaged and 20 out of the 24 students (83) received a 4.0 or more on average. The standard was met. The responses were reviewed by the Department Chair and rated according to a rubric (see Rubric 1) These responses were averaged and 20 out of the 7 of the 9 students (78%) received a 18 or more. The standard was met. Interpretation of Results Extent this Learning Outcome has been achieved by students: This outcome was met. 5

6 Program strengths and opportunities for improvement relative to assessment of outcome: The courses seem to be successful in developing students who have the knowledge and skills to understand the cybersecurity field and to be able to develop a policy that implements that cybersecurity posture. Discuss planned curricular or program improvements for this year based on assessment of outcome: No changes in these two courses are anticipated until more students have taken the courses. 6

7 Outcome and Past Assessment Learning Outcome 2: Review and understand the legal, regulatory, policy, and ethical issues related to securing cyberspace and ensuring the privacy of personally identifiable information (PII) Is this outcome being reexamined? No Assessment Activity Outcome Measures Performance Standard Data Collection Analysis Direct: In IT570: Cybersecurity: Laws, Policy, and Compliance, investigate the laws and regulations that cover an individual s privacy on the Internet. 80% of students were able to identify all laws and regulations that cover personal privacy in a specific industry in an online assignment. Assignments were downloaded from Blackboard before being graded by the instructor. The department chair measured the laws and regulations noted against a consolidated list of general and industry-specific laws and regulation. Students were expected to notify at least 9 of the most relevant laws and regulations. In Spring 2014, 8 (100%) of all the students met the requirement. Direct: In IT560, Cryptography, students were able to correctly identify cybersecurity ethical concerns and how they could be handled using cryptographic techniques. 80% of cybersecurity students were able to identify the ethical dilemma in a realistic scenario and to mitigate the risk through a cryptographic solution. The topic was discussed on the discussion board, with students not being able to see the other submissions until they had entered their response. The outcome was met. The professor downloaded the discussion topics and anonimized the student who submitted the discussion, including details for only the 6 students in the MS Cybersecurity program. A panel of three cybersecurity practitioners looked at the responses and graded them from 0 (n response) to 5 (excellent), with greater than 4 being adequate. 5 of 6 (83%) students received a score of 4 or more. The outcomes were met. Interpretation of Results 7

8 Extent this Learning Outcome has been achieved by students: Both of the outcomes were met. Program strengths and opportunities for improvement relative to assessment of outcome: The program seems to satisfactorily cover the privacy and ethics requirements of the learning outcomes. Discuss planned curricular or program improvements for this year based on assessment of outcome: There are no changes planned this early in the program. 8

9 Outcome and Past Assessment Learning Outcome 3: Communicate effectively with others, including technologists and managers in cybersecurity, IT, and users and managers in the business context Is this outcome being reexamined? No Assessment Activity Outcome Measures Performance Standard Data Collection Analysis Direct: Evaluation of communication activities in the IT585. Managing Technical People" which is a group project Students rate 70% of the team members as effective or very effective in the communication process on the project in IT585. Managing Technical People, 8 out of 10 on the rubric (Rubric 2) is considered as effective A questionnaire is given as a confidential evaluation by the student of the team as a whole and each member of the team to assess the communication skills of the team members. There were 12 students in Spring individuals rated their team or at least one of their team members as ineffective or only partially effective in their communications in team work leaving 8 with a positive feeling about their team and its members (66%). The most common complaint was collaborative writing efforts. The standard was NOT met. Direct Performance on an individual project in IT530, Computer Security Instructor rated 70% of the student on their individual reports in IT530, Computer Security, as effective (3 or 4) Department chair examined the reports submitted in the courses against Marymount s writing intensive standards. There were 11 students in Fall 2013 and 16 students in Spring (59%) of the students received a score of 3 or 4 (competent or expert) on average for their reports. This standard was NOT met. Outcome Measures Performance Standard Data Collection Analysis 9

10 Interpretation of Results Extent this Learning Outcome has been achieved by students: This outcome was not met.. Program strengths and opportunities for improvement relative to assessment of outcome: Communication online is difficult. The discussion board seems to work but not other group activities or written assignments. Discuss planned curricular or program improvements for this year based on assessment of outcome: The faculty will address how to introduce writing techniques into the graduate program more effectively. Attachments Rubric 1: Evaluation of Cybersecurity Policy in IT570,Cybersecurity: Law, Policy and Compliance Requirements Engineering Attribute Measure Scoring Student Score Rate the complexity of the topic selected for the policy document and its applicability to today's cybersecurity environment Evaluate and score as follows: excellent, advanced, good, basic, poor, or no submission 5, 4,3,2,1, and 0 Rate the depth of coverage of the policy in the report Review the form and format of the document to ensure the quality and clarity of the policy document Evaluate the language used in the report and its ability to be understood Evaluate and score as follows: extensive coverage, good coverage, adequate coverage, limited coverage, poor coverage, and no coverage Evaluate and score as follows: effectively organized, organized, poorly organized, or no submission Evaluate and score as follows: well written (no jargon or jargon explained), 10 5,4,3,2,1, and 0 3,2,1, and 0 4,3,2,1, and 0

11 by its audience, the average user in the organization Evaluate the report with respect to best practice policy standards adequately written (jargon mainly explained), includes some jargon, and full of jargon, and no submission Excellent conformance,, good conformance, average conformance,, below par conformance, and no conformance 4, 3, 2, 1, and 00 Total score Calculate score Scores from 0 through 21 Rubric 2: Evaluation of Communication in Projects Attribute Measure Scoring Student Score Review the discussion results looking for evidence that the student was engaged in the project activities Grade as fully engaged, partially engaged, or lacks engagement or was not present Factor a, values 0, 1, 2 Review the class participation records on Blackboard to look for evidence of participation by each student in weekly group activities Review the written communications for readability and ability to be understood by non-technical people Review of group project reports and their accuracy and completeness Mark as team leader, high levels of participation, participates effectively particularly in areas of expertise, some participation, little or no participation Mark as well developed documentation, satisfactorily developed documentation, or inadequate or missing Accurately reflects the findings of the project that can be understood by a non-technical person, accurately reflects project but is too technical in nature, does not accurately reflect project or not submitted 11 Factor b, values 0, 1, 2, 3 Factor c, values 0, 1, or 2 Factor d, values 0, 1, 2, or 3 Total score Calculate score a+b+c+d with students

12 receiving a score of 0 through