Department: Educational Leadership and Instructional Technology

Transcription

1 Overview: Department: Educational Leadership and Instructional Technology Report Preparer: Dr. Farough Abed Program Name and Level: Educational Technology Masters of Science Program Assessment Questions: 1. URL: 2. LO Changes: There were no changes in the learning outcomes. 3. Strengths: The assessment process is working very well and data collected by faculty and tabulated by program director is effective and reliable. In addition, focus group from student body meet annually to discuss how to strengthen the courses, projects, provide more relevancy to the public school, and further underscore the design (instruction and visual design) of the courses. 4. Improvements: In general, students are receiving additional tutorials as well as more lab time to improve their skills and build more confidence. The data gathered from 2013 is an opportunity for the faculty to improve, clarify, and enhance the overall graduate students learning outcomes and the graduate program as a whole. The following is a description of changes for the program: 1. Courses must exhibit strong linkage in the sequence of courses. Since the program is designed as a strong developmental model, linkage insures that the unity among courses is maintained. 2. Redundancy must be eliminated among courses/projects in the area of technology. 3. All courses must reflect the program goal of effective design, production, and evaluation of instructional materials using technology. All courses will provide extra remediation through stand-alone instruction and lecture recordings A critical change was the delivery of instruction either online classes or as a hybrid course. Due to the nature of the graduate students lifestyles, work locations, and learning styles, online became available in the Fall, Spring and Summer terms. For instance, the EDT 500 Fall class offers one section as an online hybrid experience. Summer courses such as EDT 532, and 598 are online courses and have become even more critical since ET graduate students have summer jobs or family commitments. As a side note, graduate students still have a strong preference for face-

2 to-face courses. Group interaction, observing other students work, and direct feedback were some of the reasons expressed in comments. EDT 514 & EDT 533 Distance Learning II Through several years of review of the program via focus groups, and student course evaluation, the program has decided to withdraw two courses: EDT 514 and 533 from its requirements. Instead, graduate students will see the objectives associated with these courses in all course. By imbedding the objectives into the classes, students will have stronger association and richer meaning. Summary of program revision The current requirement for a Masters in Educational Technology stands at 30 credits. This is a change from 36 credits. This was an academic decision (stronger linkage among courses). Graduate students in the focus group consistently argued that the program should move toward a 30 credits to allow early graduation and meaningful financial commitment. 1. EDT 500 Instructional Design and Evaluation I 2. EDT 501 Message Design and Production I 3. EDT 510 Design Tools 4. EDT 512 Computer-based Instruction 5. EDT 521 Interactive Multimedia for Instruction I 6. EDT 522 Instructional Design and Evaluation II 7. EDT 531 Interactive Multimedia for Instruction II 8. EDT 532 Distance Learning and Networking I 9. EDT 598 Inquiry in Educational Technology 10. EDT 597 Final Project P lan NOTE: EDT 514 as now an online course that is offered as a non-major course. One of the unique changes to the program and to Central CT State University is laptop loaning. ET students are supported with fully loaded new Mac laptops to augment their classroom experience. The laptops have made the technology universal what they see and do in the classroom is available at home or at work through their laptops. The cost savings in hardware and software has been a tremendous economic boost to students. This coming academic year, starting in the Fall, graduate students will be using ipads to support various experiences in the graduate courses. The ipads are not loaned to the students like the laptops, but are available in all of the classrooms. The inclusion of ipads was not meant to show the latest toy but to link apps development, ibook authoring, diagnostic tools, and other productivity solutions to effective instruction.

3 LO #1 The students will be able to understand and apply technology skills in the development of instruction Assessment Instruments: Faculty initially use project evaluations to reflect course outcomes as well as the learning outcomes. Students apply their skills (instructional design, production skills, and evaluation) to realistic classroom needs/problems to demonstrate how well they synthesized their skills. The assessment of classroom projects is based on previously agreed rubrics. The data for each learning outcome is collected using scoring rubrics specific to each course on the basis of 1. Mini hands-on projects (50%) 2. Major final hands-on project (40%) 3. Class participation (10%) 4. Capstone project At the end of each semester, faculty is required to complete a learning outcome form for each graduate course. Interpretation: Faculty input data using students projects unto Learning Outcome Form and submit to program director who gathers all the data forms, tabulates the data, and interpret them. This is similar procedure for all learning outcomes. Results: Conclusion Technology skills are technical in nature, requiring mastery of software and hardware to support production of effective instruction. Programming, video and audio, digital publishing, and web are types of software programs that are considered tools and delivery systems that graduate students understand and apply throughout their classes. The process of understanding and applying these technical skills takes time and practice. To support this learning outcome, laptops that are configured with all types of software programs are loaned to the students with the understanding that practice makes perfect. The majority of students (85%) exceeded expectation, whereas 14% somewhat met expectation. This is expected: the 14% require more time and even more direct one-on-one experience. Even with equal lab time (40% of class sessions is lab) to work on class projects, this percentage of students lacks confidence. They have the skills but lack confidence to move toward an independent status.

4 Evidence: The data collected from all classes using the Learning Outcome form for each learning outcome is presented in the chart below. Changes: Students are provided with more practice and additional exercises to complete at home using the laptops along with online tutorials. Students have additional lab time with faculty to work oneon-one.

5 LO #2 The students will be able to understand and apply the instructional Design Process Assessment Instruments: Faculty initially use project evaluations to reflect course outcomes as well as the learning outcomes. Students apply their skills (instructional design, production skills, and evaluation) to realistic classroom needs/problems to demonstrate how well they synthesized their skills. The assessment of classroom projects is based on previously agreed rubrics. The data for each learning outcome is collected using scoring rubrics specific to each course on the basis of 1. Mini hands-on projects (50%) 2. Major final hands-on project (40%) 3. Class participation (10%) 4. Capstone project At the end of each semester, faculty is required to complete a learning outcome form for each graduate course. Interpretation: Faculty input data using students projects unto Learning Outcome Form and submit to program director who gathers all the data forms, tabulates the data, and interpret them. This is similar procedure for all learning outcomes. Results: Conclusion The instructional design process is critical to the development of effective instruction and students not only need to understand but apply it throughout their projects. This is the foundation learning outcome that becomes the glue to all of the students development as instructional designers. 95% of students exceeded expectation in all the courses in understanding and applying this process. 4% somewhat met that expectation and finally 1% did not meet the expectation. The data is healthy and shows that the majority of the ET graduate students performed exceedingly well. The group of students who fell in the second and third category is expected. Delays in submitting projects and weakness in grasping and applying the instructional design process are some of the reasons. Instructional design is a natural process for most teachers and trainers in the program to grasp and apply.

6 Evidence: The data collected from all classes using the Learning Outcome form for each learning outcome is presented in the chart below. Changes: To deal with the small percentage of students who somewhat meet the expectations and did not meet expectation, there will be an increase in tutorials and exercises for them to practice. Students are provided with more practice and additional exercises to complete at home using the laptops along with online tutorials. Students have additional lab time with faculty to work oneon-one.

7 LO #3 The students will be able to understand and apply the production skills in the development of instruction Assessment Instruments: Faculty initially use project evaluations to reflect course outcomes as well as the learning outcomes. Students apply their skills (instructional design, production skills, and evaluation) to realistic classroom needs/problems to demonstrate how well they synthesized their skills. The assessment of classroom projects is based on previously agreed rubrics. The data for each learning outcome is collected using scoring rubrics specific to each course on the basis of 1. Mini hands-on projects (50%) 2. Major final hands-on project (40%) 3. Class participation (10%) 4. Capstone project At the end of each semester, faculty is required to complete a learning outcome form for each graduate course. Interpretation: Faculty input data using students projects unto Learning Outcome Form and submit to program director who gathers all the data forms, tabulates the data, and interpret them. This is similar procedure for all learning outcomes. Results: Conclusion Production skills are the applied side of the program. It allows the instructional design skills and the technical skills to interact in a process call production. Graduate students throughout the program learn to produce their design plan and follow a production process that reflects time management, production blueprint, resource allocation, and technical requirements. In other words, did they do it on time, did it match the design plan, and was it as effective as the plan articulated? Production skills allows students to see an end product that they can evaluate and compare with the design plan and understand why sometimes the end product is not the same as the plan. The more students complete projects, the more the design plan and end product become the same. It is a synthesis process. This learning outcome is solid. 82% exceeded expectation while 13% somewhat met expectation. The difference in percentage reflects confidence, practice, project management, independent work, and leadership skills.

8 Evidence: The data collected from all classes using the Learning Outcome form for each learning outcome is presented in the chart below. Changes: To deal with the small percentage of students who somewhat meet the expectations and did not meet expectation, there will be an increase in tutorials and exercises for them to practice. Students are provided with more practice and additional exercises to complete at home using the laptops along with online tutorials. Students have additional lab time with faculty to work oneon-one.

9 LO #4 Students will be able to understand, apply and evaluation standards to various instructional programs Assessment Instruments: Faculty initially use project evaluations to reflect course outcomes as well as the learning outcomes. Students apply their skills (instructional design, production skills, and evaluation) to realistic classroom needs/problems to demonstrate how well they synthesized their skills. The assessment of classroom projects is based on previously agreed rubrics. The data for each learning outcome is collected using scoring rubrics specific to each course on the basis of 1. Mini hands-on projects (50%) 2. Major final hands-on project (40%) 3. Class participation (10%) 4. Capstone project At the end of each semester, faculty is required to complete a learning outcome form for each graduate course. Interpretation: Faculty input data using students projects unto Learning Outcome Form and submit to program director who gathers all the data forms, tabulates the data, and interpret them. This is similar procedure for all learning outcomes. Results: Conclusion Graduate students learn and apply formative and summative evaluation strategies. Formative evaluation is used to assess how effective the design plan is: instructional strategies, instructional techniques, etc. Summative evaluation is brought in after the production is finished and there is a project to assess whether or not learning has taken place for students using this product. Once the data is collected, then students revise their project accordingly. Students understand and apply their own rubrics in creating formative and summative evaluation rubrics to better manage the effectiveness of the end project. Some rubrics are universal and are easily available online while others are specific to the project itself. The process of learning to evaluate their project underscores that good instructional designers are also good assessors. Some of the graduate students might not go on and design learning experiences but review and assess other products. The result for this learning outcome mirrors the production and design learning outcomes. The majority of students (98%) exceeded expectation and only 2% somewhat met expectation.

10 Evidence: The data collected from all classes using the Learning Outcome form for each learning outcome is presented in the chart below. Changes: To deal with the small percentage of students who somewhat meet the expectations and did not meet expectation, there will be an increase in tutorials and exercises for them to practice. Students are provided with more practice and additional exercises to complete at home using the laptops along with online tutorials. Students have additional lab time with faculty to work oneon-one.

11 LO #5 Students will be able to understand and apply The technology integration process Assessment Instruments: Faculty initially use project evaluations to reflect course outcomes as well as the learning outcomes. Students apply their skills (instructional design, production skills, and evaluation) to realistic classroom needs/problems to demonstrate how well they synthesized their skills. The assessment of classroom projects is based on previously agreed rubrics. The data for each learning outcome is collected using scoring rubrics specific to each course on the basis of 1. Mini hands-on projects (50%) 2. Major final hands-on project (40%) 3. Class participation (10%) 4. Capstone project At the end of each semester, faculty is required to complete a learning outcome form for each graduate course. Interpretation: Faculty input data using students projects unto Learning Outcome Form and submit to program director who gathers all the data forms, tabulates the data, and interpret them. This is similar procedure for all learning outcomes. Results: Conclusion Although the main focus of the program is the design, production, and evaluation of effective instruction, it is important to also focus on the integration skills where students learn how to implement a product (software or hardware) that is purchased or off the shelf. What is the process of implementing a product? What type of landmines can one expect? What are the stages of integration and its impact on the lesson plan or a design plan? Integration skills are valuable since they support the finding that most of the time teachers do not have the time to design and produce effective instruction and are more inclined to use existing products. This learning outcome data reflected a dedicated course EDT Integrating Technology in the curriculum. Results from the focus group, data from learning outcome forms, and student course evaluation revealed that it would be best to have integration skills imbedded throughout the courses and not

12 taught in one course. The program course requirement dropped EDT 514 and revamped all course objectives for the academic year to reflect integration objectives. This is a good move to insure that the best instructional designer also has great integration skills. For the academic year , 98% exceeded expectation and 2% somewhat met expectation. Students consistently mastered the integration process much more readily than designing a project from scratch. Evidence: The data collected from all classes using the Learning Outcome form for each learning outcome is presented in the chart below. Changes: To deal with the small percentage of students who somewhat meet the expectations and did not meet expectation, there will be an increase in tutorials and exercises for them to practice. Students are provided with more practice and additional exercises to complete at home using the laptops along with online tutorials. Students have additional lab time with faculty to work oneon-one.

13 LO #6 Students will be able to understand and apply inquiry skills in educational technology research Assessment Instruments: Faculty initially use project evaluations to reflect course outcomes as well as the learning outcomes. Students apply their skills (instructional design, production skills, and evaluation) to realistic classroom needs/problems to demonstrate how well they synthesized their skills. The assessment of classroom projects is based on previously agreed rubrics. The data for each learning outcome is collected using scoring rubrics specific to each course on the basis of 1. Mini hands-on projects (50%) 2. Major final hands-on project (40%) 3. Class participation (10%) 4. Capstone project At the end of each semester, faculty is required to complete a learning outcome form for each graduate course. Interpretation: Faculty input data using students projects unto Learning Outcome Form and submit to program director who gathers all the data forms, tabulates the data, and interpret them. This is similar procedure for all learning outcomes. Results: Conclusion Although the main focus of the program is the design, production, and evaluation of effective instruction, it is important to also focus on inquiry skills. An excellent instructional design is also a great researcher but not in the sense of applied research. The focus of this learning outcome is

14 more about what the literature in educational technology is saying. What are the findings on the effectiveness of instructional technology? The more students have reliable data from research studies, the stronger their design plan becomes. Their design decision is based on objective findings that are tied to research and avoid any subjectivity. The majority of students (90%) exceeded expectation while 8% somewhat met expectation and 2% did not meet expectation. To read and make sense of research articles and interpret them accurately takes some understanding of statistics and numbers in general. Some 2% have difficulty in this area. To improve on this, tutorials and stand-alone exercises are added to help future students. Evidence: The data collected from all classes using the Learning Outcome form for each learning outcome is presented in the chart below.

15 Changes: To deal with the small percentage of students who somewhat meet the expectations and did not meet expectation, there will be an increase in tutorials and exercises for them to practice. Students are provided with more practice and additional exercises to complete at home using the laptops along with online tutorials. Students have additional lab time with faculty to work oneon-one. LO #7 Students will be able to understand and apply leadership skills in applying instructional technology in their respective working environments. Assessment Instruments: Faculty initially use project evaluations to reflect course outcomes as well as the learning outcomes. Students apply their skills (instructional design, production skills, and evaluation) to realistic classroom needs/problems to demonstrate how well they synthesized their skills. The assessment of classroom projects is based on previously agreed rubrics. The data for each learning outcome is collected using scoring rubrics specific to each course on the basis of 1. Mini hands-on projects (50%) 2. Major final hands-on project (40%) 3. Class participation (10%) 4. Capstone project At the end of each semester, faculty is required to complete a learning outcome form for each graduate course. Interpretation: Faculty input data using students projects unto Learning Outcome Form and submit to program director who gathers all the data forms, tabulates the data, and interpret them. This is similar procedure for all learning outcomes. Results: Conclusion There are two components to leadership skills. One is project management and the other is leadership as a change agent in students respective working environments. The first set of skills is taught and monitored throughout the program since most in-class projects require planning and managing projects. Did they finish on time? What unforeseen obstacles did they come across that were not part of the plan? These and other questions can be taught and take on a major focus in this outcome. It is the second area: Leadership as change agent that is challenging. The issue is twofold:

16 monitoring the students as change agents and looking at evidence of change; and resistance to such roles in schools and businesses. The program is adding successful stories of ED Tech working in the school system. Also, the program is initiating an association for Ed Tech majors that will meet to discuss issues, solutions, ideas, etc. with one another. This will be monitored by an Ed Tech faculty member. Evidence: The data collected from all classes using the Learning Outcome form for each learning outcome is presented in the chart below.

17 General Education : Not applicable.