STUDENT ASSESSMENT OF TECHNOLOGY: VIDEO-CONFERENCING

Transcription

1 STUDENT ASSESSMENT OF TECHNOLOGY: VIDEO-CONFERENCING Institutional Research Spring

2 INTRODUCTION Summer semester 2008, USF Sarasota Manatee initiated the use of video conferencing technology in the classroom to broadcast courses simultaneously on the main campus in Sarasota and on the South County site in Venice. The Colleges of Arts & Sciences, Business, and Education have all taught courses using the technology. Upgrades to the system were made during the Spring 09 semester and are detailed in Appendix C. METHOD of ASSESSMENT All students were given the opportunity to complete the paper and pencil questionnaire during class time. Both scaled questions and open ended questions were used on the surveys. Some demographic information was collected as well. To view the questionnaire itself, see Appendix B. SUMMARY The purpose of this assessment was to continue to investigate the experiences of the students enrolled in the video conferencing courses offered by USF Sarasota Manatee. Comments collected during the Spring 09 semester on the advantages of the video conferencing were nearly the same as those received during the Summer and Fall 08 semesters. Findings indicated that student perceived benefits included (1)saving gas, money, and time by staying at the campus closest to their home/work and (2)more students served with the opening of the additional section, when the course may not have been held otherwise. Disadvantages included: (1)technology glitches, (2)instructor difficulties in engaging remote site students, (3)loss of class time due to connecting/reconnecting equipment, and (4)problems related to the use of different media types such as DVDs and PowerPoints. All but one of the survey s scale statements were rated lower in Spring 09 then in the preceding semesters and in some instances the differences were significant. Survey results have shown a continual drop in overall student agreement that they would recommend video conference courses to others (summer: 74%, fall: 64%). This pattern continued in the Spring 09 semester with an overall agreement of under 50%. However, it should be noted that the South County attendees did rate this statement significantly higher than those attending the Sarasota campus. Other differences between sites were found. For instance, 79% of those attending the South County site stated they would prefer to attend a video conference course than drive a further distance to attend a face to face course, compared to only 35% of those attending the Sarasota campus. In addition to rating the recommendation of video conference courses significantly higher than the Sarasota campus attendees, the South County respondents also rated the following statements significantly higher: (1) The audio was clear and comprehensible when the instructor was remote and (2) This course met my expectations. 2

3 POPULATION DESCRIPTION Response Rate 103 of the 158 students enrolled in the spring course sections completed the survey. Summer 2008 Fall 2008 Spring 2009 OVERALL ATTENDING SARASOTA CAMPUS 78% (62:80) 55% (65:119) 63% (72:115) 76% (76:100) 51% (82:160) 65% (103:158)* *Two respondents did not indicate site taking ATTENDING SOUTH COUNTY SITE 67% (14:20) 40% (17:42) 67% (29:43) Student Profile (For a complete listing of demographic information see Appendix B) The majority of students : Summer Fall Spring Were female in gender 67% 45% 58% Were age years old 57% 54% 51% Attended the Sarasota Manatee Campus 82% 79% 70% Student s College Summer Fall Spring CAS 21% 43% 35% COB 37% 54% 55% COE 41% 0% 6% US 1% 3% 2% Majors Spring semester s top three majors reported were finance (15%), accounting (11%), and criminology (11%). 3

4 Q2. Is this the first video conference course you have taken? Yes, first video conferenced course taken. 100% 80% 60% 64% 40% 20% 0% 50% 17% All Attending SM Campus Attending Sth. County Summer Fall Spring Q4. Which option are you more likely to do: a. Drive a further distance to attend a face-to-face course OR b. Attend a video-conferenced course closer to your home/work. Prefer to attend the video conference course. 100% 80% 60% 79% 40% 20% 46% 35% 0% All Attending SM Campus Attending Sth. County Summer Fall Spring 4

5 Student Perceptions Students were asked to answer the following questions using a scale of 1 4, 0 Strongly Disagree (SD), Disagree (D), Agree (A), Strongly Agree (SA), or Don t know (DK). Student perceptions of video conferencing varied as shown by the standard deviations calculated for the scale question means. Spring was the first semester in which some statements were not agreed to by the majority of the respondents. In addition, only one improvement was found in the rating of the scale statements during the Spring 09 semester in comparison to the Summer and Fall. In some instances the mean ranking showed a significant decline. Of special concern is the declining number of students strongly agreeing/agreeing that they would recommend videoconferenced courses, especially those taking courses on the Sarasota campus. 1a) I believe I learned as much in this class as I would have in a class that did not use video conferencing. Summer** 86% Fall 81% Spring** 67% *Significant difference summer/spring (z= 2.910, p<.05) 1b) I felt as free to ask questions and make comments in this class as I do in classes that do not use videoconferencing. Summer 80% Fall 79% Spring 76% c) I felt my experience at the video conference side was equivalent to my experience at the live class. Summer** 68% Fall** 63% Spring** 44% *Significant difference summer/spring (z= 3.771, p<.05), fall/spring (z= 2.294, p<.05) 5

6 1d) The video images were clear & comprehensible when the instructor was remote. Summer** 71% Fall** 74% Spring** 54% *Significant difference summer/spring (z= 3.358, p<.05), fall/spring (z= 3.926, p<.05) 1e) The audio was clear & comprehensible when the instructor was remote. Summer** 67% Fall** 64% Spring** 43% *Significant difference summer/spring (z= 3.515, p<.05), fall/spring (z= 3.352, p<.05) 1f) The instructor was effective at delivering this video conference course. Summer** 92% Fall 88% Spring** 87% *Significant difference summer/spring (z= 2.118, p<.05) 1g) All technicians were effective and helpful. Summer 84% Fall 85% Spring 87%

7 1h) This course met my expectations. Summer** 93% Fall 88% Spring** 82% *Significant difference summer/spring (z= 3.599, p<.05) 1i) I would recommend video conference courses to others. Summer** 74% Fall** 64% Spring** 45% *Significant difference summer/spring (z= 3.764, p<.05), fall/spring (z= 2.811, p<.05) *Those that answered Don t Know were not included in mean score or standard deviation. n= those that answered Strongly Disagree through Strongly Agree. **Statistical significance. Mann Whitney ρ<.05 SPRING SEMESTER SIGNIFICANT DIFFERENCES BY SITE* 1e) The audio was clear & comprehensible when the instructor was remote. Sarasota Mean Score Respondent Site South County Mean Score h) This course met my expectation i) I would recommend video conferenced courses to others *Statistical significance. Mann Whitney 1e) z= 2.284, 1h) z= 2.825, 1i) z= 2.737, ρ<.05 7

8 APPENDIX A. Student Video-conference Course Survey B. Complete Statistical Analysis C. Improvement made for Spring 2009 semester D. Planned Future Improvements 8

9 APPENDIX A: STUDENT VIDEO CONFERENCE COURSE SURVEY DIRECTIONS: To help the USF Sarasota Manatee Campus better serve you the student, please take a moment to provide feedback on your experience with video conferencing this semester. Though you may have had the majority of your class sessions face to face with the instructor, answer the following based on when you were on the receiving end of the video conference. Thank you. 1. Please indicate whether you AGREE or DISAGREE with the following statements, if you do not have enough information to judge, mark DON T KNOW Strongly Disagree Disagree Agree Strongly Agree Don't Know a. I believe I learned as much in this class as I would have in a face to face class. b. I felt as free to ask questions and make comments in this class as I do in face to face classes. c. I felt my experience in the course was equivalent to the experience of those in the live class. d. The video images were clear & comprehensible when the instructor was remote. e. The audio was clear & comprehensible when the instructor was remote. f. The instructor was effective at delivering this videoconference course. g. All technicians were effective and helpful. h. This course met my expectations. i. I would recommend video conferenced courses to others. 2. Is this the first video-conference course you have taken? Yes No 3. Which is your preferred USF site? South County MCC Venice Sarasota-Manatee Main Campus 4. Which option would you be more likely to do: a. Drive a further distance to attend a face-to-face course OR_ b. Attend a video-conferenced course closer to your home/work. 5. Your City of Residence: Zip Code: 6. City Where You Work: Zip Code: 7. Gender: Male Female 8. Age: Under College: Arts & Sciences Business Education Undergraduate Studies 10. Major: 11. Level: Junior Senior Master s Non-Degree 9

10 PLEASE TURN OVER 12. What are the advantages of video-conferenced courses? 13. What improvements could be made to video-conference courses? 14. Please feel free to provide any additional comments about video-conferencing. 10

11 Appendix B: Complete Statistical Analysis of Student Survey Question % Strongly Agree (4) 1a) I believe I learned as much in this class as I would have in a class that did not use videoconferencing. 1b) I felt as free to ask questions and make comments in this class as I do in classes that do not use video conferencing. 1c) I felt my experience at the video conference side was equivalent to my experience at the live end. 1d) The video images were clear & comprehensible when the instructor was remote. 1e) The audio was clear & comprehensible when the instructor was remote. 1f) The instructor was effective at delivering this videoconferenced course. 1g) All technicians were effective and helpful. 1h) This course met my expectations. % Agree (3) % Disagree (2) % Strongly Disagree (1) % Don t Know (0) N w/dk Mean Score (Scale 1 4) St. Dev N w/out DK 32% 35% 23% 9% 1% % 46% 18% 5% 1% % 27% 41% 11% 5% % 44% 37% 5% % 34% 45% 75 5% % 47% 8% 2% 3% % 40% 10% 2% 1% % 55% 15% 3% 1% i) I would recommend videoconferenced courses to others. 17% 28% 25% 21% 9% *Those that answered Don t Know were not included in mean score or standard deviation. n= those that answered Strongly Disagree through Strongly Agree. 11

12 Q2. Is this the first video conferenced course you have taken? All (n=103) SM Campus (n=72) Sth. Cty. Campus (n=29) a. Yes 50% 64% 17% b. No 46% 35% 79% No Response 4% 1% 4% Q4. Which option would you be more likely to do: a. Drive a further distance to attend a face-to-face course OR- b. Attend a video-conferenced course closer to your home/work. All (n=103) SM Campus (n=72) Sth. Cty. Campus (n=29) 50% 64% 17% 46% 35% 79% No Response 4% 1% 4% 12

13 Students Attending Sarasota Manatee Campus CITY WORKING IN Anna Maria Brade nton Ellenton Pt. Charl otte SRQ Venice Port Charlotte Sarasota Not Working Braden River 1 Bradenton CITY OF RESIDNECE Myakka 1 1 Nokomis 1 North Port 1 Osprey 1 Palmetto 1 1 Sarasota Venice 2 Students Attending South County Site CITY WORKING IN Engle wood Fort Myers North Port Placida Pt. Charlotte Punta Gorda Sarasota Venice Not Working CITY OF RESIDENCE Englewood 1 1 Nokomis 2 North Port Port Charlotte 1 1 Rotunda West 1 Venice 7 2 Q7. Gender All Respondents (n=103) SM Campus (n=72) Sth County Campus (n=29) Male 39% 39% 41% Female 58% 58% 55% Not Reported 3% 3% 4% 13

14 Q8. Age Range All Respondents (n=103) SM Campus (n=72) STH County Campus (n=29) Under 21 3% 3% 4% % 54% 43% % 25% 25% % 11% 18% % 4% 0% % 3% 7% 60+ 1% 0% 3% Q9. Major % of all (n=103) Finance 15% Accounting 11% Criminology 11% GBA 10% Q10. Student s College % of all (n=103) CAS 35% COB 55% COE 6% US 2% Psychology 9% Management 8% Marketing 7% Int l Business 6% ISS 6% Elementary Ed 5% English 5% All others (1 reporting) 7% Q11. Level % of all (n=103) Junior 69% Senior 29% Non Degree 1% No Response 1% 14

15 Appendix C: Improvements made for Spring 2009 semester Tutorial developed to familiarize faculty with use of the technology. A205 multiple programming changing to the touch panels to correct problems specifically with the sending of content within a video conference and the functionality within the video conference controls. A205 reconfigured microphones to reduce the amount of noise carried to the remote site. A205 added a second TV for professors (has been installed should be active by next week, need an s video splitter). A205 on site tech in the room during class. A205 on site tech is using xpanel to control room functions from a laptop (xpanel is a software version of the touch panel in the front of the room). A205 changed setting within the Polycom system (examples: changed monitor layout, changed sending content settings, etc). Venice multiple programming changing to the touch panels to correct problems specifically with the sending of content within a video conference and the functionality within the video conference controls. Venice changed microphone from table top to ceiling mic for better audio. Venice remote tech in Sarasota during class Venice remote tech is using xpanel to control room functions from a laptop (xpanel is a software version of the touch panel in the front of the room). Venice changed setting within the Polycom system (examples: changed monitor layout, changed sending content settings, etc). B240 changed microphone from table top to ceiling mic for better audio B240 replaced blown bulb on projector, now has better picture without needing to turn down lights. B240 changed setting within the Polycom system (examples: changed monitor layout, changed sending content settings, etc). B240 on site tech in the room during class. B240 on site tech is using wireless touch panel to control room function 15

16 Appendix D: Planned Future Improvements Locations B240 and Venice need to have the s video output and RGB output switched on the Polycom units, this will involve some programming changes as well as re wiring. This will allow the content to be displayed on the LCD projector in B240 and the Big TV in Venice at optimum quality. Currently the B240 location has the content being sent out of the RGB output which goes to the small TV in front of the room and the Venice location has the content being sent out of the s video to the large TV (quality is not optimal). Additional programming changes need to be made for B240 to correct some issues with touch panel sending the correct information to the Polycom unit and processor in the room. Additional programming needed for A205 and Venice to provide better feedback from button presses on the touch panels. Continued microphone adjustments in all locations to achieve optimal audio quality (no feedback, no tinning of voices, hearing yourself audio loop) this also might entail moving the microphones around (classroom placement). Continued lighting adjustments in B240 to improve classroom conditions (so you don t have to turn the lights down all the way to see the projector because then you can t see what you are writing in your notebook). 16