Exam Performance in a Hybrid Course: A Model for Assessing Online and In Class Exam Delivery Modes

Transcription

1 Exam Performance in a Hybrid Course: A Model for Assessing Online and In Class Exam Delivery Modes Susan M.Siegmann[1],Danyelle Moore[2],Christopher P.Aquino[3] ABSTRACT This study involved the impact of different assessment techniques in a higher education hybrid classroom. The study assessed the differences in exam results when student exams were proctored in-class versus online. In a freshmen level international business course twenty-four students participated in the study. The study was conducted with three conditions; students were administered four fifty-question multiple-choice exams with periodic retesting (N=24), a post-exam survey was distributed after exam one which provided feedback regarding student perceptions (N=24), and student feedback regarding the preference of the modes of delivery of exams was noted by the professor throughout the course (N=24). The proctored in-class retest scores for the students after taking the exam online indicated that they retained the material better than when the exam was first administered in a proctored in-class format. In general, exam scores were higher when the exam was administered online verses proctored in-class. The implications of these findings and the model developed by the authors to capture them are discussed herein. Keywords:0 hybrid classroom, blended teaching, exam delivery method, onlineexamintegrity,studentperceptions,highereducation,business [1]Associate Professor of Business, Niagara County Community CollegeInstructional Support Specialist Niagara University [2]Instructional Support Specialist, Niagara University [3]Assistant Professor of Accounting, Niagara University INTRODUCTION0 ExamPerformanceinaHybridCourse:OnlineversusProctoredInGClassExams Asassessmentofstudentperformancehasevolved,sohastheresearchassociatedwithit.Arelevantand timelyquestionlinkedtotheissueofassessmentiswhetherthemodeofdeliveryofanexamimpactsstudent performance.inotherwords,ifanexamisadministeredonlineratherthanproctoredingclass,willstudent performancebeaffected? ThisstudyinvolvesacomparativeanalysisofonlineandproctoredinGclassassessmentsinahighereducation hybridinternationalbusinessfreshmenlevelcourse.forthepurposesofthisstudy,a hybridcourse is consideredacoursethatemploysbothproctoredingclassandonlinedeliverymethods(babsonsurvey ResearchGroup,Pearson,SloanGC,2012).Thewayinwhichaprofessorchoosestoincorporatetechnologyina hybridcoursemayvary,buttheywillalmostalwaysusealearningmanagementsystemsandotherwebgbased toolstoattempttoaddvaluetothelearningexperience.johnson(nmc"horizon"report,"2013)states, technologyisgoingtocontinuetobeapartoftheclassroom whichshouldencourageprofessorstofind morewaystoincorporateit.theincreasinguseoftechnologyintheclassroommeansprofessorsmust determinehowitcanbeusedeffectivelyforthedeliveryofcoursecontent.inaddition,technologyallows studentstoexploreandlearninwaystheywouldnototherwisebeableto.forexample,goingonafieldtrip toanartmuseuminaforeigncountrymaynotbeanoption,buta virtual fieldtriptothemuseumbywayof theinternetmaybefeasible. 70

2 Oneofthepossiblechangescausedbytheincreaseduseoftechnologyineducationisthemodeofdeliveryof assessmentactivities.changingfromproctoredingclasstoonlineexamsmayallowforincreasedprofessorg studentinteractionduringhybridclasstimebecausetheexamcannowbeadministeredoutsideofregular classtime.thisaddsmoretimeforstudentgstudentandstudentgprofessorinteraction(orcommunity building.)respondingtothediverseneedsanddemandsofstudents(e.g.,distancelearning,reducedtravel timeandcost,necessityformoreflexibilityinschedules,etc.),manyhighereducationprogramshaveadded moreonlineandhybridcourses.(garrison&kamuka,2004).thisprovidesgreateropportunityforcreatinga collaborativeatmospherefilledwithengageddiscussionintheclassroom. Inthisstudy,theprimarypurposeforassessmentactivitiesistomeasuretheamountofmaterialastudenthas retainedandcanrecall.theprofessoradministeredbothonlineandproctoredingclassassessmentsto determinewhetherthemodeofdeliveryforexamsplayedapartinlearningretentionandrecall.thisstudy tookplaceatatwogyearstateschoolofhighereducationandinvolvedatotaloftwentygfourstudentsintwo freshmenlevelinternationalbusinessclassesusingthesameteachingmethods(samecourse). Literature0Review0 ThetransitioninthedeliveryofcoursesfromtraditionalproctoredinGclasstoonlineandhybridinhigher educationisdrivingtheneedforincreasedresearchofteachingandassessmenttechniques.today,threein tencollegestudentsreporttakingatleastoneonlinecourse,thatisupfromoneintenin2003(stengel,2012). Thecollegesoftodayneedtobepreparedtomanagetheincreaseindemandforonlinecontent. Existing literaturesuggeststheneedtosearchforpedagogicalapproachestoonlineeducationthatimprovethequality ofstudentlearning,stimulatefacultyintellectualgrowth,andenhanceoverallacademicproductivity (Bishop, 2003).Effectiveassessmentisacriticalcomponentofqualitylearningandacademicproductivity. Onewaycollegesareaddressingtheincreaseddemandforonlinecontentdeliveryisbyofferinghybrid courses.ahybridorblendedcourseisacoursewherea substantialproportionofthecontent(30g79%)is deliveredonline,typicallyusesonlinediscussions,andtypicallyhasareducednumberofproctoredingclass meetings (Allen&Seaman,2011).TheofferingofhybridcoursesreducestheamountofproctoredinGclass classtimeandchallengesstudentstoworkwiththematerialoutsideofthetraditionalclassroom.thehybrid formatallowstheadministrationofexamstobemovedtoanonlineformat(fromproctoredingclass)ifthe professorchooses.inaddition,withappropriatecontrolsinplace,itmaintainsthesummativeassessment environmentasafinaldisplayofstudentlearning(morgan&o Reilly,1999).Thedecisiontoadministerexams online(versesproctoredingclass)inahybridclassallowstheopportunityforgreaterstudentgprofessor interactionduringtheproctoredingclasstime.however,studentsmaynotbeasfamiliarwithstudyhabitsfor preparationofanonlinetestastheyareforatraditionalingclasstest(hawk,2007). Thehybridenvironmentchallengesstudentstotakegreaterownershipoftheirlearningbecausetheyoften mustloginandworkwiththematerialontheirownschedule.inthisenvironment,studentsmaybegiven controloverthetimingandenvironmentoftheirexambydesigningtheclasstoincludeonlineassessments. Thiscandecreasestressandaidstudentsinmoreefficientlycompletingtheirexamtasks.Itmayallow studentstooptimizetheirsleepgwakecycle resultinginbetterexamperformance (Hartley&Nichols,2008). Inaddition,theprofessorGstudentinteractioncanincreaseintheclassroombecausestudentsshouldbebetter preparedaftercompletingthedeepthinkingactivitiesduringtheonlineportionofthehybridcourse (Kahneman,2011).Itbecomescriticaltoemploytheappropriatetoolforthetaskwhenaskingstudentsto workontheirowntolearnthematerial(graham,2005)(kenney,2012). Kirtman(2009)foundthat changedoesnot necessarilymeanareductioninlearning. Learningisimportant tothestudentwhentheyareinvestedinitandfeelthattheyhavethetimetodoit(prince&felder,2006). 71

3 Whenaprofessoradministersanexamonline,his/herstudentscantakeitwhenitismostconvenientforthem todoso(withinthewindowoftheexamtime). Differentpeoplefunctionmoreefficientlyatdifferenttimesof day,andthiscanaffecttheirperformanceonanytask. (Smithetal.,2002).Intermsofteachingandlearning, thismeansthatsomeprofessors,andsomestudents,willfunctionbetteratdifferenttimesofday.italso meanssomestudentswillbemoresuccessfulonexamsatdifferenttimesofday(hartley,2008). HollisterandBerenson(2009)foundtherewerenodifferencesincourseorexamperformancewhen comparingaproctoredversesunproctoredonlineexamenvironmentusingactivitygbasedexams.however,it shouldbenotedthehollisterandberensonstudydidnotusemultiplegchoiceexams,aswasdoneinthisstudy. Thoughtestenvironmenthasreceivedsignificantattentionfromresearchers,noinstancesofpreviouswork werefoundwherethemodeofexamdelivery(onlinevs.proctoredingclass),test/retest,andhybridclassroom criteriawerecombined.thesetopicsarethefocusofthisstudy. Method0 0 ThepopulationunderstudyconsistedofundergraduatestudentsenrolledintwofifteenGweekhybridfreshmen levelinternationalbusinesscoursesatatwogyearcollege.theresearchteamconsistedofaclassroom professorteachingthetwointernationalbusinesscoursesinwhichthestudywasconductedandtwo collaboratingmembersfromanearbyfourgyearmastersgrantinginstitution.thefirstwasafacultydeveloper withknowledgeandinterestintheareaofonlinetesting.thesecondwasanaccountingprofessorwhowas interestedinlearningmoreabouttheintegrityofonlineassessment. Theclassroomprofessortaughttwosections,backtoback,ofthesameinternationalbusinesscourse.The earliersectionwillbereferredtothroughoutthisstudyasclassaandthelaterasclassb.theprofessor administeredfourexamsthroughoutthesemester.eachexamconsistedoffiftymultiplegchoicequestions; wasofequalvalueintermsofvolume,complexityandcontributiontotheoverallgradeoftheclass;andnone werecomprehensiveinnature.theexamswereadministeredintwodifferentmodes,onlineandproctoreding class,andatdifferenttimesthroughoutthesemester. Theexamdates(excludingretests)wereprovidedtostudentsonthesyllabusatthebeginningofthesemester withanexplanationthatanyoftheexamscouldbeadministeredonlineorproctoredingclass.astheyentered classroomonthedayoftheexam,studentswerenotifiedwhethertheywouldbetakingtheexamproctored ingclass(atthattime)oronline(theyhaduntilmidnightthatnighttocompletetheexamonlineinthelearning ManagementSystem).Assuch,allstudentsshouldhavebeenpreparedfortheproctoredinGclassexamwhen theyarrivedtoclassonthepregannouncedexamday.thisdesignwasemployedasatattempttoovercome thereducedperformanceofonlineexamsassociatedwithoverconfidenceandassociatedlimitedpreparation. Inotherwords,ifstudentsknowanexamwillbeonlinetheremaybeatendencynottoprepareaswellforthe exambecausetheyknowtheywillhaveaccesstooutsideresourceswhiletakingtheexam.bynotknowing whichformatwouldbeemployedandwhattimeofdayitwouldbeadministered(eitheringclassatthetimeof theclassofafterclassuntilmidnightonthedayoftheexam),thestudentspreparedasifitwasaningclass exam.thispreparationislikelytohavebeenmorerigorousthaniftheyknewtheexamwasgoingtobeonline. Inpart,thisstudyisaresponsetothecallformoreresearchbytheHollisterandBerenson(2009)study,the differencebeingmultiplegchoicequestionswereemployedhereinwhilehollisterandberensen(2009)used activitygbasedassessmenttechniques.xuandjaggars(2013)foundthatifstudentsknewinadvancethekind oftesttheyweregoingtotake,theoutcomewasaffected.therefore,inthisstudystudentswerenotpreg notifiedoftheexamformat.however,itshouldbenotedthatbecausestudentsdidnotknowtheformatof theexamaheadoftimetheywerelikelypreparedtotaketheingclassexam.iftheexamwasthenannounced 72

4 tobeonlinetheyhadextratime(untilmidnightthatday)tostudymore(ifdesired)beforecompletingthe exam. AllonlineexamswereadministeredwithcontrolsprovidedbytheLearningManagementSystemusedforthis course.thesecontrolswereputinplacetomaintainconstructvalidity(shuell,1986).morespecifically, constructvaliditywasmaintainedbyusingthesametestquestionsandanswerchoices,thenscramblingthe orderofbothinthedifferenttestingenvironments.thiswasdonetohelpensureexamswouldbeequally rigorousduringanyfutureretestingofthematerial.however,becausethesameexamquestionsandanswers wereusedforinitialexamsandretests,therewastheriskthat questionmemorization (Squire,1992)and/or priming mightaffecttheoutcomesofthestudy.primingreferstothephenomenonthatonceanobjecthas beenperceivedorprocessed,itcanbemoreeasilyperceivedorprocessedthenexttimeitisencountered (Baddeley,2004). TheLearningManagementSystemwassettolimitthetimeastudentcouldspendonanonlineexamtosixty minutesandtorandomizethequestionsandanswersforeachstudenttakingtheexam.allproctoredingclass examswereadministeredwiththesamecontrolsfortime(sixtyminutes)andthesamequestionsandanswers astheonlineexam,butthequestionswerenotrandomizedforeachstudenttakingtheexam.duringthonline exam,studentsviewedtheentireexamatonceallowingforbacktrackingtoemulatethepaperandpencil testingenvironment.however,duetothescramblingfeatureofthelearningmanagementsystemforonline exams,questionsandanswersforproctoredingclassexamsweredeliveredinadifferentorderthanforonline exams.samavatietal(2012)recommendsreducingthetimeallowedforonlineexamsto65g75%ofthetime allowedforproctoredingclassexamstoaccountforthefactthatstudentshaveaccesstooutsideresources whiletakingonlineexams.thiswasnotdoneinthisstudy.hillierandfluck(2013)notethereisapprehension onthepartofacademicsandmanagementadoptingnewfullyelectronicprocessesforhighstakesexams insteadofthefamiliarpaperbasedprocesses.thisisabarrierthatneedstobeaddressedwithreliabledigital systemsandprocedurestomakeasmoothtransitionfrompentokeyboard.duringthetimeaprofessor choosestoconvertanexamthesupportoftheteachingandlearningcenterandexperiencedcolleaguesare imperativebecauseoftheuncertaintyandlearningcurvewithnewtechnologytheprofessorisexperiencing. Tominimizetheimpactofquestionmemorizationand/orpriming,allretestswereadministeredoneweek aftertheinitialexams.testgretestreliabilityisameasureoftheconsistencyofapsychologicaltestor assessment(shuell,1992).thisisusedtodeterminetheconsistencyofatestacrosstime.retestmethod, whereatestisgiventothesameindividualsafteracertainamountoftimehaspassed,isoneoftheeasiest waystoascertainthereliabilityofmeasurements.normallythecorrelationofmeasurementsacrosstimewill notbeperfectbecauseofthedifferentexperiencesandattitudesthatrespondentshaveencounteredfromthe timeofthefirsttest(key,1997).inthisstudy,thismeansstudents experiencesthroughouttheweekmay haveaffectedtheirperformanceontheretest. Thehybridcourseunderstudywasdesignedso60%ofthefinalgradewasequaltotheaverageofthefour exams(15%each).thestudentscouldnotdoharmtotheirgradeduringretesting,buttherewasanincentive totrytoperformwellonaretestbecausethehigherofthetwoscores(theinitialtakeortheretest)was averagedintotheirfinalgrade.theremaining40%ofthecoursegradeconsistedofaseriesofcasestudy activitiesthatwassupplementaltotheseassessmentactivities. TheexamdeliverymodeforClassAwasonlinethenaproctoredinGclassretestforexamone,online(noretest) forexamtwo,proctoredingclassthenaproctoredingclassretestforexamthree,andonline(noretest)for examfour.examdeliverymodeforclassbwasproctoredingclassthenaproctoredingclassretestforexam one,online(noretest)forexamtwo,proctoredingclassthenaproctoredingclassretestforexamthree,and online(noretest)forexamfour(seetable1:examresults). 73

5 FollowingtheexamoneproctoredinGclassretest,allstudentswereaskedtocompleteashortsurveyrelating tothedifferentexamdeliverymodes.thesurveyconsistedofeightdemographicandopengendedquestions. Studentscompletedthesurveyinclasswhiletheywerewaitingtoreceivetheirretestscoreonexamonefrom theprofessor.thepurposeofthesurveywastocollectfeedbackonthemodeofdeliveryandexpectedresults fromthestudentsintheclass. Findings0 Belowisatableoutliningtheaverageexamscoresachievedbystudentsinthetwointernationalbusiness classesunderstudy.thedeliverymodeoftheexamislistedwhereol=onlineandic=proctoredingclass.ifan examwasretested,therearetwoexamscoresintheboxandthedeliverymodeofbothexamsisalsoreported inthesamebox.forexample,classaexamonewasoriginallyadministeredasanonlineexam,butwas retestedunannouncedingclassaweeklater.essentially,therewereatotaloffourunannouncedretests Table1:ExamResults ClassA (n=12) ClassB (n=12) Examone OL81.38%/IC63.8% IC74%/IC59.5% Examtwo OL80% OL71.6% Examthree IC58.4%/IC58% IC55.4%/IC61% Examfour OL78.4% OL73.6% AsindicatedinTable1,examonewasinitiallyadministeredinClassAonline(81.38%)andproctoredinGclassas aretest(63.8%)exactlyoneweeklater.thestudentsearnedaclassaverageof17.58%pointsmoreonthe initialonlineexamthantheydidontheproctoredingclassretest.thesameexamwasadministeredtoclassb proctoredingclassbothtimes(74%/59.5%)andtheclassaveragewas14.5%pointsmoreontheinitial proctoredingclassexamthanontheproctoredingclassretest.examtwowasadministeredinclassa(80%)and ClassB(71.6%)onlineonly(noretest).ExamthreewasadministeredinClassA(58.4%/58%)andClassB (55.4%/61%)proctoredinGclassforboththeinitialandretestexams.ExamfourwasadministeredtoClassA (78.4%)andClassB(73.6%)onlineonly(noretest).Bothclassesearnedhigheraveragescoresonallonline examswhencomparedtotherespectiveproctoredingclassequivalentexams.theexamscoresareconsistent withotherassignmentsinthecoursewhereclassaconsistentlyoutperformedclassb. AlowerscoreontheproctoredinGclassretestforClassAonexamone(initialonline/retestproctoredinGclass) mayhaveoccurredbecausestudentshadaccesstoadditionalresourceswhentakingtheexamonline.alower scoreontheproctoredingclassretestforclassbonexamone(initialproctoredingclass/retestproctoreding class)mightbeexplainedbythefactthatretentiondissolvesquicklywhenstudentsmoveawayfromthe testingdate(johnson&mayer,2009).classaoutperformedclassbonboththeinitialexam(onlineforclassa andproctoredingclassforclassb)andtheproctoredingclassretest(proctoredingclassforbothclassaand ClassB.)ThismayindicatethattheonlinedeliveryoftheexaminsomewayhelpedtheClassAstudentsbetter learnandrecallthematerial

6 Survey0Results0 The Online Journal of Distance Education and e-learning Volume 2, Issue 4 Afteradministeringtheretestonexamone,studentsweregivenashortsurveytocomplete.Amajorityof studentswhocompletedthesurvey(n=24)sharedpositiveremarksaboutonlinetesting(seeappendixafor thesurvey).therewasverylittlevariationbetweentheresponsesinclassawhencomparedtoclassb.for example,whenaskedtofinishthefollowingsentence, Thedifferencebetweenthegradesismostlikely because,studentsrespondedwithanswersdemonstratingtheirawarenessofthepossibilityofgreater learningtakingplaceduringonlinetestingasfollows: Acomfortableenvironmenthelpsstudentswithtestanxietydobetterontestsandretain moreinformation. Investigationofthematerialwhiletakingthetestincreasesthelearningofthematerial. Studentsliketheindependenceofonlinetesttaking. ParallelsrealGlifeexperienceinthefieldoftraining Lessstressful Prefertheabilitytoresearchthematerialthattheyarenotfamiliarwith LearnmoreinGdepthastheyreadthematerialtheydidnotknowonthetest Studentswerealsoaskedonthesurveyiftheyfeltthegradesearnedonlinewereatrueindicatorofstudent understandingofthematerial.theyansweredasfollows: Youneedtounderstandthematerialtoworkonlineefficiently. Itisuptothestudenttotakeresponsibilitytostudywithoutbeinginclass. Youmustreviewmorebecausethereislessteacherinteraction. Iwouldn thavegottenan84onthesecondtest/retestifihadn tdoneanystudyingfortheprevious test.irememberedmostofthematerial. Youareonyourownmostofthetimeanditisuptoyoutolearnit. Twopositiveaspectsofonlinetestingwererevealedthroughthestudentsurvey.First,onlinetestingisoften lessstressfulforstudentsand,second,theabilitytoresearchthequestionswithinspecifiedtimeconstraints mimicstherealworldbusinessenvironment.theseresultswereconsistentwiththosereportedinastudyby Greenberg,etal.(2008). Discussion00 Inthisstudy,theaveragegradesforonlineexamswerehigherineverycase.Thedifferenceinexamscores betweenonlineandproctoredingclassexamscouldbeexplainedbydifferentpreparationtechniquesthat studentsuseforonlineversusproctoredingclassexams(hawk,2007).hawkfoundthatstudentsaremore familiarwithproctoredingclassexamsandknowhowtostudyandprepareforthem.however,inanother 75

7 studyaccesstooutsideresourcescompensatedfortheunfamiliarformatofonlineexams(miller,rainer& Corley,2003). Anotherpossibleexplanationforbettertestresultsintheonlineenvironmentcouldbetimeofday(Hartley& Nichols,2008).Studentswererequiredtotakeonlineexamsafterclassonexamdaybutbeforemidnight. Theywereinformedofwhichitwouldbeonlywhentheyenteredtheclassroomonthedayoftheexam.Ifthe examwastobeonline,theycouldchoosethetimethatbestfittheirscheduleandmoderatedtheirstressors. Ifthiswasafactorinthedifferenceinperformanceinthisstudy,itimpliesthatconvenienceshouldbebuilt intothestructureofahybridclass.withflexibilityoftimetheresearchersdidlookatsubmissiontimesfor onlineexamsandfoundsubmissiontimeswerenotclosetooneanotherleadingtotheconclusionthat studentsintheclassesdidnotworktogetherwhileactuallycompletingtheexam. Inthisstudytheinclassretestgradeswereessentiallythesameregardlessofwhetherstudentstooktheinitial examinclassoronline.thisistrueforexamoneandexamthree.theseresultsareconsistentwiththose foundinthe2008studybygreenberg,etal.wheretheystated: Ourquantitativeanalysisresultedinnosignificantdifferencesbetweenthemeanscoresofthecontentitems onthecomprehensivefinalthatwereinitiallyassessedwithaproctoredingclassexamandthoseinitially assessedwithanonlineexam.thiswasanimportantfindingbecauseitsuggestedthattraditionally administeredexamsdidnotnecessarilyresultinbetterperformanceonatraditionallyadministered comprehensivefinalgandthesamedegreeofusefulnessintermsofperformanceonfinalexamsforonline exams. NumerousstudieshaveproposedthatstudentperformanceononlineandproctoredinGclassassessmentsis statisticallythesame(hollister&barenson,2009),(larson&chungghsien,2009),(stowell&bennett,2010), (Mentzer,etal.,2007).LarsonandSung(2009)employedcontrolsforproctoringintheonlinesectionofthe course.thestudyrequiredstudentstofindaproctoratauniversityorcollege,library,ormilitaryinstallation andtheproctorwasrequiredtocheckphotoidentification.hollisterandberenson(2009)foundtherewereno differencesincourseorexamperformancewhencomparingaproctoredversesunproctoredonlineexam environmentusingactivitygbasedexams.however,asmentionedearlier,thehollisterandberensonstudydid notusemultiplegchoiceexams,aswasthecaseinthisstudy. Commentsfromthesurveyadministeredinthisstudyindicatedthatacomfortableenvironmenthelpsstudents performbetteronexamsandhelpsthemtobetterretaintheinformationbeingtestedon.thismaybe becausethestudentsoftodayareoftenmorecomfortablewithtechnologythantheyarewithpenciland paper. ResearchontestingviacomputergoesbackseveraldecadesandsuggeststhatformultipleGchoice tests,administrationviacomputeryieldsaboutthesameresultsasviapaperandpencil. (Bunderson,1989). However,morerecentresearchshowsthatforyoungpeoplewhohavegonetoschoolwithcomputers, nationalandstatetestsadministeredviapaperandpencilcanyieldsevereunderestimatesofstudents'skillsas comparedwiththesametestsadministeredviacomputer. (Russell,1999).Testanxietyduringanonline assessmentisshowntobelowerthanthatfeltbystudentswhentakingpencilandpaperassessments(stowell &Bennett,2010). Perhapsthemostsignificantcontributionofthisstudyisthedesignoftheretest.Inthestudy, studentsenteredtheclassroomonexamdaywithoutknowingwhethertheywouldbetakingtheexam proctoredingclassatthattimeoronlinelaterintheday.thismeanttheyneededtoprepareasifitwasgoing tobeaproctoredingclassexam.ifitturnedintoanonlineexam(laterthatday)theyhadthebenefitofboth proctoredingclassexampreparationandtheuseofoutsideresourceswhiletakingtheonlineexam.thismay bethebeststructureforahybridclassassessment.inotherwords,ifahybridclassisdesignedsostudentsare unawareofwhethertheexamwillbeproctoredingclassoronline,theywillhavetoprepareforaproctoreding classexam.iftheexambecomesanonlineexam(laterthatday,)performancewilllikelybegreaterthanifit 76

8 waseitheranexpectedonlineorexpectedproctoredingclassexambecausestudentswillhavethebenefitof themorerigorousexampreparation(expectedproctoredingclassexam)andtheuseofoutsideresources (onlineassessmentresourcessuchastextbook,notes,internet,etc.) Limitations0and0Recommendations0for0Future0Research0 Thisstudywaslimitedbythesmallsamplesize(n=24)andbecauseitwasconductedatasingleinstitutionand inasinglecourse.arecommendationforfutureresearcheffortsinthisareawouldbetoadministertoalarger groupacrossinstitutionsanddepartmentsorreplicatethestudyinmultiplecourses.thelimitationofasmall samplesizemayalsohaveaffectedthesurveyresultsifthestudentsfeltidentifiable.assuch,futureresearch inthisareausingstudentperceptionsurveyswouldbenefitfromtheuseofonlineanonymoussurveying. Thepossibilityofstudentsstudyingforthesecondretestisalsoalimitationandmayhaveaffectedtheexam twoproctoredingclassretestscore.inotherwords,theelementofsurprisemayhavebeenlessenedafterthe retestprocesshadbeenusedearlierinthecourse.tocontrolforthisfactor,examsshouldbeproctoredonline andretestedmultipletimesifpossible.thisshouldbeemployedinbothclassesasclassbdidnothavethe onlineexamfollowedbyaningclassretestforthisstudy. Itisalsoimportantforfutureresearchersinthisareatomaintainconstructvalidityiftheydecidetochange examquestionsandanswerstofurthereliminatethepotentialforquestionmemory.retestingoneveryexam, thoughpossible,isnotrecommendedbecausestudentswouldexpecttheopportunitytoimprovetheirgrade andwouldlikelypreparefortheretest.however,thiswouldoccuronlyifstudentswerenotpleasedwiththeir initialexamscore.thismayskewtheretestresults.itispossiblethiswasanissueontheretestofexamthree inthisstudy. Achangeinexamformatmayalsoimprovethevalidityoftheresults.Forexample,ratherthanutilizingall multiplegchoicequestions,futurestudiesmaychoosetoadministershortgansweroressayquestionsforinitial orretests.futurestudiesmayalsoconsidertheuseofasecuredonlinetestingenvironmentsuchaslockdown Browserorliveproctoringtominimizetheuseofoutsidematerials. Conclusion0 A2011studybyAllenconcludedthelevelofstudentsatisfactionisapproximatelythesameforbothonlineand facegtogfacecourses.studentsparticipatinginthisstudyreportedthattheyweresatisfiedwiththeonline testingenvironmentandthatitdecreasedtestanxiety.overalltheprofessorandstudentsinthisstudyhad positiveexperienceswithonlinetesting.thissupportsthehypothesisthatmodeofdeliveryofexamsdoes affectoverallperformance.basedonthequalitativeandquantitativedatacollectedinthisstudy,thestudents earnedahigheraveragegradeandperformedbetterwhentheywereassessedusingonlineexams.thiswas supportedbyallonlineexamscoreswhencomparedtotheirproctoredingclassequivalents. Onlineexamsallowforgreaterflexibilityinscheduling.Inaddition,studentsenjoyusingmoderntechnology andreceivingimmediatefeedbackoftheirtestperformance.thesefactorsmeanonlineexamsareoftenless stressfulforstudentshowever,studentsarealsomorecomfortablepreparingfortraditionalingclassproctored exams(hawk,2007).thisislikelybecausethemajoritytheexamstheyfaceearlyinlifeareadministeredin thisformat.assuch,werecommenddesigninghybridcourseswithrandomtestingmodes,alternating betweenonlineandproctoredingclassformats.thiswillallowstudentstoprepareforexamsinamannerthat ismostcomfortableandfamiliartothem(ingclass)andto take atleastsomeportionsoftheirexamsina mannerthatislessstressfultothem(online.)thisprovidesa bestofbothworlds assessmentapproachtoa hybridclass.inaddition,therandomizationofmodeofdeliverymayincreaseperformanceonallexams. TheprimarypurposeofthisstudywasnottoquantitativelyproveordisprovewhetherinGclassoronline assessmentissuperior,ratheritwastoestablishandapplyaprocess/modelfordoingso.theresearchersfeel 77

9 thiswasaccomplishedandthatthissurpriseretestingmodelhasvalue.however,weacknowledgethatfurther applicationofthemodelonamuchlargerscaleandusingprovenquantitativemethodologywouldbeofgreat benefittoanacademiccommunitythatissteadilyandintentionallymarchingtowardaworldwithmuch greateremphasisonhybridandonlineteachingandlearning.assessmentproceduresinthisnewworldmust bevalidatedand bestpractice modelsneedtobedeveloped.allofthismusttakeplaceveryquicklybecause thepaceofchangetoonlineandhybridclassesisrapid.thisstudycreatedamodeltobeemployedonalarger scaleacrossinstitutionstomoveforwardthisrapidlychangingfield. REFERENCES Allen,E.,&Seaman,J.(2011,November).Babson"Survey"Research"Group:"Going"the"distance:"Online"education" in"the"united"states,"2011[pdf].retrievedfrom Allen,I.E.,&Seaman,J.(2013,January).Babson"Survey"Research"Group:"Changing"course:"Ten"years"of"tracking" online"education"in"the"united"states[pdf].retrievedfrom Baddeley,A.D.(2004).Thepsychologyofmemory.InA.D.Baddeley,M.D.Kopelman,&B.A.Wilson (Authors),The"essential"handbook"of"memory"disorders"for"clinicians(Rev.ed.).Hoboken,NJ:Wiley. Bourne,J.,&Moore,J.C.(Eds.).(2003).SloanQC:"Vol."4."Elements"of"quality"online"education:"Practice"and" direction[pdf].retrievedfromhttp://sloanconsortium.org/publications/books/eqoe4summary.pdf Engelbrecht,J.,&Harding,A.(2004).CombiningonlineandpaperassessmentinawebGbasedcoursein undergraduatemathematics.journal"of"computers"in"mathematics"and"science"teaching,23,217g232. Garrison,D.R.,&Kanuka,H.(2004).Blendedlearning:Uncoveringitstransformativepotentialinhigher education.the"internet"and"higher"education,7(2),95g105.retrievedfromsciencedirectdatabase. Graham,C.R.(2006).Blendedlearningsystems.InC.J.Bonk&C.R.Graham(Authors),The"handbook"of" blended"learning:"global"perspectives,"local"designs.sanfrancisco,ca,usa:pfeiffer. Greenberg,K.,Lester,J.,Evans,K.,Williams,M.,Hacker,C.,&Halic,O.(2008).Studentlearningwith performancegbased,proctoredingclassandlearnergcentered,onlineexams.international"journal"of"teaching" &"Learning"In"Higher"Education,20(3),383G393. Hartley,J.,&Nicholls,L.(2008).Timeofday,examperformanceandnewtechnology.British"Journal"of" Educational"Technology,39(3),555G558. Hawk,T.F.,&Shah,A.J.(2007).Usinglearningstyleinstrumentstoenhancestudentlearning.Decision" Sciences"Journal"of"Innovative"Education,5(1),1G19. Hillier,M.,&Fluck,A.(2013).ArguingagainforeGexamsinhighstakesexaminations.RetrievedFebruary19, 2014,fromTransformingExamswebsite: Hollister,K.K.,&Berenson,M.L.(2009).Proctoredversusunproctoredonlineexams:Studyingtheimpactof examenvironmentonstudentperformance.decision"sciences"journal"of"innovative"education,7(1),271g Johnson,C.I.,&Mayer,R.E.(2009).Atestingeffectwithmultimedialearning.Journal"of"Educational" Psychology,101(3),

10 Kahneman,D.(2011).Thinking,"fast"and"slow.NewYork,NY:Farrar,StrausandGiroux. Kenney,J.L.(2012).Gettingresults:Smallchanges,bigcohortsandtechnology.Higher"Education"Research"&" Development,31(6),873G899. Key,J.P.(1997).ModuleS2Correlation.RetrievedFebruary19,2014,fromResearchDesigninOccupational Educationwebsite: Larson,D.K.,&Sung,C.GH.(2009).Comparingstudentperformance:OnlineversesblendedversesfaceGtoGface. Journal"of"Asynchronous"Learning"Networks,13(1),31G42. Mentzer,G.A.,Cryan,J.,&Teclehaimanot,B.(2007).Twopeasinapod?AcomparisonoffaceGtoGfaceand webgbasedclassrooms.journal"of"technology"&"teacher"education,15(2),233g246.retrievedfromeducation Sourcedatabase. Miller,M.D.,Rainier,R.K.,&Corley,J.K.(2003).PredictorsofengagementandparticipationinanonGline course.retrievedfebruary19,2014,fromonlinejournalofdistancelearningadministrationwebsite: Morgan,C.,&O'Reilly,M.(1999).Assessing"open"and"distance"learners.London:KoganPage. NMC"horizon"report":"2013"higher"education"edition[PDF].(2013).Retrievedfrom Prince,M.J.,&Felder,R.M.(2006).Inductiveteachingandlearningmethods:Definitions,comparisons,and researchbases.journal"of"engineering"education,95(2),123g tb00884.x Russell,M.,&Haney,W.(1997).Testingwritingoncomputers.Education"Policy"Analysis"Archives,5. Samavati,H.,Stumph,C.F.,&Dilts,D.(2012).Interestingtimes:Relationbetweentesttimesandstudent performanceinonlinecourses.journal"of"higher"education"theory"&"practice,12(6),74g80.retrievedfrom EducationSourcedatabase. SchoenfeldGTacher,R.,McConnell,S.,&Graham,M.(2001).DonoharmGAcomparisonoftheeffectsofonGline vs.traditionaldeliverymediaonasciencecourse.journal"of"science"education"&"technology,10(3),257g265. RetrievedfromEducationSourcedatabase. Shuell,T.J.(1986).Cognitiveconceptionsoflearning.Review"of"Educational"Research,56(4),411G Shuell,T.J.(1992).Designinginstructionalcomputingsystemsformeaningfullearning.Adaptive"Learning" Environments,85,19G54. Siemens,G.(2005,August10).Connectivism:LearningasnetworkGcreation.RetrievedFebruary20,2014,from elearnspacewebsite: Smith,C.S.,Folkardb,S.,Schmiedera,R.A.,Parraa,L.F.,Spelten,E.,&Almiral,H.(2002).Investigationof morning eveningorientationinsixcountriesusingthepreferencesscale.personality"and"individual" Differences,32(6),949G968. Squire,L.R.(1992).Declarativeandnondeclarativememory:Multiplebrainsystemssupportinglearningand memory.journal"of"cognitive"neuroscience,4(3),232g

11 Stengel,R.(2012,November29).Reinventingcollege.Time,180(18),31.RetrievedfromAcademicSearch Completedatabase. Stowell,J.R.,&Bennett,D.(2010).Effectsofonlinetestingonstudentexamperformanceandtestanxiety. Journal"of"Educational"Computing"Research,42(2),161G171.RetrievedfromEducationSourcedatabase. Thorndike,E.L.(1913).Educational"psychology.NewYork,NY:Teacherscollege,ColumbiaUniversity. Xu,D.,&Jaggars,S.S.(2013).Theimpactofonlinelearningonstudents courseoutcomes:evidencefroma largecommunityandtechnicalcollegesystem.economics"of"education"review,37,46g57. Appendix(A: Post-Exam Survey 1) My ACTUAL online exam grade was (please circle) , 89-80, 79-70, 69-60, 59-50, 49-40, 39-30, 29-20, or 9-0 2) I EXPECT to earn a (please circle) higher or lower in-class exam grade. 3) The difference between the grades is most likely because PLEASE STOP HERE UNTIL YOU HAVE RECEIVED YOUR GRADED IN CLASS EXAM 4) My ACTUAL in-class exam grade is 5) This difference between the expected and actual in-class exam grade is most likely because 6) This exam grade difference between the online and actual in-class is most likely because 7) Do you expect the grade differences you have seen for yourself to be similar or different when compared to those of your classmates? Circle: Similar Different Please give at least three reasons why this may be. 1) 2) 3) 8) Do you feel grades earned in online/hybrid classes are a true indicator of a student understanding of the material? Please explain. (Optional Question) Your emotions as you completed this survey can best be described as: (Optional Question) In general, how do you feel about online/hybrid classes verses traditional classes? THANK YOU FOR YOUR PARTICIPATION IN THIS VERY IMPORTANT SURVEY 80