An Evaluation of Accelerated Learning in the CMU Open Learning Initiative Course Logic & Proofs

Transcription

1 AnEvaluationofAcceleratedLearning inthecmuopenlearninginitiativecourse Logic&Proofs ChristianD.Schunn&MellisaM.Patchan LearningResearchandDevelopmentCenter UniversityofPittsburgh May31,2009

2 TableofContents EXECUTIVESUMMARY... 3 THETYPICALINTRODUCTIONTOSYMBOLICLOGICCOURSE... 4 SYLLABIANALYSIS... 4 SamplingofUniversities... 4 CodingSyllabi... 5 SURVEYANALYSES... 5 Participants... 5 Survey... 5 RESULTS... 5 SyllabiAnalyses... 5 SurveyAnalyses... 7 SUMMARY...10 INCLASSVS.ONLINEINLARGECOURSESTUDIES...11 PARTICIPANTS...11 Fall Spring RESULTS...13 SUMMARY...13 SAMETIME/MORECONTENTINACOMMUNITYCOLLEGE...15 CONTEXT&PARTICIPANTS...15 ATTRITIONRATES...15 TIMEONTASK...15 PERFORMANCEONMIDTERMEXAMS...16 FINALEXAMPERFORMANCE...16 SUMMARY...16 ACCELERATIONFORCOMMUTERSTUDENTSWITHOPTIONALRECITATIONSESSIONS...18 CONTENTLEARNINGRESULTS...18 ATTENDANCE...18 SUMMARY...19 SAMETIME/MORECONTENTWITHHYBRIDINSTRUCTIONINAUNIVERSITY...20 CONTEXT&PARTICIPANTS...20 TIMEONTASK...21 PERFORMANCEONFINALEXAM...21 SUMMARY...22 REFERENCES

3 ExecutiveSummary CMU sopenlearninginitiativeseekstocombineexpertiseinthecognitivescienceof learningwithdisciplinaryinstructionalexpertiseandhumancomputerinteractionto produceonlinelearningenvironmentsthatcanbroadlyimproveinstruction.thecourse Logic&Proofs(L&P)isonesuchcourse,developedunderthestewardshipofWilfriedSieg fromthephilosophydepartmentofcarnegiemellonuniversity. ThecurrentreportdescribesevaluationsofL&P,specificallytestingthehypothesisthatuse ofl&pinahybridinstructionalmode(onlineinstructioncombinedwithreducedand strategicallytargetedface to faceinstruction)canleadtoacceleratedlearning,eitherinthe formofmorecontentlearningwithinthesameamountofstudenttimeorthesamecontent learningwithinashortertimeperiodofstudenttime. Beforebeginningtheevaluationexperiment,weconductedacontextanalysistodetermine therealmarketofthelogic&proofscourse:whatkindsofuniversitiestendedtoteachthis overallcourse,whatcontenttendedbecovered,andwhatkindsoftoolsforproof constructionwerealreadybeingused.overall,l&p likecontentistaughtinamajorityof universitiesintheus,butwithsomevariationbyuniversitytype.further,l&pcontains somecontentthatmanycoursesdidnotcover.thus,theapplicabilityofthecourseishigh, andtheopportunitiesforaccelerationviamorecontentcoveragealsoexist. Theevaluationswereconductedintwophases.First,twoevaluationswereconductedata largepublictier oneinstitution,showingthatface to faceinstructionproducedequivalent learningoutcomestoonline onlyinstruction.inaddition,onlineinstructionappearedto resultinamuchloweredattritionrate,providingefficienciesforstudentsofanotherform. Second,anevaluationwasconductedofacceleratedlearningwasconductedatthree institutions,eachrepresentingdifferenttypesofinstitutions,usingdifferentformsof accelerationandevaluationlogicssuitedtothecontext(e.g.,randomassignmentto conditionvs.naturalexperiment,morecontentinequaltimevs.equalcontentinlesstime). Onestudyfoundstudentsatacommunitycollegewereabletolearnsignificantlymore contentandsomecontenttohigherperformancelevelsusinghybridinstructioncompared withface to faceinstruction.asecondstudyfoundthatcommuterstudentsataregional campusofapublicuniversitysigningupforonlineinstructionwereresistantto participatinginhybridinstructionandtherewasnotaclearassociatedbetweenamountof hybridinstructionandstudentlearning.thethirdstudyfoundthattraditionalstudents fromprimarilycomputerscienceandengineeringbackgroundsatanationaluniversity actuallyshowedsmallbutsignificantdeclinesinexamperformancewithhybridinstruction relativetoface to faceinstruction. 3

4 TheTypicalIntroductiontoSymbolicLogicCourse Inordertoprovideacomprehensivedescriptionofthetypicalintroductiontosymbolic logiccourse,weexaminedthecoursesintwosteps:1)wesystematicallyexaminedcourse syllabiand2)webroadlysurveyedthosewhotaughtsymboliclogiccourses. SyllabiAnalysis SamplingofUniversities AlistofuniversitiesintheUnitedStateswascompiledusingthedatabasefromthe CarnegieFoundationfortheAdvancementofTeaching,US&WorldNewsReport s America sbestcolleges,andwikipedia.additionalinformationabouteachschoolwasalso recorded(seetable1).arandomsampleof40ofeachtypeofuniversity(i.e.national, LiberalArts,Comprehensive,HBU,and2 yearuniversities)andallnineofthetribal Collegeswascollected.Atotalof209USuniversitieswereexamined. Table 1. Additional information collected on each school. Category Source Category Definition Funding CarnegieFoundationforthe Privatenot forprofit AdvancementofTeaching Privatefor profit Public Attendance CarnegieFoundationforthe Full time AdvancementofTeaching Medium&High Part time Transfer CarnegieFoundationforthe Hightransfer AdvancementofTeaching Lowtransfer Size CarnegieFoundationforthe Verysmall&Small AdvancementofTeaching Medium&Large Type US&WorldNewsReport National Offersbothundergraduate andgraduatedegrees LiberalArts Offersmainlyundergraduate degreesintheliberalarts Comprehensive Offersmainlyundergraduate degreeswithlessthan50% inliberalarts Wikipedia HistoricallyBlack University TribalCollege Rank US&WorldNewsReport Top Tier3&4 4

5 CodingSyllabi Foreachofthe209universities,theuniversitywebsitewasvisitedtoseeifthe universityhadaphilosophydepartmentandofferedasymboliclogiccourse.ofthe114 universitiesthatofferedsymboliclogic,64syllabiwerecollectedeitherbybeingpostedon awebsiteorbycommunicationwiththeinstructor.thebook,topicscovered,andtypeof assignmentswererecordedusinginformationcontainedinthesyllabus. SurveyAnalyses Participants MembersoftheAssociationforSymbolicLogicwererecruitedtocompletethe onlinesurvey.thisassociationisaninternationalorganizationwithover1500members fromover60differentcountries.themajorityofthemembers(52%)resideintheunited States.Responseswerecollectedfrom199participantsfrom39differentcountries.Over halfoftheresponderswerefromaphilosophydepartment,28%werefromamathematics department,and13%werefromacomputersciencedepartment.theremaining participantswerefromotherdepartments. Survey Thesurveyconsistedof26questionsdividedintofourmainsections.Themainfocusofthe surveywastodeterminehowsymboliclogiccourseswerebeingtaughtandwhether technologicaltoolswerebeingused.thefirstsectionaskedforgeneralinformationabout theresponder(e.g.,institutionname,department,country)andaboutthecourse(e.g.,who offersthecourse,howoftenitisoffered,howmanystudentstakeit,whatmajorandlevel arethetypicalstudents).thesecondsectionfocusedonthecoursespecifics,suchasthe primarytextbookandtopicscovered.thefinalquestioninthissectionaskedwhether educationalsoftwarewasused.basedontheresponse,thenextsectionfocusedoneither whyeducationalsoftwarewasnotusedorhowtheeducationalsoftwarewasbeingused. Thelastsectionfocusedonfutureeducationalsoftwareuse.Iftheresponderindicatedthat heorshewouldbeconsiderusingafully computer basedandhighlyinteractivecourse, theyweredirectedtoafinalfollow upquestionabouthowtheywouldlikelyusethistype ofcourse. Results SyllabiAnalyses Tobeginouranalyses,wewantedtoknowhowmanyuniversitiesofferedasymboliclogic coursethroughthephilosophydepartment.inordertoprovideanaccurateestimate,we firstdeterminedhowmanyuniversitiesofferedthecoursefromourrandomsample.out of209universities,55%(ormorespecifically114universities)offeredasymboliclogic course.becausethepercentageofuniversitiesofferingthecoursediffereddependingon thetypeofuniversity,wecalculatedaweightedestimateofuniversitiesofferingsymbolic logic:1258universities(seetable2). 5

6 Table 2. Estimated number of universities offering symbolic logic Liberal 2 year Comp. HBU Arts National Tribal Total #ofschools Percentageofschools offeringsymbolic logic 48% 38% 40% 75% 85% 0% 55% Estimated#of schoolsoffering symboliclogic Wewerealsointerestedinwhatvariablesmightpredictwhethertheuniversitiesoffereda symboliclogiccourse(i.e.,type,level,funding,attendance,transfer,size,andrank).the LiberalArtsandNationaluniversitiesweremorelikelytoofferasymboliclogicthanthe2 year,comprehensive,andhbuuniversities(seefigure1).thepublicuniversitieswere morelikelytoofferasymboliclogiccoursethantheprivate,not for profituniversities, whoweremorelikelytoofferitthantheprivate,for profituniversities.universitiesthat havemostlyfull timestudentsweremorelikelytoofferasymboliclogiccoursethan universitiesthathavemostlypart timestudents.mediumandlargeuniversitiesweremore likelytoofferthecoursethansmallandverysmalluniversities.top tieruniversitieswere morelikelytoofferthecoursethantier3and4universities.therewerenodifferencesin level(i.e.,2 yearversus4 year)ortransferrates(i.e.,hightransferratesversuslow transferrates). Figure 1. Frequency of offering symbolic logic by different type of universities Next,thesyllabiwerecodedtoseewhichtopicswerecovered.Nineofthesyllabididnot includeascheduleorlistoftopicstobecovered,sotheywereexcludedfromtheanalyses. Therewere12possibletopicsofinterest.Fiveofthesetopicspertainedtosententiallogic (i.e.,syntax,symbolization,semantics,derivations,andmetamathematics),andsevenof 6

7 thesetopicsconcernedpredicatelogic(i.e.,syntax,symbolization,semantics,derivations, identity,functions,andmetamathematics).overall,therewerenodifferencesbetweenthe typesofuniversitiesinthenumberoftopicstheycovered(m=7.3,sd=2.3).however,the typesofuniversitiesdiddifferinthenumberofsententiallogictopicscovered(seefigure 2).TheComprehensiveuniversitiescoveredmoresententiallogictopics(M=4.2,SD=0.4) thananyothertypeofuniversity,andthehbuuniversitiescoveredthesecondmost sententiallogictopics(m=4.0,sd=0).therewerenodifferencesinthenumberofpredicate logictopicscovered(m=3.6,sd=1.8). Figure 2. Number of sentential topics covered by type of university Finally,weexaminedwhichbookswereusedtoteachsymboliclogic.Twenty fivedifferent bookswerefound.threeinstructorschosetonotuseabook,butinsteadtodistributetheir ownnotes.themostpopularbookwas AConciseIntroductiontoLogic byhurley,which wasusedby13courses.otherbooksusedinmultiplecoursesincluded: TheLogicBook bybergman,moor,andnelson(6courses), Language,Proof,andLogic bybarwiseand Etchemendy(5courses),and IntroductiontoLogic bycopiandcohen(5courses). SurveyAnalyses Whiletheintentionofthesurveywastodeterminehowsymboliclogiccourseswerebeing taughtandwhethertechnologicaltoolswerebeingused,thefocusofthecurrentanalysisis tojustprovideacomprehensivedescriptionofthetypicalcourse.therefore,wewillonly bereportingresultsfromthefirsthalfofthesurvey.thefirstsectionofthesurveyfocused ongeneralaspectsofthecourse(e.g.,whooffersthecourse,howoftenitisoffered,who takesit).thesecondsectionfocusedonthecoursespecifics,suchastheprimarytextbook andtopicscovered. Whooffersthecourse?Accordingtoourparticipants,thedepartmentmostlikelytooffera symboliclogiccourseisthephilosophycourse(53%).otherdepartmentsthattypically offerthecourseincludethemathematics(28%)andcomputerscience(13%). 7

8 Howoftenisthecourseoffered?Theparticipantshadtheoptiontochoosebetweenevery semester/quarter,onceeveryyear,onceeverytwoyears,orlessoften.typically(63%), thesymboliclogiccoursesareofferedonceayear.lessfrequently,thecourseisoffered everysemester(26%)andeverytwoyears(9%). Whotakesthecourse?Inordertobetterunderstandwhotakesthecourse,weasked participantshowmanystudentstypicallytakethecourse,whatmajorarethey,andwhat levelarethey.theaveragenumberofstudentstakingasymboliclogiccourseis68 (SD=83),whichsuggeststhattensofthousandsofstudentsintheUSaretakingacoursein symboliclogiceachsemester.therewasalargerangeofresponses,fromaslittleas2 studentstoasmuchas550students.notsurprisingly,mostofthestudentstakingthis courseweremajoringinphilosophy(63%),computerscience(50%),ormath(45%). Othermajorsincludehumanities(26%),socialsciences(17%),engineering(10%),natural science(8%),andbusiness(5%).studentsaremostlikelytobejuniors(51%).however, studentsatalllevelstaketheclass(sophomores40%;freshman39%;seniors35%;and graduatestudents7%). Textbooks:Outofthe97participantswhoreportedtheirtextbook,almost60different textbookswerementioned.fourteenparticipantsstatedthattheydidnotuseatextbook, andnineofthoseparticipantsusedtheirownnotesinstead.themostpopulartextbooks usedwasbarwise&etchemendy s Language,Proof,andLogic (11participants), Enderton s AMathematicalIntroductiontoLogic (7participants),andBergmann,Moor,& Nelson s TheLogicBook (5participants). Topics Covered: Participantsindicatedmanydifferenttopicsthattheytypicallycoverintheirsymboliclogic class(seetable3). Table 3. Topics typically covered in a symbolic logic course Sentential:syntax&semantics 93% Sentential:truthfunctionalcompleteness&normal forms 69% Sentential:naturaldeductionproofs 63% Sentential:truthtrees(tableaux) 38% Sentential:completeness 56% Predicate:syntax&semantics 95% Predicate:naturaldeductionproofs 64% Predicate:truthtrees(tableaux) 33% Predicate:identity&definitedescriptions 55% Predicate:introductionoffunctionsymbols 39% Predicate:completeness 46% Aristotelianlogic 12% Modallogic 10% Intuitionisticlogic 13% Additionalsententialandpredicatelogic 24% Basicincompletenessandundecidabilityresults 19% 8

9 Modeltheory 25% Settheory 17% Computabilitytheory 12% Prooftheory 25% Topicsincomputerscienceormathematics 14% Topicsinmethodologyandphilosophicallogic 19% Othertopics 25% TopicWishList:Finally,wealsoaskedparticipantswhethertheywishedtheycouldcover additionaltopics.onlyhalfoftheparticipantsrespondedthattheywouldliketocover additionaltopics.theseadditionaltopicswerecategorizedintooneofninedifferent groups:1)additionalsententialandpredicatelogic,2)basicincompletenessand undecidabilityresults,3)modeltheory,4)settheory,5)computabilitytheory,6)proof theory,7)topicsincomputerscienceormathematics,8)topicsinmethodologyand philosophicallogic,and9)othertopics(seetable4). Table 4. Topics wish list Additionalsententialandpredicatelogic 12% Basicincompletenessandundecidabilityresults 18% Modeltheory 8% Settheory 4% Computabilitytheory 11% Prooftheory 16% Topicsincomputerscienceormathematics 7% Topicsinmethodologyandphilosophicallogic 23% Othertopics 12% DifficultiesinTeaching:Participantsdescribedmanydifficultiesthattheyhaveencountered whileteachingthesematerials.themostcommondifficultiesincluded adiversestudent backgroundwithawiderangeofabilities (16%), studentshaveweakmathematical/ logicalbackground (15%),and studentsarenotmotivatedanddonotcomeprepared (15%).Someoftheparticipantsindicatedthatcertaintopicsarespecificallydifficulty,such asformalreasoning(e.g.,naturaldeduction,proofs)andformallanguage(e.g., symbolization,translations). UseofEducationalSoftware:Mostoftheparticipants(81%)saidtheydidnotuse educationalsoftwareinthistypeofcourse.manyreasonsweregiven,butthemostpopular explanationsincluded softwareisnotuseful don tlikethesoftware don tneedthe software (28%)and notimetolearnthesoftware itdistractsfromthematerials (15%). Severalparticipantsindicatedthattheywouldbewillingtousethesoftwareifcertain conditionsweremet,suchas ifthesoftwarecouldhandlespecificfunctionslikesat solvers,proofs,andmodels (18%), softwarewaseasytolearn (11%),and iftherewere teachingassistantsandlabs (11%). 9

10 Thosewhodouseeducationalsoftwareindicatedwhichsoftwaretheyused.Themost commonsoftwareusedweretheonesincludedwiththebarwise&etchmendybook (Tarski sworld,fitch,andboole),whichwereusedby22%oftheparticipants.there seemedtobethreemainusesofthesoftware:assessment(62%),demonstrations/ examples(29%),andoptionalpractice(9%).whilealmost30%oftheparticipants indicatedthattheyhadnoproblemswiththesoftware,someparticipantsindicateda coupledifficultiesthattheyhaveusingthesoftware,suchas studentsdon tlikeitbecause itwastooeasyortheydidn tgetenoughfeedback (29%), thereweretechnicalproblems withthesoftware (24%)and iteliminatesthinking (18%). Theparticipantswhoalreadyuseeducationalsoftwarewereaskedwhethertheywould considerusingafullycomputer basedandhighlyinteractivesoftware.approximately 40%oftheparticipantssaidtheywouldconsideritundercertaincircumstances:asa replacementforastandardtextbook(62%),asthebasisforenrichedinstruction(55%),as thebasisforindividualizedinstruction(43%),andasafullreplacementforalecture course(18%). Summary Wefoundthatsymboliclogic,whilenotuniversallytaught,isfrequentlytaughtatalltypes ofuniversitiesexcepttribalcolleges,andthuswewishedtoexplorehybridinstructionina rangeofuniversitytypes.further,thetotalpoolofuniversitiesteachingasymboliclogic courseiswellover1000.themostcommonsettingforteachingsymboliclogicwasina philosophydepartment,andthusweselectedphilosophydepartmentsasoursettingfor experimentation.thereseemedtobegeneralopportunitiesforacceleratedlearningvia morecontentcoverageasthesurveysuggestedthatanumberoftopicsfoundinthel&p coursewerenotalwaystaught.further,asignificantproportionofinstructorsseemed opentotheuseofatoollikel&pasareplacementofanexistingtextbook. 10

11 InClassVs.OnlineinLargeCourseStudies InFall2007andSpring2008,studieswereconductedcomparingstudentspurelyusing L&Pwithnoface to facecomponentagainststudentsinatraditionalface to faceclass. Bothstudiesinvolvedthesameclassandsameinstructor.TheuniversitywasalargetoptierpublicresearchinstitutionwithahighproportionofEnglish as a SecondLanguage learners.theinstructorwasafull timelecturer,withsignificantresearchexperiencein symboliclogic,significantteachingexperiencewithsymboliclogic,andsignificant experiencewithvarioustoolsforonlineinstruction.theclasswastraditionallyaverylarge class,andthusincludedanumberofsimpleonlinetoolsembeddedintotheface to face class(e.g.,onlinequizzes).bothsectionshadaccesstotasforhelpwithexamsand homework. Participants Fall2007 Condition Online In class Startingcourse Finishingcourse %retentionrate 90% 75% Anewsectionofthecoursewascreatedshortlybeforethebeginningofthesemester, producingfewerthattypicalstudentsinthecourse.studentsvolunteeredforin class versusonlineinstructionduringthefirstweekofclass.apretestincludingdemographics wasgiventoassesspre existingdifferencesbetweenstudentsineachcondition.critically, therewasnodifferencebetweentheperformanceonpretestoftheolistudents(m=8.7, SD=3.2,N=45)andtheIn classstudents(m=9.1,sd=2.8,n=49).onalldemographic dimensions,therewereatmostminordifferences(i.e.,10%orless)betweensections(i.e., gender,year,gpa,major,priorexperiencewithlogiccourses,priorexperiencewithformal proofs,priorexperiencewithweb basedorprogrammingenvironments).therewasa higherattritionrateinthein classcondition,likelyproducingaperformancebiasinthat condition. Spring2008 Condition Online In class Startingcourse Finishingcourse %retentionrate 99% 41% 11

12 Topics/ScheduleforFall2007andSpring2008 Week In class Online Differences 1 Introductiontocourse& reasoningvalidity Introductiontocourse& reasoningvalidity 2 Quantifierfreelogic: symbolizationandsemantics 3 Quantifierfreelogic:formal syntaxandlogicalconcepts Midterm 1 Sententiallogic:syntaxand symbolization Sententiallogic:semantics (truthfunctionsandlogical concepts) SymbolizationNOT,AND,OR,IF...THEN,IFFsententiallogic.Syntax. Truth tablesemantics.conceptsvalidity,logicaltrue,logicalfalse, logicalindeterminate 4 Quantifierfreelogic:semantic tableau 5 Quantifierfreelogic: derivations/proofswith sententialconnectives 6 Quantifierfreelogic:proof strategies Midterm 2 Sententiallogic: derivations/proofs Sententiallogic:derivations withindirectrules Sententiallogic:strategiesand derivedrules truthtrees/tableauandderivationsnot,and,or,if...then,iff. 7 Predicatelogic:syntaxof quantification 8 Predicatelogic:translationsof quantification 9 Predicatelogic:quantification semantictableau Midterm 3 Sententiallogic:metatheory Predicatelogicwithout quantifiers:syntaxand translationsofquantification Predicatelogic(with quantifiers):syntaxand translationsofquantification, truthtrees derivationsnot,and,or,if...then,iff,symbolization,syntax, semantics,tableauall,existspredicatelogic 10 Predicatelogic:quantification semantictableaucont. Predicatelogic:derivations 11 Predicatelogic:derivations Predicatelogic:derivationswith strategiesandderivedrules 12 Predicatelogic:derivationswith strategies 13 Predicatelogic:identity (semantictableauand derivationswithidentity) Final Exam Predicatelogic:identity (numericquanifiers,definite descriptions,andderivations withtruthtrees) Predicatelogic:function symbols Comprehensive onlinedelayscoveringiff onlinecoverstruthtreesone weekearlier onlineaddsderivedrules onlineaddsmetatheory onlineaddsderivedrules onlineaddsfunctionsymbols 12

13 Studentsregisteredforface to faceversusonlineinstructioncourses,withadditional recruitingfortheonlinecoursetakingpartjustpriortothebeginningofthesemesterfrom studentsenrolledinthein classcourse.thelecturecoursewasmuchlargerthaninthe priorsemester,reflectingmoretypicalenrollmentsinthiscourseatthisuniversity. Interestingly,attritionintheonlinecoursewasalmostnon existent,whereasthein class conditionhadveryhighattrition(althoughattypicallevelsforthislarge,rigorouscourse). Thisdifferentialattritionratelikelyproducesverysignificantperformancebiasesinfavor ofthein classcondition.thesamepre testwasadministered.therewasnosignificant differencebetweenonline(m=10.8,sd=4.7,n=65)andthein classstudents(m=10.4, SD=4.5,N=83)onthepre test.therewerealsonodemographicdifferencesgreaterthan 10%betweenconditions. Results Inbothsemesters,studentshadthreeexamsduringthesemester,onecumulativefinal exam,andweeklysurveysregardingtimespentattendinglecture,readingmaterial,getting TAhelp,doinghomeworkonthecomputer,anddoingpracticeexamquestions.Theexams wereconservativelyselectedbasedonpriorexamsfromtheinclasssection,producinga likelyminorbiastowardstheinclasssection.somequestionsweremodifiedfortheonline sectiontoreflectminordifferencesinlogicnotationsusedinthetwosections. Semester Fall2007 Spring2008 Condition Online Inclass Online Inclass Exam1 68% 69% 72% 70% Exam2 44% 55% 55% 60% Exam3 55% 58% 54% 54% FinalExam 63% 62% 62% 65% Totaltime 8hrs/wk 7hrs/wk 8hrs/wk 9hrs/wk Inthefall,therewerenosignificantdifferencesbetweensectionsonanyoftheexams, althoughatrendtowardsbetterperformanceintheinclassconditiononexam2.online studentsreportedslightlyhighertotaltimeonthecourse,primarilyreflectinglargespikes intimeontaskinweeks5,6,8,and9.thesemomentsreflectthebeginningofmore difficultproofconstructions.asaresultofthesereportedspikesineffort,thehomework contentwasadjustedforthefollowingsemester. Inthespring,therewereagainnosignificantdifferencesbetweensectionsonanyofthe exams,althoughtherewasthesametrendtowardsbetterperformanceintheinclass conditiononexam2.thistimetheonlineconditionreportedmoretotaltimeperweekin theweeklysurveys,althoughthedifferencewasnotstatisticallysignificant. Summary Theresultsofthesetwostudiessuggestthatsimilarlearningoutcomescanbeobtainedin thiskindofsettingfromapureonlinecourseasfromaprimarilyface to faceinstruction coursewithroughlyequivalenttime on task.notehoweverthatthiscomparisonis conservativebecausethereweremuchhigherattritionratesintheface to faceinstruction 13

14 case.further,althoughwedidnothaverandomassignmenttocondition,thehuge differencesinattritionratearestronglysuggestedofanimportantretentionratebenefitof thel&pcourseovertheexistingtraditionalinstructionmodel.largelecturecourseswith rigorouscontent(ascanbeseenbytherelativelylowexamperformanceacrosstheboard) arefrequentlysubjecttohighattritionrates,andthusthisretentionratebenefitcouldbea veryimportantresultforimprovingtheefficiencyofuniversityeducation. 14

15 SameTime/MoreContentinaCommunityCollege The first accelerated learning study involved trying to cover significantly more content in a community college context. As would commonly be the case in the community college context, the instructor had a high load (4 courses/semester) of moderate enrollment courses (25 to 30), no research experience with symbolic logic, a modest teaching experience with symbolic logic, and little experience with online tools in instruction. The existing course typically covered less than half of the L&P full course: it included only sentential logic and with relatively little inclusion of full proofs even within sentential logic. The instructor s goal for the current study was to cover as much of the full L&P course as possible in a hybrid instruction mode: one lecuture per week, with students experience primary instruction through online instruction and practice. Student performance on the shared content would be compared against prior semester performance on exams derived from previous semester exams (3 sections from the prior semester, n=85 students, against 3 sections of the current semester, n=85 students). Context&Participants Participants included students registered for one of three sections of an Introduction to Symbolic Logic course offered at a midwestern U.S. community college (see Table 5 for background information i.e., gender, year, major, GPA, current course load, reason for taking course, goal grade, and computer experience). The majority of the participants had not experienced formal/symbolic logic, history of logic, or rigourous proofs. Only 30% of the participants took a single course that had formal/symbolic logic 11% took more than one of these courses. Only 17% of the participants took a course with history of logic 1% took more than one of these courses. Only 15% of the participants took a course with rigorous proofs 8% took more than one of these courses. Overall, the participants did not do well on the pretest (M =.31; SD = 1.59; range 1 to 8 out of 16). They only got 4% of the statement questions correct (SD = 0.36; range 0 to 1 out of 5), 15% of the argument questions correct (SD = 1.00; range 0 to 4 out of 4), 64% of the equivalence questions correct (SD = 1.08; range 0 to 5 out of 5), and none of the proof questions correct (out of 2). AttritionRates Thecommunitycollegeproducedananalysisoftheattritionratebetweenthetwo semestersinquestion.theanalysiscomparedtheseatsattritionfromtenthdaytoendof term.tenthdayseatsarebasedontotalgradesexcludingx,whileendoftermseatsare basedontotalgrades(excludingx)minuswsandn,followingthecollege sinternal definitionofattrition.theattritionratesforbothpriorandcurrentcourseswereidentical at 8.2%. TimeonTask Participants in the current semester were asked to rate the workload, difficulty, and amount of material covered in this course compared to other classes that they are taking. A 5-point scale (-2 to 2) was used, with negative numbers indicating less than other classes and positive numbers indicating more than other classes. A one sample t-test was used to compare the average (workload = -0.18; difficulty = 0.25; amount of material = 0.16) on each of the possible ratings against 0 (same as other courses). The averages were only not significantly different from 0, 15

16 indicating that the workload, difficulty, and amount of material was the same as in other classes. Further, while difficulty and amount of material might have trended towards being higher than average, workload (critical for an accelerated learning study) trended towards lower than average. PerformanceonMidtermExams The prior semester had three midterm exams and an optional final exam, while the current semester only had one midterm exam (basically the content of the prior midterm 1) and one required final exam (including content from prior midterms 2 and 3 plus new content not previously covered). The current semester performed at considerably higher levels on their midterm (M = 81%; SD = 15) than those who took the comporable first midterm exam in the prior semester (M = 69%; SD = 20), t(155)=4.24, p<.0001, Cohen s D=0.7. FinalExamPerformance Thepriorsemesterdidnothavearequiredfinalexam,sothefinalexamforthecurrent (fall2008)semesterwascreatedfromthe secondandthirdmidtermfromthespring semester.additionalquestionswerealso includedthatcoverednewmaterialthatwas notcoveredinthepriorsemsester.this materialincludedbeingabletoreflectand explainstrategiesusedtosolveaproof, constructingatruthtree,andtranslations usingpredicatelogic.thetwosectionswere analyzedseparately. Therewerenosignificantdifferencesbetween thespringsemester sperformance(m=75%, SD=26)andthefallsemester sperformance (M=72%,SD=16)onmaterialthatwascoveredbybothsemesters,t(150)= 0.72,p=.47.The fallsemesterdidfairlywellontheextraquestions(m=78%;sd=18). Summary Inacommunitycollegecontext,acceleratedlearningwasachievedusinghybrid instruction:someofthepreviouslycoveredcontentwaslearnedtogreaterlevelsand considerableadditional,moreadvancedcontentwasalsolearned.thisgainincontent learnedtookplaceoverthesamesemestertime,withoutcreatinganatypicallyhigh workloadcourseandwithoutanyriseinthecourseattritionrate. 16

17 Table 5. Background information for the Community College Acceleration Study Gender 34 male; 28 female Year 19 freshmen; 29 sophomores; 12 junior; 0 seniors; 1 5 th year or above Major GPA Current Course Load Reason for Taking Course Goal Grade Computer Experience 1 one course; 2 two courses; 3 three courses; 2 four courses; 39 five courses; 15 six courses 18 meets the university requirement; 17 thinks the subject is interesting; 14 major requirement; 6 covers skills I want to develop; 4 fits well into schedule; 1 expect the course to be challenging; 1 hear the teacher was good; 1 prerequisite 50 A; 9 B; 3 no goal grade 15 commented on a blog/wiki 13 used educational software 27 worked with spreadsheets 3 worked with statistical packages 4 proficient in computer programming 17

18 Accelerationforcommuterstudentswithoptionalrecitationsessions InmanyregionalcampusesofpublicinstitutionsintheUS,alargeproportionofthe studentspreferonlineinstructionbecauseitallowsthemtoworkand/orcareforfamily memberswhileattendingcollege.inthiscontext,itisdifficulttorequirehybridinstruction. Thisstudyexaminedwhethertherelativelyamountofhybridinstructionwouldallow studentstocovermorecontentwithinthesemester. Atthisuniversity,L&Phadbeenusedinonlineonlyformatforanumberofyears,although thecurrentinstructorwasnewtoteachingthiscourse.asinpriorsemestersofthel&p courseinthiscontext,studentswereallowedtocompletethecourseattheirownpace. Additionally,gradesfocusedoncontentcoverageratherthanthemoretypicaldegreeof mastery:themorethatstudentscompleted,thehigherthegradetheywouldbeassigned (assumingminimallevelsofproficiencyweremetoneachsectioninthefinalexam). Throughoutthesemester,optionalrecitationsessionswereofferedtohelpthestudents completetheworkinatimelyfashion.itwaspredictedthatifstudentstookadvantageof therecitationsessions,thentheywouldlikelycompletemorematerial. ContentLearningResults Foreachstudent,wecalculatedamountofmateriallearnedbystudentsasthenumberof chaptersforwhichtheyreceivedapassinggradeinthefinalexam.unlikewhatwas hypothesized,therewasnotasignificantrelationshipbetweennumberofrecitations attendedandamountofmaterialcompleted(r=.1,p>.4). Attendance Oneclearproblemwiththisstudy,however,isthatthemajorityofthestudentsdidnot attendanyoftherecitationsessions,andonlyacoupleofstudentswereregularattendees (seetable6).basedonthedistributions,cutoffsforagroupingvariableseemedtobe thosewhodidnotattendanysessions(n=23),thosewhoattendedonetothreesession(n =2),andthosewhoattendedfourormoresessions(N=11). 1 However,analysesofexam performanceinvolvingthesegroupswerenotsignificanteither(p>.6),althoughtherewas aslighttrendforthezerosessiongrouptodolesswellthantheothertwogroups. Increasesinfinalgradeswereofferedasincentivetoattendtherecitationsessions.If studentsattendedsixormoresessions,thentheywouldreceiveabumpintheirfinalgrade (i.e.,b wouldbeincreasedtoab).ifstudentsattendedtwelveormoresessions,thenthey wouldreceivetwobumpsintheirfinalgrade(i.e.,b wouldbeincreasedtoab+). Consideringthesecutoffs,fourgroupswerecreated:thosewhodidnotattendanysessions (N=23),thosewhoattendedatlessthansixsessions(N=4),thosewhoattendedatleast sixsessionsbutlessthan12(n=7),andthosewhoattended12ormoresessions(n=2). Again,therewerenosignificantdifferencesbetweentheseconditions(p>.6) 1 Total N for this analysis is less than the amount indicated in the table. There was some loss due to students not taking the final exam. 18

19 Table 6. Distribution of attendance at the recitation sessions. number of sessions numberof studentsin attendance cumulativenumberof studentsinattendance (increasing) cumulativenumberofstudents inattendance(decreasing) Summary Evenwithagradeincentiveforattendingrecitationsections,studentsinthisuniversity contextwereunlikelytodoso.thesmallamountofattendenceinhybridinstructionwas notclearlyassociatedwithincreasesinamountofmateriallearned.insufficentstudents completedthepre testforustoassesswhetherrecitationattendencewascorrelatedwith weakerperformance.however,thatremainsaclearpossibleexplanationforwhyrecitation attendencewasnotcorrelatedwithbettercourseperformance. 19

20 SameTime/MoreContentwithHybridInstructioninaUniversity Thecontextwiththehighestlikelihoodofteachingsymboliclogicisthenationaluniversity. Relativetothecommunitycollegecontext,theinstructorsarelikelytobemorefamiliar withsymboliclogiccontent,theirstudentsaremorelikelytobealreadybelearningsome ofthemoreadvancedcontentfoundinl&p,andtheinstructorshavelowerteacherloads butperhapslargerclasses.however,therearesomeopportunitiesforcoveringmore contentusingl&pthanwiththeexistingcourseasthesurveyofcoursesdidfindthatnot allofl&pcontentwastypicallycovered.further,inthiscontextstudents,studentswere previouslyattendingtraditionalsymboliclogicclasses,andthushybridinstructionis possible. Theparticularcontentthatwasnotpreviouslyfoundinthestudiedcontextinvolvesome aspectsofmetamathematicsandstrategicelementsofproofconstruction. Context&Participants ParticipantsincludedstudentsregisteredforanIntroductiontoSymbolicLogiccourse offeredata4 th tieruniversity.two thirdsofthesestudentsweresophmoresfrom computerscienceorengineeringbackgrounds,andthuswererelativelywellmatchedto learningthiskindofcontent(e.g.,halfthestudentshadpriorexperiencewithrigorous proofs). Studentswereofferedanopportunitytoparticipateinthisstudy.Outofthetotalof81(63 male)registeredstudents,31studentsvolunteered(22male).thevolunteerswere randomlyassignedtooneoftwoconditions:face to face(n=10)orhybrid(n=21).the 50non volunteerswerecompletedthesameworkastheface to faceparticipants. Thefirstanalysescomparedthebackgroundandpriorknowledgeofthevolunteers(i.e., collapsingtheface to faceandhybridconditions)tothenon volunteers.therewerevery fewdifferencesbetweenthetwoconditions.thevolunteersoutperformedthenonvolunteersonthepriorknowledgeportionofthepretest(volunteers:m=10.1,sd=2.1; non volunteers:m=8.9,sd=2.5;t(79)= 2.2,p=.03) especiallyonthestatements questions(volunteers:m=3.1,sd=1.1;non volunteers:m=2.6,sd=1.3;t(79)= 2.1,p=.04).Thevolunteersonlyhadslightlymorerigorousproofclassesthanthenon volunteers (volunteers:m=0.7,sd=1.1;non volunteers:m=0.3,sd=0.6;t(79)= 1.9,p=.07). Therewerenosignificantdifferencesforknowledgequestionsinvolvingarguments, equivalence,orproofs,gender,yearinschool,major,totalcoursescurrentlytaking,overall gradepointaverage,computerexperience,andhowmanypreviousformallogicand historyoflogiccoursestheytook. Becausethereweresofewdifferencesbetweenthevolunteersandnon volunteers,the non volunteersandface to facestudentswerecollapsedforalltheremaininganalyses. Therewerenotevenmarginallysignificantdifferencesinbackgroundandpriorknowledge betweentheresultingface to facestudentsandhybridstudentgroups. 20

21 TimeonTask Attheendofthesemester,studentscompletedasurveyaboutthecourseworkload.Six students surveyswerenotincludedintheanalysis:threecouldnotbeidentifiedfromthe initialclassroster,twodidnotprovidenames,andonedidnotcompletehalfthesurvey. Overall,therewerenodifferencesinthetotaltimeontaskbetweentheface to face students(m=14.1,sd=6.4)andthehybridstudents(m=11.1,sd=3.0),t(55)=1.6,p=.11.notsurprisingly,theface to facestudents(m=1.9,sd=0.8)spentmoretimein lecturethanthehybridstudents(m=0.5,sd=0.7),t(55)=5.5,p<.0001).theyalsospent moretimeontheirhomework(face to face:m=3.0,sd=1.3;hybrid:m=1.9,sd=0.5; t(55)=2.9,p=.01).however,thehybridstudentsspentmoretimeinrecitation(face toface:m=1.5,sd=0.8;hybrid:m=2.1,sd=1.0;t(54)= 2.2,p=.03). Studentswerealsoaskedfourquestionsabouttheirperceptionsofthecourse(i.e.,how interestingwasthecoursecomparedtoothercourses;howhelpfulwastherecitation comparedtoothercourses;howdifficultwasthematerialcoveredinthiscoursecompared toothercourses;howlikelyareyoutotakeafollow upcoursethatbuildsonthis materials).theyweretoratethequestionsonascaleof1(notatall)to5(very much/likely).therewerenodifferencesbetweenthethreeconditions. PerformanceonFinalExam Allstudentstookthesamefinalexam,whichcoveredthefollowingtopics:Symbolization intorl(multiple choice);translationfrompl(multiple choice);otherquestions (multiple choice);symbolization(short answer);translation(short answer);other Questions(short answer);derivationjustifications;derivationviolations;truthtable Method;Derivation;TreeMethod;Translation&Tree/Derivation.Eachofthesetopcis wereanalyzedseparately.threestudents examswerenotincludedintheanalyses becausetheycouldnotbeidentifiedoninitialtheclassroster. Twoversionsofthetestweredistributedwithineachgroup.Theonlydifferencebetween thetwoversionswastheorderinwhichthequestionswerepresented.therewasno significantdifferencesinperformancebetweenthetwoversions(version1:m=43.0,sd= 14.7;version2:M=47.5,SD=14.2;t(60)= 1.2,p>.2)andthusthetwoversionswere collapsedfortheremaininganalyses. Overall,theface to facestudents(m=47.6,sd=13.2)outperformedthehybridstudents (M=37.3,SD=16.6)onthefinalexam,t(60)=2.4,p=.02).Theydidbetterontheshortanswersymbolizationquestions(face to face:m=3.5,sd=1.9;hybrid:m=2.2,sd=2.0; t(60)=2.2,p=.03),thederivationviolationquestions(face to face:m=2.5,sd=0.9; hybrid:m=1.4,sd=1.4;t(60)=3.2,p=.001),thetruthtablemethodquestions(face toface:m=3.8,sd=1.6;hybrid:m=2.7,sd=2.3;t(60)=2.0,p=.05),andthederivation questions(face to face:m=15.6,sd=7.5;hybrid:m=10.4,sd=8.1;t(60)=2.2,p=.03). Thehybridstudentsdidslightlybetteronthemultiple choicesymbolizationquestions (face to face:m=2.3,sd=1.7;hybrid:m=3.2,sd=1.8;t(60)= 1.7,p=.10). 21

22 Summary Inthiscontext,therewasnoapparenttestperformancebenefitforhybridinstructionover traditionalface to faceinstruction.onemightarguethatbetterperformancewouldhave resultedfromhybridinstructionhadtheinstructorandtahadmorefamiliaritywiththe L&Psystem.Goodhybridinstructioninvolvesinstructorslearninghowtomakesenseof feedbackthesystemprovidesonstudentlearningandthenprovidingtargettedinstruction. ItispossiblethattheinstructorandTAcontinuedtouseoldinstructionalstylesinthenew context.however,theresultsdosuggestthatgainsmaynotbefoundimmediatelyonthe switchtothiskindofhybridinstructioninthiskindofcontext(contentknowledgeable instructorsworkingwithrelativelystrongstudents). 22

23 Acknowledgements WewouldliketoacknowledgethetirelesscontributionsofWilfriedSieg.Thisevaluation effortwasamajorundertakingthatcouldnothavebeendonewithoutin depthinputfrom himateverystepoftheway.inaddition,dawnmcglauglin,davinlafon,leslieburkholder, RebeccaThall,DaveBeisecker,andVictoriaRogersprovidedconsiderablehelpalongthe way. 23

24 References Carnegie Foundation for the Advancement of Teaching ( US & World News Reports America s Best Colleges ( Wikipedia ( es) 24