Transcription

1 1 M.P,LeRouxB;ForewordbyP.Suppes: Excerptfrom significanceteststobayesianinference",bern, RouanetH,BernardJ.M,LecoutreB,Lecoutre PeterLang. "Newwaysinstatisticalmethodology:from

2 Chapter4 Introductionto HenryRouanetandMarie-ClaudeBert CombinatorialInference justasonecantreatofelectricitywithoututteringthewordfrog. OnecouldtreatofProbabilitywithoututteringthewordChance, Introduction PaulValery earlyeighties:seerouanet,bernard,lecoutre(1986)androuanet, inferencethatourmath&psygrouphasbeendevelopingsincethe ThischapterisanintroductiontoCombinatorialInference,orSettheoreticInference(SeeSection4.3),analternativetofrequentist Bernard,LeRoux(1990).Itsmotivationistoprovideresearchers torialprocedures,thedissociationmadeinearlierchaptersbetween withaframeworkthatcanbeusedwhenthe\validityassumptions" ofthecommonproceduresarenotmet.inthemakingofcombina- algorithmandstatisticalframeworkisputtouse.roughlyspeaking,thealgorithmsofcombinatorialprocedurescoincidewiththose

3 98 ofconventionaltests,whiletherandomframeworkisdiscarded.as aresult,indataanalysis,itwillbepossibletokeepmanyfamiliaralgorithms,whiletheconclusionsofcombinatorialinferenceare HenryRouanetandMarie-ClaudeBert geneitytests(section4.2).thenwewilloutlinethemakingofcom- binatorialinferenceanddiscussrelatedviewpoints(section4.3). formalizedinanonprobabilisticway. WewillrstpresentTypicalitytests(Section4.1),andHomo- statedintermsofnewconcepts,suchastypicalityandhomogeneity, terizethetypicalityproblem(1.2),and,intheelementarycontextof 4.1 Inthisrstsection,wepresenttypicalitysituations(1.1)andcharac- TypicalityTests inference(1.5).last,theextensiontosamplingfromadistribution thehypergeometrictypicalitytestforarelativefrequency(1.4).we willproceedbymakingsomegeneralcommentsaboutcombinatorial nitesampling,wepresentthetypicalitytestforthemean(1.3)and 4.1.1TypicalitySituations willbeoutlined(1.6). pointed.canthecommitteebedeclaredtobeatypicaloftheclub withrespecttothemeanage,orthesexratio,etc.? Considerthefollowingsituations: Committee.Amongthemembersofaclub,acommitteeisap- amongthefrench,thepercentageofred-hairedpeopleisaround 10%,canitbeinferredthatthegroupofschoolboysisatypicalof of20schoolboysgeto,7ofwhomarered-haired.assumingthat Schoolboys.AttheLouvrestationoftheParismetro,agroup Frenchschoolboys? resort;7ofthedayswererainydays.anadvertisementclaimsthat thepercentageofrainydaysinaugustis10%.canthetouristinfer thatthevacationperiodwasatypicaloftheadvertisedclimate? Vacation.Atouristhasspentaperiodof20daysinAugustina

4 chologisthasfoundthatforacertaintaskthemeangradeofher CombinatorialInference Giftedchildren.Inafollow-upstudyof5giftedchildren,apsy- 99 toclaimthathergroupofgiftedchildrenisontheaveragesuperior tothereferencechildren? groupis30,withansdof6,whereasforareferencepopulationof childrenofthesameagethemeanisknowntobe25.issheentitled 4.1.2TheTypicalityProblem Theprecedingsituationsexemplifywhatwecalltypicalitysituations. raised,intuitivelyformulatedasfollows:\canthegroupofobservationsbeassimilatedtothereferencepopulation,orisitatypicalof alsoaknownreferencepopulation.somestatisticisconsidered,such asthemeanofavariableofinterest.thenthetypicalityproblemis Insuchasituationthereisagivengroupofobservations,andthereis thegroupofobservations(withrespecttothepopulation,according tosomestatisticofinterest)?" it?";ormorespecically:\howcanatypicalitylevelbeassessedfor Yettheconventionalstatisticalframeworkisnotvalid,sincenorandomnessisassumedinthedatageneratingprocess.IntheCommitteeexample,thegroupunderinvestigationisasubset butnota Intypicalitysituations,itistemptingtodoasignicancetest. ofthereferencepopulation. randomsubset,asarule ofthereferencepopulation;inthegifted anissuethatisperfectlydistinctfromrandomness.asanexample, Childrenexample,thegroupunderinvestigationisnotevenasubset supposeatrialjuryof9membersthathappenstoincludenotasingle woman;evenifthejuryhasbeenlawfullyconstitutedbyrandom Evenforarandomsample,thetypicalityissuemayberaisedas ideawillbetocomparethegroupofobservationstothesamplesof isnottypicalwithrespecttothesex-ratio. sorting,itscompetencemightbequestionedonthegroundsthatit thereferencepopulation,wheresamplesaresimplydenedassubsets ofthepopulation. Inordertooerasolutiontothetypicalityproblem,thebasic

5 FiniteSampling:TypicalityTestfortheMean HenryRouanetandMarie-ClaudeBert toaniteset.letndenotethesizeofthegroupofobservations,and Inthecontextofnitesampling,thetermpopulationwillalwaysrefer Nthesizeofthereferencepopulation.Incombinatorialinference, asampleofthepopulationwillbedenedasann-elementsubsetof thereferencepopulation,andthesetxofall Nnn elementsubsets denesthesamplespace.wewillnowdescribethetypicalitytest denotetheobservedmeanofthegroupofobservations,themean associatesitsmeanm(x),denesthemeanasastatistic.letmobs inthecaseofanumericalvariable,takingthemeanasastatistic ofthereferencepopulation,andsuppose toxourideas that ofinterest.themappingmonxthat,witheachsamplex2x, mobs).letp=p(mmobs)(observedupperlevel) than(orequalto)mobs,thatis,whichsatisfytheproperty(m mobs>.thenweconsiderthesampleswhosemeansaregreater deningtheleveloftypicalityforthemean,withrespecttothereferencepopulation.thesmallerthevalueofp,thelowerthetypicality. observationswillbedeclaredtobeatypicalofthereferencepopulation,withrespecttothemean,upwiseatlevel(one-sided).when mobs<,theobservedlowerlevelwillbesimilarlyconsidered,and Foranybetween0and1/2,ifP(Mmobs),thegroupof betheproportionofthosesamples.thisproportionwillbetakenas thetypicalityleveldenedaccordingly.agroupofobservationswill bedeclaredatypicalifitisatypicalwhetherupwiseordownwise. 67;70;70).Thereare 93=84samples(ofsizen=3),whosemeans Example.CommitteewithN=9,n=3(Rouanetetal,1990,p. generatethesamplingdistributionofthestatisticm:seefigure ).Letacommitteeof3memberswithmeanagemobs=69tobe comparedtotheclubof9memberswithages(58;61;64;64;64;67; Chapter2,sincewehave2=84<0:025but2=84>0:005(signicant property(m69);hencep=2=84=0:024.thusforany2=84, theresultissignicant.takingtheconventionalgriddescribedin Byinspection,itisfoundthatoutofthe84samples,2satisfythe

6 CombinatorialInference Figure4.1:SamplingdistributionofMean results?),weconcludethatthecommittee indeedanygroupof observationswithmeanmobs=69 isatypicaloftheclubwith havep=5=84.since5=84>0:025(nonsignicantresult:ns),we Asanotherexample,consideragroupofobservationswithmean mobs=68,togetherwiththesamereferencepopulation;thenwe respecttotheage,ontheupperside,atthelevel.025(one-sided). cannotconcludethatthegroupofobservationsisatypical,atthe conventionaltwo-sided.05level. Fundamentaltypicalityproperty.Theconstructionofthetypicalitytestextendstoanynumerical(orsimplyordinal)statistic, takingasatypicalityindextheproportionofsamplesthataremore extremethan(orasextremeas)thedata,withrespecttothisstatistic.thetypicalitytestcanbeappliedtoeverysampleofthereferencepopulation,viewedasaparticulargroupofobservations.then thattheproportionofthesesamplesisatmost \atmost"rather than\equalto",owingtothediscretenessofthesamplingdistribution. foranyspecied,thetestwillseparateoutthosesampleswhichare atypicalatthe-level.thefundamentaltypicalitypropertystates

7 HypergeometricTypicalityTestforaRelative Frequency HenryRouanetandMarie-ClaudeBert Withcategorizeddata,combinatorialinferenceleadstoexplicitformulas.Toillustrate,wewillpresentthetypicalitytestforarelative frequency,inthecontextofnitesampling. acharacterofinterest,hencetheobservedrelativefrequencyofthis characterfobs=a=n.supposethatinthereferencepopulationof sizen,thecorrespondingrelativefrequencyis'0=a=n.the Supposethatinagroupofnobservations,aobservationspossess fobstothereferencefrequency'0forapopulationofsizen.let F(statistic)bethemappingonthesamplespacethat,witheach combinatorialtesthereamountstocomparingtheobservedfrequency samplex,associatesitsrelativefrequencyf(x).thenumberof Thealgorithmofthetypicalitytestforarelativefrequencyistheone samplesforwhichtheproperty(f=a=n)holdsis Aa N A hencetheobservedupperlevelpisgivenby p=p(fa=n)=pna0=a Aa0 N A n a0= Nn n a; usedintheclassicalhypergeometrictestforfrequencies;recasting thelattertestinacombinatorialframeworkamountstoretainingits ofwhomarewomen;isthecommitteeatypicaloftheclubwith ofaclub,6arewomen;acommitteeof5membersisappointed,4 Example.CommitteewithN=20,n=5.Amongthe20members algorithmwhilediscardingtheprobabilisticinterpretation. respecttothesexratio?wehaveheren=5,fobs=4=5(=:80), '0=:30,N=20.Thereare 205=15504samples,andamong them =216forwhichF4=5.Hencep= 216=15504=:014.Takingconventionallevels,sincepliesbetween thecommittee indeedanygroupof5observationswithobserved.025and.005,thefrequencyfobs=4=5=:30issignicantlyhigher (inacombinatorialsense)thanthereferencefrequency'0=:30,at the.025level(one-sided).intypicalityterms,itisconcludedthat over-representationofwomeninthecommitteeiscalledfor. frequencyfobs=4=5 isatypicaloftheclub,withrespecttothe sexratio,attheone-sidedlevel.025(s?).aninterpretationofthe

8 CombinatorialInference quencies,includingthederivationofcombinatorialcondencelimits, Amorethoroughpresentationofcombinatorialinferenceforfre- 103 canbefoundinrouanetetal.(1986)androuanetetal.(1990). Typicalitytestanddescriptivestatistics.Theconceptoftypicalitythatwehavedenedhereisinharmonywiththatof\typical value"ofadistribution:mean,median,etc.moreimportantly,assessingthetypicalitylevelofagroupofobservationsappearsasthe directextension,forn1,ofthenaturalstatisticalprocedureof assessingtheperformanceofanindividualinagiventaskbymeans oftheproportionofscoresexceedingthescoreofthatindividualin asn>1,itisnotadescriptivestatistic inthetechnicalsense areferencepopulation.foragroupofobservations,however,the typicalityleveldependsonthesizeofthegroup,therefore,assoon 4.1.5RemarksonCombinatorialInference hand,thelinkbetweencombinatorialnotionsandthoseoffrequentistinferenceisapparent,whenthedatasetisasamplefromthe Fromtypicalityteststofrequentistinference.Ontheother thustoberegardedastherststageofinductivestatistics. thatwasspeciedearlierinchapter1.combinatorialinferenceis frequentistnitesamplingtheoryexceptthatnoprobabilitiesare encearejustthe\unorderedsampleswithoutreplacement"ofthe population.thesamplesfromapopulationincombinatorialinfer- attachedtothem;foreachpropertyofinterest,whatisassessedinsteadissimplytheproportionofsamplesforwhichthatproperty samples(subsets)haveequalprobabilitiesofbeingextracted.then holds.theconventionalfrequentistframeworkwillbe\recovered" theconversionpropertyholds:undertheassumptionofrandomsam- is inthenitetheoryconsideredhere wenowsupposethatall ifweintroducetheadditionalassumptionofrandomsampling,that pling,theproportionofsamples(subsets)satisfyingacertainprop- ertybecomestheprobabilitythatasamplewillsatisfythatproperty. Thus,fortheprecedingnumericalexample,undertherandomsamplingassumption,theproportionofsamplesforwhichtheproperty

9 comestheprobabilitythatarandomlyextractedsamplesatisesthis 104 (M69)holds aproportionthatwewrotep(m69) be- HenryRouanetandMarie-ClaudeBert sameproperty(m69) aprobabilitythatwemayalsowrite P(M69),nowreadingP\probability"insteadof\proportion". Theconversionpropertythustransformscombinatorialprocedures intofrequentistones.clearly,itnotjust\amatterofsemantics"to speakoftheproportionofsamplessatisfyingacertainproperty,orof theprobabilitythatasamplesatisesthisproperty.therststatementisvalidwithoutrestriction,whereasthesecondonedemands theadditionalassumptionthatallsamplesareequallyprobable. canceformulationscanberetained,whilequaliedas\combinato- rial",sincenoprobabilisticinterpretationisintended. directextensionofdescriptivestatistics,technicallyitinvolvesalgo- rststageofinductivedataanalysis. rithmsoffrequentistinference.combinatorialinferenceisthusthe Summarizing:Whileconceptually,combinatorialinferenceisa Inference.Wheneveracombinatorialproceduretechnicallycoincides withthealgorithmofafrequentistprocedure,thefamiliarsigni- TheforegoingdiscussionappliestothewholeofCombinatorial Thetypicalitytestcanbeextendedtosamplespacesdenedby 4.1.6SamplingfromaDistribution samplingfromadistribution(ratherthanfromapopulation),by generalizingthenotionofasampleandmakinguseofmathematical convergencetheorems. comparisonofanobservedrelativefrequencyf=a=ntoareference Thebinomialtypicalitytest.Asarstexample,letustakethe previouslywrittenhypergeometricexpressionoftheupperlevelconvergestothebinomialexpressiosiderasequenceofpopulations,suchthatthepopulationfrequency '=A=Napproaches'0whenNtendstowardinnity.Thenthe value'0,whennopopulationsizenisspecied.wemaythencon- p=p(fa=n)=pna0=a'0a0(1 '0)n a0

10 Eventhoughthenumberofsamplesisnotniteinthelimit,thisbinomialexpressionmaybetakenasdeningaproportionofsamples, CombinatorialInference 105 whennopopulationsizeisspecied.intuitively,theproceduremay bethoughtofasaninferenceforanarbitrarilylargepopulation.a WhenA=Napproaches'0,weagainobtainthebinomialexpression. proportionofthosethatsatisfy(fa=n)ispna0=a(an)a0(1 AN)n a0. mentinapopulation.thenumberofsuchsamplesisnnandthe relatedapproachconsistsinconsideringorderedsampleswithreplace- Samplingfromadistribution.Infrequentistinference,samples highlysignicantresult(s??),leadingtoaconclusionofatypicality. 20,f=7=20and'0=:10.WethenndP(F7=20)=:0001,a Examples.Schoolboys,Vacation.Forbothexampleswehaven= areconsideredeventhoughthereisnorelevant(nite)population, anddenedintermsofindependentidenticallydistributed(i.i.d.) randomvariables.incombinatorialinference,asamplefromadistributionwillbedenedinmeasure-theoreticterms,namelyasan fromdistributions e.g.samplesformanormaldistribution elementofaproduct-measurespace(un;n),whereuisameasurablespaceandisapositivemeasureoftotalmass1overu.even thepropertyofthesamplespace.asaconsequence,theintuitive thoughthenumberofsamplesmaybeinnite,theproportionofthose thatsatisfyagivenpropertyiswell-denedbythe(n)-measureof formulationsofthenitetheorymaybecarriedover(rouanetetal, 1990,p.103). Student'scombinatorialt-test.Asanexample,letusrecastStudent'st-test comparinganobservedmeantoareferencemean value incombinatorialterms.takingasasamplespacethesamplesofsizenfromanormaldistributionofmean,theclassical Studentproperty,incombinatorialterms,reads:theproportionof samplesforwhichtheratiom forany,wehave Nowconsideragroupofnnumericalobservations,andareference P(M S=pn>t)= S=pnexceedstisequalto;thatis, meanvalue0.lett=m 0 S=pn.ThankstotheStudentproperty,we

11 106 mayassessthetypicalityofthisgroup,forthemean,withrespectto anynormalreferencedistributionofmean0.thusforthestudent HenryRouanetandMarie-ClaudeBert 4:06.Hencetheone-sidedobservedlevelisp=0:0014(S**).In combinatorialinference,thep-valueisinterpretedasthetypicality and0=0(referencemean),andtheobservedt-ratioistobs= data(cf.chapter2),wehaven=10,mobs=1:58(observedmean), levelofthegroupofobservations,forthemean,withrespecttoa normaldistributionofmean0.theconclusionofthecombinatorial testisthatthegroupisatypical,forthemean,onthepositiveside, combinatorialone,itisareference.inthecombinatorialtest,there ofanormaldistributionofmean0,atthe.005level(one-sided). isnovalidityissue,eventhoughthereisarelevanceisssue,since Inthefrequentisttest,normalityisanassumption,whereasinthe Thesemanticdierencewiththefrequentistt-testisapparent. oftenbebetterjustiedthanthefrequentistone,becausethenormaldistributionisaprivilegedreferenceinmanysituations.letus n=9,mobs=30,s=6,0=25(meanscoreofreferencechildren), andhencetobs=2:5.theresultissignicantatthe0.025level isbasedonnormalizedscores awidespreadpsychometrictechnique thepsychologistisentitledtoclaimthathergroupofgifted (one-sided)(s?).insofarasthedistributionforreferencechildren childrenisontheaveragesuperiortothereferencechildren.this (1991,p.A20) forcommonproceduressuchasthet-test. exampleillustrateshowthecombinatorialframeworkmayoerreal, interesting,andplausiblesettings toparaphrasefreedmanetal. thechoiceofaparticulardistributionasareferencemaybemore takeforinstancethegiftedchildrenexampleofsection1.1,with orlessappropriate.inthisconnection,thecombinatorialt-testwill Inthissection,wewilloutlinehomogeneitytestsalonganapproach 4.2 similartotheoneusedfortypicality.werstpresenthomogeneity HomogeneityTests situationsandstatethehomogeneityproblem(2.1).thenwewill

12 presentthehomogeneitytestsfortwobasicstructures(2.2),and relatedcombinatorialtests(2.3).thenwewillexposethepassage CombinatorialInference HomogeneitySituations fromcombinatorialinferencetofrequentistinference(2.4)andto thebayesianframework(2.5). severalteachinggroups.attheendofthecourse,anexamisgiven Considerthefollowingsituations. totheparticipants,revealingsubstantialdierencesamongthemean scoresofthegroups.canitbesaidthatthegroupsareheterogeneous Summerschool.Participantsinasummerschoolareallocatedto withrespecttotheirmeanscores? icationinthewagesystemisintroducedinaworkshopofafactory. Forthe12workersintheworkshop(\subjects"s1throughs12), theoutputs(numberofitemsperhour)arethefollowing(aafter Wagemodication(adaptedfromFaverge,1956,p.88).Amod- modication,bbefore): Themeanoftheindividualoutputdierences(\after" \before") s1:a220,b203s2:a226,b222s3:a254,b246s4:a246,b221 is8.92,andthe(corrected)s.d.9.59.thusdescriptively,thereisa s5:a296,b287s6:a222,b224s7:a293,b275s8:a247,b246 substantialmeanincrease(0.67timesthes.d.).arethetwogroups s9:a240,b246s10:a269,b258s11:a236,b216s12:a199,b197 ofscores(\before"and\after")heterogeneous? Thehomogeneityproblem.Theprecedingsituationsexemplify bemerged,oraretheyheterogeneous?";\canalevelofhomogeneitybeassessed?"asinthetypicalityproblem,itistemptingto observations,andsomestatisticofinterestisconsidered.thehomogeneityproblemisraised,intuitivelyformulatedas\canthegroups whatwecallhomogeneitysituations.thereareseveralgroupsof thatwereoriginallydevisedwithinafrequentistframeworkandjust mogeneitytests,onemay,likefortypicality,takesignicancetests dosomeconventionalsignicancetest.yetagain,norandomnessis assumedinthedatageneratingprocess.togetcombinatorialho-

13 108 retaintheiralgorithms.forourpurposehere,wewilltaketheclassicalpermutationtests,orfisher-pitmantests,initiatedbyfisher HenryRouanetandMarie-ClaudeBert signtest,ranktests,fisher'sexacttestfora22table,etc.are Permutationtests.Thefamiliarnonparametrictests,suchasthe (1987,p.17-21). andbypitman(1937);forabriefhistoricalaccount,seeedgington variantsofpermutationtests,forwhichexplicitformulascanbe derivedandtablescanbeconstructed.thishasrenderedthose testsapplicablebeforethecomputerera.bycontrast,forthebasic Fisher-Pitmantests,aconsiderableamountofcomputationisrequired,evenformodestsamplesizes.Thisformidablecomputatiorialcomputationscanbecarriedout.Forintermediatesizes,Monte Carloprocedures,thatis,computersamplingfrompermutationdistributions,canbeused.Forlargedatasets,approximatemethods involvingclassicaldistributionsareoftenavailable. elusivelytreatedintextbooks.readerswhoarenottooclearabout permutationtestswillnditadvantageoustogetacquaintedwith mutationtestsinthefrequentistframeworkisintricateandoften Leavingasidethecomputationalobstacle,thejusticationofper- obstacle,whichhaslonghinderedthefulluseofpermutationtests, isbeingovercomenowadays.forsmalldatasets,exactcombinato- themthroughthecombinatorialframework,whoselogicisstraightforwardandwilllightentheslipperypathsleadingtofrequentist interpretations HomogeneityTestsforTwoBasicStructures groups.thisexchangeabilityprincipleleadsustoconsiderthebaselinedatasetobtainedbydisregardingthesubdivisionintogroups, belongingtoagroupmighthavebelongedaswelltoanyoneofthe andthentoconstructallpossibledatasetsobtainedbyreallocating theobservationsofthisbaselinedatasettothegroupsinallpossible Tosaythatseveralgroupsarehomogeneousamountstosayingthat thesubdivisionintogroupsmaybeignored,thatis,anyobservation ways.technically,thisamountstoapplyingapermutationgroupto

14 theobserveddataset,thusgeneratingasetofpossibledatasets allofthesamestructureastheobservedone orprotocolspace, CombinatorialInference 109 againstwhichtheobserveddataset(observedprotocol)willbesituated.theremainderofthetestprocedurewillbethesameasfor thestatisticofinterestiscalculated,andtheproportionofprotocols forwhichthisstatisticismoreextremethan(orasextremeas)the typicalitybyhomogeneity.foreachprotocolintheprotocolspace, typicalitytests,replacing\samplespace"by\protocolspace"and pendsonthedesignstructure.hereafterwedescribehomogeneity observedvaluedenesthelevelofhomogeneityofthegroups. tests,rstforthestructureoftwoindependentgroups(summer School),thenforthatoftwomatchedgroups(Wagemodication). Thepermutationgroupusedtogeneratetheprotocolspacede- DisregardingthisGroupfactor,thederivedbaselinedatasetisthe 1990,p.116).Considerseveralindependentgroupsofobservations, thatisthedesignwheresubjectsarenestedwithinagroupfactor. Independentgroupdesign(Nestingstructure)(Rouanetetal., poolofthegroups.hereafterwedescribeindetailthecaseoftwo groups,g1andg2,ofsizesn1andn2;thederivedbaselinedatasetis byreallocatingn1ofthepoolofn1+n2observationstog1andthe othern2tog2;itthuscomprises n1+n2 thepoolofthen1+n2observations.theprotocolspaceisgenerated TakingthedierenceofmeansDasthestatisticofinterest,the thefollowingnumericaldataset:g1:3;8;10;10;g2:1;1;2;5;5. Forinstance,supposetherearetwogroupsofsizes4and5,with n1protocols. observedvalueofthisstatisticisdobs=7:75 2:8=4:95.The poolofg1andg2isthegroupof9observations(writteninincreasingvalues):g1g2:1;1;2;3;5;5;8;10;10.applyingthepermutation group,9!=5!4!=126protocolsareconstructed.thusstartingwith g1:3;8;10;10;g2:1;1;2;5;5(observeddataset),andpermutingthe rstobservationsofg1andg2,wegettheprotocol:g1:1;8;10;10; g2:3;1;2;5;6,etc.foreachprotocolthevalueofdiscalculated: isfoundthatoutofthe126protocols,thereare3forwhichthedif- thus4.95(forthedataset),then4.05,etc.then,byinspection,it

15 110 ferenceofmeansisgreaterthanorequaltotheobserveddierence; hence:p(ddobs)=3=126=:024.since3=126liesbetween.025 HenryRouanetandMarie-ClaudeBert and.005,itisconcludedthatthetwogroupsareheterogeneous equivalenttoatypicalitytest,takingthepoolofthetwogroups g1beinghigherthang2 atlevel.025(one-sided)(s?). groupsasa\sample." (baselinedataset)asa\referencepopulation"andoneofthetwo Thehomogeneitytestfortwoindependentgroupsisseentobe thetwogroupsare\separated,"inthesensethatallobservationsof aremoreextremethanallotherprotocols.foranextremaldataset, onegroupexceedallobservationsoftheother.thenthehomogeneity Ofspecialinterestareextremaldatasets,thatis,datasetsthat levelissimply1/ n1+n2 sizes,itisreadilyseenthatforn1=n23,twoseparatedgroups forn1=n2=4,theyareheterogeneousattheone-sidedlevel.025 cannotbesaidtobeheterogeneous(atthetwo-sidedlevel.05);that n1.takingforsimplicitytwogroupsofequal Matched-groupdesign(crossingstructure)(Rouanetetal.,1990, (S?);andthatforn1=n25,theyareheterogeneousattheonesidedlevel.005(S??). unitisnestedin indeedisconfoundedwith thecrossingof p.121).nowconsiderthestdesign,wherensubjectsarecrossed withafactort(\treatments",or\trials",etc.).eachexperimental factorssandt.therefore,disregardingfactort,thederivedbaselinedatasetischaracterizedbythesolestructureofthenestingotionswithineachpairinallpossibleways.theprotocolspacethus unitswithinfactors(restrictedexchangeability).hereafterwedeal thenthegroupofpermutationsisdenedbypermutingtheobserva- comprises2nprotocols. withthecaseofatwo-levelfactort,i.e.thematched-pairdesign; 4096protocols.LetDdenotethemeanoftheindividualoutput dierences\after before"(statisticofinterest).fortheobserved datasetwehavedobs=8:92.thebaselinedatasetisthesetof12 Forinstance,fortheWagemodicationdata,thereare212= unorderedpairs(writteninincreasingvalueorder):

16 CombinatorialInference s5:287,296s6:222,224s7:275,293s8:246,247 s1:203,220s2:222,226s3:246,254s4:221, Applyingthepermutationgroup,startingwiththeobserveddataset, weget,bypermutingthetwoobservationsofsubjects1(hereafter writteninboldfacecharacters): s9:240,246s10:258,269s11:216,236s12:197,199 Then,amongthe4096protocols,thenumberforwhichDisgreater thanorequalto8.92iseasilyfound usingacomputerprogam suchastheinferprogramdescribedinrouanetetal.(1990) to s1:a220,b203s2:a226,b222s3:a254,b246etc. be20,hencetheproportionp(ddobs)=0:0049(one-sided).at the.005level(one-sided),itisconcludedthatthematchedpairsare homogeneitylevelissimply1/2n.itisreadilyseenthatforn5, forwhichallindividualdierenceshavethesamesign;thenthe heterogeneous(s??),\after"beinghigherthan\before". thematchedpairsofanextremaldatasetcannotbesaidtobe Hereagain,ofspecialinterestareextremaldatasets,here,those theyareheterogeneousattheone-sidedlevel.025(s?);andthatfor n8,theyareheterogeneousattheone-sidedlevel.005(s??). heterogeneous(atthetwo-sidedlevel.05);thatforn=6andn=7, Studentdata,forwhich9dierencesarestrictlypositive,andone isnull,hencep(ddobs)=1/29=:0020.theconclusionof heterogeneityisattainedattheone-sidedlevel.005(s??).itmaybe Anexampleofanextremaldatasetisprovidedbytheclassical noticedthatthehomogeneitylevel.0020diersfromthevalue.0014 foundforthetypicalitylevelwithrespecttoanormaldistribution, 4.2.3RelatedCombinatorialTests obtainedbystudent'st-test.suchadiscrepancyisnotsurprising, sincethetwotestsanswerdierentquestions. Structureddata.Theapproachofhomogeneitytestsextendsto varioussortsofstructureddatacommonlyencounteredinplanned experimentationorobservation.inordertoinvestigateafactorof interest,thegeneralprincipleremainsthesame:constructthebaselinedatasetbyremovingthisfactorfromthestructure,thengener-

17 112 atethespaceofallprotocolssharingtheoriginalstructure,bymeans ofapermutationgroupassociatedwiththatstructure. HenryRouanetandMarie-ClaudeBert alsoappliestoaproblemakintohomogeneity,namelytheindependenceproblem(rouanetetal.,1990,p ),inconnectionwith Combinatorialindependencetest.Thecombinatorialapproach ofthelargestdepartmentsoftheu.c.atberkeley,therewere191 thebivariatestructureinobservations. menand393womenwhoappliedforadmissiontograduateschool; (fromfreedman&lane,1983).inthe academicyear,atone Asanexample,letusconsiderthefollowingSexbiassituation 54menand94womenwereadmitted,henceanappreciabledierence inpercentages(28%formenvs24%forwomen).canitbesuspected thattherewasasexbiasintheuniversity'sadmissionpolicy?in ture,consistshereofthetwoderivedsetsof584observationsper- tainingtoeachoneoftheseparateattributessexandadmission.in thecombinatorialindependencetest,theprotocolspacewillconsist Thebaselinedataset,obtainedbyremovingthebivariatestruc- SexandAdmissionbesaidtobeindependentorassociated?" termsofindependence,theproblemreads:\canthetwoattributes ofallpossiblematchingsbetweenthosetwosets.fortwodichotomousattributes,thealgorithmoftheindependencetestamountsto Fisher'sclassicalexacttest,andinturn,whenthenumberofobser- valueofthe2statistic,forthecorresponding22table,isfound isnotlowenoughtobedeclaredsignicant(atconventionallevels). tobe2obs=1:29,hencep(2>2obs)=0:26.theobservedlevel vationsislarge,tothefamiliar2-test.inthepresentexample,the thatthereisanassociationbetweensexandadmission.theu.c. Incombinatorialterms,theconclusionisthatitcannotbeinferred 4.2.4FromCombinatorialtoFrequentistInference atberkeleycannotbechargedwithsexbias. Theprecedingdiscussionreinforcestheviewofcombinatorialinferenceastherststageofinductivedataanalysis.Insomesituations, theconclusionsreachedthroughcombinatorialinferencemaybefelt

18 tobesucient.oralternatively,itmaybewishedtoprolongthem CombinatorialInference byprobabilisticconclusions.takinghomogeneitysituationsonce 113 again,wearegoingtodiscusshow,startingwithacombinatorial conclusion,frequentisttestscanbeconstructed. esisexpressing,inintuitiveterms,thatthefactorofinterest\hasno eect."then,inordertomakeastatementaboutthishypothesis, wewilltrytobuild aswedidfortypicalitytests afrequentist Withthenotionofhomogeneitywemayassociateanullhypoth- frameworkentailingaconversionproperty,thatis,forthatmatter, fortypicalitytests.tobeginwith,morethanonesingleframework transformingproportionsofprotocolsinto(frequentist)probabilities. Forhomogeneitytests,thingsarenotasstraightforwardastheyare maybedevised.belowwesketchtwoframeworks bothclassical forcomparingtwoindependentgroups,thatsharethesame algorithmbutrestondierentassumptions,andleadtodierent interpretationsofthenotionof\noeect". samplefromsomeunknowncontinuousparentdistribution the randomsamplingmodeloftheconventionalfrequentistkindisassumedforeachgroup,thatis,eachgroupisassumedtobearandom Randomsamplingandconditionaltest.Inthisframework,a continuityassumptionbeingmadetodisposeoftheproblemofties. tical.underthenullhypothesis,thepoolofthetwosamples our baselinedatasetdenedinsection2.2 canberegardedasasinglesample(ofsizen1+n2)fromthecommonparentdistribution. Thenullhypothesisstatesthatthetwoparentdistributionsareiden- Therefore,conditionallytothebaselinedataset,all n1+n2 generatedbypermutationareequallyprobable. clusionofheterogeneity.undertherandomsamplingmodel(con- ditionaltest),thenullhypothesistestedisthatthetwogroupsare Thus,fortheSummerSchooldata,wehaveacombinatorialcon- n1protocols hypothesisisnotcompatiblewiththedata(atlevel.025,one-sided, torialconclusionbecomesthefrequentistconclusionthatthisnull samplesfromtwoidenticalparentdistributions,andthecombina- thatis:s?).

19 114 situation aswellasforindependencesituations:fisher'sexact Formally,aconditionaltestcanbedevisedforanyhomogeneity HenryRouanetandMarie-ClaudeBert doesnotreallypertaintosomeconjecturalparentpopulations,but thermore,inmanyhomogeneitysituations,thequestionofinterest testfora22tableisclassicallyjustiedasaconditionaltest. rathertotheexperimentalunitsathand.thus,forthesummer Randomsampling,however,maynotbearealisticassumption.Fur- performanceoftheparticipants.similarly,inthesexbiasexample(section2.3),theindependencequestionisraisedaboutthe584 Factor specically,thesourcesofvariationlinkedwiththedivisionintogroups:dierentteachers,etc. hashadaneectonthe School,therealquestionistoinvestigatewhetherornottheGroup Thereisabroadrangeofsituationswheretherandomsampling lationfromwhichthesestudentswouldbesupposedtobeextracted. studentsunderconsideration,ratherthantosomeconjecturalpopu- takenupintherandomizationmodel,inwhichnounderlyingparent Randomizationtests.Theconcernjustmentionedisundoubtly assumptioneitherisunrealisticorinducesthewrongquestion. notdependonwhichconditionisappliedtothatunit.underthis distributionisassumed,andtheinferencesoughtonlypertainsto theunitsthatappearintheexperiment.thenullhypothesisnow statesthatforeachunit,thetwoobservationsthatcanbemadedo theyhadbeenassignedtotheothergroupratherthantothegroup unknownscoresthattheparticipants(units)wouldhaveobtainedif stance,inthesummerschoolexample,theparametersarenowthe nullhypothesis,all n1+n2 n1protocolsareagainequiprobable.forin- towhichtheywereactuallyassigned.theprimaryfrequentistjusticationofthetest,now,isthephysicalactofrandomization,by hypothesisconsideredwillbethatall9participantswouldhaveobtainedthesamescoresinthegrouptowhichtheywerenotassigned. inthesummerschoolexample,supposetheparticipantshavebeen whichconditionshavebeenallocatedtoexperimentalunits.thus Then,fromtheheterogeneityconclusionofthecombinatorialtest,it assignedtogroupsbymeansofarandomdevice.thenthenull

20 data.conditionalandrandomizationtestsarefurtherdiscussedby maybeinferredthatthisnullhypothesisisnotcompatiblewiththe CombinatorialInference 115 Statusofrandomization.Themethodologicalstatusofrandomizationasanexperimentalprocedurehasbeenmatterofdebate. Insensitivedomainslikemedicalresearch,randomizationraisesimtalrandomizationgeneratesaconsensusabouttheprobabilityof observablesunderprivilegiednullhypotheses,andthisisoftena deniteadvantage,inresearchareaswhereknowledgeislimitedor controversial.thisstatisticaladvantageissometimeserectedasa principle,alongwhich whenrandomsamplingislacking randomizationisamustforstatisticalinferencethatmightbedrawn fromdata.wedonotadheretothisprinciple,ifonlybecausethere CoxandHinkley(1974,p ). menseethicalproblemsthatarebeyondthescopeofthisbook. Conningourselvestostatisticalissues,itisafactthatexperimen- \conditions" towhichunitsmaybeallocatedornot butare aretoomanysituationsthatarenotamenabletorandomizationand forwhichstatisticalinferencestillappearsdesirable.onesuchsituationisthenestingstructurewhenthegroupsdonotpertainto naturalgroups,suchasboysandgirlsinaclassroom,etc.othersituationsarethecrossingstructuresuchasthebeforeandafterdesign (seenextsubsection),thebivariatestructure(leadingtothecombinatorialindependencetest),etc.insuchsituations,shouldone renouncestatisticalinferencejustbecauserandomizationisoutof thequestion?wethinknot.werstlyproposecombinatorialinference,asanonprobabilisticstatisticalinferencethatisapplicable 4.2.5TowardtheBayesianFramework rethoughtalongthelinewesketchbelow. inanycase.wethensuggestthatprobabilisticinferencemightbe notmet. aimingatovercomingthelimitationsoffrequentistinference,when randomnessassumptions(randomsamplingorrandomization)are Inthissubsection,wesubmitreectionsandtentativesuggestions

21 116 Therandomizationparadox.IntheWageModicationexample, thequestionofinterestistoassesstheeectivenessofwagemodicationforthegroupofthe12workersintheworkshop.now,ifwe HenryRouanetandMarie-ClaudeBert taketherandomizationprincipleseriously,thelackofrandomization inthebeforeandafterdesignprecludesinterpretingheterogeneityin termsofsome\noeect"hypothesispertainingtothegroupof12 workersbelongingtog1and\after"observationsonlyonthe6workersbelongingtog2.wewouldthenhavetwoindependentgroupsof 6observationseach,towhichanunobjectionablerandomizationtest toassesstheeectivenessofthewagemodication.equivalently, startingwiththefullmatched-pairdatasetathand,wemayran- ourstatisticalanalysistothese12observations.nowthebeforeand domlysample6\before"observationsand6\after"onesandconne randomlydividedthe12workersintotwogroupsg1andg2of6workerseach,andproceededtomake\before"observationsonlyonthe6 workers.nowinsteadofthebeforeandafterdesign,wemighthave mightbeappliedandallowone,inthecaseofasignicantresult, icalthatusingallavailableinformationshouldprecludeasortof conclusionthatwouldbeauthorizedusingonlypartialinformation. thanthepreceding\randomizationdesign."itthusseemsparadox- afterdesign,wheresubjectsaretheirowncontrols,issurelybetter mogeneitysituations,startingwiththeremarkthatwhateverformal The\noeect"hypothesis.Leavingasiderandomnessassumptionsoffrequentistmodels randomsamplingandrandomization alike letustakeanewlookatthe\noeect"hypothesisinhoualisshownthesetofprotocolsgeneratedfromthebaselinedata set,andaskedtoguesswhichoneoftheprotocolsistheobserved meaningisgiventothishypothesis,thebaselinedatasetdoesnot dataset.ifthisindividualbelievesthatthereisnoeect,thenall containinformationaboutthishypothesis.nowsupposeanindivid- protocolswillbeequiprobableforthatindividual andunderthe beliefthatthereisaneecttheywillpresumablynotbeequiprobable henceaconversionpropertyfromproportionstoprobabilities,validregardlessofanyrandomnessassumption.wesubmitthis

22 conversionpropertytobetakeninallsituationsastheoperational CombinatorialInference probabilisticcharacterizationofthenullhypothesisof\noeect". 117 Wehopethatreaderswillfeelwithusthatthischaracterizationof thenullhypothesisisnatural.thereasonforwhichitisnotclassical isthattheprobabilitiesinvolvedmaynotinterpretableaslong-run frequencies;theybasicallyexpressdegreesofbeliefinparticularsituations.inbayesianterms,thoseprobabilitiesarepredictiveand oftenpresented asinthelatechaptersofthisbook asanenlargementofthefrequentistone,thatis,asasuperstructurethatis conditionaluponthenullhypothesis.thebayesianframeworkis bypassingtheintricaciesofthefrequentistframework(s). directwayfromcombinatorialinferencetothebayesianframework, Chanceformulations.Whenthe\noeect"hypothesisiscompatiblewiththedata(nonsignicantresult),itiscommonlysaid ofcoincidence,orluck,fortuitousness,uke,etc.,suggesting,byim- befruitless.whenonthecontrarythe\noeect"hypothesisisnot thattheresult\mighthaveoccurredbychance" i.e.asamatter addedtoafrequentistmodel.theforegoingdiscussionsuggestsa plication,thatattemptingtointerprettheeectanyfurtherwould thatgoesbacktolaplace,atleast.asalaplace-inspiredexample, theresult\isnotduetochance,"whichmeansthatattemptinginterpretationisinorder.suchformulationshavealong-standinghistory compatiblewithdata(signicantresult),itiscommonlysaidthat following12-charactersequence:kindergarten.thereasonthat leadsustothinkthatthisarrangementisnotduetochance,laplace supposethatachildusingatypewriterforthersttimecomposesthe normoreprobable,andwewouldthennotsuspectanyparticular wouldexplain,cannotbethefactthat,physicallyspeaking,itisless probablethantheothers,because,ifthewordkindergartenwere notinuseinanylanguage,thisarrangementwouldbeneitherless intentionalratherthanduetochance(laplace,1825/1986,p.229). SuchLaplaciancommentsagainpointtotheBayesianframework. causeinconnectionwithit.butasthewordisinuseamongus,itis incomparablymoreprobablethatthearrangementofcharactersis

23 118 TheMakingofCombinatorialInference HenryRouanetandMarie-ClaudeBert portions.yetthesemanticsofprobabilitiesreferstouncertainty, Bothprobabilitiesandrelativefrequenciesareisomorphic,thatis, theyobeythesameformalrulesofamoregeneralcalculusofpro FrequenciesandProbabilities andthatoffrequencies,toobservedstatisticaldata.toconfusetwo isomorphicentitiesistocommitastructuralfallacy1. theprobabilisticlanguageisusedtointroducetheoreticaldistribuilationofprobabilitiestofrequencies.inrouanet(1982),wediscussedthefallaciousconverseassimilation,whichisconveyedwhen InAppendix2ofChapter1,wediscussedthefallaciousassimtions,suchasthenormaldistribution.TherststeptowardCombinatorialInferencethusconsistsincharacterizingsuchdistributions interpretedas\theproportionofstandardscoresgreaterthan1.96 tributions.alongthisline,thenotationp(z>1:96)=0:025is is2.5%."thereareindeedsomestatisticaltextbooksthatadopt as\stylized"frequencydistributions,insteadof\probability"dis- suchanonprobabilisticpresentation,aboveall,thosewritteninthe psychometrictradition,suchasfaverge(1956).inourstatistical teaching,wehaveconstantlyadheredtothistradition,asreected inlecoutreandlecoutre(1979),andtheninrouanet,bernard,le Roux(1990,chapters2and3). Admittedly,nonprobabilisticformulationsofstatisticalinferenceare 4.3.2TheCrucialStep isolation,andthebasiccombinatorialstructuresofstatisticalinfer- occasionallyfoundintextbooks.forinstance,thesentence\95percentofcalculatedcondenceintervalswillcovertheparameter's value"iscommonlyfound.nonetheless,suchsentencesappearin encearemaskedbytheprobabilisticphraseology.virtuallyallsta- 1.AsanexampleofstructuralfallacydiscussedbyJereys(1961):Heatand vaporobeythesamedierentialequations,butitdoesnotfollowfromthis thatheatisavapor.

24 tisticaltextbooksstresstheprobabilisticframeworkandrandomness assumptions.therandomnesshabitissorootedthat\sample"is CombinatorialInference 119 oftenusedasasynonymof\randomsample"!incidentally,suchan insistenceonrandomnessisfurtherevidencethatthechangefrom \randomness"character,andreplacetheprobabilisticformulations ondone,whichconsistsinstrippingtheconceptofasampleofits probabilitiestoproportionsisnotjusta\matterofsemantics". bytheformulationsintermsof\proportionsofsamples."wetook Toarriveatcombinatorialinference,thecrucialstepisthesec- thisstepintheearlyeighties,whenwestartedteachingintroductory statisticalinference. Thedicultiesofteachingstatisticalinferencearewell-known,and indeed,theteachingmotivationshavebeenstronginourmakingof 4.3.3TeachingMotivations thereectionsofourcolleaguesandourselvesthatthealgorithms combinatorialinference.intheearlyeighties,theideaemergedin oftheelementaryinferenceprocedurescouldbetaughtimmediately followingdescriptivestatistics,droppingthetraditional\probabilityprerequisites".suchastrategy,wefelt,wouldallowstudents interpretations.westartedteachingproportionformulations,and toconcentraterstoncomputationalaspects,withoutbeingprematurelyconcernedwiththeconceptualdicultiesofprobabilistiproach,andarstpresentationofitwasmadeattheinternational ConferenceonTeachingStatisticsheldinSheeld(England),with phraseset-theoreticinferencewascoinedtorefertothenewap- devisinginterpretationsintermsoftypicalityandhomogeneity.the theprovocativetitle\teachingstatisticalinferencewithoutprobabillowed:rouanetetal.(1986),andthenthereferencebookrouanetityprerequisites"(rouanetetal,1992).amoredetailedpaperfol- withnopreviousknowlegeofeitherprobabilityorstatisticalinfer- Bernard,LeRoux(1990),withitscompanionteachingsoftwareinfer.AttheUniversityReneDescartes,CombinatorialInferencehas beentaughtcontinuouslysince1982,bothtopsychologystudents

25 120 ence,andtomathematicalstudentsasacomplementofthestandard statisticalcurriculum2. HenryRouanetandMarie-ClaudeBert torialdataanalysis,whichemphasizesalgorithmsinsteadofprob- abilisticmodels.thistrendhasbeenespeciallyactiveinthearea 4.3.4CombinatorialDataAnalysis AgrowingtrendinstatisticsinthelastfewyearshasbeenCombina- phraseologyisoftenmisleading.itsoonbecameclearthat\settheoreticinference"waspartofthistrend apointwelltakenby ofclassicationandisseentonaturallyincludeallthosetechniques suchashalf-split,jacknife,bootstrap,etc.wheretheprobabilistic Arabieetal.(1996,p.5)andothers.TheRouanetetal.book (1990)thusappearstobetherstIntroductiontostatisticalinferencewrittenalongthelineofCombinatorialDataAnalysis.In ordertoemphasizethisconnection,ovefranksuggestedwecalled theapproach\combinatorialinference",andwehavenowdenitely 4.3.5TowardRecognitionofCombinatorialInference adoptedthiswelcomesuggestion. usedinsituationswherethefrequentist\validityassumptions"are pavingthewaytorecognition ofcombinatorialinferencebythe communityofresearchers.firstly,statisticalproceduresareoften Avarietyofreasonsconcurthatshouldfacilitatetheacceptance commentmadebyapsychologist:\butthisisjustwhatihavealwaysdone!"3secondly,thetermsoftypicalityandhomogeneityare notmet.byprovidingassumption-freeinterpretations,combinatorialinferencemakessenseofcommonpractice.hereisarevealing 2.Similarly,attheUniversityReneDescartes,anintroductiontoBayesian sonaturalthattheyarespontaneouslyadopted.thirdly,following 3.Thiscommentcuriouslyechoestheone(byastatistician)reportedbyFreedmanandLane(1982):\ThisisjustwhatIhavealwaysthought!" inferencehasbeentaughtsince1993,asanextensionofclassicalsignicance testing,inthelineofthelastchapterofrouanet,bernard,leroux(1990) andofthesubsequentchaptersofthepresentbook.

26 typicality;fromthisviewpoint,statisticaltypicalityappearsasthe Rosch'swork,cognitivepsychologistshavebeendeeplyinterestedin CombinatorialInference 121 applicationofthegeneralnotiontocollectiveobjects.fourthly,there weprefertostressthatcombinatorialinferencedoesnotrequireunveriableassumptionsactness"mustbequalied,ofcourse.student'st-testwas(andstill isthecurrentprestigeofexacttestsinstatistics.themagicof\ex- is)anexacttesttoo!ratherthanstrikingupthe\exactnesscant", OnceCombinatorialInferencehadtakenshape,westartedinquiring 4.3.6RelatedViewpoints aboutrelatedapproaches.then,leavingasidetheabundanttechnicaldevelopmentsaboutpermutationtests,monte-carloprocedures, etc.,wediscoveredthattherehavebeenreallyfewpublicationsdevelopingconceptualviewpointsakintocombinatorialinference.in whatfollowswesketchthreesuchsignicantcontributions4. familiartoeconometricstatisticiansforhisfamous1954paradoxin DecisionTheory also,perhaps,forthenobelprizehegotin1989. Nowintheearlyeighties,MauriceAllaisvigorouslydenouncedthe MauriceAllaisandnonprobabilisticmodels.Allais'nameis confusionoffrequencieswithprobabilitiesinthecurrentinterpretationofeconometricmodels.hereiswhatwereadinallais(1983): anysimilarterm...allthefundamentaltheoremsoftheso-called Probabilitytheory,theBernoullilawoflargenumbers5,orthecen- \Theso-calledmathematicaltheoriesofprobabilitycouldallbepresentedwithouteverusingthewordschance,probable,random,or 5.InRouanetetal.(1990),westatetheBernoullilawoflargenumbersin 4.Otherreferencesarealsoworthmentioning,suchasMatheron(1989),areferencethatdidnotescapeShafer's(1994)attentiontrallimittheoremofconvergencetothenormallaw,thelawofiterreference otherthanallais thatwouldsuggest(evenremotely)that termsofthelimitproportionofcentralpaths.wearenotawareofanysingle suchcombinatorialformulationsofstandardprobabilitytheoremsarenot onlypossiblebuthighlymeaningful.

27 122 atedlogarithm,thearcsinelaw,etc.areonlyasymptoticproperties offrequencydistributionsbasedoncalculationsofcombinatorialtechniques."(author'sitalics).toenhancehisclaim,allaisexhibitsa HenryRouanetandMarie-ClaudeBert quasi-periodicmodel hencefullydeterministic whosepredictionscouldtypically(andfallaciously)beinterpretedintermsofa Edgingtonandnonrandomsamples.Startingfromthefact stochasticmodel.inspiteoftheauthor'snotoriety,allais'message wentvirtuallyunnoticed andunchallenged. thatnonrandomsamplesarewidelyusedinexperimentation,and buildingonthedistinctionbetweenrandomsamplingandrandomization cf.section2.4 EugeneEdgington,innumerouspublicationssuchasEdgington(1987,1995),hascogentlyandvaliantly defendedthepositionthatfrequentistinferencemaybeperformed innonrandomsamples,wheneverrandomizationisavailable.weare basicallyinfullsympathywithapositionwhichstressesastatisticalframework i.e.randomization whichisbadlyneglected. Asacounterpart,thephysicalactofrandomizationseemstobe foredgingtonanecessaryrequirement(alongthe\randomization Section2.5. DavidFreedmanandnonstochasticsettings.InFreedmanand principle"),andthisdepartsfromourviewpoint,aswediscussedin Lane(1982,1983),theauthorsconsiderthefollowingproblem.\Data areobtainedinanonstochastic[i.e.nonrandom]setting,andfor throughexamples suchasthesexbiasexample(section2.3) bedismissedasanartifact,ordoesitrequireamoresubstantialexplanation?"thesolutionsuggestedbytheauthors,andillustrated someattributeofthisdata,thequestionisraised:canthisattribute comesverycloseinspirittocombinatorialinference.onemayregretthatthosethought-provokingpapershavenotbeenfollowedby frequentistviewpoint. inallrespectsamostcommendablebook isconnedtothe systematicdevelopments,andthattheintroductorystatisticalbook writtenbydavidfreedmanandhiscolleagues(freedmanetal,1991)