REVIEW ARTICLE TRENDS i Spor Scieces 014; 1(1: 19-5. ISSN 99-9590 The eed o repor effec size esimaes revisied. A overview of some recommeded measures of effec size MACIEJ TOMCZAK 1, EWA TOMCZAK Rece years have wiessed a growig umber of published repors ha poi ou he eed for reporig various effec size esimaes i he coex of ull hypohesis esig (H 0 as a respose o a edecy for reporig ess of saisical sigificace oly, wih less aeio o oher impora aspecs of saisical aalysis. I he face of cosiderable chages over he pas several years, eglec o repor effec size esimaes may be oed i such fields as medical sciece, psychology, applied liguisics, or pedagogy. Nor have spor scieces maaged o oally escape he grips of his subopimal pracice: here saisical aalyses i eve some of he curre research repors do o go much furher ha compuig p-values. The p-value, however, is o mea o provide iformaio o he acual sregh of he relaioship bewee variables, ad does o allow he researcher o deermie he effec of oe variable o aoher. Effec size measures serve his purpose well. While he umber of repors coaiig saisical esimaes of effec sizes calculaed afer applyig parameric ess is seadily icreasig, reporig effec sizes wih o-parameric ess is sill very rare. Hece, he mai objecives of his coribuio are o promoe various effec size measures i spor scieces hrough, oce agai, brigig o he readers aeio he beefis of reporig hem, ad o prese examples of such esimaes wih a greaer focus o hose ha ca be calculaed for o-parameric ess. KEY WORDS: spor sciece, effec size calculaio, parameric ess, o-parameric ess, mehodology. Received: 1 Sepember 013 Acceped: 15 February 014 Correspodig auhor: maciejomczak5@gmail.com 1 Uiversiy School of Physical Educaio i Pozań, Deparme of Psychology, Polad Adam Mickiewicz Uiversiy i Pozań, Faculy of Eglish, Deparme of Psycholiguisic Sudies, Polad Wha is already kow o his opic? Esimaes of effec size allow he assessme of he sregh of he relaioship bewee he ivesigaed variables. I pracice, hey permi a evaluaio of he magiude ad imporace of he resul obaied. A effec size esimae is a measure worh reporig ex o he p-value i ull hypohesis esig. However, o every research repor coais i. Afer he ull hypohesis has bee esed wih he use of parameric ad o-parameric ess (saisical sigificace esig, measures of effec size ca be esimaed. A few remarks o saisical hypohesis esig Sudies i spor scieces have addressed a wide specrum of opics. Empirical verificaio i hese areas ofe makes use of correlaio models as well as experimeal research models. Jus like oher scholars coducig empirical research, researchers i spor scieces ofe rely o ifereial saisics o es hypoheses. From he poi of view of saisics, he hypohesis verificaio process ofe comes dow o deermiig he probabiliy value (p-value, ad o decidig wheher he ull hypohesis (H 0 is rejeced (a es of saisical sigificace [1,, 3, 4]. I he case of rejecig he ull hypohesis (H 0, a researcher Vol. 1(1 TRENDS IN SPORT SCIENCES 19
TOMCZAK, TOMCZAK will accep a aleraive hypohesis (H 1, which is ofe referred o as he so-called subsaive hypohesis as a researcher formulaes i based o various crieria applicable o heir ow sudies. Such a approach o hypohesis verificaio has is origi i Fisher s approach (p-value approach ad he Neyma-Pearso framework o hypohesis esig ha was developed laer (fixed-α approach. Below, based o Araowska ad Ryel [5, p. 50], we prese he wo approaches (Table 1. Rejecig he ull hypohesis (H 0 whe i is i fac rue is wha Neyma ad Pearso call makig a Type I error (kow as false posiive or false alarm. To corol for Type I error, or i oher words, o miimize he chace of fidig a differece ha is o really here i he daa, researchers se a appropriaely low alpha level i heir aalyses. By coras, failig o rejec he ull hypohesis (H 0 whe i is acually false (ad should be rejeced is referred o as a Type II error (kow as false egaive. Here, icreasig he sample size is a effecive way of reducig he probabiliy of obaiig a Type II error [1,, 3]. The preseed approach o hypohesis esig has bee a commo pracice i may disciplies. However, reporig he p-value aloe ad drawig ifereces based o he p-value aloe is isufficie. Hece, saisical aalyses ad research repors should be supplemeed wih oher esseial measures ha carry more iformaio abou he meaigfuless of he resuls obaied. Why he p-value aloe is o eough? or O he eed o repor effec size esimaes Thaks o some of is advaages, he cocep of saisical sigificace esig has prevailed i he empirical verificaio of hypoheses o he exe ha may areas have sill see oher vial saisical measures go largely urepored. I spie of recommedaios o o limi research repors o preseig he ull hypohesis esig ad reporig he p-value oly, o his day a relaively large umber of published aricles have o goe much beyod ha. By way of illusraio, a mea-aalysis of research accous published i oe presigious psychology joural i he years 009 ad 010 showed ha almos half of he aricles reporig a Aalysis of Variace (ANOVA did o coai ay measure of effec size, ad oly a mere quarer of he surveyed research repors supplemeed Sude s -es aalyses wih iformaio abou he effec size [6]. Spor scieces have see comparable pracices every ow ad he. As already poied ou, givig he p-value oly o suppor he sigificace of he differece bewee groups, or measuremes, or he sigificace of a relaioship is isufficie [7, 8]. The p-value aloe merely idicaes wha he probabiliy of obaiig a resul as exreme as or more exreme ha he oe acually obaied, assumig ha he ull hypohesis is rue [1]. I may circumsaces, he compued p-value depeds (also o he sadard error (SE [9]. I is ow well esablished ha he sample size affecs he sadard error ad, as a resul of ha, he p-value. As he size of a sample icreases, he sadard error becomes smaller, ad he p-value eds o decrease. Due o his depedece o sample size, p-values are see as cofouded. Someimes a resul ha is saisically sigifica maily idicaes ha a huge sample size was used [10, 11]. For his reaso, he value of he p-value does o say wheher he observed resul is meaigful or impora i erms of (1 he magiude of he differece i he mea scores of he groups o some measure, or ( Table 1. Fisher s ad Neyma-Pearso s approaches o hypohesis esig The Fisher approach o hypohesis esig (also kow as he p-value approach formulae he ull hypohesis (H 0 selec he appropriae es saisic ad specify is disribuio collec he daa ad calculae he value of he es saisic for your se of daa specify he p-value if he p-value is sufficiely small (accordig o he crierio adoped, he rejec he ull hypohesis. Oherwise, do o rejec he ull hypohesis. The Neyma-Pearso approach o hypohesis esig (also kow as he fixed-α approach formulae wo hypoheses: he ull hypohesis (H 0 ad he aleraive hypohesis (H 1 selec he appropriae es saisic ad specify is disribuio specify α (alpha ad selec he criical regio (R collec he daa ad calculae he value of he es saisic for your se of daa if he value of he es saisic falls i he criical (rejecio regio, he rejec he ull hypohesis a a chose sigificace level (α. Oherwise, do o rejec he ull hypohesis. 0 TRENDS IN SPORT SCIENCES March 014
The eed o repor effec size esimaes revisied. A overview... he sregh of he relaioship bewee he ivesigaed variables. Relyig o he p-value aloe for saisical iferece does o permi a evaluaio of he magiude ad imporace of he obaied resul [10, 1, 13]. I geeral erms, here are good eough reasos for researchers o suppleme heir repors of he ull hypohesis esig (saisical sigificace esig: he p-value wih iformaio abou effec sizes. Give saisical measures, a large umber of effec size esimaes have bee developed ad used o his day. As reporig effec size esimaes is beeficial i more ha oe way, below we lis he beefis ha seem mos fudameal [6, 1, 14, 15, 16, 17, 18]: 1. They reflec he sregh of he relaioship bewee variables ad allow for he imporace (meaigfuless of such a relaioship o be evaluaed. This holds boh for relaioships explored i correlaioal research ad he magiude of effecs obaied i experimes (i.e. evaluaig he magiude of a differece. O he oher had, applyig a es of sigificace oly ad saig he p-value may solely provide iformaio abou he presece or absece of a differece, is impac ad relaio, leavig aside is imporace.. Effec size esimaes allow he resuls from differe sources ad auhors o be properly compared. The p-value aloe, which depeds o he sample size, does o permi such comparisos. Hece, he effec size is criical i research syheses ad meaaalyses ha iegrae he quaiaive fidigs from various sudies of relaed pheomea. 3. They ca be used o calculae he power of a saisical es (power saisics, which i ur allows he researcher o deermie he sample size eeded for he sudy. 4. Effec sizes obaied i pilo sudies where he sample size is small may be a idicaor of fuure expecaios of research resuls. Some recommeded effec size esimaes I he prese secio we provide a overview of a umber of effec size esimaes for saisical ess ha are mos commoly used i spor scieces. Sice parameric ess are frequely used, measures of effec size for parameric ess are described firs. The, we describe effec size esimaes for o-parameric ess. Reporig measures of effec size for he laer is more of a rariy. Aside from ha, i he overview below we omi he measures of effec size ha are mos popular ad widely repored for parameric ess. I spor scieces examples of he mos popular esimaes of effec size iclude correlaio coefficies for relaioships bewee variables measured o a ierval or raio scale such as he Pearso s correlaio coefficie (r. Nor do we prese effec size measures popular ad widely used, amog ohers, i spor scieces, calculaed for relaioships bewee ordial variables such as he Spearma s coefficie of correlaio. Some measures of effec size preseed below ca be calculaed auomaically wih he help of saisical sofware such as Saisica, he Saisical Package for he Social Scieces (SPSS, or R. Ohers ca be calculaed by had i a quick ad easy way. Effec size esimaes used wih parameric ess The Sude s -es for idepede samples is a parameric es ha is used o compare he meas of wo groups. Afer he ull hypohesis is esed, oe ca easily ad quickly calculae he value of he poi-biserial correlaio coefficie wih he help of he Sude s -es (provided ha he -value comes from comparig groups of relaively similar size. This coefficie is similar o he classical correlaio coefficie i is ierpreaio. Usig his coefficie oe ca calculae he popular r (η. The formula used i compuig he poi-biserial correlaio coefficie is preseed below [1, 6, 19]: r r η + df + df value of Sude s -es, df he umber of degrees of freedom ( 1 1 + 1; 1, he umber of observaios i groups (group 1, group r poi-biserial correlaio coefficie r (η he idex assumes values from 0 o 1 ad muliplied by 100% idicaes he perceage of variace i he depede variable explaied by he idepede variable Ofe used here are he effec size measures from he so-called d family of size effecs ha iclude, amog ohers, wo commoly used measures: Cohe s d ad Hedges g. Below we provide a formula for calculaig Cohe s d [1, 19, 0, 1]: Vol. 1(1 TRENDS IN SPORT SCIENCES 1
TOMCZAK, TOMCZAK x d x 1 σ d Cohe s idex meas of he firs ad secod sample σ sadard deviaio of a populaio x 1, Normally, we do o kow he populaio sadard deviaio ad we esimae i based o he sample. Give ha, i is possible here o use he esimae of sadard deviaio of he oal populaio. I his case, o esimae he effec size oe ca compue he g coefficie ha uses he weighed pooled sadard deviaio []: g x x ( + ( 1 s 1 s 1 1 1 + 1 1, he umber of observaios i groups (group 1, group s 1, s sadard deviaio i groups (group 1, group rough arbirary crieria for Cohe s d ad Hedges g values: d or g of 0. is cosidered small, 0.5 medium, ad 0.8 large [1] Whe i comes o he depede-samples Sude s -es, i is possible o compue he correlaio coefficie r. For his purpose, he above-preseed formula for calculaig r for idepede samples is adoped. However, he r coefficie is o loger he simple poi-biserial correlaio, bu is isead he correlaio bewee group membership ad scores o he depede variable wih idicaor variables for he paired idividuals parialed ou [3, p. 447]. Addiioally, oce he depede-samples Sude s -es has bee used, i is possible o calculae he effec size esimae g, where [1, ]: g D SSD 1 D mea differece score SS D sum of squared deviaios (i.e. he sum of squares of deviaios from he mea differece score I ur, o compare more ha wo groups o raio variables or ierval variables, Aalysis of Variace (ANOVA is used, be i oe-way or muli-facor ANOVA (provided ha he samples mee he crieria. The effec size esimaes used here are he coefficie η or ω. To compue he former (η, we may use he ANOVA oupu from popular saisical sofware packages such as Saisica or SPSS. Below we prese he formula [1, 6, 4]: η SS SS SS ef sum of squares for he effec SS oal sum of squares η he idex assumes values from 0 o 1 ad muliplied by 100% idicaes he perceage of variace i he depede variable explaied by he idepede variable Oe of he disadvaages of η is ha he value of each paricular effec is depede o some exe o he size ad umber of oher effecs i he desig [5]. A way ou of his problem is o calculae he parial ea-squared saisic ( η p, where a give facor is see as playig a role i explaiig he porio of variace i he depede variable provided ha oher effecs (facors prese i he aalysis have bee excluded [6]. The formula is preseed below [1, 6, 4]: η p SS ef SS ef ef + SS SS ef sum of squares for he effec SS er sum of squared errors I he same way, oe ca calculae he effec size for wihi-subjec desigs (repeaed measures. However, boh coefficies η ad ( η p are biased ad hey esimae he effec for a give sample oly. Therefore, we should compue he coefficie ω ha is relaively ubiased. To calculae i by had oe ca use he ANOVA oupu ha coais values of mea square (MS, sum of squares (SS, ad degrees of freedom (df. For bewee-subjec desigs he followig formula applies [4]: er df ω ef ( MSef MSer SS + MS MS ef mea square of he effec MS er mea square error SS he oal sum of squares degrees of freedom for he effec df ef er TRENDS IN SPORT SCIENCES March 014
The eed o repor effec size esimaes revisied. A overview... For wihi-subjec desigs ω is calculaed usig he formula [4]: df ω ef ( MSef MSer SS + MS MS ef mea square of he effec MS er mea square error MS sj mea square for subjecs df ef degrees of freedom for he effec The parial omega-squared ( ω p is compued i he same way boh for he bewee-subjec desigs ad wihi-subjec desigs (repeaed measures usig he formula below [4]: ω ef ef er p ef ef ef er sj df ( MS MS df MS + ( df MS Boh η ad ω are ierpreed similarly o R. Hece, hese measures muliplied by 100% idicae he perceage of variace i he depede variable explaied by he idepede variable. Effec size esimaes used wih o-parameric ess Now we ur o o-parameric ess. Various effec size esimaes ca be quickly calculaed for he Ma- Whiey U-es: a o-parameric saisical es used o compare wo groups. I addiio o he U-value, he Ma-Whiey es repor (oupu coais he sadardized Z-score which, afer ruig he Ma- Whiey U-es o he daa, ca be used o compue he value of he correlaio coefficie r. The ierpreaio of he calculaed r-value coicides wih he oe for Pearso s correlaio coefficie (r. Also, he r-value ca be easily covered o r. The formulae for calculaig r ad r by had are preseed below [6]: Z r r η Z Z sadardized value for he U-value r correlaio coefficie where r assumes he value ragig from 1.00 o 1.00 r (η he idex assumes values from 0 o 1 ad muliplied by 100% idicaes he perceage of variace i he depede variable explaied by he idepede variable he oal umber of observaios o which Z is based Followig he compuaio of he Ma-Whiey U-saisic, oe ca also calculae he Glass rak-biserial correlaio usig average raks from wo ses of daa ( _ R 1, _ R ad sample size i each group. Some saisical packages ex o he es score produce he sum of raks ha ca be used o calculae mea raks. To ierpre he calculaed value oe ca draw o he ierpreaio of he classical Pearso s correlaio coefficie (r. Here he followig formula applies [1]: ( R R r 1 1+ _ R_ 1 mea rak for group 1 R mea rak for group 1 sample size (group 1 sample size (group r correlaio coefficie where r assumes he value ragig from 1.00 o 1.00 For aoher o-parameric es, he Wilcoxo sigedrak es for paired samples, agai, he Z-score may be used o calculae correlaio coefficies employig he formula give below (where is he oal umber of observaios o which Z is based [6]. r O he oher had, oce he Wilcoxo siged-rak es has bee compued, oe ca also calculae he rakbiserial correlaio coefficie usig he formula [1]: Z R1+ R 4 T r + ( + 1 R 1 sum of raks wih posiive sigs (sum of raks of posiive values R sum of raks wih egaive sigs (sum of raks of egaive values T he smaller of he wo values (R 1 or R he oal sample size r correlaio coefficie (which is he same as r coefficie i is ierpreaio Vol. 1(1 TRENDS IN SPORT SCIENCES 3
TOMCZAK, TOMCZAK For he Kruskal-Wallis H-es, a o-parameric es adoped o compare more ha wo groups, he easquared measure (η ca be compued. The formula for calculaig he η esimae usig he H-saisic is preseed below [6]: H k+ η H k 1 H he value obaied i he Kruskal-Wallis es (he Kruskal-Wallis H-es saisic η ea-squared esimae assumes values from 0 o 1 ad muliplied by 100% idicaes he perceage of variace i he depede variable explaied by he idepede variable k he umber of groups he oal umber of observaios I addiio, oce he Kruskal-Wallis H-es has bee compued, he epsilo-squared esimae of effec size ca be calculaed, where [1]: E R H ( 1/( + 1 H he value obaied i he Kruskal-Wallis es (he Kruskal-Wallis H-es saisic he oal umber of observaios E R coefficie assumes he value from 0 (idicaig o relaioship o 1 (idicaig a perfec relaioship Also, for he Friedma es, a o-parameric saisical es employed o compare hree or more paired measuremes (repeaed measures, a effec size esimae ca be calculaed (ad is referred o as W [1]: χ w W Nk ( 1 W he Kedall s W es value χ w he Friedma es saisic value N sample size k he umber of measuremes per subjec The Kedall s W coefficie assumes he value from 0 (idicaig o relaioship o 1 (idicaig a perfec relaioship. Also, i spor scieces i is quie commo pracice o use he chi-square es of idepedece (χ. Havig esed he ull hypohesis (H 0 wih a χ es of idepedece, oe may assess he sregh of a relaioship bewee omial variables. I his case, Phi (φyoula, compued for ables where each variable has oly wo levels, e.g. he firs variable: male/female, he secod variable: smokig/o-smokig ca be repored, or oe ca repor Cramer s V (for ables which have more ha rows ad colums. The values obaied for he esimaes of effec size are similar o correlaio coefficies i heir ierpreaio. Agai, popular saisical sofware packages calculae Phi ad Cramer s V. Below we prese he formulae for such calculaios [1, 6]: ad for Cramer s V: φ V χ χ df ( s df s degrees of freedom for he smaller from wo umbers (he umber of rows ad colums, whichever is smaller χ he calculaed chi-square saisic he oal umber of cases The Phi coefficie ad he Cramer s V assume he value from 0 (idicaig o relaioship o 1 (idicaig a perfec relaioship. Coclusios I he prese coribuio we have re-emphasized he eed o repor esimaes of effec size i cojucio wih ull hypohesis esig, ad he beefis hereof. We have preseed some of he recommeded measures of effec size for saisical ess ha are mos commoly used i spor scieces. Addiioal emphasis has bee o effec size esimaes for o-parameric ess, as reporig effec size measures for hese ess is sill very rare. The prese paper may also serve as a poi of deparure for furher discussio where pracical (e.g. cliical magiude (imporace of resuls i he ligh of he codiioigs i a chose area will come io focus. 4 TRENDS IN SPORT SCIENCES March 014
The eed o repor effec size esimaes revisied. A overview... Wha his paper adds? This paper highlighs he eed for icludig adequae esimaes of effec size i research repors i he area of spor scieces. The overview coais various ypes of effec size measures ha ca be calculaed followig he compuaio of parameric ad o-parameric ess. Sice reporig effec size esimaes whe usig o-parameric ess is very rare, his secio may prove paricularly useful for researchers. Some of he effec size measures give ca be calculaed by had quie easily, ohers ca be calculaed wih he help of popular saisical sofware packages. Refereces 1. Kig BM, Miium EW. Saysyka dla psychologów i pedagogów (Saisical reasoig i psychology ad educaio. Warszawa: Wydawicwo Naukowe PWN; 009.. Cohe J. The earh is roud (p <.05. America Psychologis. 1994; 49(1: 997-1000. 3. Cohe J. Thigs I have leared (so far. America Psychologis. 1990; 45(1: 1304-131. 4. Jascaiee N, Nowak R, Kosrzewa-Nowak D, e al. Seleced aspecs of saisical aalyses i spor wih he use of Saisica sofware. Ceral Europea Joural of Spor Scieces ad Medicie. 013; 3(3: 3-11. 5. Araowska E, Ryel J. Isoość saysycza co o aprawdę zaczy? (Saisical sigificace wha does i really mea?. Przegląd Psychologiczy. 1997; 40(3-4: 49-60. 6. Friz CO, Morris PE, Richler JJ. Effec size esimaes: curre use, calculaios, ad ierpreaio. Joural of Experimeal Psychology: Geeral. 01; 141(1: -18. 7. Drikwaer E. Applicaios of cofidece limis ad effec sizes i spor research. The Ope Spors Scieces Joural. 008; 1(1: 3-4. 8. Fröhlich M, Emrich E, Pieer A, e al. Oucome effecs ad effecs sizes i spor scieces. Ieraioal Joural of Spors Sciece ad Egieerig. 009; 3(3: 175-179. 9. Alma DG, Blad JM. Sadard deviaios ad sadard errors. Briish Medical Joural. 005; 331(751: 903. 10. Sulliva GM, Fei R. Usig effec size or why he p value is o eough. Joural of Graduae Medical Educaio. 01; 4(3: 79-8. 11. Bradley MT, Brad A. Alpha values as a fucio of sample size, effec size, ad power: accuracy over iferece. Psychological Repors. 013; 11(3: 835-844. 1. Brzeziński J. Badaia eksperymeale w psychologii i pedagogice (Experimeal sudies i psychology ad pedagogy. Warszawa: Wydawicwo Naukowe Scholar; 008. 13. Durlak JA. How o selec, calculae, ad ierpre effec sizes. Joural of Pediaric Psychology. 009; 34(9: 917-98. 14. Shaughessy JJ, Zechmeiser EB, Zechmeiser JS. Research Mehods i Psychology. 5h ed. New York, NY: The McGraw-Hill; 000. 15. Aars S, va de Akker M, Wikes B. The imporace of effec sizes. Europea Joural of Geeral Pracice. 014; 0(1: 61-64. 16. Ellis PD. The esseial guide o effec sizes: Saisical power, mea-aalysis, ad he ierpreaio of research resuls. Cambridge: Cambridge Uiversiy Press; 010. 17. Lazarao A. Power, effec size, ad secod laguage research. A researcher commes. TESOL Quarerly. 1991; 5(4: 759-76. 18. Hach EM, Lazarao A, Jolliffe DA. The research maual: Desig ad saisics for applied liguisics. New York: Newbury House Publishers; 1991. 19. Rosow RL, Rosehal R. Effec sizes for experimeig psychologiss. Caadia Joural of Experimeal Psychology. 003; 57(3: 1-37. 0. Cohe J. Some saisical issues i psychological research. I: Wolma BB, ed., Hadbook of cliical psychology, New York: McGraw-Hill; 1965. pp. 95-11. 1. Cohe J. Saisical power aalysis for he behavioral scieces. d ed. Hillsdale, NJ: Lawrece Erlbaum; 1988.. Hedges LV, Olki I. Saisical mehods for meaaalysis. Sa Diego, CA: Academic Press; 1985. 3. Rosow RL, Rosehal R, Rubi DB. Corass ad correlaios i effec-size esimaio. Psychological Sciece. 000; 11(6: 446-453. 4. Lakes D. Calculaig ad reporig effec sizes o faciliae cumulaive sciece: a pracical primer for -ess ad ANOVAs. Froiers i Psychology. 013; 4: 863. 5. Tabachick BG, Fidell LS. Usig mulivariae saisics. Upper Saddle River, NJ: Pearso Ally & Baco; 001. 6. Cohe BH. Explaiig psychological saisics. 3rd ed. New York: Joh Wiley & Sos; 008. Vol. 1(1 TRENDS IN SPORT SCIENCES 5