An Overview of Experimental Design

Transcription

1 A Overview of Experimetal Deig I. Hypothei Tetig While much reearch i Biology coit of data collectio for decriptive purpoe, there i a burgeoig tred toward collectig iformatio with the hope of awerig particular quetio or to recat iformatio collected for decriptive purpoe i light of particular hypothee. Thi i due to a umber of caue, amog them; the growig body of decriptive iformatio o all apect of atural hitory ad the iappropriatee of exitig data et to awer pecific quetio. It may alo repreet a icreae i the awaree of cietit of the logical tructure of Scietific Method. Whatever the caue, the effect i that biologit are akig quetio ad deigig their reearch effort to awer quetio. Hece, our cocer with akig awerable quetio ad for developig procedure upo which to bae probabilitic iferece regardig the awer to thee quetio. However, before we dicu particular procedure ad their applicatio, ome dicuio of the form of the quetio we ak ad the poible outcome of our attempt to awer a particular quetio i warrated. A. Null, teted, ad alterative hypothee Whe preeted a a tatemet, rather tha a quetio, the quetio of iteret i a particular reearch program i called the "teted hypothei." It may be either of the geeral form; "Factor A i repoible for pheomeo B," or "Factor A i ot repoible for pheomeo B." Thi latter form, which i the egatio of ay relatiohip betwee Factor A ad pheomeo B i alo called a "ull hypothei." Each of thee hypothee ca erve a the teted hypothei or a the correpodig alterative hypothei. The alterative hypothei i that et of hypothee implied by the rejectio of the teted hypothei. Oce agai, a ull hypothei i imply a hypothei of "o effect" or "o relatiohip" betwee ome factor of iteret ad ome obervable pheomeo. B. Itimacy, advocacy, ad impartiality i hypothei tetig Give a pecified teted ad alterative hypothei, with what goal i mid hould evidece be gathered to "tet" the hypothei. Should our "tet" be a attempt to fid evidece coitet with our teted hypothei or to fid evidece icoitet with our teted hypothei? Do we wih to prove or diprove our teted hypothei? A a liguitic coveiece ad a a widepread practice, may cietit trive to prove particular hypothee, or at leat ay they do. However, for both logical ad practical reao it may be deirable to trive to falify the teted hypothei. Firt, the practical reao. If the reearcher attempt to prove the teted hypothei by gatherig evidece i it upport, they ru the rik of igorig evidece that might

2 cotrovert the teted hypothei. Furthermore, if the teted hypothei i ot a ull hypothei, the proce of attemptig to prove the teted hypothei amout to advocatig the teted hypothei. Thi may lead the reearcher to develop "pet theorie" ad eek to upport them rather tha expoig thee theorie to critical crutiy. Such itimacy betwee reearcher ad hypothei ca impair the reearcher' ability to cat aide favorite, yet uteable hypothee. Logically peakig, Hume ad Popper illutrate the aymmetry that exit betwee proof ad diproof. The weight of evidece foud coitet with a particular hypothei i o match for the itace icoitet with that hypothei. That igle itace i ufficiet to falify the hypothei where o amout of evidece coitet with a hypothei i ufficiet to prove it. So to iure a impartial, detached evaluatio of competig hypothee ad to efficietly ad rigorouly ae the relative merit of competig hypothee, oe hould attempt to gather evidece capable of falifyig the teted hypothei, ot evidece deiged to prove the teted hypothei.. The tatitical hypothei, error, ad power A tatitical hypothei i a tatemet about a tatitical populatio that, o the bai of iformatio obtaied from the oberved data, oe eek to refute. A tatitical tet i a et of rule whereby a deciio about the hypothei i reached. Aociated with the deciio rule i ome idicatio of the accuracy of the deciio reached by followig the rule. The meaure of accuracy i a tatemet about the probability of makig the correct deciio whe certai coditio are true i the populatio i which the hypothei applie. The accuracy of the deciio baed upo iformatio upplied by a experimet deped to a great extet upo the deig of the experimet. The deciio rule are et up by the experimeter ad deped upo what the experimeter coider to be the critical boud for arrivig at the wrog deciio. However, the tatitical hypothei doe ot become fale whe it exceed the critical boud, or i it true whe it doe ot exceed the boud. The deciio rule are merely guide i ummarizig the reult of the tatitical tet - followig the guide eable the experimeter to attach probability tatemet to their deciio. The probability tatemet aociated with the deciio rule of the tatitical tet are predictio a to what may be expected to be the cae if the experimet were repeated may time. The logic of a tatitical hypothei tet i a follow:. Oe aume the teted hypothei to be true. Oe examie the coequece of thi aumptio i term of a amplig ditributio that deped upo the truth of thi hypothei. 3. If, a determied from the amplig ditributio, the oberved data have relatively high probability of occurrig, the deciio i made that the data do ot cotradict the hypothei.

3 4. If the probability of a oberved data et i low whe the teted hypothei i true, the deciio i made that the data cotradict the teted hypothei. 5. Agai, the teted hypothei i ofte tated i uch a way that whe the data cotradict it, the experimeter ha demotrated the preece of ome experimetal effect. The experimeter ha bee able to ullify the teted hypothei, i favor of the alterative hypothei that ome effect i detectable. Voila, a ull teted hypothei. The level of igificace, α, defie the probability level that i too low to warratupport of the teted hypothei. It i oe of the deciio rule. If the probability of occurrece of the oberved data (whe the teted hypothei i true) i maller tha the level of igificace, the the data cotradict the hypothei beig teted, ad the deciio i made to reject the teted hypothei. Thi rejectio i equivalet to upportig oe of the poible alterative hypothee that are ot cotradicted by the data. If the teted hypothei i ymbolized by H 0, ad the et of alterative hypothee that remai teable whe H 0 i rejected i H a : the the deciio rule i a tatitical tet ca be pecified with repect to rejectio or o-rejectio of H 0 :. The rejectio of H 0 may be regarded a the acceptace of H a.. The o-rejectio of H 0 may be regarded a a rejectio of H a. If the deciio rule reject H 0 whe it i i fact true, the rule ha led to a erroeou deciio. The probability of makig uch a error i at mot equal to α, the level of igificace. Thi kid of error i kow a a Type I error; rejectig the teted hypothei whe true. If the deciio rule do ot reject H 0, whe it i i fact fale, it alo lead to a erroeou deciio. Thi kid of error i kow a a Type II error; failig to reject the teted hypothei whe it i fale. The potetial magitude of a Type II error deped i part upo the level of igificace of the tet, ad i part upo which of the alterative hypothee the data actually upport. Aociated with each poible alterative hypothei i a differet probability of a Type II error. Type I error ca oly occur if the deciio i made to reject H 0 ad Type II error may occur whe the deciio i made to ot reject H 0. The probability of makig a Type I error i uder the direct cotrol of the experimeter, ice the experimeter et the level of igificace, α. However, Type II error i cotrolled idirectly, primarily through the deig of the experimet. If poible the teted hypothei i tated i uch a way that the more cotly error i the Type I error, ice it magitude ca be directly cotrolled by the experimeter. Thi i why the teted hypothei i ofte tated a a ull hypothei, ice rejectio of the teted ull hypothei whe it i true amout to fidig a experimetal effect whe oe exit. Such a error could have a great impact o a reearch program ice it will mot likely 3

4 lead the experimeter to coider the quetio awered. Better to be coervative ad fail to fid a experimetal reult. The experimeter would the be forced to repeat the experimet, poibly with modificatio, or to perform other experimet to tet the ame hypothei. Err o the ide of iocece. Neverthele, it i bet, to try to miimize both ource of error. However, Type I ad Type II error are ot idepedet. The maller the probability of a Type I error, α, the larger umerically the potetial Type II error ca be. The relatiohip betwee Type I ad Type II error ca be bet repreeted graphically. The rejectio regio for H 0 i defied relative to the amplig ditributio of the tatitic of iteret whe H 0 i true (olid lie, a). The dahed lie repreet the amplig ditributio of the ame tatitic whe a particular alterative hypothei i true, H a. β, the probability of a Type II error i that area uder the dahed curve that lie withi the regio of o-rejectio of the amplig ditributio of the tatitic whe H 0 i true. Part b illutrate the effect of alterig a o the value of β. If α i maller, the area of the dahed curve that fall withi the regio of o-rejectio of the amplig ditributio whe H 0 i true i larger, hece β i larger whe α i maller. The power of a tet i equal to (l - β ). The power i the area uder the dahed curve that fall i the regio of rejectio of the amplig ditributio whe H 0 i true. Sice β i the probability of failig to reject the teted hypothei whe fale, ( - β ), the power i the probability of rejectig the teted hypothei whe fale. = P (rejectig whe fale) + P (failig to reject whe fale) Power β Power = ( - β ) Power i the probability that the deciio rule reject H 0 whe a pecified H a i true. The cloer the H a to H 0 (the greater the overlap i the correpodig amplig ditributio) the lower the power of the tet with repect to that particular alterative. A well-deiged experimet will both be coervative, have low α, ad have high power, β, with repect to all alterative hypothee which are i a practical ee differet from H 0. For a H 0 of µ = µ, ad a H a of µ = µ , thi H a may ot be, for all practical purpoe, a differet hypothei tha H 0. Hece power with repect to thi alterative i of o practical coequece. The mot commo mea to miimize the probability of a Type II error ad to icreae the power of a tet relative to all poible reaoable alterative hypothee, for fixed α, i to icreae the ample ize or replicatio i the experimet. Thi i becaue the diperio of the amplig ditributio of a tatitic decreae by a factor of ( N ). Hece, the overlap i the amplig ditributio of the teted ad alterative hypothee decreae a N icreae. 4

5 While a iordiate emphai ha bee placed o the level of igificace of a tatitical tet, the power of a tet i uually igored. Thi i partially due to the reluctace of experimeter to preet reult i which a teted ull hypothei could ot be rejected, ad to a lack of iformatio a to what cotitute a reaoable alterative hypothei. While it i eay for the experimeter to cotrol Type I error, ad the experimeter may wih to perform a coervative tet (miimizig the probability of a Type I error), thee tet may uffer from a complete lack of power to dicrimiate betwee the teted hypothei ad reaoable alterative hypothee. If thi i true the the bet olutio may be to allow higher Type I error i order to icreae the power of the tet veru fixed alterative. I uch itace α value of 0., 0., or eve 0.3, may be reaoable. However, give a properly deiged experimet i which the power of the tet ha bee ivetigated for pecified alterative for pecified α value, ay deired power ca be obtaied imply by providig a adequate ample ize, eve though thi may be very expeive. Give a iitial prelimiary urvey the eceary ample ize to dicrimiate a particular alterative hypothei ca be etimated. Gree (979, p.43) outlie uch a procedure for a x factorial experimet i which a tet of the iteractio betwee time ad treatmet wa the tatitical hypothei of iteret. D. The tatitical hypothei tet ad it relatiohip to rulig hypothee or theorie I practice our hypothei tet uually ivolve a pecific et of obervatio regardig the effect of a particular factor (ay, oil moiture) o a particular repoe variable (ay, crop yield). If our teted hypothei i i the form of a ull hypothei, H 0 : Soil moiture cotet ha o effect o crop yield, a logical alterative hypothei might be that; H a : Soil moiture cotet effect crop yield. Two outcome are poible from our tet, we ca either reject or fail to reject the teted hypothei. If we reject our ull hypothei, ad our obervatio were derived from cotrolled experimet o that oly oil moiture wa allowed to vary amog replicate field, the we might afely coclude that oil moiture doe affect crop yield, or at leat that our meaure of oil moiture ad crop yield would ugget o. However, if we fail to reject our ull hypothei we caot coclude that oil moiture ha o effect o crop yield, rather oly that from the data at had oe i ot compelled to poit that it doe. Poibly a better-cotrolled or deiged experimet would have ucceeded i falifyig the teted ull hypothei. How do thee experimetal outcome relate to the geeral hypothei that oil moiture effect plat growth ad yield? I the itace where we rejected the ull hypothei we have demotrated that thi appear to be true for oe crop, give our experimetal deig. I the itace of failig to reject the ull hypothei we are back to the drawig board to deig a more critical experimet. I either evet, a igle experimet i iufficiet to lead u to believe that the truth cotet of the alterative or teted hypothei i high. Repeated attempt to reject or failure to reject uch a hypothei with more ad more critical experimet are eceary to etablih it veriimilitude. It i eldom that a igle experimet will have a major impact o how cietit i a diciplie view a rulig theory. 5

6 II. A Typology of Evidece A. No-experimetal reearch. Data-dredgig Data dredgig, a decribed by Selvi ad Stuart (966), occur i the proce of examiig data et that are ofte collected for other purpoe. If a hypothei ad a hypothei tet are tated prior to the examiatio of the data, ad the reult of the tet, regardle of the outcome are reported, the data dredgig ca be ueful ad reult i a coiderable avig i effort. Why collect more data to tet a hypothei if a adequate et already exit? However, if a pecific quatitative hypothei i ot tated i advace, but rather emerge durig the data aalyi, perhap alog with a ovel "tet" variable, the the tregth of the tet i compromied, ice a mechaitic explaatio for the reult mut alo be developed a poteriori. I the iitial tage of tudy o a ew topic thi may be a ueful proce to help develop ew hypothee ad to formalize critical hypothei tet. But, whe the topic ha bee the ubject of much tudy, a pecific a priori hypothei hould be available for tetig. are mut alo be take whe egagig i three other type of data dredgig; "oopig ", "fihig", ad "hutig. Soopig i tetig a large et of hypothee. The problem arie ice ome tet are expected to be igificat by chace aloe, ad becaue the hypothee may ot be idepedet. Thee problem alo occur with experimeter geerated data et. Fihig i chooig tet variable baed o a examiatio of the data rather tha becaue of their importace to a a priori hypothei. Alo, by relegatig variable to two clae, thoe choe ad thoe dicarded, the iterpretatio of the reult are clouded. I the abece of a pecific a priori hypothei, why ot report the tet for all variable? Hutig i the proce of earchig through may data et to fid ome relatiohip worth tetig. We ever kow how may data et were foud ot to diplay the deired relatiohip ice egative reult are eldom reported.. Ucotrolled Obervatio By ucotrolled obervatio I mea obervatio o a tet variable uder experimetal coditio which caot be compared to a et of obervatio obtaied o the ame tet variable i the abece of the experimetal coditio. Ucotrolled obervatio are ofte, but ot alway, experimeter geerated. They arie either becaue of poor experimetal deig, or becaue of the ature of the problem uder tudy. A example of thi latter problem ca be ee i tudie of tred i global atmopheric chemitry. 6

7 Thee tudie potulate recet athropogeic chage i atmopheric chemitry baed o obervatio o preet coditio ad kowledge of the icreae i athropogeic iput. The oly available cotrol obervatio are theoretical or budgetary predictio removig the athropogeic iput, or a recotructed fragmetary hitorical ad prehitoric record. While thee might be the oly ort of "cotrol" obervatio poible i thee circumtace, the abece of good etimate of experimetal ad cotrol parameter ad their variace make rigorou hypothei tetig difficult. Similar problem plague explaatio of the impact of itroduced plat ad aimal o ative populatio. Uually o iformatio i available o populatio tred i the ative biota prior to the itroductio of the exotic pecie. The oly cure for ucotrolled obervatio i to geerate cotrolled obervatio. Thi ca be accomplihed by repeatig a experimet with proper cotrol, if poible, geeratig theoretical expectatio of the cotrol obervatio by either determiitic model or by Mote arlo procedure, or by eekig obervatio that may be coidered "atural cotrol." While the firt of thee three alterative i mot deirable, it i ofte ot poible. Uig theoretical expectatio i oly a reaoable a the exitig theory. The ue of atural cotrol i plagued with the problem of determiig if the o called "atural cotrol" obervatio were recorded uder coditio that truly differ from the experimetal coditio i oly that the experimetal coditio are abet. Ecologit have tried for the lat two decade to ue atural cotrol i o called "atural experimet". However, i mot cae o effort wa made to demotrate the imilarity, i all but the experimetal coditio, of the cotrol ad experimetal obervatio. B. Experimetal Evidece Experimetal evidece i data collected by the experimeter for the expre purpoe of awerig a particular quetio or to tet a particular hypothei. I do ot mea to ugget that all experimetal evidece i created equal a a bai for caual iferece. I fact, I ca dicer everal kid of experimet that i the order I will preet them repreet a icreaig degree of itervetio o the part of the experimeter ito the workig of ature. Ad, I believe a icreaig ability to itimately coect caue ad effect.. otrolled obervatio otrolled obervatio are collected by deig to tet a particular hypothei. The deig iclude ample uder the experimetal coditio of iteret ad uder putative cotrol coditio (lackig the experimetal treatmet). However, the obervatio are derived from a amplig program that ivolve ature oly paively. The oly activity of the experimeter i to make the obervatio, aalyze the data, ad iterpret the reult. For example, I coducted a amplig program to determie if the amout of folivory o White Oak tree i related to the timig of leafig ad leaf developmet i the prig. Much of the folivory that occur o woody plat, i geeral, occur durig prig whe leave are youg ad upple. If the amout of damage 7

8 received by Oak tree i determied by the age of the foliage relative to the emergece time of leaf feedig iect, the tree that either foliate ufficietly prior to iect emergece ad feedig, or after iect emergece may receive le damage. To tet thi hypothei, I ampled group of tree imilar i ize, but differig i the timig of foliatio. Three clae of foliatio were etablihed, early, mid, ad late, which by makig iter-compario erve a correpodig cotrol a well (mid ad late erve a cotrol for early, etc.). I thi "experimet", I have iterveed oly to record my obervatio o folivory ad foliatio. I have either altered the leafig time of the tree to oberve the ubequet damage received, or have I maipulated the herbivore to create or detroy a patter of ychroou emergece ad foliatio. The key to ditiguih cotrolled obervatio from a more elaborate experimet i the paive role of both ature ad the experimeter. I a much a the experimetal obervatio have good cotrol obervatio, uch amplig program ca be a reaoable bai for caual iferece. However, the dager exit that the cotrol obervatio are ot true cotrol, ice the experimeter may ot be able to iure that ubject ued for experimetal obervatio differ oly i the experimetal treatmet from the ubject ued i the cotrol obervatio. I the above example it i coceivable that ome apect of leaf chemitry, either utritioal quality, the cocetratio of volatile compoud (which may be ued i hot locatio ad timulate feedig) or the cocetratio of chemical that deter feedig may co-vary with time of foliatio. Thee chemical characteritic may be the proximal caual aget repoible for ay oberved relatiohip betwee foliatio ad folivory. Foliatio time may either affect thee characteritic or imply, a I aid, co-vary with them. I either itace, further experimet would be eceary to determie the actual caual mechaim. Therefore, although more covicig tha iferece baed o ucotrolled obervatio it i till difficult to bae firm caual iferece purely o cotrolled obervatio.. Meurative Experimet The ext tep i experimetal itervetio I call meurative experimet (eu Hurlburt 984). They ivolve the experimeter ad a part of ature a bit more actively i the hypothei tet, but oly to paively meaure aother part of ature. A commo example of a meurative experimetal techique i ecology i the ue of litter bag to examie the rate of litter decompoitio i aquatic eviromet or o the foret floor. The experimeter package a bit of ature i thee litter bag (which have meh that allow coloizatio by bacteria ad ivertebrate who alog with chemical weatherig are repoible for the litter decompoitio) ad expoe the bag uder differet experimetal coditio or time to determie if igificat variatio i decompoitio rate i detectable betwee the experimetal coditio of iteret. I thi cae, the experimetal itervetio ito the workig of ature i olely to create a replicable amplig device with which to paively meaure a atural proce. 8

9 A with cotrolled obervatio, meurative experimet uffer becaue we caot uiquely aociate a caual mechaim with the variety of experimetal ituatio ito which we have placed our meurative device. Were we to fid differet rate of litter decompoitio i temperate ad tropical foret, to what would we attribute thee differece? The lit of reaoable caue i legthy. Oce agai further experimet are eceary to etablih the pecific caual mechaim ivolved. 3. Maipulative Experimet I a maipulative experimet, the experimeter may exercie total cotrol over a portio of ature to create all the deired experimetal ad cotrol coditio. To repeat the folivory experimet metioed above a a maipulative experimet would ivolve direct modificatio of the foliatio time ad expoure of all tree to idetical herbivore populatio, poibly via maive rearig ad releae of leaf-eatig iect. The litter decompoitio experimet would ivolve a erie of experimet i which everal factor uch a temperature, humidity, bacterial populatio, fugal populatio, ad ivertebrate populatio are cotrolled or allowed to vary igly or i combiatio to tet for imple effect ad iteractio of factor i determiig decompoitio rate. Obviouly, it i ofte eaiet to perform maipulative experimet i laboratorie or experimetal ecloure where there i ome hope of ucce i actually cotrollig the multitude of evirometal ad biological variable. If it i i ay way poible to perform a maipulative experimet, it i preferred over the previouly metioed kid of experimetal exercie. However, it i extremely difficult to perform thee kid of experimet uder field coditio. Maipulative experimet do, however, have a greater ability to aociate caue ad effect ice if properly deiged ad executed they remove the poibility of co-varyig potetial caual factor. Maipulative experimet, eve where logitically feaible, are ot without their problem. The mot importat of which i the dager of itroducig ome experimetal artifact via ome apect of the maipulatio. Thi problem alo beet meurative experimet, but to a leer degree ice the experimetal itervetio i ature i le dratic. III. Allocatio of amplig effort A. What i a amplig program to do? A tatitical populatio i the collectio of all elemet about which oe eek iformatio, or about which oe deire to make ome iferece. It i icumbet upo the experimeter to tate a priori the populatio of elemet about which they wih to make ome tatemet. It i crucial that thi populatio i defied i advace of deigig a amplig program or experimet. The reao for thi will be apparet, hortly. 9

10 Uually oly a mall portio, or a ample, of thi populatio ca actually be oberved. It i from data o the elemet of the populatio that are member of the ample that cocluio or iferece are draw about the characteritic of the etire tatitical populatio. Quatitie computed from ample data are commoly termed tatitic while thoe characterizig populatio are kow a parameter. Sample tatitic the erve two role: ) to decribe the data obtaied i the ample ) to etimate or tet hypothee about characteritic of the populatio Had we eumerated the value of a particular characteritic for all elemet i a populatio, ad the tabulated the frequecie with which the elemet of the populatio take o differet value, the reultig tabulatio would be the populatio ditributio of the character of iteret. The populatio ditributio ca be decribed by thi ort of eumeratio or by a erie of parameter. The umber of parameter eceary to decribe a particular ditributio deped o it form, but i i geeral a more parimoiou approach tha eumeratio. For example, the Poio ditributio ca be pecified by oe parameter ad the ormal ditributio by two parameter. If we are itereted i etimatig a populatio mea, µ, the ample mea, x, geerally provide a good etimate. Similarly the ample tadard deviatio,, provide a good etimate of the populatio tadard deviatio, σ. The preciio of thee etimate deped o four factor: ) the ize of the ample ) the maer of amplig 3) the characteritic of the uderlyig populatio 4) the priciple ued i etimatig the parameter If a ample i draw uch that: ) all elemet of the populatio have a equal chace of beig draw at all time, ad ) all poible ample of ize have a equal (or fixed ad determiable) chace of beig draw, the, the ample i a radom ample of ize from the uderlyig populatio. Of coure to meet the coditio the populatio mut be defied i advace or a amplig program that iure that all elemet have a equal chace of beig draw caot be deiged. 0

11 Radom ample iure that all elemet i a populatio are at equal rik of beig ampled ad that the probability of amplig ay idividual elemet from the populatio i idepedet of which other elemet may or may ot be ampled. Suppoe 0,000 ample each of elemet are draw from a populatio. Sample mea, x, ad variace,, could be computed from each ample. The tabulatio of the frequecie with which our ample tatitic take o differet value i the amplig ditributio of the tatitic. I thi itace, we have determied the ditributio empirically. The form of thee ditributio deped i part upo the amplig method. A with populatio ditributio amplig ditributio ca be decribed more ecoomically with parameter tha by eumeratio. Frequetly the parameter of the amplig ditributio of a tatitic are related to the parameter of the uderlyig populatio. The mea or average value of the amplig ditributio of a tatitic ad it tadard deviatio i the tadard error of the tatitic. The form of the amplig ditributio a well a the magitude of it parameter deped o: ) the form of the populatio ditributio, ) the maer of amplig, ad 3) the ize of the ample. If, for example, the uderlyig populatio i ormally ditributed ad the ample are radom ample, the if oe draw a large umber of ample, the amplig ditributio of the ample mea, x, will be approximately ormal with mea, µ, ad tadard error σ. Thi ame coequece i derivable mathematically baed o the propertie of radom ample. Thi i the importace of radom ample - they permit the etimatio of amplig ditributio from purely mathematical coideratio without eceitatig the laboriou kid of eumeratio I have metioed. The key apect of radom amplig which allow thi i that radom ample eure all elemet of the populatio are at equal rik of beig ampled ad the probability that ay igle elemet i ampled i idepedet of which other elemet are ampled. Statitic obtaied from ample draw by other amplig pla which are ot radom have amplig ditributio which are either ukow or which ca oly be approximated with ukow preciio. Good approximatio to amplig ditributio required if oe i to evaluate the preciio of the iferece made from ample data. Of coure, the etral Limit Theorem geeralize thi reult for populatio that are o-ormal. The amplig ditributio of a tatitic derived from a o-ormal populatio i alo approximately ormal ad the approximatio improve a the ample ize icreae. For the ample mea, x, it expected value i till, µ, ad it tadard error, i σ. I the cotext of hypothei tetig, the role of amplig i to eable the experimeter to dicover omethig about the amplig ditributio of the tatitic of iteret, uder the

12 experimetal coditio of iteret, baed o the uderlyig populatio of iteret. Thi i becaue the tatitical hypothei tet i baed upo the ample etimate of the parameter of the amplig ditributio of the tatitic, ot upo the ample etimate of the uderlyig populatio parameter. The cloe relatiohip betwee the parameter of the amplig ditributio of a tatitic ad the populatio parameter etimated by the ample tatitic ted to obcure thi fact. The tatitical hypothei tet ivolve a compario of amplig ditributio. Upo thi compario iferece about populatio ditributio ad their parameter ca be made. B. Bia, preciio ad radom amplig So we ample to etimate populatio parameter ad to lear through kowledge of the amplig ditributio of our tatitic jut how good our etimate are. Two criteria are commoly ued to judge the accuracy of a etimate: bia ad variace. A tatitic i a ubiaed etimate of a parameter if the expected value of the amplig ditributio of the tatitic i equal to the parameter of which it i a etimate. Bia therefore, i a property of the amplig ditributio ot of a igle tatitic. Thi implie that i the log ru the mea of a tatitic computed from a large umber of ample of equal ize will be equal to the parameter, if it i ubiaed. I additio to iurig that the elemet icluded i a ample are idepedet, radom amplig alo help to prevet biaig etimate of populatio parameter. If all elemet i a populatio were ot at a equal rik of beig ampled it i eay to ee that value ytematically above or below the true populatio value may be repreeted diproportioately i the ample. The preciio of a etimator i meaured by the tadard error of it amplig ditributio. The maller the tadard error the greater the preciio. The tadard error i oly a good meaure of preciio if the amplig ditributio i aymptotically ormal. If thi i true, the the bet-ubiaed etimator i the oe with the mallet tadard error. Thi i called a miimum variace ubiaed etimator. Icreaig ample ize will alo icreae the preciio of a etimator.. The prelimiary urvey Oce the tatitical populatio of iteret ha bee defied, the attribute to be examied are elected, ad the experimetal coditio decided upo, the experimeter i left with the tak of decidig where to ivet amplig effort. Firt ad foremot, thi i dictated by the quetio of iteret. Aumig limited reource, there i o reao to exped extra effort to tet acillary hypothee that are ot of preig iteret. It i eay to compromie all the hypothei tet you wih to perform by attemptig to deig a all ecompaig amplig pla which allow you to tet everal hypothee, but oe with ay power. You jut caot awer all the importat quetio i biology i oe MS. or Ph.D. thei. Believe me, I tried. State the quetio you wih to awer ad rak them i importace. If the cot of ample collectio or proceig i time or moey, or the iheret variatio i the attribute of the populatio you wih to tudy are high the pare dow the umber of quetio o that at leat

13 ome ca be awered with adequate cofidece ad power. Secod try out your amplig gear to ae it accuracy ad to etimate the cot per ample. I ature you ca ret aured that appearace will be deceivig ad that field work alway cot more i time ad moey tha aticipated. If you are uig ome ort of amplig gear that you caot ormally oberve durig it operatio, try to oberve it behavior at leat oce. If there i ay ubjective compoet to ample electio or ay other apect of the collectio, ortig, or eumeratio of ample have more tha oe oberver repeat the ame procedure to ee if ay ytematic bia i beig itroduced. Third, carry out a prelimiary urvey o that you ca etimate the amout of variatio to be expected uder each et of experimetal coditio. If you kow where the variatio lie i your ubject populatio you ca icreae your replicatio to improve the preciio of your etimate ad thereby (by decreaig the tadard error of the amplig ditributio) icreae the power of your tet agait fixed alterative for fixed value of α. D. Optimal allocatio of amplig effort A I metioed before, the preciio of our etimate of populatio parameter deped upo the form of the populatio ditributio, the maer of amplig, ad the ize of the ample. The experimeter ha cotrol oly over thee lat two apect. So the allocatio of amplig effort mut ivolve variatio i the maer of amplig ad the ize of the ample. Sample ize Icreaig ample ize will icreae the preciio of our etimate by decreaig the tadard error of our ample tatitic. Icreaig ample ize hould ot decreae our etimate of the tadard deviatio of the uderlyig populatio. For fixed α, thi icreae the power of a hypothei tet agait all alterative. For example, the ample ize required to be 95% cofidet that our etimate of the ample mea lie withi a allowable error, L, of the true populatio mea i: = 4σ L where i ample ize ad σ i etimated by the ample tadard deviatio. Icreaig preciio i yoymou with decreaig the allowable error, L, ad for fixed cofidece we mut icreae to achieve icreaed preciio. Samplig maer The reult cocerig amplig ditributio that I metioed earlier hold for other type of amplig tha jut imple radom amplig. It i ufficiet for the amplig 3

14 method to ample all elemet idepedetly ad with kow probabilitie. Thee probabilitie eed ot be equal for all elemet of the populatio (a i imple radom amplig), a log a we take accout of thee probabilitie whe cotructig our etimate. Samplig pla that follow thee criteria are kow a probability amplig. Simple radom amplig beig the mot commo of thee. Two other commoly ued method of probability amplig are tratified amplig ad -tage amplig. Stratified amplig ivolve dividig a populatio ito a umber of part, called trata, drawig imple radom ample from each trata, ad computig the parameter of iteret a a weighted mea of the parameter etimate from each trata. For the ample mea we have x t = k h= N N h x h, where i the total umber of elemet i the hth tratum, x h i the ample mea for the hth tratum ad k N h h= i the ize of the populatio. Stratified amplig i ueful becaue differece betwee trata mea do ot cotribute to the tadard error of the mea, x. That i, the amplig error arie olely becaue of variatio amog elemet withi trata. If we ca tratify a otherwie heterogeeou populatio ito trata which are fairly homogeeou, we ca icreae the preciio of our etimate over that achievable by imple radom amplig. The ize of the ample we chooe i ay tratum i determied by the experimet. Thi freedom of choice allow the experimeter to allocate amplig effort efficietly. Thi cotrol over the allocatio of amplig effort i ofte the pricipal reao for the gai i preciio derived from tratificatio. If equal fractio of the elemet i each tratum are ampled the weightig factor are equal for all trata ad we eed ot modify our ample tatitic to accout for the uequal probabilitie of amplig elemet i differet trata. Thi i kow a tratified amplig with proportioal allocatio of amplig effort. The optimum allocatio of amplig effort i a tratified deig i ot ecearily a proportioal allocatio program where h /N h i equal for all trata. Rather the optimal olutio i to take h elemet proportioal to N σ h h h, where σ h i the withi tratum tadard deviatio, ad h i the cot per ample i the hth tratum. Thi method give the mallet tadard error of the etimated ample tatitic for a give total cot of amplig. I other word, take a larger ample i a tratum that i uuually variable (σ h large), ad a maller ample where amplig i uuually expeive ( h large). If the withi trata tadard deviatio are all approximately equal ad the cot of amplig i each trata i alo equal the thi method reduce to the method of proportioal allocatio. Of coure, i order to allocate effort optimally rough etimate of tadard deviatio ad cot mut be made. 4

15 Two tage amplig I a two tage amplig program the ample i derived by firt collectig a ample of primary amplig uit, ad the by ub-amplig withi each of thee uit. The oak tree experimet I decribed earlier i a example of a two-tage amplig program. The primary uit are the tree elected from the foret ad the ub uit are the leave or leaf cluter ampled withi a tree. Sometime two-tage amplig i the oly practical amplig method. O a live oak tree 5 meter i height I oce couted 4,000 leave o jut oe brach. Obviouly eumeratig all the leave o the tree would be very tediou. I geeral, it i eay to ample the primary amplig uit but difficult to ample the ub-uit. The obervatio o each ub-uit i coidered to be the um of two idepedet term. Oe term, aociated with the primary uit, ha the ame value for all ecod-tage uit i the primary uit, ad varie from oe primary uit to the ext with variace σ. The ecod term, which erve to meaure differece betwee ecod tage uit varie idepedetly from oe ub-uit to the ext with variace σ. If a ample coit of, primary uit from each of which ub-uit are draw, the the ample a a whole cotai idepedet value of the firt term ad value of the ecod term. The variace of the ample mea, x, per ub-uit i: σ σ σ x = +. Thee two compoet of variatio ca be etimated from a aalyi of variace. MSPrimary Uit - MS Sub - uit σ = =, = = ( M Sub - uit) σ, σ. x = x = + Therefore, we ca juggle the umber of primary ad ecodary uit to miimize But what choice of value i bet? Naturally the awer to thi quetio deped o the relative cot of primary ad ecodary amplig uit. If the cot aociated olely with amplig primary uit i ad the cot aociated with amplig ecodary uit i the the total cot ( T ) of a -tage program i x. T = + If advace etimate of thee idividual cot ad of the variatio due to each amplig tage are kow the oe ca allocate amplig effort to miimize the tadard error of 5

16 6 the tatitic of iteret for fixed cot, or to achieve a deired preciio of our etimate by miimizig the product ( ) V T + + = where V i the variace of the ample mea i thi cae ad T i total cot. Sice drop out of thi expreio we ca olve for the value of that miimize thi expreio: ( ) ( ) 0, = = V T 0 = + + = = = The for kow total cot T, + = T, ad for kow total variace V, ( ) [ ] { }V + =. Therefore, the value of required for a optimal allocatio of amplig effort ca be obtaied, ad a imilar value for ca alo be obtaied cotiget o beig able to pecify the total cot or variace deired. IV. Experimetal Deig I have tried to illutrate that the goal of a amplig program i both to produce ubiaed etimate of populatio parameter ad to lear omethig about the amplig ditributio appropriate for the uderlyig populatio ditributio. Alo, the reao for chooig a particular amplig program i to improve the power or eitivity of the tatitical hypothei tet motivatig the amplig.

17 I tetig a tatitical hypothei oe ue amplig ditributio which are largely choe for mathematical coveiece (i.e., whoe form ca be pecified if certai precoditio are met by the amplig program). Oe propoe a model, impoe pecific coditio upo the model, ad derive the model' coequece i term of amplig ditributio which are valid give the propertie aumed for thee amplig ditributio. To the extet that the model ad coditio impoed upo it approximate the actual experimet, the model ca be ued a a guide i drawig iferece from the data. To ue model that allow the propertie of the amplig ditributio to be pecified i advace, the experimet mut be deiged to meet the precoditio aociated with the particular model. If a experimet doe ot meet the pecificatio of exitig model, the experimeter may be able to develop a model tailored to the pecific experimet. However, the reultig data mut till be aalyzed. If amplig ditributio with kow ad maageable characteritic appropriate for a experimet ca be derived, the pecific model ca lead to iferece with kow preciio. Without kowledge of the propertie of the appropriate amplig ditributio, iferece draw from a experimet have ukow preciio. The aalyi of experimetal data i depedet upo the experimetal deig ad the amplig ditributio appropriate for the uderlyig populatio ditributio. The deig, i part, determie what the amplig program will be. For tadard deig the amplig ditributio eceary to tet the hypothee of iteret have kow ad maageable propertie (i.e., aymptotic ormality), which lead to the widepread ue of thee deig. Alterative deig are ofte available for a experimet havig pecified objective. Depedig upo the pecific ituatio, oe deig may be more efficiet - that i have power i the aociated tet ad arrower cofidece iterval - for a give amout of experimetal effort. The goal i plaig experimet i to fid the deig that i mot efficiet per uit effort relative to the primary objective of the experimet. Icreaig ample ize, improvig meauremet techique, ad itroducig variou kid of cotrol all may decreae experimetal error ad therefore improve power. Which method reult i the greater icreae i power for a give uit of effort will deped upo coditio uique to each experimetal ituatio. A examiatio of purely tatitical apect of experimetal deig will help the experimeter fid the model bet uited for their experimet. The model choe hould allow the experimeter to reach deciio regardig all the objective of the experimet. Whether or ot a particular model actually correpod to a pecific experimetal ituatio require a i-depth kowledge of the ubject matter addreed by the experimet. A careful aemet of the adequacy of alterative model may lead the experimeter to more fully udertad the ource of variatio iheret i the experimet. Thi may ultimately lead to a better deig ad therefore a more clear-cut iterpretatio of the experimetal reult. 7

18 Five criteria for evaluatig experimetal deig ca be tated.. The model choe ad it uderlyig aumptio hould be appropriate for the experimetal material.. The deig hould provide a much iformatio a poible with regard to the major objective of the experimet for a give amout of experimetal effort 3. The deig hould provide ome iformatio with regard to all the experimetal objective. 4. The deig mut be feaible withi the workig coditio that exit for the experimeter. 5. The aalye baed upo the deig hould provide uambiguou iformatio o the primary objective of the experimet. I the followig dicuio everal broad categorie of experimetal deig will be preeted. The beefit ad cot of chooig oe particular category of deig over aother will be examied. A. Factorial Deig Factorial experimetal deig ivolve the compario of the effect of two or more factor actig imultaeouly o a commo repoe or criterio variable. A factor ca be coidered a et of related treatmet or related claificatio. Each member of the et of related treatmet belogig to Factor A i coidered a level of Factor A. The pricipal advatage of uig a factorial deig veru a erie of igle factor experimet i that it allow oe to examie the effect of the iteractio of each factor combiatio o the criterio variable. The preece of a iteractio effect attributable to the combiatio of factor above ad beyod the effect of the factor igly ca be determied. However, the additioal effort eceary to tet a hypothei of iteractio ca be coiderable. For example, if five replicatio are made at each level to tet for the effect of - four level factor igly the uch a deig require a total of 40 replicatio. To tet for the effect of - four level factor ad their iteractio require 80 replicatio. If prior iformatio idicate that o iteractio exit, a factorial deig will ot be a ecoomical a everal igle-factor deig. Figure -4 illutrate the data layout ad aalyi of variace for a igle factor ad a two-factor factorial experimet. I each itace a equal umber of idepedet ad radomly ampled elemet are ampled at each factor level or combiatio of factor level. Fully factorial deig, thoe with o cofoudig betwee factor ad idepedet obervatio at all factor level, are the mot commo ad widely ued deig. Other kid of factorial experimet are ometime ueful. 8

19 Figure. Sigle Factor Factorial Experimet - data layout Treatmet Treatmet Treatmet k X X X 3 X X X X 3 X X k X k X 3k X k Figure. Sigle Factor Experimet - ANOVA Table Source of SS df MS F Variatio Treatmet SS treat k- SS treat /(k-) MS treat /MS error Error SS error k-k SS error /k-k Total SS total k- SS total /k- Figure 3. Two Factor Factoral Experimet - data layout Factor B Factor A Level Level Level p Level X X X 3 X X X 3 X p X p X p3 x Level X X X 3 X X X p Level 3 Level r X r X rp X r X rp X r3 X rp3 X r X rp 9

20 Figure 4. Two Factor Factorial Experimet - ANOVA Table F-Ratio Source of Variatio SS df MS Model I Model II Model III (A fixed, B radom) Factor A SS A p- SS A /(p-) MS A /MS e MS A /MS AB MS A /MS AB Factor B SS B r- SS B /(r-) MS B /MS e MS B /MS AB MS B /MS e AB SS AB (p-)*(r- SS AB /(p-)*(r-) MS AB /MS e MS AB /MS e MS AB /MS e Iteractio ) Withi cell SS e pr*(-) SS e /pr*(-) (error) Total SS total Occaioally i executig a igle-factor experimet a limited umber or amout of primary amplig uit are available to receive the experimetal treatmet. Thoe that are available may ot be coidered trict "replicate" becaue ucotrolled variatio exit betwee primary uit prior to the experimet. I order to icorporate eough replicate for each experimetal treatmet it i ofte eceary or maybe eve deirable to ue more that oe primary uit. The bet deig i thi ituatio i a radomized complete block deig. Each primary uit i coidered a block ad each treatmet i radomly aiged to ub-block withi each block. A -factor aalyi of variace i performed with block a oe factor, i order to remove variatio due to block from the experimetal error. If hypothei tet are oly performed o the treatmet effect the the blockig factor ca be coidered a fixed factor. If aalyzed i thi maer the treatmet block iteractio i implicitly coidered to be zero. B. Neted Deig Three kid of eted deig are ued i agricultural ad pychological reearch ad have may applicatio i biology. Thee deig are hierarchical, plit-plot, ad repeated meaure. The primary purpoe of thee deig i to elimiate ucotrolled variatio due to a priori differece i primary amplig uit from the etimate of experimetal error. I thi ee we ca ee that thee deig are a way to remove cofoudig variatio by addig claificatory cotrol or trata. Aother reao for the ue of thee kid of eted deig i biological reearch i that we ofte wih to make iferece cocerig hierarchically arraged eviromet, habitat, ad pecie..hierarchical Factor oider the example depicted i Figure 5, igorig for the momet the high ad low marh categorie. We wih to tet the hypothei that ome characteritic, ay above groud bioma, doe ot differ betwee etuarie. We have etimate of bioma per marh i each etuary. Our Factor B, marhe, i ot completely croed with Factor A, 0

21 etuarie, ice o marh i foud i both etuarie. The marh factor i eted withi each level of the factor etuarie. Sice all level of the factor marh do ot occur i combiatio with all level of the factor etuarie, we caot examie the effect of a marh by etuary iteractio. The degree of freedom ad SS for etuarie ca be computed a i a ormal -Way ANOVA. The SS for marhe i computed a the um of the SS marhe withi level of factor A ad the SS marhe withi level of Factor A. The degree of freedom for each of thee compoet i (q - ) where q i the umber of marhe i each etuary. If marhe are coidered a radom factor the F ratio to tet the hypothei that σ α = 0 i F =MS A /MS B. If marhe i a fixed factor the F i MS A /MS W. Deig with more level of etig are poible. If we iclude data o bioma at locatio high ad low i each marh the reultig deig i a partially hierarchical deig. The high-low factor i ot eted withi marhe or etuarie, but rather completely croed with them. If we coider the etuary ad high-low factor fixed ad marhe radom the thee hypothee may be teted: a etuary effect, a high-low effect, ad a high-low/etuary iteractio. A outlie of the degree of freedom, mea quare, ad F ratio are give i Figure 6. Note that the withi cell variatio ha bee partitioed ito two orthogoal compoet which are ued a error term to evaluate differet hypothee. Figure 5. Neted Aalyi of Variace - Data Layout Etuary Etuary Etuary 3 Marh Marh Marh 3 Marh 4 Marh 5 Marh 6 Marh 7 Marh 8 Marh 9 High Low

22 Figure 6. Neted Aalyi of Variace - ANOVA Table Data Layout Source of Variatio SS df MS F Etuarie SS Etuarie (p-) SS Etuarie (p-) MS Etuarie MS Marhe w Etuarie Marhe withi Etuarie SS Marhe w Etuarie p*(q-) SS Marhe w Etuarie p*(q-) High-Low SS High-Low r- SS High-Low (r-) MS High-Low MS Marhe w (Etuary by High-Low) Etuary by High-low iteractio SS Etuary by High-Low (p-)*(r-) SS Etuary by High-Low (p-)*(r-) MS Etuary by High-Low MS Marhe w (Etuary by High-Low) Marhe withi (Etuarie by High - Low) SS Marhe w (Etuary by High- Low) p*(q-)*(r-) SS Marhe w (Etuary by High-Low) p*(q-)*(r-). Split Plot Deig Split-plot deig are equivalet to repeated meaure deig ad are ued widely i agriculture. They are ueful whe oe of the treatmet i more difficult to apply tha the other, or at leat that oe-treatmet i eaier to apply at a larger cale. Figure 7 depict the layout of a plit-plot deig. Thi deig i imilar i form to the radomized complete block deig except that i thi itace a treatmet level i applied to each whole block or plot. Withi each whole plot, each level of a ecod treatmet i radomly aiged to ub-plot. The effect of factor A are cofouded with differece betwee whole plot while the effect of factor B are part of the withi plot variatio. The etimate of the effect of Factor B are free from variatio due to whole plot. The iteractio betwee Factor A ad B i alo free from whole-plot effect. The aalyi of thi deig i outlied i Figure Repeated Meaure Deig I repeated meaure experimet obervatio are made o the ame ubject at all level of at leat oe factor. For example, the paired t - tet ca be coidered the implet itace of a repeated meaure deig. Each ubject i oberved before ad after the applicatio of ome treatmet. The advatage of uch a deig i that the ubject act a a elf-cotrol. Variatio betwee ubject that occur for reao urelated to the experimet ca the be removed from oe etimate of experimetal error. Thi may lead to a more eitive tet of the hypothei of iteret. For example i a t - tet with u-correlated obervatio (o repeated meaure) the etimate of