Enhancing the Quality of Price Indexes A Sampling Perspective

Transcription

1 Enhancng the Qualty of Prce Indexes A Samplng Perspectve Jack Lothan 1 and Zdenek Patak 2 Statstcs Canada 1 Statstcs Canada 2 Abstract Wth the release of the Boskn Report (Boskn et al., 1996) on the state of U.S. prce ndexes, there has been an ntense debate on ways to mprove ther qualty. In Canada and the U.S., and certanly n other countres, there have been questons rased about the conceptual underpnnngs, the defntons and the manner, n whch the data are collected and processed. Major strdes have been made n assessng the applcablty of standard busness survey methodologes n an area that has long been domnated by judgment and subject-matter expertse. In the U.S., n partcular, the coverage, collecton and samplng processes have experenced major nnovatons. Wth the growng mportance of the servce sector, Statstcs Canada s developng a new set of Servce Producer Prce Indexes that may ncorporate many of these nnovatons. Statstcs Canada s usng ths opportunty to nvestgate ssues such as mprovng the frame and sample coverage, ntroducng two-stage random samplng technques, mprovng outler detecton technques and ntroducng estmaton of samplng varance. Ths paper sketches out our progress to date wth emphass on the samplng desgn, n partcular, a comparson of PPS (probablty proportonal to sze) versus SRS (smple random samplng). The results of the smulaton study favour PPS but further research s requred to gan a more thorough understandng of how ths advantage s affected by dmnshng sample sze, non-response, and outler treatment. Hence, the paper also outlnes some possbltes for future development. Key Words: Prce Index, Smulaton Study, Probablty Samplng 1. Introducton One of the mandates of a central statstcal agency s to collect nformaton for the complaton of the natonal accounts, whch n turn s used to measure changes n the well-beng of the natonal economy. To acheve ths goal, a number of busness surveys gather data on economc nputs and outputs at current dollar values. To evaluate real changes n the state of the natonal economy, one must elmnate the effects of nflaton by convertng the current dollar values to constant (deflated) dollars based upon a standard base year s prces. Prce ndexes are a tool that s used to convert all nputs and outputs collected n current dollars to constant dollars for a selected pont n tme. A Servces Producer Prce Index (SPPI) measures the rate of change n the prces of specfc servces bought and sold by producers wthn a specfc ndustry group. In theory, thousands of these ndexes mght exst and t s the natonal accountants responsblty to apply them to the nformaton derved from busness surveys. In practce, the natonal accountants reduce the complexty of the task by assumng that a specfc ndex s representatve of a group of commodtes wthn a group of ndustres. Defnng the groupng procedure s a complex task that combnes knowledge of data avalablty, conformty wth nternatonal standards and the natonal accountants judgment. Collectng data for a specfc SPPI s an mportant matter. In practce, SPPIs adopt a two-stage samplng procedure that samples representatve products from a selected sample of representatve establshments wthn an ndustry group. The prce changes for these sampled products are then collected for a selected tme perod and a weghted average for the producer group s computed. It s assumed that out of the mllons of transactons takng place durng the reference perod these selected transactons wll represent the average prce changes experenced by producers wthn the ndustry group. Once frame and samplng strateges have been establshed, one must defne formulas for computng prce changes at the sample unt level and at the aggregate group levels. Because prce changes are ratos, the defnton of the aggregate rato s not straghtforward. The relatonshp between the unt ratos and the aggregate ratos s non-lnear, and mathematcal operatons are often non-transtve and non-commutatve. Also, dfferent formulas can have dfferent economc propertes and nterpretatons. In addton to the above ssues, there s much debate gong on concernng the how to ncorporate nformaton nto SPPIs on product creaton, product obsolescence, and the effect of changes n qualty of 1454

2 the product. Overall, estmatng SPPIs s a challengng task. Snce ts ncepton n the late 1800s, prce ndex estmaton has been domnated by judgmental and heurstc methodologes. Startng n the md-1900s, scentfc methodologes such as samplng started to enter the feld and most central agences contnue to move away from judgmental strateges. In the last 50 years, major strdes have been made n ntroducng standard busness survey methodologes nto prce ndex estmaton. The 1980s and 1990s were a tme of great turmol n the feld, partcularly for the consumer prce ndex (CPI). Throughout the world, renowned economsts were crtczng the foundaton and methodologcal underpnnngs of the current methods for calculatng prce ndces. In response to these crtcsms, techncal experts from around the world were brought together to dscuss these ssues and offer new drectons. As a result of these dscussons, many natonal statstcal agences are revtalzng ther prce ndex estmaton strateges. New ndexes are beng ntroduced, new data sources are beng developed to enhance the coverage and scentfc statstcal methodologes are beng augmented. As survey methodologsts, the area that concerns us the most s mprovng the qualty of our frames, ncreasng the use of scentfc probablty samplng strateges, and ntroducng qualty measures. By means of a smulaton study, ths paper compares two probablty samplng strateges, stratfed smple random samplng wthout replacement (SSRSWOR) and probablty proportonal to sze samplng, commonly used n practce n the context of estmatng a prce ndex. The study was done to answer the queston of whether SSRSWOR presents a vable alternatve to PPS samplng, whch s the domnant probablty samplng strategy used for prce ndex estmaton. In the next secton, some of the advantages and dsadvantages of SSRSWOR versus PPS samplng wll be dscussed. In Secton 3, the survey used for buldng the smulaton populaton wll be descrbed. Sectons 4 and 5 present the manner, n whch the smulaton populaton was constructed and the smulaton study was carred out. Some observatons and concludng remarks are offered n Secton SSRSWOR or PPS Samplng? Samplng strateges can vary markedly wth systematc, smple random, PPS, judgmental and purposve samplng all beng used wthn the same agency. In addton, each of these strateges could be used ndependently n the two stages, plus at tmes the frst or second stage may be deleted. The frames used by these samplng strateges can vary strkngly as well. Ths s partcularly true for the samplng of commodtes at the second stage where judgment s frequently used to select commodtes. The current redesgn of prces programs around the world faces many challenges worthy of several papers and t s dffcult to do justce to the subject n a sngle paper. The area that we want to focus on s whch samplng strategy s the most approprate for prce ndex estmaton, SSRSWOR or PPS samplng (Ohlsson, 1998). Each strategy has ts merts. SSRSWOR s easy to understand and thus t s less of a black box. In addton, there s a wealth of classcal survey lterature documentng how one should mplement a two-stage samplng procedure and how to calculate the varances assocated wth ths strategy. PPS samplng s man advantage s that t s selfweghtng. Economc theory states that the change n the measured prce must be economcally weghted by the frm s or the commodty s contrbuton to ndustry output. The PPS samplng weghts are typcally the nverse of these economc weghts and thus these two weghts cancel. Ths mples that the pont estmate s a smple average, and that the estmaton formulas are decomposable at all levels of aggregaton. Another advantage of PPS samplng s that t reduces sgnfcantly the response burden mposed on small frms. The complexty of prce ndex formulas and the fact that hstorcally most samples have been judgmental n nature have not been conducve to computng varances of prce ndces. Qualty ndcators assocated wth estmates based on judgmental samples are more lkely to follow smple rules of thumb rather than a theoretcally sound procedure. In the absence of sold qualty measures, judgng the precson of an ndex becomes elusve. Movng away from non-probablty samplng strateges s the frst step on the way to beng able to assess the qualty of prce ndces n a scentfc manner. Comparng the propertes of SSRSWOR and PPS samplng methods of selectng random samples for estmatng a prce ndex has the added beneft of 1455

3 allowng the methodologst to evaluate sample sze versus precson consderatons. There are many low level ndces currently n producton that are based on a sample of 10 or fewer (ths s typcal for prce ndex samples used by statstcal agences around the world) unafflated prce quotes at the end of the producton cycle. Ths s often deemed suffcent to compute an ndex of publshable qualty. 3. Wholesale SPPI The frst n the seres of new SPPI surveys that Statstcs Canada s developng s the Wholesale servces ndex. It was started two years ago and has recently entered ts sxth cycle. The target populaton conssts of all busnesses operatng n Canada that have at least one establshment wthn ther structure coded to one of the n-scope North Amercan Industry Classfcaton System (NAICS). To reduce response burden and to restrct unts wth a neglgble mpact on ndex calculaton from enterng the sample, the establshments wth the smallest revenue that contrbute a maxmum of 5% to the populaton revenue were elmnated from the target populaton. Ths reduced the target populaton to a survey populaton of approxmately 3 wholesale establshments. The survey populaton was stratfed by trade group (a collecton of 3- and 4-dgt NAICS codes used by the Monthly Wholesale Trade Survey) and by sze based on estmated annual revenue. The ntal sample sze was drven by budgetary and resource consderatons as per the recommendaton of the Producer Prce Index Manual, Internatonal Monetary Fund (2004). Once suffcent data have been collected, the sample may be re-allocated to strengthen areas wth hgh samplng varance. The sample sze was set at 3,000 unts, and was allocated to 17 trade groups accordng to an x- proportonal allocaton (Banker, 1988). Subsequently, the unts were allocated to three sze strata, (a) certanty, (b) large take-some, and (c) small take-some. In each take-some stratum, a sample was selected usng PPS samplng, gvng larger unts hgher probablty of selecton. Each partcpatng establshment was asked to provde prcng nformaton on three tems that generate the most annual revenue. It s beleved that the varable of nterest, gross margn, s hghly correlated wth revenue. To reduce response burden, monthly data are collected on a quarterly bass. To ensure that the collected nformaton does not contan grossly anomalous values, basc outler detecton s performed. At the end of collecton, there are three a trplet complete and valdated observatons for each tme perod for each respondent. These observatons form the bass for the frame used n the smulaton study. The specfc detals of frame creaton follow n the next secton. 4. A Note On Frame Creaton And Samplng To create the samplng frame for the smulaton study, all complete trplets collected n the 15-month perod from October 2005 to December 2006 were extracted from the Wholesale SPPI database. Each trplet had to pass certan crtera to be deemed sutable for the smulaton study. The current and prevous nput and output prces and gross margns (output prce nput prce) had to be vald and greater than zero. The ratos of current to prevous nput and output prces (prce relatves) had to fall n the nterval [1/3, 3]. Ths last step was performed to elmnate extreme suspect data ponts. All the observatons passng the above crtera were pooled nto one aggregate data set that represented the prelmnary smulaton frame. The sze dstrbuton on the prelmnary frame was adjusted to more closely match the sze dstrbutons that were observed n the actual frms on the survey frame. Ths resulted n many small unts beng cloned several tmes as the PPS samplng scheme that was used to draw the sample favours the selecton of larger unts. The resultng smulaton populaton contaned some 3 establshments. Now, suppose that, based on sample (s) of y-values, we want to estmate a populaton (U) weghted mean Y = ay / U a U = A U y = IL, whch s a defnton of the Laspeyres Index, wth A = a / a beng the relatve weght of unt. If U π s the probablty that a unt s n the sample, then a sample based estmate of I L s I ˆ 1 a y 1 π / π a =. s s In the context of estmatng a prce ndex, the observed values y are elemental prce ndces computed at the unt level, and the relatve weghts a represent the relatve mportance of a unt n the aggregate formula. Typcally, a s a fxed measure of sze revenue (or employment, as s the case for other statstcal agences such as the U.S. Bureau of Labor Statstcs) avalable for all unts n the populaton. 1456

4 The sample based weghted mean can be seen as a 1 1 rato estmator, I ˆ z = π / π a = Z ˆ / A ˆ s s, where z = ay and Î denotes a prce ndex. Now, we can appeal to classcal estmaton theory to proceed wth varance estmaton, be t n the context of SSRSWOR or PPS samplng. Informaton on how to estmate the varance of a rato estmator n the context of SSRSWOR and PPS (Posson) can be found n Särndal et al. (1992). 5. Smulaton Study The populaton from whch samples were drawn was based on the sample currently used for the Wholesale SPPI. An advantage of usng real data s beng able to assess the mpact of varous types of non-response on the estmated varances (current study wll be extended to compare stratfed SRSWOR and PPS samplng under varous types of non-response, and shrnkng sample szes). A total of samples, each comprsng 3,000 unts, were generated for each samplng scheme. For PPS, n partcular the Posson varant, a unt s selected wth a probablty proportonal to revenue (or another sze measure avalable unversally),.e., π = nx / X, where π s the probablty of selectng a unt, X s the assocated sze measure, and n s the desred sample sze. If nx / X 1, the unt s selected wth certanty and the correspondng total sze measure s adjusted accordngly. Ths process s repeated untl all certanty unts have been dentfed. For SSRSWOR, the Lavallée-Hdroglou (1988) algorthm was used to dentfy optmal stratum boundares. The survey frame was dvded nto one certanty and two probablty strata. The desred sample sze was parttoned usng Neyman allocaton. Wthn each non-certanty stratum, unts were selected wthout replacement usng the same probablty of selecton. To assess the propertes of each samplng scheme n terms of precson of the resultng estmates, both emprcal samplng bas ( ˆB ) and varance ( V ˆ ) were computed. They were defned as and ˆ 1 B = ( Iˆ IL) = 1 2 ( ), = 1 ˆ 1 V = Iˆ Iˆ 1 where Iˆ = I ˆ. To assess the propertes of an estmator, one typcally computes the relatve emprcal samplng bas, but n the context of estmatng a prce ndex, dvdng a very small change by a number close to one does not change the results. The parameter I L s the true populaton Laspeyres prce ndex formulated as follows PQ P I A A y, U ck bk ck L = = U bk = U bk k PQ U bk bk Pbk where the subscrpts b and c denote the base and current perods, P denotes prce, Q s quantty, and A bk s referred to as the base perod economc weght relatve contrbuton of an entty (busness unt or tem) to the overall ndex computed at some level of aggregaton (ndustry, provnce, etc.) defned as PQ bk bk / PQ U bk bk. The populaton ndex I L s the quantty that most statstcal agences estmate n practce. In economc lterature, one can fnd many dfferent ndex formulas that have been defned over the years to estmate the true prce movement whle stll beng operatonally feasble and delverable n a tmely fashon. The nterested reader may consult the Producer Prce Index Manual, Internatonal Monetary Fund (2004) for a bref overvew of the most commonly used n practce ndces. We confne the smulaton study to the geometrc mean ndex (Jevons Index) and the arthmetc mean ndex (Laspeyres Index) as both have a number of desrable propertes, not the least of whch s easy mplementaton. Economc theory tells us that the Laspeyres Index s typcally upward based and hence t provdes an upper bound on the true ndex. The geometrc mean s always lower than the arthmetc mean and as such, t s closer to the true ndex. The two tables below show the emprcal samplng bas and standard devaton correspondng to the two ndces n the context of SSRSWOR and PPS samplng. It should be noted that the bas assocated wth the Jevons Index s economc as both samplng methods are unbased. The overall bas s negatve, whch only confrms the relatonshp between the 1457

5 arthmetc and geometrc means. Both samplng methods produce smlar levels of precson wth PPS samplng margnally outperformng SSRSWOR. However, ths gan n performance s not statstcally sgnfcant. In Table 2. the economc bas dsappears as the defnton of the varable of nterest concdes wth the populaton parameter. Once agan, PPS samplng outperforms SSRSWOR wth precson results beng almost dentcal to those reported n Table 1. The smulaton study was desgned to measure monthly prce movement and although the dfferences between the two samplng schemes are small, they may become sgnfcant when annualzed. Assumng that monthly changes are relatvely constant over tme, the annual rate of change estmated by SSRSWOR s 0.8% compared to 2.3% for PPS samplng when usng the Jevons Index. The annual rates are obtaned by frst correctng for the bas, and then multplyng the dfference between one and the estmated parameters by twelve. Both numbers seem plausble as estmates of annual prce movement and need to be compared to the true prce ndex, whch wll be nvestgated shortly. Table 1. Geometrc mean (Jevons Index) at unt level Trade Group Bas Bas A B C D E F G H I J Overall Table 2. Arthmetc mean (~ Laspeyres) at unt level Trade Group Bas Bas A B C D E F G H I J Overall Several other ndex formulas were studed n addton to the two exposed n ths paper. Apart from the economc bas showng consderable range elmnatng some ndex formulatons as not beng vable, the statstcal propertes under both desgns were comparable. The smulaton results dd not change apprecably when gross margn, the true varable of nterest that would not be avalable outsde of a smulaton study, was used as the sze measure for sample selecton and economc weghtng. Ths s most lkely due to a large overall sample sze. 6. Concludng Remarks The two tables show very lttle dfference n the statstcal propertes of a prce ndex based on ether stratfed SRSWOR or PPS samplng. Ths s true n the deal settng of the current smulaton study. To nject a touch of realty nto the smulaton study, we would lke to nvestgate what happens when real lfe phenomena are ncorporated such as mperfect sze measure, other types of msclassfcaton, nonresponse, outlers that may go undetected, and varous types of mputaton for mssng data. We would also lke to modfy the populaton fle to allow for the computaton of a true prce ndex to verfy the conjecture that the results reported for Laspeyres Index also hold for the true prce ndex,.e., statstcally no dfference between PPS and SRS. It s not clear how sample sze would affect the results. To assess how robust the two desgns are aganst dmnshng samples, the smulaton study needs to be extended to cover sample szes that are typcally seen n real prce ndex surveys. The sample szes may be reduced even further to dentfy the breakng pont of each desgn. It should be noted that De Haan (1998) and Dorfman (2006) produced smulatons usng a lmted set of scanner data that ndcate that a smple cut-off strategy performs as well as SRS. We also hope to explore ths opton usng our dataset. Acknowledgements The authors would lke to thank the followng for ther comments and suggestons, whch contrbuted to mprove greatly the fnal verson of the paper: Fred Barzyk, Jean-Franços Beaumont, George Beelen, Sylve Gauther, Perre Lavallée, and Wesley Yung. The vews expressed n the paper are those of the authors and do not necessarly reflect the offcal 1458

6 poston of Statstcs Canada. All remanng errors are those of the authors. References Banker, M (1988), Power allocatons: determnng sample szes for subnatonal areas. The Amercan Statstcan, 42, Boskn M. J., Dulberger E. R., Gordon R. J., Grllches Z. and Jorgenson D. (1996), The Boskn Commsson Report: Toward A More Accurate Measure Of The Cost Of Lvng. De Haan, J., Opperdoes, E., Schut, C. M. (1998), Item Selecton n the Consumer Prce Index: Cut-off Versus Probablty Samplng. Survey Methodology, 1, Dorfman, A. H., et al. (2006), On Sample Survey Desgns for Consumer Prce Indces. Survey Methodology, 2, Internatonal Monetary Fund, Internatonal Labour Organzaton. (2004), Producer Prce Index Manual: Theory and Practce, Washngton, DC: Internatonal Monetary Fund. Lavallée, P. and Hdroglou, M. A. (1988), On the Stratfcaton of skewed populatons. Survey Methodology, 14, Ohlsson, E. (1998), Sequental Posson Samplng. Journal of Offcal Statstcs, 14, Särndal, C.-E., Swensson, B., Wretman, J. (1992), Model Asssted Survey Samplng. New York: Sprnger-Verlag. 1459