Implementation of consumer electronics scanner data in the New Zealand CPI

Transcription

1 Implemenaon of consumer elecroncs scanner daa n he New Zealand CPI Paper presened a he Oawa Group, Tokyo, Japan May 2015 Frances Krsnch Senor Researcher, Prces Un, Sascs New Zealand P O Box 2922 Wellngon, New Zealand frances.krsnch@sas.gov.nz

2 Crown copyrgh Ths work s lcensed under he Creave Commons Arbuon 3.0 New Zealand lcence. You are free o copy, dsrbue, and adap he work, as long as you arbue he work o Sascs NZ and abde by he oher lcence erms. Please noe you may no use any deparmenal or governmenal emblem, logo, or coa of arms n any way ha nfrnges any provson of he Flags, Emblems, and Names Proecon Ac Use he wordng 'Sascs New Zealand' n your arbuon, no he Sascs NZ logo. Lably The opnons, fndngs, recommendaons, and conclusons expressed n hs paper are hose of he auhor. They do no represen hose of Sascs New Zealand, whch akes no responsbly for any omssons or errors n he nformaon n hs paper. Caon Krsnch F (2015, May). Implemenaon of consumer elecroncs scanner daa n he New Zealand CPI. Paper presened a he Oawa Group, Tokyo, Japan.

3 Absrac From he Sepember 2014 quarer, Sascs New Zealand began usng scanner daa o measure he prce change of 12 consumer elecroncs producs, ncludng elevsons, compuers, and cellphone handses. The monhly scanner daa, from marke research company GfK, conans comprehensve nformaon on produc characerscs. Ths nformaon on characerscs allows us o use he Impuaon Törnqvs rollng year GEKS (ITRYGEKS) mehod o derve non-revsable qualy-adjused prce ndexes. These ndexes are free of chan drf, use all he nformaon n he daa, and reflec he mplc prce movemens of new and dsappearng producs (de Haan and Krsnch, 2014a). We wll dscuss aspecs of he mplemenaon of consumer elecroncs scanner daa n he New Zealand CPI, whch s a quarerly measure, ncludng: how we deal wh havng only wo monhs of he quarer avalable n me for producon he dervaon of a quarerly ndex from monhly scanner daa he monorng and analyss process communcang he new approach o users. 1. Inroducon Sascs New Zealand began usng scanner daa for consumer elecroncs producs n he New Zealand Consumers Prce Index from he Sepember 2014 quarer. Monhly-aggregaed scanner daa s purchased from marke research company GfK for welve consumer elecroncs produc caegores ncludng elevsons, compuers, and cellphone handses. Prce ndexes are derved from he scanner daa usng he Impuaon Törnqvs rollng year GEKS (ITRYGEKS) mehod of de Haan and Krsnch (2014a). The ITRYGEKS ndex s an exenson of he RYGEKS of Ivancc e al (2011) whch s based on me-dummy hedonc ndexes o ensure ha he mplc prce movemens of new producs enerng he marke, and old producs dsappearng from he marke, are refleced. The developmen of producon processes has been an erave one whch s sll evolvng. Ths enables unancpaed ssues o be deal wh flexbly and effcenly. The paper s srucured as follows: Secon 2 gves he background o he mplemenaon of consumer elecroncs scanner daa n he New Zealand CPI. Secon 3 dscusses he mplemenaon, wh a descrpon a he caegores of producs nroduced, how we deal wh paral daa, and usng monhly aggregaed daa for a quarerly ndex and processng me. Secon 4 dscusses he decson o use he ITRYGEKS ndex. Secon 5 presens he erave approach o he developmen of he processng sysem. Secon 6 oulnes how he mplemenaon of scanner daa n he New Zealand CPI has been communcaed o users. Secon 7 concludes. 2. Background

4 Sascs New Zealand has been usng scanner daa from marke research company GfK o nform expendure weghs and sample selecon snce Over he las fve years acve research on prce measuremen from scanner daa has ncluded collaboraon wh Sascs Neherlands on he Impuaon Törnqvs rollng year GEKS (ITRYGEKS) mehod (de Haan and Krsnch, 2014a). The New Zealand CPI Advsory Commee of 2013 recommended ha Sascs NZ pursue he use of scanner daa n he NZ CPI n her Recommendaon 11: Recommendaon 11 of he CPI Advsory Commee of 2013:. Tha Sascs NZ endeavours o use real ransacon (scanner) daa o measure prce change n he CPI.. Tha he mehods used algn wh nernaonal bes pracce.. Tha arrangemens be n place o ensure he connuy of real ransacon daa supply. v. Tha here s a suable conngency plan n he even ha he supply of real ransacon daa s unavodably nerruped. v. Tha Sascs NZ perodcally auds prces n real oules o manan confdence ha he real ransacon daa mees qualy requremens. Sascs New Zealand decded o ncorporae consumer elecroncs scanner daa from GfK no he New Zealand CPI from he Sepember 2014 quarer usng he ITRYGEKS mehod, for welve produc caegores. 3. Implemenaon n Sepember 2014 quarer 3.1 Produc caegores ncorporaed no he NZ CPI From he Sepember quarer 2014, Sascs New Zealand ncorporaed real ransacon daa, or 'scanner daa', no he CPI, o measure prce change for consumer elecroncs producs. The 12 produc caegores are: hea pumps deskop compuers lapop compuers able compuers mul-funcon devces cellphone handses dgal cameras dgal camera memory cards elevson ses se-op boxes for elevson ses DVD, Blu-ray players and player/recorders home heare and sereo sysems. In erms of expendure weghng: hea pumps conrbue a ffh of he major household applances class, whch conrbued 0.71 percen o he all groups CPI expendure wegh for he June 2014 quarer cellphone handses conrbue over 90 percen of he elecommuncaon equpmen class, whch conrbued 0.29 percen o he all groups CPI 4

5 he remanng 10 caegores conrbue four-ffhs of he audo-vsual and compung equpmen subgroup, whch conrbued 1.16 percen o he all groups CPI. The scanner daa s beng suppled by marke research company GfK. GfK adds nformaon abou produc feaures, or characerscs, o he daa colleced from realers. Toal monhly sales values and quanes sold are avalable, across realers, for each produc, along wh exensve nformaon abou he characerscs of each produc. There are hundreds of housands of ransacons represened n he scanner daa each monh. The number of characerscs n he daa range from 10 for dgal camera memory cards o 77 for dgal cameras, as shown n able 1 for Augus The number of dsnc producs n he daa a Augus 2014 ranges from 24 for se-op boxes for elevsons, o 445 for lapop compuers. Table 1. Number of characerscs and dsnc producs a Aprl 2014 Produc caegory Characerscs Producs Hea pumps Deskop compuers Lapop compuers Table compuers Mul funcon devces Cellphone handses Dgal cameras Dgal camera memory cards Televson ses Se op boxes for elevson ses DVD, Blu ray players, and player/recorders Home heare and sereo sysems Incomplee daa avalable n me for producon The New Zealand CPI s a quarerly ndex. One of he lmaons of he GfK scanner daa s ha only he frs wo monhs of he quarer are avalable n me for producon. There are four opons for how o deal wh hs lmaon n producon. We assessed each of hese opons agans he benchmark of he ndex f we had all hree monhs of he quarer ncorporaed. 1. Esmae he ITRYGEKS ndex on he frs 2 monhs of he quarer, wh complee backdaa feedng no he esmaon (e he 3rd monh wll be ncorporaed no he 13- monh esmaon wndow for he followng quarer) 2. Esmae he ITRYGEKS ndex usng 3 monhs of daa, lagged by one monh 3. Esmae he ITRYGEKS usng only he mddle monh of each quarer 4. Esmae he ITRYGEKS usng only he frs wo monhs of each quarer Opon 1 performed beer han he oher opons on wo summary measures: 1. The average of he absolue relave dfferences beween each opon s movemen and ha of he benchmark. 2. The average of he absolue dfference beween each opon s movemen and ha of he benchmark. 5

6 Therefore hs s he approach aken we base he ndex on he frs wo monhs of daa of he laes quarer along wh complee backdaa for he prevous four quarer. Ths s shown below n fgure 1. Fgure 1. Daa ncorporaed no each quarer's ndex a he me of producon 2014Q3 2014Q4 2015Q1 2013Q3 2013Q4 2014Q1 2014Q2 2014Q3 2014Q4 2015Q1 The fgure shows ha, for example, o calculae he quarerly prce movemen for he hrd quarer of 2014 (e he op row), we use full quarerly daa from each of he quarers from quarer hree of 2013 hrough o quarer wo of 2014, and he frs wo monhs of daa from he hrd quarer of Quarerly ndexes from monhly daa The consumer elecroncs scanner daa s aggregaed o a monhly level. The New Zealand CPI s a quarerly ndex, so we can eher derve he quarerly ndex from he monhly ndex, or preaggregae he daa o a quarerly level before dervng a quarerly ndex. Whle s useful o produce he monhly ndex as par of he monorng and analyss processes, s concepually more approprae for he quarerly ndex o be derved from quarerly prces and quanes, as ensures he prces for producs sold n each monh of he quarer are appropraely weghed for prce deflaon, o produce quarerly volume ndexes n he Naonal Accouns. In pracce, he wo approaches gve very smlar resuls. 4. Mehodology 4.1 The need for new mehods Consumer elecroncs s a rapdly changng produc class, so s parcularly mporan ha we use mehods ha wll appropraely adjus for he change n qualy of wha s beng purchased. Also, because consumer elecroncs producs can have shor lfe-cycles, s mporan ha new producs are nroduced no he ndex n such a way ha he mplc prce movemen assocaed wh her nroducon s approprae. Tha s, f a new produc has an nroducory prce ha s low, relave o s se of feaures, hen hs s a prce decrease ha he prce ndex should reflec. In addon o hs requremen for approprae qualy adjusmen, he CPI s non-revsable. Ths places anoher consran on mehods for creang prce ndexes from scanner daa usng sascal models, where he models are generally based on a pas wndow of daa usually a year endng wh he mos recen quarer. The wo key reasons why radonal ndex number mehods do no work n he case of scanner daa are: The hgh level of churn or producs appearng and dsappearng n he marke. 6

7 Volale prces and quanes due o dscounng, whch leads o a bas called chan drf when chaned superlave ndexes such as a chaned Törnqvs, are used o connually updae he baske n he presence of hgh produc urnover. 4.2 The rollng year GEKS Ivancc, Dewer, and Fox (2011) proposed a mehod for producng prce ndexes from scanner daa ha uses all he prces and quanes n he daa, and s free of chan drf 1. Called he rollng year GEKS (RYGEKS) ndex, s based on he Gn, Eleö and Köves, and Szulc (GEKS) ndex used for mullaeral spaal prce ndexes such as purchasng prce pares whch compare prces n dfferen counres a a pon n me. Whn some esmaon wndow 2 he RYGEKS ndex beween perods 1 and 2 s he geomerc mean of all he superlave blaeral ndexes (such as he Törnqvs ndex or, as used n Ivancc e al (2011), he Fsher ndex) beween: 1 and all he oher perods n he wndow, and 2 and all he oher perods n he wndow. The monhly RYGEKS based on a 13-monh rollng esmaon wndow s as follows: For he frs wndow, e T=0 o 12, he RYGEKS ndex s equal o he GEKS ndex: P 0T RYGEKS P 0T GEKS 12 0 T P P (1) Where j P s any superlave ndex (eg a Törnqvs ndex) beween perods and j. From =13 onwards, RYGEKS lnks on he mos-recen movemen from he GEKS calculaed on he nex wndow (e from =1 o 13, hen from = 2 o 14, and so on) as follows: P 0,13 RYGEKS P 0,12 GEKS ,, o, ,,13 1 P P P P P P P 0,14 RYGEKS and so on. P 0,13 RYGEKS ,,14 P P 1 13 (2) However, a lmaon of he RYGEKS mehod s ha does no reflec he mplc prce movemens of new and dsappearng producs enerng and leavng he marke. For example, f he nal prce of he laes model of a moble phone s hgh relave o s se of feaures hen hs mplc prce ncrease s no refleced n he RYGEKS ndex. 1 Though non-ransve due o he rollng wndow aspec of he mehod so has he poenal o suffer from wha we could call splce-drf hough hs s lkely o be relavely nsgnfcan. 2 The esmaon wndow s commonly one year plus a perod e 13 monhs or 5 quarers. 7

8 4.3 The mpuaon Törnqvs rollng year GEKS An exenson of he RYGEKS ndex, called he mpuaon Törnqvs rollng year GEKS (ITRYGEKS) was proposed n de Haan and Krsnch (2014a). Lke he GEKS, he ITRYGEKS s able o ulse all he nformaon n he daa, whle remanng free of chan-drf. In addon, reflecs he mplc prce movemens of new and dsappearng producs. Unlke he RYGEKS descrbed above, whch s based on superlave ndexes such as he j Törnqvs or Fsher - e he P n equaons (1) and (2) he ITRYGEKS ndex s based on blaeral me-dummy hedonc ndexes. The Törnqvs n s name refers o he fac ha s algebracally equvalen o a Törnqvs ndex based on real and predced prces, as shown below n equaon (4). A blaeral me-dummy hedonc ndex beween any perods 0 and s derved from a sascal model based on he daa from perods 0 and. 3 The esmang equaon for he blaeral me dummy hedonc model s: ln p K D z (3) k 1 k k Where: ln p s he log of he average monhly prce for produc n perod s he nercep erm D has he value 1 f he observaon relaes o perod ( 0) and he value 0 f he observaon relaes o perod 0 z s he quany of he kh characersc, or feaure, for produc. 4 By defnon, hs wll be he same n boh perods 5 s an error erm wh an expeced value of 0 Snce model (3) conrols for changes n he produc characerscs, qualy-adjused prce change beween perods 0 and. exp s a measure of And, when he average 6 expendure shares are used as weghs for he mached producs and half of he expendures shares for he unmached producs (n he perods hey are avalable) he ITRYGEKS ndex from perod 0 o can be expressed as follows: P 0 ITRYGEKS 0 U p 0 p 0 0 s s s 2 pˆ U p 0 D UN p 0 pˆ s 2 (4) 3 Ths can be generalsed o any wo perods and j 4 For caegorcal characerscs, such as hose on he consumer elecroncs scanner daa, each characersc k wll be represened by a se of dummy varables correspondng o all possble values of characersc k ha s, varables whch are se o 1 (or 0) n he presence (or absence) of ha value of he characersc. 5 A produc corresponds o a dsnc se of characerscs, or feaures. So, any change n characersc would resul n a dfferen produc. 6 Across boh me perods. 8

9 Where: U 0 U 0 D s he se of mached producs wh respec o perods 0 and s he se of 'dsappearng' producs wh respec o perods 0 and ha s, producs ha exs n perod 0 bu no n perod s he se of 'new' producs wh respec o perods 0 and U 0 N s s he expendure share of produc n perod Formula (4) demonsraes ha he ITRYGEKS ndex s equvalen o a Törnqvs ndex for he mached producs and, for new and dsappearng producs, he ITRYGEKS ndex apples a Törnqvs formula o prces predced from blaeral me-dummy hedonc models for he perod n whch here s no prce avalable. The dervaon of (4) s gven n de Haan and Krsnch (2014). The decson o use he ITRYGEKS ndex for he consumer elecroncs scanner daa was made on he bass ha, a he me, hs was a well documened and undersood mehod. Snce ha decson was made, has become clear ha eher of he mullaeral me-dummy (TD) hedonc ndex or he fxed-effecs wndow-splce (FEWS) ndex would also be feasble mehods o use for hs consumer elecroncs scanner daa n producon. 4.4 The poenal o use eher TD or FEWS ndexes Recen research by de Haan and Krsnch (2014a) has shown ha he mullaral me dummy (TD) hedonc ndex s a geomerc form of a qualy-adjused un value (QAUV) ndex whch, emprcally, s shown o be vrually he same as he harmonc QAUV. Krsnch (2014) has shown ha he FEWS ndex can produce fully qualy-adjused nonrevsable prce ndexes whou explcly ncorporang any characerscs, by leveragng off he longudnal nformaon n he daa. However, here s currenly no ncenve o use he FEWS ndex over mehods such as he ITRYGEKS or TD whch requre explc nformaon on he full se of characerscs. Ths s because here s confdenalsng n he daa where model names are overwren wh a radebrand denfer when more han a ceran percenage of ha model are sold hrough jus one realer. Therefore he full se of characerscs n he GfK daa s requred o derve he produc denfers whch would be requred for he FEWS ndex. 5. Developng he processng sysem 5.1 An erave approach o he processng sysem The approach o developng he processng sysem for prce measuremen from consumer elecroncs scanner daa has been an erave one. Processes for analyss and monorng have connued o evolve over he me ha scanner daa has been n producon as some unancpaed ssues have arsen. For example, changes n he codng of characerscs can cause spurous movemens n he ndex f no denfed, as hey are nerpreed as a complee replacemen of producs wh new producs a he me of he change. Idenfyng changes n codng pracces s non-rval wh such a large number of characerscs across all produc caegores. Wh such a new developmen as he use of scanner daa for prce measuremen, he flexbly and adapably of a user-wren prooype sysem s a rsk-averse and effcen approach o 9

10 ake. Once he sysem has all he necessary analyss, monorng and esmaon componens bedded no a sreamlned process, wll be approprae o ncorporae as par of a more formally developed processng sysem for Prces bg daa more generally. Ths sysem wll also ncorporae fuure use of supermarke scanner daa; posal survey daa such as ha for used cars whch use a hedonc esmaon mehod; overseas rade daa such as elevsons and moble phones; and web-scraped onlne daa. I wll be bul wh he flexbly o ncorproae oher fuure bg-daa sources. 5.2 Monorng and analyss processes Daa s exraced from GfK s sarrack sysem no excel fles, whch are hen mpored o SAS. Before he producon of quarerly prce ndexes usng he ITRYGEKS mehod, here are a number of monorng and analyss processes run, whch nclude: Checks ha ses of characerscs have remaned he same snce las quarer Tme seres of monhly quanes sold for each produc caegory Tme seres of monhly average prces (unadjused for qualy change) for each produc caegory Idenfcaon of oulers, n erms of boh movemens and levels for he laes quarer, and ncorporang nformaon from he full longudnal record. Tme seres of dsrbuons across key characerscs, for each produc caegory, n erms of boh quanes sold and expendure shares Monhly ITRYGEKS qualy-adjused prce ndexes Tme seres of urnover sascs e couns of new producs, dsappearng producs, and mached producs beween each par of adjacen monhs If one of he monorng or analyss oupus suggess furher nvesgaon, oher processes can hen be nvoked: The quarerly (or monhly) prce ndexes can be run for subses of he daa. For example, we could run he quarerly ITRYGEKS for dfferen brands of elevson, o deermne wheher an unusual prce movemen s conrbued o by all brands, or wheher one parcular brand s drvng he resul. We can run he prce ndexes excludng oulers eher full longudnal records of oulyng producs, or parcular oulyng values of producs. So, f ouler deecon process denfes a poenal error, we can es he mpac on he ndex of ouler removal. 6. Communcang he new approach o users A number of dfferen forums were used o communcae he move o scanner daa for measurng prce change for consumer elecroncs n he CPI. Sascs New Zealand releases a quarerly elecronc newsleer called Prce Index News. In he Aprl 2014 newsleer (Sascs New Zealand, 2014a) we sgnalled our nenon o nroduce scanner daa no he CPI A sources and mehods documen led Measurng prce change for consumer elecroncs usng scanner daa was released on he Sascs NZ n November 2014 (Sascs New Zealand, 2014b) and hs was laer lnked o by he January 2015 edon of he Prce Index News (Sascs New Zealand, 2015). A number of shor arcles were wren o communcae he developmen o dfferen groups: n he newsleer of he New Zealand Assocaon of Economss (NZAE, 2014); he newsleer of he Inernaonal Assocaon of Survey Sascans (IASS, 2015); and he Exper Daa Users newsleer of Sascs NZ (2014c). By explanng he developmen n boh plan englsh and wh lnks o echncal deal, he nformaon needs of a range of dfferen audences have been caered for. 10

11 7. Concluson Scanner daa was ncorporaed no he New Zealand CPI for consumer elecroncs producs, from he Sepember 2014 usng he ITRYGEKS mehod. In addon o obanng suable daa, and deermnng an approprae ndex mehodology, some praccal ssues arsng from he melness of he daa and approprae levels of aggregaon needed o be addressed. An erave approach o he developmen of a processng sysem has enabled he analyss and monorng processes o evolve n response o unancpaed ssues, such as he poenal for codng of characerscs o change over me. A number of dfferen forums were used o communcae he new developmen o dfferen user groups, a dfferen levels of echncal deal. The producon processes for consumer elecroncs scanner daa n he New Zealand CPI are a prooype sysem whch wll nform he more formal developmen of a Prces bg daa processng sysem ha wll encompass a range of daa sources ncludng supermarke scanner daa, rade daa, elecommuncaons blls daa and onlne daa. References de Haan, J, & Krsnch, F (2014a). Scanner daa and he reamen of qualy change n nonrevsable prce ndexes. Journal of Busness & Economc Sascs, 32:3, de Haan, J, & Krsnch, F (2014b, Augus). Tme-dummy hedonc and qualy-adjused un value ndexes: Do hey really dffer? Paper presened a he conference of he Socey for Economc Measuremen, Chcago. Ivancc, L, Dewer, WE, & Fox, KJ (2011). Scanner daa, me aggregaon and he consrucon of prce ndexes. Journal of Economercs, 161(1), IASS (2015) Survey Sascan Newsleer Jan Avalable a hp://sass.org/home/servces/he-survey-sascan/ Krsnch, F (2011, May). Measurng he prce movemens of used cars and resdenal rens n he New Zealand consumers prce ndex. Paper presened a he 12h meeng of he Oawa Group, Wellngon, New Zealand. Avalable a hp:// 2011/agenda.aspx Krsnch, F (2013, May). Usng he rollng year me-produc dummy mehod for qualy adjusmen n he case of unobserved characerscs. Paper presened a he 13h meeng of he Oawa Group, Copenhagen. Avalable a hp:// Krsnch, F (2014, May). The FEWS ndex: Fxed effecs wh a wndow splce. Paper presened a he meeng of he group of expers on consumer prce ndces, Geneva, Swzerland. Avalable a hp:// NZAE (2014) Asymmerc Informaon (newsleer of he New Zealand Assocaon of Economss) ssue 41 Dec 2014 Avalable a hp:// Sascs New Zeland (2011). Updang he CPI used cars ndex hedonc model. Avalable a hp:// Sascs New Zealand (2013). Consumers prce ndex advsory commee 2013 dscusson paper. Avalable a 11

12 hp:// Sascs New Zealand (2014a). Prce Index News: Aprl Avalable from hp:// Sascs New Zealand (2014b). Measurng prce change for consumer elecroncs usng scanner daa. Avalable from Sascs New Zealand (2014c) Exper Daa Users newsleer November ssue: Avalable a hp://a.smarmalpro.com/webv/c4jvzbh2dh#elecronc Sascs New Zealand (2015) Prce Index News: January Avalable a hp:// 12