Quantification of qualitative data: the case of the Central Bank of Armenia

Transcription

1 Quantfcaton of qualtatve data: the case of the Central Bank of Armena Martn Galstyan 1 and Vahe Movssyan 2 Overvew The effect of non-fnancal organsatons and consumers atttudes on economc actvty s a subject of great nterest to both polcymakers and economc forecasters. The Busness Tendency and Consumer Surveys of the Organsaton for Economc Co-operaton and Development (OECD) are mportant sources of nformaton about household opnon and non-fnancal organsatons expectatons concernng current economc condtons and future developments. The nformaton collected n such surveys s manly qualtatve because respondents are asked to assgn qualtes, rather than quanttes, to the varables of nterest. For example, n a busness tendency survey, respondents are asked to assgn qualtes to the value of ther order books, such as hgher than normal, normal or below normal. It s generally much easer for respondents to gve qualtatve rather than quanttatve nformaton. As a result, the questonnares can be completed quckly and results of the surveys can be publshed earler than the results of tradtonal statstcal surveys. Ths s one of the man advantages of qualtatve surveys. Thus, before computng the fnal ndces, t s necessary to quantfy the qualtatve data collected, and t s very mportant to select the rght quantfcaton methods. Quantfcaton of qualtatve data Quanttatve analyss s the numerc representaton and manpulaton of qualtatve observatons for the purpose of descrbng and explanng the event that those observatons reflect. The analyss of how economc agents form ther expectatons about economc varables has been treated as one crucal ssue n explanng many mportant economc trends. Whle there s a vast lterature on ths topc, no consensus has been acheved among researchers on how to quantfy the expectaton survey data. There are at least four man approaches to convertng the results of qualtatve surveys to standard quanttatve varables. The frst method s some varant of the probablstc approach. The prncple behnd ths approach s that the respondents reply that the value of the reference varable x can be descrbed by a certan statement (eg x stays stable) f t les between two known thresholds 1 2 Central Bank of Armena, Statstcs Department, Vazgen Sargsyan str 6, Yerevan 0010, Republc of Armena. E-mal: m.galstyan@cba.am Central Bank of Armena, Statstcs Department, Vazgen Sargsyan str 6, Yerevan 0010, Republc of Armena. E-mal: vmovssyan@cba.am 202 IFC Bulletn No 33

2 (eg ± 5 per cent around ts ntal value). Thus, by assumng that the functonal form of the underlyng probablty dstrbuton of opnons and expectatons about x s known, the average value of x can be expressed as a functon of the gven thresholds. A measure of heterogenety of opnons and operators uncertanty can also be derved wthn the same analytcal framework. The second approach s based on regresson technques amed at estmatng the value of x underlyng each qualtatve answer. Ths method requres the regresson of a standard quanttatve measure of x aganst the tme seres of percentage of people who gave each qualtatve answer. The thrd method regards the percentages of each qualtatve answer as a functon of a common latent measure of x observed by respondents. The usual multvarate technques can help n estmatng the dynamcs or the sectoral varatons (but not the absolute level) of the latent factor affectng the opnons and expectatons expressed by the ntervewed operators. The tme seres of percentages of answers collected n qualtatve surveys are closely correlated. In the frst place, ths fact mples that the latent varable approach s possbly sound and relable. However, t also suggests that even very sophstcated methods, based on complcated transformatons of orgnal percentages, tend to produce ndcators that follow the common trend and cycles that can be easly deduced by any tme seres of percentages, or a smple combnaton thereof. Ths explans and justfes the wdespread use of the balance between the percentages of optmstc and pessmstc answers. The fourth method s the calculaton of dffuson and composte ndces. Ths method s a summary measure desgned to facltate the analyss and forecast of busness cycles combnng the behavour of a group of economc ndcators whch represent dfferent economc actvtes such as producton and employment. Dffuson ndces aggregate the drectons of change of a selected seres to detect a busness cycle phase, whle composte ndces aggregate the percentage changes of a selected seres to detect the volume of a busness cycle. The emprcal verfcaton of performance of the varous methods s mxed. Generally, no one procedure outperforms another, even f some authors have ponted out the sharp neffcency of balance statstcs and others have noted that dynamc regresson models are generally superor. Table 1 provdes a swft comparson of dfferent approaches. Table 1 Quantfcaton methods Method Man assumptons Advantages Drawbacks Probablstc The functonal form of opnons about the relevant varable x s known. Respondents reply x dd (wll) reman stable f x les between two gven thresholds. Addtonal assumptons are requred n a polychotomous case. The results depend only on the observed percentages of answers and only to a mnor extent on the The tme seres of results may be very volatle f some specal combnatons of answers occur. assumptons about the probablty dstrbuton of the varables and the thresholds assumed by Other nformaton on x s completely neglected, even when t s avalable. respondents. The treatment of polychotomous questons may be complcated. IFC Bulletn No

3 Table 1 (cont) Quantfcaton methods Method Man assumptons Advantages Drawbacks Regresson Latent factor Dffuson and composte ndces Respondents also attach to each qualtatve answer a reference value of x. Reference values can be estmated by usng regresson models. A sngle common latent factor drves each percentage of answers. Dffuson ndces measure one half of the respondents reportng no change and all respondents reportng postve answers. The reason why a group of ndcators combned nto a composte ndcator should be more relable over a perod of tme than any of ts ndvdual components s related to the nature and causes of busness cycles. It s very general, regardless of the wordng of questons and the number of answers authorsed. Integraton nto standard econometrc models s straghtforward. A reference quanttatve tme seres s needed. Estmaton can be flawed by multcollnearty and numercal convergence problems. It s very general. Very short tme seres of answers cannot be treated. In prncple, no extraneous nformaton s needed. However, they can be exploted as well. The same quantfed ndcator may be used n both prelmnary analyss and econometrc modellng. Easy to compute The performance of ndvdual ndcators wll then depend on the causes behnd a specfc cycle. Some ndcators wll perform better n one cycle and others n a dfferent cycle. It s therefore necessary to have sgnals for many possble causes of cyclcal changes, and to use all potental ndcators as a group. These ndces are more volatle than ndces constructed wth the methods lsted above. Of the methods descrbed, we use a dffuson and composte ndces method for constructon of the consumer confdence ndex (CCI), the economc actvty ndex (EAI) and the busness clmate ndex (BCI). Purposes and tasks of the surveys In response to the wdespread belef that consumers opnons and expectatons nfluence the drecton of the economy, a growng number of studes have set out to analyse the relatonshp between consumer atttudes and economc varables. 204 IFC Bulletn No 33

4 Takng ths nto consderaton, the estmaton of household expectatons regardng the economy, as an ultmate prvate sector drver of market economes, s an mportant factor n the organsaton and mplementaton of macroeconomc polces. For observaton of household perspectves on the current economc stuaton and estmaton of ther expectatons regardng future shfts n the economy, the Statstcs Department of the Central Bank of Armena (CBA) has conducted quarterly consumer confdence surveys snce the frst quarter of The man purpose of the surveys s to estmate consumer behavour n the lght of ther expectatons of current and future economc condtons, and to calculate the CCI. In order to acheve these goals, the followng tasks are performed: Analyss of household opnons regardng the overall economc stuaton (current and future) Analyss of household opnons regardng ther materal securty (current and future) Calculaton of ndces of current and future condtons. The Statstcs Department of the CBA also conducts busness tendency surveys, also known as economc actvty and busness clmate surveys. The man purpose of these surveys s to ask managers of non-fnancal organsaton about the current status of ther busness and ther plans and expectatons for the near future. These surveys provde nformaton that s valuable to the respondents themselves and to economc polcymakers and analysts. Although they do not provde precse nformaton on levels of output, sales, nvestment or employment, they can be used to predct changes n these aggregates, and for that reason, they are partcularly useful for analysng the busness cycle. Survey methodology Consumer confdence survey The survey s conducted n the second month of each quarter, wth tme-ndependent samples of households, and covers all Armenan households. In order to facltate the analyss of the evoluton of phenomena over tme, startng wth the next survey, a part of the sample wll comprse households ntervewed n prevous surveys (panel households). Panel households wll represent about 40 per cent of the sample. Data are collected from households by means of telephone ntervews. The survey sample sze ranges from 1,700 to 2,000. The samplng of survey has stratfed one stage sample desgn wthout replacement: The whole unverse was dvded nto admnstratve subdvsons called strata. The cty of Yerevan was dvded nto communtes (strata) and ts regons nto dstrcts. The sample unts are selected randomly from each stratum. The sample sze of each stratum s proportonal to ts populaton. Survey questons are drafted wth the am of elctng useful nformaton wthout mposng an undue burden on respondents. The questons are generally qualtatve and have a threepont scale of response (ncrease, stable, decrease). Quanttatve questons are also ncluded, but generally confned to demographc aspects of households. The questonnares also contan a queston about household ncome, but because of the senstvty of ths queston, t s suggested that households stuate ther ncome wthn one of the gven ranges. IFC Bulletn No

5 Economc actvty and busness clmate surveys As ndcated, the man purpose of the quarterly EAIs and BCIs s to analyse the expectatons and perspectves of economc agents concernng each branch of the economy (ndustry, constructon, trade and servces). For the economc actvty and busness clmate surveys, the sample of non-fnancal organsatons s consttuted by a non-probablty samplng method: cuttng off the tal. The sample comprses the largest organsatons that account for at least 80 per cent of the gross proft of a partcular branch or segment of the economy. Thus, for the frst quarter of 2009, the survey sample conssts of 832 companes (Fgure 1). Fgure 1 Structure of non-fnancal organsatons sample Industry Trade Servces Constructon Small Medum Large The survey s conducted by telephone, letter and facsmle. Furthermore, as the four surveyed branches account for the largest share of Armena s GDP (Fgure 2), t s also mportant to analyse the correlaton between the EAI, BCI and value added of the respectve branches. Fgure 2 Weghts of four surveyed branches n GDP INDUSTRY CONSTRUCTION TRADE SERVICES Fgure 2 shows that the four branches together account for about 80 per cent of GDP. 206 IFC Bulletn No 33

6 Tables 2 and 3 descrbe all the steps n mplementng the dffuson and composte ndces method for constructon of the consumer confdence, economc actvty and busness envronment ndces. Phase 1 Phase 2 Phase 3 Balance of current condtons for each communty Balance of future condtons for each communty Total balances for each communty Weghted balance of all households BA current Table 2 CCI constructon 3 j1 ( Answer current pos 1 2 Answer current neu current BA balance of current condtons for -th communty current Answer pos postve answers to each current queston current Answer neu neutral answers to each current queston 3 future future 1 future BA ( Answer pos Answer neu ) j1 2 future BA balance of current condtons for -th communty future pos Answer postve answers to each future queston future Answer neu neutral answers to each future queston BA 1 2 BA current BA future BA average balance of answers to all questons for -th communty pop WBA W BA WBA weghted balance of answers for all households W weght of populaton n -th communty pop ) Phase 4 Index computaton (CCI) WBA Index WBA and 0 refer to current and base perod, respectvely. IFC Bulletn No

7 Phase 1 Phase 2 Table 3 Three-step weghted method of EAI and BCI constructon Balance of answers Balance of answers for branch BA BA jq p k1 W EmplK ( Answer pos 1 2 Answer jq balance of j -th segment of -th branch for q -th queston W EmplK weght of k -th organsaton s employees n all organsatons of that segment Answer pos postve answers to each queston Answer neu neutral answers to each queston WBA q n j1 BA j RW WBA q weghted balance of -th branch questons for q -th queston RW weght of proft of the j -th segment n -th branch j j neu ) Phase 3 WBA WBA Average balance Average of answers 4 WBA average balance of answers to all questons Average 4 q 1 ncluded n -th branch q Phase 4 Phase 5 Balances of answers for the overall economy Index computaton (EAI, BCI) TBA 4 1 WBA Average VA TBA weghted balance of answers for the overall economy VA weght of -th branch value n cumulatve value of all four branches TBA Index TBA and 0 refer to current and base perod, respectvely. To nvestgate the possble relatonshps between economc actvty and busness clmate questons, we need to analyse the correlatons between the branches varables (questons) and the value added for each branch. Correlaton matrces are presented n the appendces. We have to underlne that here, n correlaton analyss, the varables concernng the future expectatons of respondents are taken wth a + 1 lag: for example, n ndustry, the expectaton of respondents (taken at quarter t ) regardng the demand for ther products for 208 IFC Bulletn No 33

8 the t 1-th quarter s correlated wth the actual growth of ndustry value added for the t 1- th quarter. As we can see, n both ndustry and constructon, almost all ndvdual economc actvty questons have strong postve correlatons wth the quarterly growth rate for that branch. In trade, only one queston (VolumeC) s sgnfcantly correlated (0.719) wth the growth of trade value added. From Table 4 t s obvous that n ndustry, constructon and trade, the branch economc actvty analyss better descrbes the approprate branch of the economy than busness clmate analyss. In partcular, n constructon, the coeffcent of correlaton between the weghted balances of economc actvty questons and the constructon growth rate equals (t s sgnfcant at the 0.01 level). Table 4 Correlaton matrx of EA and BC weghted balances of answers (WBA) and growth of branches EA WBA BC WBA Industry growth 0.719** Constructon growth 0.842** 0.540* Trade growth 0.668** Servces growth ** The correlaton s sgnfcant at the 0.01 level (2-taled). * The correlaton s sgnfcant at the 0.05 level (2-taled). Concluson To summarse the analyss, we can conclude that the household and corporate sector surveys conducted by the CBA snce 2005 are mportant sources of nformaton that descrbes the man sectors of the natonal economy. The ndces, calculated by the dffuson and composte ndex method, partcularly the economc actvty ndces of ndustry, constructon and trade, can be used as leadng growth ndcators for the correspondng sectors value added. Ths shows that the selected quantfcaton method (dffuson and composte ndex method) works well for the ntended purposes. We should menton that we have short tme seres of composte ndces (data for 14 quarters), and ths can lead to overestmatng the relablty of the results. In the future, wth enlargement of the survey database, t wll be possble to obtan more relable estmates of ndcators that can be used for prelmnary forecastng of the development of the Armenan economy. IFC Bulletn No

9 210 IFC Bulletn No 33 Appendx 1: Correlaton matrx (ndustry) Balance of economc actvty questons Balance of busness envronment questons Balance of other questons VolumeC VolumeF StockF DemandF RsksC RsksF SubndC SubndF PrceC WageF EmployeeF VolumeC 1 VolumeF 0.777** 1 StockF 0.575* 0.578* 1 DemandF 0.807** 0.905** 0.633* 1 RsksC ** RsksF * ** 1 SubndC 0.592* * ** 0.606* 1 SubndF 0.585* * 0.680** 0.677** 0.521* 0.657** 1 PrceC ** * * 1 WageF 0.555* * 0.605* 0.846** EmployeeF 0.606* * * 0.682** 0.781** 0.700** ** 1 Ind growth 0.753** 0.726** * ** The correlaton s sgnfcant at the 0.01 level (2-taled). * The correlaton s sgnfcant at the 0.05 level (2-taled).

10 IFC Bulletn No Appendx 2: Correlaton matrx (constructon) Balance of economc actvty Balance of busness envronment questons questons Balance of other questons VolumeC VolumeF DemandF RsksC RsksF SubndC SubndF PrceC WageF EmployeeF VolumeC 1 VolumeF 0.870** 1 DemandF 0.843** 0.931** 1 RsksC RsksF SubndC 0.565* 0.555* SubndF 0.611* 0.779** 0.787** PrceC ** 0.646** * 1 WageF 0.678** 0.826** 0.896** ** 0.624* 1 EmployeeF 0.764** 0.926** 0.952** ** 0.760** 0.879** 1 Const growth 0.764** 0.833** 0.785** ** * 0.819** ** The correlaton s sgnfcant at the 0.01 level (2-taled). * The correlaton s sgnfcant at the 0.05 level (2-taled).

11 212 IFC Bulletn No 33 Appendx 3: Correlaton matrx (trade) Balance of economc actvty Balance of busness envronment questons questons Balance of other questons VolumeC VolumeF DemandF RsksC RsksF SubndC SubndF PrceC WageF EmployeeF VolumeC 1 VolumeF DemandF 0.584* 0.727** 1 RsksC RsksF * 1 SubndC 0.650** 0.523* 0.634* SubndF ** 0.619* PrceC WageF EmployeeF * * * 1 Trade growth 0.719** ** The correlaton s sgnfcant at the 0.01 level (2-taled). * The correlaton s sgnfcant at the 0.05 level (2-taled).

12 IFC Bulletn No Appendx 4: Correlaton matrx (servces) Balance of economc actvty Balance of busness envronment questons questons Balance of other questons VolumeC VolumeF DemandF RsksC RsksF SubndC SubndF PrceC WageF EmployeeF VolumeC 1 VolumeF 0.661** 1 DemandF 0.732** 0.865** 1 RsksC 0.553* 0.527* 0.532* 1 RsksF SubndC 0.546* * 0.527* SubndF * 0.554* PrceC WageF EmployeeF Serv growth * ** The correlaton s sgnfcant at the 0.01 level (2-taled). * The correlaton s sgnfcant at the 0.05 level (2-taled).

13 Appendx 5: Abbrevatons of varables VolumeC VolumeF StockF DemandF RsksC RsksF SubndC SubndF PrceC WageF EmployeeF Ind growth Const growth Trade growth Serv growth Volume change (current) Volume change (future) Stock change (future) Demand change (future) Rsks change (current) Rsks change (future) Economc stuaton of segment (current) Economc stuaton of segment (future) Prce change (current) Average wage change (future) Employees change (future) Quarterly growth rate of ndustry value added Quarterly growth rate of constructon value added Quarterly growth rate of trade value added Quarterly growth rate of servce value added 214 IFC Bulletn No 33

14 References Abeyasekera, Savtr (2000): Quanttatve analyss approaches to qualtatve data: why, when and how, Statstcal Servces Centre, Unversty of Readng. Enrco D Ela (2005): Usng the results of qualtatve surveys n quanttatve analyss, Workng Paper no 56. Henzel, Steffen, and Tmo Wollmershäuser (2005): An alternatve to the Carlson-Parkn method for the quantfcaton of qualtatve nflaton expectatons: evdence from the Ifo World Economc Survey, Ifo Insttute for Economc Research (Ifo), Unversty of Munch, Workng Paper no 9, June. Nlsson, Ronny (2000): Confdence ndcators and composte ndcators, paper for presentaton at the Centre for Internatonal Research on Economc Tendency Surveys (CIRET) Conference n Pars, October. Schenker, Rolf (2008): Methods of quantfyng qualtatve data: a survey of recent project research and results, ETH (Swss Federal Insttute of Technology), 29 July. IFC Bulletn No