Validation of Compensation of Employees Survey

International Comparison Program Validation of Compensation of Employees Survey Draft version Operational Guide

Contents INTRODUCTION... 3 1.1. DATA TO BE COLLECTED... 3 1.2. OCCUPATION TYPES... 3 1.3. AVERAGE REMUNERATION... 4 VALIDATION PROCESS... 4 INTRA-COUNTRY VALIDATION... 5 1.4. INITIAL DATA VALIDATION... 5 1.4.1. Within the same level of experience... 5 1.4.2. Between different levels of experience... 6 1.5. FINALIZATION OF DATA... 8 INTER-COUNTRY VALIDATION... 8 1.6. INITIAL DATA VALIDATION... 9 1.7. VALIDATION TABLE ANALYSIS... 12 1.8. FINALIZATION OF DATA... 12 GLOBAL VALIDATION... 12 ANNEX: COMPARISON OF 2011 ITEMS WITH 2005 ITEMS... 13 2

Validation of Compensation of Employees Survey 1 1. Introduction 1.1. Data to be collected As explained in the previous chapter on Government ( Approach and Data Requirement ), three sets of data are required to be submitted for the survey on compensation of government employees. The first is Compensation of Government, which is a set of remuneration data of government employees. The second is Government Expenditure data, which is a necessary source to obtain weights. The last is Pay & Employment Structure Indicators, which provides useful information for the validation purpose. PPPs for compensation of government employees are calculated using these data sets. Table1: Data to be collected Compensation of Government Employees Government Expenditure Pay & Employment Structure Indicators From Official government pay scales Final government accounts Relevant official statistics For 44 typical occupations (4 levels of experience each) 3 categories of government (General = Central + Subnational) General indicators Data Data Collection Form (DCF): Basic pay Cash allowances Income in kind Employers social security contributions Information on hours worked Questionnaire: Wages and salaries in cash Employers contribution to social security funds Benefits in kind Information related to fixed capital formation Aggregated indicators: General Indicators such as GDP and population Government Recurrent Expenditures Indicators Wage bill Indicators Employment Indicators Note Covers 3 BH (Health, Education and Collective services) Report separately for Health, Education and Collective services Additional ratios will be computed automatically (ICP kit) 1.2. Occupation types The list of occupations for ICP 2011 government compensation survey has 44 occupations and, for each occupation, the remunerations for 4 levels of experience, 0-year experience (starting salary), 5-year 1 This chapter is prepared by Mizuki Yamanaka, with input from Michel Mouyelo-Katoula, Nada Hamadeh, and Virginia Romand. 3

experience, 10-year experience, and 20-year experience, will be recorded. Therefore, the total number of items for this survey will be 176 (44 times 4), if all the items are available in a country. Some of the 44 occupations, such as senior government official, driver, or office cleaners, are used for the calculation of PPPs for the three basic headings (BHs), collective service, health service, and education service. Others, such as hospital doctor, university teacher, or database administrator, are used only for one of the three BHs. In sum, 31 occupations are used for the calculation of BH PPPs of government collective service, 18 are used for government health service, and 16 are used for government education service. 1.3. Average remuneration In most countries, governments have official national pay scales. In this case the number of observations per occupation will be one. However, if a country does not have a unified national pay scale and subnational governments have different pay scales, multiple observations would be collected. In this case, the average remuneration for each occupation would be the weighted average of the sub-national pay scales, weighted by the number of employees at sub-national level. 2. Validation Process Intra-country validation Inter-country validation Global validation The validation of data from the compensation of government employees survey has three stages, intracountry validation, inter-country validation and global validation as same as the validation of household consumption survey data does. After the compilation of annual data in a country, intra country validation is implemented by the NCA. This stage involves initial data validation and the finalization of data. Unlike data validation for household consumption survey, there are no statistical tests at this stage. This is because, as mentioned before, the number of observations would be relatively small. The NCAs should submit their data to their RCA after data validation at national level. Then, at regional level, inter-country validation is conducted by the RCA and the NCAs. At this stage, initial data validation, validation table analyses, temporal analysis, and finalization of data should be implemented. After data has been transmitted from RCAs to the Global Office, the Global Office will conduct the global level validation in cooperation with the RCAs and NCAs. In the following sections, validation steps that are unique to this survey are mainly explained. Steps which are similar to the ones for Household Consumption validation are shown just with an arrow. For a detailed explanation on them, please refer to the chapter on Household consumption validation. 4

3. Intra-country Validation Intra-country validation Inter-country validation Global validation After data collection has finished, country level data validation will be conducted by NCAs. As mentioned, because of the lower number of observations for each item, this level of validation for compensation of government employee survey does not involve statistical tests. However, there are some validation steps that are unique to this survey and they are explained below. 1.4. Initial data validation Inital data validation Finalization of data Step 1 Step 2 Step 3 Add remunerations and metadata to data collection tool Check added remunerations and metadata for errors and discrepancies Check that remunerations are plausible within the same occupation 1.4.1. Within the same level of experience It is recommended to follow the process specified earlier (1.3 average remuneration) in order to obtain remuneration data. Then, in most cases, only one observation will be recorded after the process. However, in the case of each sub-national government having different salary scales, comparisons between the remuneration of the same occupation with the same level of experience should be done before obtaining the weighted average for each occupation. Even if sub-national governments have different pay scales, remuneration for the same occupation with a same level of experience cannot differ so much. Box 1: Example of validation within the same occupation and within the same level of experience Location Occupation Level of experience Remuneration Observation 1 Municipal A Database administrator Observation 2 Municipal B Database administrator 5-year experience 35,256 5-year experience 74,662 In the above case, the remuneration in Municipal A more than doubles compared to that in Municipal B, for the same occupation with the same level of experience. The data shows an extreme difference and need to be flagged for further investigation on the possibility of having a price error or a product error. If there is an acceptable economic reason behind the differential, the price should not be removed or edited. 5

If there are multiple observations under an item, after all the observations are verified, they need to be averaged. In doing so, it is recommended to obtain data for the number of employees from which the remuneration data is quoted and take a weighted average, as mentioned earlier. All these processes and data used in the process are needed to be noted and reported to the RC when the data are submitted to the RC. 1.4.2. Between different levels of experience Normally, in an economic environment, one s remuneration increases if he or she gains work experience in the same course of work, or at least, it does not decrease. By comparing remunerations within the same occupation and between different levels of experience, the plausibility of this economic aspect of the observations should be checked. Box 2: Example of validation within the same occupation and between different levels of experience Location Occupation Level of experience Remuneration Observation 1 Municipal C Hospital doctor 0-year experience 50,653 Observation 2 Municipal C Hospital doctor 5-year experience 45,367 In a private hospital, it is not possible that a doctor with less experience earns more, since in some economies the salary they obtain would be completely based on the competence or popularity of each doctor. However, usually, government has a set of salary scales and it would be unusual if the salary of 5-year doctor is lower than that of a doctor who started working just recently. Therefore, these observations would need to be flagged for further investigation. Step 4 Check that remunerations are plausible between related occupations In the 44 occupations, there are several occupations that have relations in terms of skill, knowledge, proficiency or position, which would result in a difference in their remunerations. Usually, becoming a hospital doctor needs advanced degree and proficiency compared to becoming a hospital nurse. Therefore, in most cases, a hospital doctor earns a larger salary than a nurse does. Also, there are some occupations that usually come with higher/lower salary compared to other occupations, even if there is no direct relation between the occupations. A typical example of this is a member of parliament. The remuneration for a member of parliament is expected to be higher than that of personnel professional or a payroll clerk. Box 3: Example of validation between related occupations Location Occupation Level of experience Remuneration Observation 1 Municipal C Hospital doctor 5-year experience 62,556 Observation 2 Municipal C Hospital nurse 5-year experience 75,698 As mentioned, in most cases, a hospital doctor earns more than a hospital nurse does. The above observations from government pay scales could have either a price error or a product error. Therefore, the above observations need to be flagged and to be checked again. 6

However, it needs to be stressed that this analysis of related occupations needs to be limited within the same level of experience. Level of skill, knowledge, or proficiency is a factor that strongly affects remuneration. However, level of experience is also a remuneration determining factor. These two factors are totally independent and should not be mixed in this course of validation. Box 4: Example of erroneous validation between related occupations Location Occupation Level of experience Remuneration Observation 1 Municipal C Hospital doctor 0-year experience 62,556 Observation 2 Municipal C Hospital nurse 20-year experience 75,698 As explained above, even though becoming a doctor needs more education and proficiency in the field, level of experience has independent influence on remunerations. Then, comparing the salaries of very experienced nurse and that of novice doctor would be very much misleading. Mixing independent factors in an analysis should be avoided. Step 5 Check that remunerations are temporally plausible by comparing them to data from previous ICP round The occupation list for the compensation of government employee survey is updated from the list of the 2005 round. The 2005 list has 39 occupations and level of experience is not necessarily specified for all the occupations. In the ICP 2011 round, we have 44 occupations. Even though there are some new items that were not on the 2005 list, most of the occupations in the 2005 list are still in the 2011. In some cases, the name of the item has been changed. Table 2: Example of comparable occupations between ICP 2011 list and ICP 2005 list ICP 2011 ICP 2005 Code Occupation Code Occupation 32 Firefighter 215 Fire Fighter 33 Policeman/woman 213 Policeman/woman 34 Prison guard 214 Prison Guard 35 Driver (general duty) 221 Chauffeur 36 Office cleaners 212 Cleaner 38 Messengers 209 Messenger 39 Army: Commander of Infantry Regiment 402 Army: Commander of Infantry Regiment Then, if CPI data for the three basic headings are available, it is recommended to adjust the 2005 remuneration data to 2011 CPI-adjusted value. This makes it easier to conduct the temporal analysis. However, even if the specific CPI data needed to deflate the 2005 data to 2011 value are not available, the pattern of government pay scales can be checked. In this case, the general economic situation of the country regarding the change in the price level needs to be taken into consideration. If the country is experiencing inflation, remunerations would also increase to reflect the inflation to some degree, and vice versa. 7

Step 6 Compare remuneration data and expenditure data by using structure indicators. (Remuneration x number of employment <==> Expenditure) Information on Pay & Employment Structure Indicators provides important clues for verifying the plausibility of remuneration data. One of the ways of using the data is to check the relevancy in the relation of remuneration, employment and expenditure. Theoretically speaking, for an occupation, the government expenditure should match the amount of the remuneration multiplied by the number of employment for the occupation. Because of the feasibility in implementation, NCAs are not asked to do research on population of each occupation for each level. However, the questionnaire of the indicators asks key elements such as number of government employees in education, health and other collective service, and their aggregates. Starting from aggregated level such as expenditure, average remuneration and the number of employee in total public sector, NCAs can verify to some degree whether the relationship of remunerations and expenditures makes sense or not. If the expenditure is too large or too small compared to the remuneration multiplied by the number of employee in each field, something could be wrong in the data. In the case, the NCA needs to check breakdown figures to explore where the problem comes from. However, it is worth stressing that those data cannot give any exact matches in numerical value. Because of the difference in the system of data collection in expenditure side and price side, and also because of feasibility issues, it only gives rough sketches of the relationship. Government Expenditure data should be validated in line with the validation of national accounts data. However, the above step also plays a role of validation of the expenditure data and Pay & Employment Structure Indicators data. If NCAs find problems in the process, not only the remuneration but also the expenditure and general indicators need to be further investigated for verification. Step 7 Analyze price data and metadata for flagged cases 1.5. Finalization of data Inital data validation Finalization of data Step 1 Confirm remuneration data, expenditure data, indicators and metadata to be intra-country validated Step 2 Submit remuneration data, expenditure data, indicators and metadata to Regional Coordinating Agency 4. Inter-country Validation Intra-country validation Inter-country validation Global validation After receiving all the data and the metadata from NCAs, the RCA takes initiative to conduct regional level validation. As mentioned in validation for HHC survey, this level of validation is a collective process 8

involving the regional coordinator and a group of countries. Remuneration data collected in countries in the region need to be checked on their accuracy and comparability between countries. Even though the RCA leads the process, active involvement of NCAs is essential, since NCAs cooperation is required in order to investigate the data when the RCA finds any potential problems. 1.6. Initial data validation Inital data validation Validation Table Analysis Finalization of data Step 1 Step 2 exchange rate Add remunerations and metadata to data validation tool Convert remunerations in local currency into base currency using annual average Inter-country validation helps to check whether there are extreme values among the average remunerations as well as discrepancies with the reported metadata for each item within a basic heading. Therefore, the average remunerations need to be converted to a same base to make them comparable. Then, firstly, the remunerations expressed in a local currency unit need to be converted and expressed in a base currency unit using annual average exchange rate. Step 3 Re-base data from all the countries to reference hours worked Next, the remunerations need to be converted into remunerations that are based on the same number of hours worked, across the countries in the region, since if they are based on different reference numbers of hours worked, they are not comparable. Box 5: Example of rebasing to reference hours worked Country Remuneration Regular Hours worked Actual Hours Worked per week per week Never land 60,000 32 25 Timberland 70,000 40 60 Before re-based, data from the two countries cannot be comparable. In the above table, if you just look at remunerations, Timberland has the higher amount. However, it is obvious that the salary in Timberland is not so high if we take into consideration the number of hours worked per week. They need to be rebased so that they represent a figure based on the same number of hours worked per week. Country Remuneration Rebased on Regular Rebased on Actual Hours (40 hours) Hours (40 hours) Never land 60,000 (a) {(a)/32}*40 = 75,000 {(a)/25}*40 = 96,000 Timberland 70,000 (b) {(b)/40}*40 = 70,000 {(b)/60}*40 = 46,667 After the conversion, it becomes clear that Never Land has higher salary for the reference number of hours. 9

In many countries it is an accepted practice that government employees work less than the regular (official) hours per week. Therefore, it is requested to report the best estimate of the hours per week actually worked by employees for each occupation. Then, for the conversion, the global office recommends to convert remunerations into ones based on actual hours worked, if the information is available. After being converted by reference hours and exchange rate, the average prices of each NCA are checked against the average prices of the other NCAs in the region. Remunerations from countries need to be brought into a data validation tool with which the RC can easily compare prices across the countries. Box 6: Example table for validation at regional level County County County C County County E A B D 10 University teacher 9,086 16,182 17,109 28,697 35,387 12 Primary school teacher 1,214 4,652 5,108 9,920 25,562 13 Secondary school teacher 1,959 5,259 4,221 14,313 26,960 16 Database administrator 1,400 17,396 3,799 4,744 23,362 30 Building caretaker 965 5,947 1,931 2,542 8,605 32 Firefighter 560 7,569 18,199 4,660 13,719 33 Policeman/woman 1,380 8,235 3,257 5,422 12,501 34 Prison guard 836 2,457 3,820 5,422 13,918 35 Driver (general duty) 999 4,308 2,531 3,558 8,024 36 Office cleaners 965 3,499 1,858 3,558 8,595 Step 4 Check that remunerations are plausible within a country First, the plausibility of remuneration data need to be validated vertically. (As in the Box 6, here vertical means within each country. ) Though what should be done in this step is almost the same as the validation done at the country level (See Step A3 and Step A4), it is important and beneficial that this is done by the RCA who has knowledge on the situation of whole the region. Step 5 Check that remunerations are plausible across the countries Next, the plausibility of remunerations data needs to be validated horizontally. (As in the Box 6, here horizontal means across the countries. ) Now that the data are based on the reference quantity and expressed in a base currency, the remunerations can be compared across the countries. The RCA needs to employ their knowledge of the economic structure and situation of each country in the region. 10

Box 7: Example table for validation at regional level County County County C County County E A B D 10 University teacher 9,086 16,182 17,109 28,697 35,387 12 Primary school teacher 1,214 4,652 5,108 9,920 25,562 13 Secondary school teacher 1,959 5,259 4,221 14,313 26,960 16 Database administrator 1,400 17,396 3,799 4,744 23,362 30 Building caretaker 965 5,947 1,931 2,542 8,605 32 Firefighter 560 7,569 18,199 4,660 13,719 33 Policeman/woman 1,380 8,235 3,257 5,422 12,501 34 Prison guard 836 2,457 3,820 5,422 13,918 35 Driver (general duty) 999 4,308 2,531 3,558 8,024 36 Office cleaners 965 3,499 1,858 3,558 8,595 For example, in the table from Box 7, there is a big discrepancy between the remuneration for firefighters in Country A and that in Country C. This could be a problem or not a problem at all. Since the payment structure and the price level of the countries differ country by country, the knowledge of RCA would be very precious to find potential problems. In this case, even though the remunerations for other occupations in Country A are much lower than those of Country C, the gap for the firefighter is still too large. Then, these would be flagged for further investigation. Though validation table analyses in the following steps will bring more objective criteria for finding potential errors, it is useful if RCAs can find any potential problems at this initial stage.. Box 8: Application of information from Pay & Employment Structure Indicators The following indicators are automatically calculated in the Pay & Employment Structure Indicators if relevant information is entered. These indicators give insights on the employment, payment and economic structure of each country and reinforce the knowledge of RCAs that is needed when comparing data across countries. (1) Wage bill Indicators Government wage bill per GDP for General/Central/Sub-national Government (2) Compression Ratios (Ratio of average total remuneration) For Health/Education/Collective respectively: Managerial-professional ratio Managerial-clerical ratio (3) Public Sector Remuneration per GDP/capita For Health/Education/Collective respectively: Managerial Professional Clerical (4) Employment Indicators Employment per capita Employment per labor force (LF) 11

Step 6 previous ICP round Step 7 Check that remunerations are temporally plausible by comparing them to Analyze price data and metadata for flagged cases 1.7. Validation Table Analysis Inital data validation Validation Table Analysis Finalization of data Identical to the household consumption survey validation process, analyses using tables such as Quaranta tables and Dikhanov tables make it possible to conduct detailed comparisons. Analytical tables provide indices for more objective validation, such as CV, CUP-ratio, etc. Please follow the validation steps in household consumption survey for actual steps to be taken. Also, temporal analyses with ICP 2005 data using indices such as PLIs and PPPs make the series of analyses more robust. 1.8. Finalization of data Inital data validation Statistical tests Finalization of data After the confirmation of remuneration data, expenditure data, indicators and metadata to be intercountry validated, the RCA needs to submit the data to the Global Office for the global level validation process across regions. 5. Global Validation Intra-country validation Inter-country validation Global validation The global office takes initiative to conduct the global level validation across regions. Similar to the household consumption survey data validation process,, active involvement of RCAs and NCAs is crucial in this final validation stage. The objective is to ensure that remunerations collected across the regions are comparable and that the linked global PPPs are plausible in wider terms. 12

6. Annex: Comparison of 2011 items with 2005 items ICP 2011 ICP 2005 Code Occupation ISCO 08 ISCO 88 Code Occupation 1 Member of parliament 1111 1110 (Not in 2005 list) 2 Senior government official 1112 1120 (Not in 2005 list) 3 Hospital manager 1120 1210 110 Hospital Chief Executive 4 Data processing manager 1330 1226 (Not in 2005 list) 5 Secondary school principal 1345 1229 305 Head Teacher 6 Government statistician 2120 2121 / (Not in 2005 list) 2122 7 Hospital doctor 2211 2221 / 101 Doctor, Head of Department* 1229 2221 / 102 Doctor, (20 years of seniority)* 1229 2221 / 103 Doctor (10 years of seniority)* 1229 8 Specialist doctor 2212 2212 (Not in 2005 list) 9 Hospital nurse 2221 2230 / 104 Nurse, Head of Department* 3231 / 3232 2230 / 105 Nurse, Operating Theatre* 3231 / 3232 2230 / 106 Nurse 3231 / 3232 10 University teacher 2310 2310 304 University Lecturer 11 Vocational education 2320 2310 / (Not in 2005 list) teacher 2320 12 Primary school teacher 2341 2331 / 302 Primary Teacher 3310 13 Secondary school teacher 2330 2320 303 Secondary Teacher 14 Government accountants 2411 2411 (Not in 2005 list) 15 Human resources 2423 2412 (Not in 2005 list) professional 16 Database administrator 2522 2131 224 Data-base Administrator 17 Judge 2612 2422 (Not in 2005 list) 18 Government economist 2631 2441 (Not in 2005 list) 19 Laboratory assistant 3212 3211 109 Laboratory Assistant 20 Auxiliary nurse 3221 2230 107 Nursing Auxiliary 21 Medical records clerk 3252 4143 (Not in 2005 list) 22 Office supervisors 3341 3431 / 202 Executive Official (skill level III) * 3439 / 3442 / 3443 / 3449 3431 / 203 Executive official (skill level IV)* 3439 / 3442 / 3443 / 13

3449 23 Medical secretary (hospital) 3344 4115 / 111 Secretary (Hospital) 4111 / 4112 24 Customs inspector 3351 3441 (Not in 2005 list) 25 Computer operator 3513 3121 204 Computer Operator 26 Secretaries (not medical) 4120 4115 / 207 Secretary (not Hospital) 4111 / 4112 27 Accounting and bookkeeping 4311 4121 205 Bookkeeping Clerk clerks 28 Payroll clerks 4313 4121 (Not in 2005 list) 29 Cooks 5120 5122 112 Cook (not Head Cook) 30 Building caretaker 5153 9141 211 Building Caretaker 31 Teacher's aides 5312 5131 (Not in 2005 list) 32 Firefighter 5411 5161 215 Fire Fighter 33 Policeman/woman 5412 5162 213 Policeman/woman 34 Prison guard 5413 5163 214 Prison Guard 35 Driver (general duty) 8322 8322 221 Chauffeur 36 Office cleaners 9112 9132 212 Cleaner 37 Kitchen helpers 9412 9132 (Not in 2005 list) 38 Messengers 9621 9151 209 Messenger 39 Army: Commander of 0110 0110 402 Army: Commander of Infantry Infantry Regiment Regiment 40 Navy: Commander of Frigate 0110 0111 404 Navy: Commander of Frigate 41 Air Force: Fighter Pilot/Wing 0110 0112 406 Air Force: Fighter Pilot/Wing Commander Commander 42 Army: Private of Infantry 0310 0113 401 Army: Private of Infantry 43 Navy: Able Seaman 0310 0114 403 Navy: Able Seaman 44 Air Force: Airman (ground 0310 0115 405 Air Force: Airman (Ground Crew) crew) *Items with the asterisk need to consider level of experience when comparing with a 2011 item. 14