Optimum Cut-off Point of the Japanese Short Version of the Effort-Reward Imbalance Questionnaire

Transcription

1 340 J Occup Health, Vol. 55, 2013 J Occup Health 2013; 55: Journal of Occupational Health Optimum Cut-off Point of the Japanese Short Version of the Effort-Reward Imbalance Questionnaire Sumiko Kurioka 1, Akiomi Inoue 2 and Akizumi Tsutsumi 3 1 Department of Health Policy and Management, Graduate School of Medical Science, University of Occupational and Environmental Health, Japan, 2 Department of Mental Health, Institute of Industrial Ecological Sciences, University of Occupational and Environmental Health, Japan and 3 Department of Public Health, Kitasato University School of Medicine, Japan Abstract: Optimum Cut-off Point of the Japanese Short Version of the Effort-Reward Imbalance Questionnaire: Sumiko KURIOKA, et al. Department of Health Policy and Management, Graduate School of Medical Science, University of Occupational and Environmental Health, Japan Objectives: The theoretical threshold (effort-reward ratio >1.0) may not be ideal for the Japanese short version of the effortreward imbalance (ERI) questionnaire. We aimed to seek the optimum cut-off point. Methods: We administered the original and short versions of the ERI questionnaire with a psychological distress scale to randomly selected workers (n=1,489) in a web-based survey. We evaluated kappa statistics and conducted receiver operating characteristics curve analyses. Cut-off values of the short version effort-reward ratios at 0.1 intervals in the range of were tested using the criterion of an effort-reward ratio >1.0 for the original version. Results: The short version questionnaire had acceptable reliabilities. When using the theoretical cut-off point, the prevalence of high-risk groups was 63.2% for the short version compared with 18.9% for the original version, and their agreement was slight. Kappa agreements and receiver operating characteristics curve analyses suggested that a short-version effort-reward ratio of around 1.3 and 1.4 was the most equivalent to the original criterion. Regression equation procedures supported the findings, and ERI defined by the cut-off values showed significant associations with an external criterion (psychological distress) with minimal estimation error. Because the highest but only moderate kappa agreements with the risk group defined by the original criterion were obtained when setting as the cut-offs, we considered >1.4 as optimal. Conclusions: This empirical investigation suggests the cut-off value Received Oct 7, 2012; Accepted Jun 10, 2013 Published online in J-STAGE Jul 26, 2013 Correspondence to: A. Tsutsumi, Department of Public Health, Kitasato University School of Medicine, Kitasato, Minamiku, Sagamihara, Kanagawa , Japan of >1.4 for the Japanese short version of the ERI questionnaire screens out the ERI group with the most compatibility with the original version. (J Occup Health 2013; 55: ) Key words: Cut-off point, Effort-reward imbalance, K6, Questionnaire, Short version, Web-based survey The effort-reward imbalance (ERI) model posits that effort at work is spent as part of a socially organized exchange process to which society at large contributes in terms of occupational rewards. The ERI model assumes that experiencing a lack of reciprocity in terms of high costs and low gains results in emotional distress and adverse health effects 1). ERI at work has been correlated with psychiatric and physical diseases, such as depression and cardiovascular diseases 2 6). The ERI questionnaire consists of three scales, effort, reward and overcommitment. The effort scale measures demanding aspects and obligations of the work environment, whereas the reward scale measures gains offered or promised via social and contractual reciprocity in terms of esteem, financial compensation, and career opportunities, including job security. The overcommitment scale measures attitudes, behaviors, and emotions that reflect excessive endeavors combined with a strong desire for approval and esteem in one s working life. Siegrist recommended the fixation of cut-off values on the basis of theoretical and mathematical considerations as follows: a ratio of the effort and reward scales weighted for the number of items where a value >1.0 indicates the critical high cost/low gain condition (i.e., high-risk for work stress) and scale scores in the upper tertile indicate a critical threshold of overcommitment 1, 7). Recently, a short version of the ERI questionnaire was developed 8). The validity of the short version has been demonstrated in European countries. The short

2 Sumiko KURIOKA, et al.: Japanese Short Version of the ERI Questionnaire 341 version has been shown to be well adapted to the ERI model, and serves as a useful tool for epidemiological studies focusing on the adverse effects of work and employment on health 8 11). In several Japanese settings, however, the prevalence of a high-risk group defined by the short version is considerably high 12). The threshold of the ERI (i.e., effort-reward [ER] ratio >1.0) may not be appropriate, although the cut-off point of the short version has not been clearly demonstrated. Discrepancies of high-risk groups defined by different versions of an occupational stress index based on the same theoretical concept may disturb occupational health practices, such as monitoring occupational stress state over time. Therefore, it is necessary to adjust the criterion of newly developed instruments to that of the prevailing original version. Because there is little information on the characteristics of high-risk groups defined by these different versions tested in the same working population, we aimed to seek the cut-off point for the short version of the ERI questionnaire that is the most equivalent to that of the original version. Subjects and Methods Participants The study population consisted of workers who were registrants of a market research company. The company registered 967,445 Japanese aged 20 year living across Japan, including 327,097 workers. Of them, a total of 7,841 workers were randomly selected and invited to the study in order to obtain a sample size of approximately 1,500. The invitation explained the study and that electronic money would be offered to the participants. As a result, a total of 1,547 workers agreed to participate (response rate: 19.7%). We asked the workers to complete a web-based selfadministered questionnaire twice. Of them, 51 workers who answered the questionnaire only once and seven workers who reached the general retirement age (i.e., 65 year) were excluded. Therefore, the final number of workers included in the analysis was 1,489 (1,023 men), and their mean age was 39.6 year (SD, 9.2). Procedure We conducted web-based self-administered surveys in which the participants were asked to answer the original and short versions of the ERI questionnaire at two-week intervals in January The participants were randomly dichotomized to avoid the influence of the sequence of the questionnaire. For one group, the original version of the ERI questionnaire and a short screening scale for psychological distress (K6 scale) 13) were provided first. Two weeks later, the short version of the ERI questionnaire and the K6 scale were provided. For the other group, the short version of the ERI questionnaire and the K6 scale were provided first. Two weeks later, the original version of the ERI questionnaire and the K6 scale were provided. Both groups were asked to answer the questions about demographic characteristics the first time. This study was approved by the Ethics Committee for Medical Care and Research, University of Occupational and Environmental Health, Japan (approval number , December 10, 2010). Measurements 1) Original version of the ERI questionnaire The original version of the ERI questionnaire consists of two components, a situation-specific component (i.e., effort and reward) and a personspecific component (i.e., overcommitment) 14). In our study, we excluded the person-specific component and focused only on the situation-specific component. This is because our aim was to seek the optimum cut-off value for the updated situation-specific components 8). Effort is measured by six items, and reward is measured by 11 items. The effort scale includes relevant features of a demanding daily work environment, and all six items are affirmative sentences for distress (e.g., I have constant time pressure due to a heavy work load ). Participants were asked to respond to the statements on a five-point Likert scale ( does not apply, does apply, but the respondent does not consider her/ himself to be distressed, does apply, and the respondent considers her/himself somewhat distressed, does apply, and the respondent considers her/himself distressed, and does apply, and the respondent considers her/himself very distressed ). The effort scale score ranges from 6 to 30, with higher scores indicating higher effort 14). The reward scale measures constructs related to financial, esteem-related, and organizational rewards (e.g., job security). Four items are for affirmative items for distress (e.g., My job security is poor ), and they are answered with the same response format as the effort scale. Seven items are for non-affirmative items for distress (e.g., I receive the respect I deserve from my superiors ), and participants were asked to respond to the statements on a five-point Likert scale ( apply, does not apply, but the respondent does not consider her/himself to be distressed, does not apply, and the respondent considers her/ himself somewhat distressed, does not apply, and the respondent considers her/himself distressed, and does not apply, and the respondent considers her/ himself very distressed ). The reward scale score ranges from 11 to 55, with higher scores indicating

3 342 J Occup Health, Vol. 55, 2013 higher reward 14). The ER ratio was computed by putting the effort score in the numerator and the reward score in the denominator. The resulting score was multiplied by a correction factor to adjust for the unequal number of items in the two scales. The first Japanese version of the ERI questionnaire was developed by using an independent back-translation procedure and has been evaluated for its reliability and validity in several Japanese surveys 4, 5). The original questionnaire of the current study was updated by modulating the first two-step response format to a one-step response format according to Siegrist 14). 2) Short version of the ERI questionnaire The short version of the ERI questionnaire is based on relevant psychometric tests drawn from a large prospective epidemiological study 8). The short version was developed to reflect three distinct features. First, each item represents a core aspect of the respective dimension (including the subcomponents salary, job promotion, job security and esteem of the reward component). Second, the frequency of the respective job content is relatively high in various occupational groups. Third, each item meets established quality criteria and exhibits a satisfactory correlation coefficient with the respective scale. The short version of the ERI questionnaire has 10 items. Effort is measured by three items, and reward is measured by seven items. Participants were asked to respond to the statements on a four-point Likert scale ( strongly agree, agree, disagree and strongly disagree ). All the items for effort are affirmative for distress. Four of the items for reward are non-affirmative for distress, and the remaining three are affirmative for distress; the scoring for the affirmative items is reversed. Scores on each scale range from 3 to 12 for effort and 7 to 28 for reward, with higher scores indicating higher effort and reward. The Japanese short version of the ERI questionnaire was developed by one of the authors (AT). He performed the Japanese translation of the questionnaire with independent back-translation to the English version. The back-translated scale was reviewed and approved by Professor Siegrist, University of Düsseldorf. 3) K6 scale The K6 scale is a short screening scale for psychological distress developed by Kessler and colleagues 13). The K6 scale has six items that ask about frequently experienced symptoms of psychological distress (e.g., feeling so sad that nothing can cheer you up) during the past 30 days. Participants were asked to respond to the statements on a five-point Likert scale ( all of the time, most of the time, some of the time, a little of the time and none of the time ). Scores range from 0 to 24, with higher scores indicating greater psychological distress. The Japanese version of the K6 scale was developed in accordance with the WHO translation guidelines by experts in psychiatric interviewing and/or psychiatric epidemiology 15, 16). The K6 scale has been evaluated for its reliability and validity in community surveys in Japan. A cut-off point of 9/10 has been used to screen for mood or anxiety disorders 17), which is what we used in this study. 4) Demographic characteristics We measured relevant confounding variables, such as sex, age (10-year intervals), educational attainment (junior high school, high school, junior college or higher professional school, and university), employment status (regular, non-regular, and others), occupation (managerial or administrative workers, professional workers, technical workers, clerical workers, sales and service workers, manual workers, and others), working hours per week (continuous variable), and net income per year (<3 million yen, <5 million yen, <8 million yen, <10 million yen, <15 million yen, and 15 million yen). Statistical analyses We calculated Cronbach s alpha coefficient for each scale. Kappa statistics were calculated to evaluate the agreement between the ERI defined by the original versus the short versions; the short version ER ratios at 0.1-intervals in the range of were tested. The following interpretations of the kappa statistic were used 18) : indicated slight agreement, indicated fair agreement, indicated moderate agreement, indicated good/substantial strength of agreement, and indicated a very good/almost perfect agreement. We then conducted receiver operating characteristic (ROC) curve analyses. Sensitivity, specificity, and the Youden index were calculated. Assuming that sensitivity and specificity are of equal importance, the Youden index indicates an optimum cut-off point as a linear function of sensitivity and specificity 19). We compared the areas under the curve (AUC) for the different cut-off values of the short version using an external criterion of an ER ratio >1.0 for the original version. We conducted two complementary analyses. An alternative method used a simple regression equation to develop correction score coefficients. The logarithmic ER ratios 20) from the original scales were regressed on the short version logarithmic ER ratios to yield a regression coefficient and a constant term that could then be used to make the short version ER ratios equivalent to the original version ER ratios. Because there is criticism that the regression method essentially mixes all sources of discrepancy together and does not allow identification of any intermedi-

4 Sumiko KURIOKA, et al.: Japanese Short Version of the ERI Questionnaire 343 ate sources 21), we repeated this procedure adjusting for demographic variables (sex [men as a reference], age [15 24-year-old group as a reference], educational attainment [junior high school group as a reference], employment status [regular group as a reference], occupation [managerial or administrative workers group as a reference], working hours per week [continuous variable], and net income per year [<3 million yen group as a reference]) (see Appendix for regression equations). To examine the validation of cut-off values, we performed logistic regression analysis using psychological distress defined by K6 as an external criterion (dependent variable). The same demographic variables as in the test of the regression equation were adjusted in the model. We analyzed the combined data of two groups because the prevalence of the high-risk group defined as an ER ratio >1.0, kappa statistics, and the chi-square test of the high-risk groups defined by each scale showed little difference when analyzed separately. All associations were inferred with an alpha level of We analyzed the data using the SPSS 19 software package (IBM, Armonk, NY, USA). Results Table 1 shows the characteristics of the 1,489 participants and summary measures of both versions of the ERI questionnaire. A total of 82.5% of participants were white-collar (managerial or administrative workers, professional workers, technical workers, clerical workers, and sales and service workers), 86.4% were regular workers, and 53.2% were university graduates. The mean working hours per week was 51.6 hours, and 26.1% of participants worked longer than 50 hours per week. The prevalence of the highrisk group defined as an ER ratio >1.0 was 18.9% for the original version and 63.2% for the short version. The mean values of ER ratios were significantly different between the original version (0.71) and the short version (1.19) (p<0.001), although they were significantly correlated (r=0.628). The mean values of the ER ratio were uniformly higher when applying the short version across all demographic categories. The effect of order showed essentially no difference between the two questionnaires. The mean values of the ER ratio of the original and the short versions were 0.71 and 1.19 among the group in which the original version of the ERI questionnaire was provided first followed by the short version. The corresponding figures were 0.71 and 1.18 for the group in which the short version was provided first followed by the original version. Cronbach s alpha coefficients for the effort and reward scales of the original version were 0.88 and 0.91, respectively, and they were 0.77 and 0.76, respectively, for the short version. The distribution of the ER ratio calculated with the original questionnaire was positively skewed, whereas the ER ratio calculated with the short version was more normally distributed (Fig. 1). Table 2 shows the prevalence of the high-risk group, kappa as a measure of consistency with the high-risk group defined by the original version, the Youden index, and the AUC of ROC analysis for different cut-off values of the short version. Figure 2 displays the ROC curve for the different cut-off values of the short version using an external criterion of an ER ratio >1.0 for the original version. Although the high-risk groups defined by each scale were significantly associated with each other (χ 2 =147.4, p<0.001), the agreement was slight (kappa=0.204) when 1.0 was used for the cut-off of the short version. The agreement between ERI definitions based on the original and short versions was moderate (kappa= ) when the cut-off of the short version was , with the agreement level then decreasing sharply in both directions away from the end values. The largest AUC and the highest Youden index of trade-off between sensitivity and specificity were found when the ER ratio of the short version was These results suggest that the cut-off value of the short version of the ERI questionnaire that was the most equivalent to that of the original version, was around 1.3 or 1.4. For complementary analyses, the logarithmic ER ratios from the original scales regressed on the short version logarithmic ER ratios yielded a regression coefficient (1.041) and a constant term ( 0.273) (R 2 =0.448). The regression equation coefficient predicted for the short version ER ratio, which was equivalent to an original version ER ratio of 1.0, was The kappa statistic for the high-risk group between the observed original scales and the predicted original scales from the short version scales using the simple regression equation adjustment was fair (~0.365). The estimate improved slightly after adjusting for demographic variables, yielding a regression coefficient (1.029) and a constant term ( 0.116) (R 2 =0.466). The regression equation coefficient predicted for the short version ER ratio, which was equivalent to an original version ER ratio of 1.0, was 1.30 (see Appendix). Multiple logistic regression analysis showed significant associations between ERI defined by the short version and psychological distress (odd ratios and 95% confidence intervals were 4.85 and when setting >1.3 as the cut-off and 5.00 and when setting >1.4 as the cut-off, respectively). Odds ratios increased in both directions away from the values, but the confidence intervals also widened,

5 344 J Occup Health, Vol. 55, 2013 Table 1. Characteristics of the study population (n=1,489) Variables Mean SD n % Mean of the effort-reward ratio Original version Short version Age years old years old years old years old years old Years of job tenure Sex Men 1, Women Occupation Managerial or administrative workers Professional workers Technical workers Clerical workers Sales and service workers Manual workers Others Employment status Regular 1, Non-regular Others Working hours/week Education Junior high school High school Junior college or higher professional school University Net-income/year <3 million yen <5 million yen <8 million yen <10 million yen <15 million yen million yen Unclear Original version Effort score Reward score Effort-reward imbalance Short version Effort score Reward score Effort-reward imbalance

6 Sumiko KURIOKA, et al.: Japanese Short Version of the ERI Questionnaire Original version Short version 4.0 Effort-reward ratio Number of workers Fig. 1. Frequency distribution of the effort-reward ratio of the original (left) and short (right) versions of the effort-reward questionnaire (n=1,489). Table 2. Prevalence of the high-risk group defined by the short version; kappa, which is a measure of consistency of the high-risk groups defined by the original version; the Youden index; and the area under the curve of receiver operating characteristic analyses for the different cut-off values of the short version (n=1,489) Cut-off value Prevalence (%) Kappa Youden index AUC* Sensitivity Specificity > > > > > > > > > > *AUC: Areas under the curve of the receiver operating characteristic analysis. suggesting that estimation errors were increased. Although an ER ratio >1.3 could be a candidate for the cut-off value of the short version, we suggest >1.4 is optimal based on the findings of agreement between the high-risk groups defined by the original version versus the short version. Discussion We conducted a set of web-based self-administered surveys in 1,489 Japanese workers to seek the optimum cut-off point of the short version of the ERI questionnaire, while defending the cut-off value of the original version. When using the theoretical cut-off value (ER ratio >1.0), the ERI prevalence defined by the short version was considerably high (63.2%), and the agreement with the ERI defined by the original version was slight. Although the agreement between ERI definitions was moderate and not necessarily ideal, the kappa statistic and ROC curve analysis suggest that >1.4 is the cut-off value of the short version of the ERI questionnaire that is the most equivalent to that of the original version. The regression equations and associations computed with an external criterion (psychological distress) supported this finding. The mean value of the ER ratio of the short version was significantly higher than that of the original version, and the prevalence of the high-risk group defined by the short version was considerably high when using the theoretical cut-off value (ER ratio

7 346 J Occup Health, Vol. 55, 2013 >1.0). The different distributions of the ER ratios indicate less discrimination of the high-risk group by the short version. Setting the cut-off value under the mean ER ratio would lead to screening more than half of the study population. There are at least two possible reasons for the different distributions of the ER ratios between the original and short versions. Item selection helps to screen as many people as possible in line with the intention behind development of the scale. For example, the effort scale excluding physical load has been found to be psychometrically appropriate in samples predominantly characterized by white collar jobs 22), such as our study sample. In fact, the mean ER ratio slightly increased to 0.73 when the physical load item was excluded from the original version of the ERI questionnaire. Another explanation for the different distributions of the ER ratios between the original and short versions may include the different format of the response categories and their scorings (i.e., weighting; scored as 1 to 5 for the original version and 1 to 4 for the short version). Scrutinizing the response patterns for the original version showed that the items chosen most frequently were does not apply and apply but somewhat distressed for affirmative items for distress, and apply and does not apply but somewhat distressed for non-affirmative items for distress. Consequently, the mean effort score was distributed between 1.91 and 2.52 and the mean reward score was distributed between 3.31 and 4.09 for the original version (slightly less distressed for the effort and reward scores). In contrast, the items chosen most often were either agree or disagree for the short version, and the mean effort and reward scores were approximately 2.5 (the middle point for the effort and reward scores). The response patterns might be explained by the respondents attitudes, such as social desirability or avoiding extreme-position responses 23). The effects of format of the response categories may be more clearly elaborated by item response theory analysis 24) or analyzing the agreement between scales of different formats with the same original 17 items. Fig. 2. The receiver operating characteristic curve for the different cut-off values of the short version using an external criterion of an effort-reward ratio >1 for the original version (n=1,489). The cut-off values of 1.0, 1.3 and 1.4 are shown in parentheses. Strength and limitations No other studies have compared the psychometric properties and agreement for the high-risk group between the original and short versions of the ERI questionnaire within the same population. We adopted a research design in which participants were randomly divided into two groups and the questionnaires to be compared were provided in reversed order. Because large changes in the work environment during a short interval (two weeks) may be small, we minimized differences in social and economic effects on the participants so that we could compare the two versions in almost the same situation. Our study population did not represent the Japanese working population. They were from more metropolitan areas, younger, higher educated, and comprised of two times as many white-collar workers as the general working population 22). These characteristics of our study population may reflect often-reported selection in web-based surveys 25, 26). Furthermore, our study population was working longer hours 22) and appeared to suffer from ERI more often than the general working population, in which the mean ER ratio was ). However, this selection issue is unlikely to affect the equivalency of the cut-off values between the original and short versions in the same data set. Finally, the less than ideal agreement of the ERI defined by the original and short versions may only be present among the Japanese working population. However, the mean ER ratio of the short version (1.19) in our study was similar to that found in Western countries 10, 11), and that of the original version (0.71) was almost the same as that reported by a study in Germany 28). Ideally, our analysis should be replicated in other cultural environments. Practical implication Because of the moderate agreements between highrisk groups defined by the original and short versions of the ERI questionnaire, occupational practitioners should be cautious in evaluation of monitored data if they change over from the original version to the

8 Sumiko KURIOKA, et al.: Japanese Short Version of the ERI Questionnaire 347 short version of the ERI questionnaire. In addition, our findings do not imply that one cannot use cut-off values other than >1.4 when using the short version as an instrument for screening the ERI high-risk group. In this case, practitioners or researchers may have to make a trade-off when choosing the cut-off value. To screen in as many risk groups as possible, a cut-off value of 1.3 could be chosen if practitioners or researchers are ready to accept more false positives. Conclusions Although the short version of the ERI questionnaire is concise and psychometrically valid, the agreement for the high-risk group between the original and short versions was slight when the theoretical cut-off point (ER ratio >1.0) was used. Based on this empirical investigation, we suggest using a cut-off value of >1.4 for the Japanese short version of the ERI questionnaire, which screens out the ERI group with the most compatibility with the original version. Acknowledgments: This study was supported by a Grant-in-Aid for Scientific Research on Innovative Areas from the Ministry of Education, Culture, Sports, Science and Technology, Japan, for the research project titled Elucidation of social stratification mechanisms and control over health inequality in contemporary Japan: New interdisciplinary areas of social and health sciences ( Social Stratification and Health project, no ). References 1) Siegrist J. Adverse health effects of high-effort/ low-reward conditions. J Occup Health Psychol 1996; 1: ) Siegrist J, Bernhardt R, Feng ZC, Schettler G. Socioeconomic differences in cardiovascular risk factors in China. Int J Epidemiol 1990; 19: ) Siegrist J, Peter R, Motz W, et al. The role of hypertension, left ventricular hypertrophy and psychosocial risks in cardiovascular disease: prospective evidence from blue-collar men. Eur Heart J 1992; 13 (Suppl D): ) Tsutsumi A, Ishitake T, Peter R, et al. The Japanese version of the Effort-Reward Imbalance Questionnaire: a study in dental technicians. Work Stress 2001; 15: ) Tsutsumi A, Kayaba K, Theorell T, et al. Association between job stress and depression among Japanese employees threatened by job loss in a comparison between two complementary job-stress models. Scand J Work Environ Health 2001; 27: ) Kuper H, Singh-Manoux A, Siegrist J, et al. When reciprocity fails: effort-reward imbalance in relation to coronary heart disease and health functioning within the Whitehall II study. Occup Environ Med 2002; 59: ) Peter R, Alfredsson L, Hammar N, et al. High effort, low reward, and cardiovascular risk factors in employed Swedish men and women: baseline results from the WOLF Study. J Epidemiol Community Health 1998; 52: ) Siegrist J, Wege N, Puhlhofer F, et al. A short generic measure of work stress in the era of globalization: effort-reward imbalance. Int Arch Occup Environ Health 2009; 82: ) Leineweber C, Wege N, Westerlund H, Wahrendorf M, Siegrist J. How valid is a short measure of effortreward imbalance at work? A replication study from Sweden. Occup Environ Med 2010; 67: ) Li J, Loerbroks A, Jarczok MN, et al. Psychometric properties and differential explanation of a short measure of effort-reward imbalance at work: a study of industrial workers in Germany. Am J Ind Med 2012; 55: ) Msaouel P, Keramaris NC, Apostolopoulos AP, et al. The effort-reward imbalance questionnaire in Greek: translation, validation and psychometric properties in health professionals. J Occup Health 2012; 54: ) Tsutsumi A, Kawanami S, Horie S. Effort-reward imbalance and depression among private practice physicians. Int Arch Occup Environ Health 2011; 85: ) Kessler RC, Andrews G, Colpe LJ, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32: ) Siegrist J, Starke D, Chandola T, et al. The measurement of effort-reward imbalance at work: European comparisons. Soc Sci Med 2004; 58: ) Furukawa TA, Kessler RC, Slade T, et al. The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychol Med 2003; 33: ) Furukawa TA, Kawakami N, Saitoh M, et al. The performance of the Japanese version of the K6 and K10 in the World Mental Health Survey Japan. Int J Methods Psychiatr Res 2008; 17: ) Kawakami N. Distribution and correlating factors of mental health status evaluated by K6 questionnaire in a nationwide study. In: Hashimoto H, editor. Studies on the study of a system for identifying and analyzing statistics on the situation of households in terms of public health. Research Report Summary sharing-fy Tokyo (Japan): Ministry of Health, Labour and Welfare; p ) Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977, 33: ) Youden W. Index for rating diagnostic tests. Cancer 1950; 3: ) Pikhart H, Bobak M, Siegrist J, et al. Psychosocial

9 348 J Occup Health, Vol. 55, 2013 work characteristics and self rated health in four post-communist countries. J Epidemiol Community Health 2001; 55: ) Karasek R, Choi B, Ostergren PO, et al. Testing two methods to create comparable scale scores between the Job Content Questionnaire (JCQ) and JCQ-like questionnaires in the European JACE Study. Int J Behav Med 2007; 14: ) Ministry of Internal Affairs and Communications. Labour Force Survey, Japan. [Online] [cited 2012 Aug 17]; Available from: URL: stat.go.jp/english/index.htm 23) Albaum G, Murphy BD. Extreme response on a likert scale. Psychological Reports 1988; 63: ) Harvey RJ MW, Markham SE. Evaluation of three short-form versions of the Myers-Briggs Type indicator. Journal of Personality Assessment 1994; 63: ) Couper M. Web surveys: a review of issues and approaches. Public Opin Q 2000; 64: ) Honda N. Quality of web survey and access panel survey: issues for utilization of web based access panel survey. The Japanese Journal of Labour Studies 2005; 551: ) Tsutsumi A. Development and application of the Japanese version of effort-reward imbalance questionnaire. Report of Research Project, Grand in Aid for Scientific Research (C), , Okayama University School of Medicine and Dentistry, Hygiene & Preventive Medicine ) Lehr D, Koch S, Hillert A. Where is (im) balance? Necessity and construction of evaluated cut-off points for effort-reward imbalance and overcommitment. J Occup Organ Psychol 2010; 83: Appendix Regression equations from the ER ratio based on the original scales of the ER ratio based on the short version; a single regression equation and a regression equation after adjusting for sex, age, educational attainment, employment status, occupation, working hours per week, and net income per year (n=1,489) are given below. Log (original ER ratio)=log (short ER ratio)* , R 2 = Log (original ER ratio)=log (short ER ratio)*1.029 sex*0.059 age* occupation* (emplo yment status)* (net income per year)* (educational attainment)* (working hours per week)* , R 2 =0.466.