Validation of the Core Self-Evaluations Scale research instrument in the conditions of Slovak Republic

Validation of the Core Self-Evaluations Scale research instrument in the conditions of Slovak Republic Lenka Selecká, Jana Holienková Faculty of Arts, Department of psychology University of SS. Cyril and Methodius Trnava, Slovakia lenka.selecka@ucm.sk, jana.holienkova@ucm.sk Abstract - The study is focused on psychometric attributes verification of the Core Self-Evaluations Scale questionnaire (CSES) and validation of this instrument in the conditions of Slovak Republic. The sample comprised of 873 participants (M=25.50, SD=5.774, 17-52). The internal consistency of the questionnaire was proved to be satisfactory (α=0.780). To investigate the construct validity like convergent and discriminant validity of the CSES questionnaire we examined the potential correlations of the Core self-evaluations (CSE) and Big Five personality traits on the sample of 255 individuals (M=29.89, SD=6.787, 20-52). Principal component analysis was used to determine suitability of particular factor model that followed the four original constructs of the questionnaire (self-esteem, generalized self-efficacy, neuroticism, locus of control). Our results demonstrated the suitability and reliability of the translated version of the investigated CSES questionnaire. Keywords - The Core Self-Evaluations Scale, self-esteem, generalized self-efficacy, locus of control, neuroticism, internal consistency, principal component analysis (PCA), validation. I. INTRODUCTION The investigation of dispositional sources of job satisfaction has a rich and diverse history. The researchers investigating the relations between individual s personality and his/her job satisfaction have employed variety of research designs, methodological approaches, measurement strategies and statistical analyses. In recent years, an increasing number of studies has been focused on three theoretical approaches [6]: (1) positive affectivity/negative affectivity, (2) the five-factor model of personality, and (3) Core Self-Evaluations model. Each of these approaches, due to its fundaments, has particular advantages in investigation of dispositional sources of job satisfaction. The first two approaches are well-established in Slovak Republic. However, the last mentioned approach is still an unknown instrument in the investigation of job satisfaction dispositional sources in the conditions of Slovak Republic. The Core Self-Evaluations model represents a relatively new concept in the dispositional approach to job satisfaction research. According to its authors [9], Core Self-Evaluations (CSE) represents a broad, latent, higher-order trait indicated by four well established traits in personality psychology: (1) selfesteem - the overall value that one contributes on oneself as a person, (2) generalized self-efficacy - an evaluation of how well an individual can perform across different situations, (3) neuroticism - the tendency to have negativistic cognitive/explanatory styles and to concentrate on negative aspects of the self, and (4) locus of control - beliefs about the causes of events in one s life - the locus is internal when individuals understand events as being contingent on their own behavior. In most cases the personality traits creating the Core Self- Evaluation model have been studied as individual constructs without discussions on their interrelationships. However, based on the results of number of studies, authors (for example [6], [10], [2], [7], [8], [11]) have discovered that due to empirically proven strong similarities among the respective traits, it might be possible to determine these traits as a one single factor comprising of four dimensions. Several studies have been undertaken to prove the reliability as well as validity of the developed construct (for example [10], [11]). In addition to the United States, its reliability and validity have been confirmed also in other countries, for example in Germany [16] or Japan [14]. Since the Core Self-Evaluation model appears as theoretically and empirically suitable instrument for job satisfaction dispositional sources investigation, we argue that verification of this concept in conditions of Slovak Republic could provide Slovak researchers a set of individual traits that would be useful predictors of job satisfaction. In the presented study we focus on verification of the internal consistency of CSES research instrument. This characteristic could provide basic information about reliability of this method in Slovak conditions. In our study we also investigate the factor structure of the translated questionnaire, as well as its validity in order to develop a standardized instrument, which is a necessity of a quality research method. Participants II. METHODS A total of 873 individuals participated in testing of the psychometric properties of CSES questionnaire in the conditions of Slovak Republic. This sample comprised of 644 females (73.8%) and 228 males (26.1%), one respondent did not indicate the gender (0.1%). The average age was 25.5 years (the youngest participant was 17, the oldest one 52 years old). - 113 -

A part of the sample (N=255; M=29,89; SD=6,787; 20-52 years) was used in testing of the construct or convergent validity of the CSES questionnaire by analyzing the correlations between CSE and Big Five personality traits. Measure In 1997, reference [9] introduced the theoretical concept of Core Self-Evaluations (CSE) in attempt to provide the personality traits that would be useful predictors of job satisfaction. Based on the theory arguments, reference [10] have in 2003 created an instrument called The Core Self- Evaluations Scale (CSES) that was used to verify the theoretical groundings. During the construction and verification of an instrument assessing the key aspect of CSE, the authors extracted from existing instruments a group of items representing four factors (self-esteem, generalized self-efficacy, neuroticism, locus of control), that have been previously identified based on personality psychology literature review. In the process of validity and components structure identification, the authors performed a series of 6 independent measurements and have developed the final 12-items scale. The total score of the scale is formed as a sum of scores for all twelve items, with respect to reverse items. Respondents indicate their level of agreement with the particular statements on 5-point Likert-type scale, with answers ranging from strong disagreement (1) to strong agreement (5). In the first stage of testing a four factor model was examined by exploratory and confirmatory factor analysis. However, alternative models identified strong intercorrelation between the first three factors, while locus of control reported considerably lower values and was therefore understood as a separate factor. In the next stage the differences between twofactor and one-factor model have been tested. However, the differences showed very low values, thus the CSES questionnaire was determined by its authors as a single-factor model [10]. The internal consistency analysis has been performed in a series of six independent measurements (N 1 =280; N 2 =175; N 3 =126; N 4 =265; N 5 =205; N 6 =126), and in all measurements it achieved values of Cronbach s alpha greater than 0.80 (α 1 =0.85; α 2 =0.83; α 3 =0.85; α 4 =0.87; α 5 =0.83; α 6 =0.81), with average value of 0.84. Also, the correlations between different items of the research instrument have been tested, and their valued ranged from 0.48 to 0.55 (average value was 0.50). Since the correlation matrix proved intercorrelation between the respective items, the authors [10] considered the research instrument to be internally consistent. The presented research comprises the creation of a new version of the CSES questionnaire, which analysis (in terms of translation into Slovak language) is a subject of the analysis. After studying the questionnaire we prepared its translated version, based on independent translations of two psychologists. The translated CSES questionnaire has been administered to 873 persons with detailed instructions how to fill the questionnaire. The data obtained (N=873) was used to test the psychometric properties of the questionnaire using principal components analysis, that analyze the correlations between manifest variables and based on them determines the latent variable. In the translated questionnaire we also examined its internal consistency and its validity. The obtained data was processed using SPSS statistical software. III. RESULTS One of the common ways of estimating reliability of a test method is the assessment of internal consistency reliability, which is counted by Cronbach's coefficient alpha [18]. Internal consistency reliability was used mainly because the original questionnaire authors [10] established the presence of sufficiently strong positive correlations among all items of the CSES questionnaire, which is a basic requirement for the internal consistency estimation. TABLE I. Mean (SD) DISTRIBUTIONAL PROPERTIES AND RELIABILITY EXTIMATES OF THE CSES QUESTIONNAIRE Internal consistency reliability (N of items) 3.101 (0.577) 0.780 (12) Note: Internal consistency estimates are Cronbach s coefficient alpha (α) reliability estimates. Table 1 presents the CSES questionnaire s descriptive statistics and its internal consistency estimate. The minimum acceptable value of reliability is to be 0.70 [17]. In this context, we can say a satisfactory internal consistency coefficient is achieved. In the comparison to the English version of the original questionnaire authors [10] the consistency of the Slovak questionnaire is slightly lower, since the authors achieved an average internal consistency of 0.840. Potential correlation relations among CSE and personality traits of Big-five model of personality were examined in order to assess the CSES construct validity. Based on the personality traits theoretical definitions it is assumed a presence of a relations (convergent validity), or on the contrary an absence of relations (discriminant validity) with total Core Selfevaluations (CSE). TABLE II. Neuroticism Extraversion CONSTRUCT VALIDITY THE CSES - NEO-FFI (CORRELATION COEFFICIENTS Openness to Agreeableness Conscientiousness experience r -0,637 0,376-0,038 0,227 0,467 p 0,000 0,000 0,555 0,000 0,000 As can be observed in Table 2, positive moderate correlations (0.376 and 0.467) among personality traits extraversion, conscientiousness and CSE are found. A negative, moderate correlation (-0.637) is found between neuroticism and total score of the Core Self-Evaluations (G- CSES). Overall, the G-CSES showed significant convergence with the three personality traits. The data thus provide support for convergent validity of CSES. Analogically we tested discriminant validity of the questionnaire, where we assumed that G-CSES should correlate weakly or not statistically significantly with openness - 114 -

to experience and agreeableness in the Five-factor model of personality traits in compliance with the conclusions of the original questionnaire authors [10]. These constructs are shown to be distinct from CSES and therefore their results support discriminant validity of the English version CSES. In Table 2 are shown the correlations between G-CSES and the two mentioned personality traits. One personality trait, namely openness to experience, doesn t significantly correlate with CSE (-0.038) and agreeableness correlates with G-CSES weakly (0.227). The openness to experience - G-CSES correlation supports discriminant validity of the Slovak version CSES. Weak correlation between agreeableness and G-CSES is contradictory. However, it is consistent with the Germanlanguage CSES validation findings [4]. Testing of the CSES construct validity indicates that the questionnaire is a valid construct, it converges with three personality traits and diverges from openness to experience. Reference [10] analysed the structure of the CSES questionnaire using Confirmatory Factor Analysis. There was suggestion that the structure could be composed by 4 or 2 factors but it was found out that these structures gave little incremental information than treating it as a single measure. We also tested the factor structure although we used Exploratory Factor Analysis. In this study, Principal Components Analysis (PCA) was used to examine psychometric properties of the CSES questionnaire. The reason to use PCA in our study is that this is not final psychometric properties testing, the study has exploratory character. In the following, we present the results of Exploratory Factor Analysis for the CSES questionnaire. The data obtained from the translated version of the questionnaire is analyzed by Principal Component Analysis. We allowed the factors to be rotated to see if there was a clear under structure to the items. We used a rotation method Promax because this rotation allows us to monitor the interdependence of the components. Kaiser-Meyer-Olkin test of Sampling Adequacy gained a sufficient value (0.875) as each value above 0.5 is acceptable [13], so it is possible to consider that PCA is in this case adequate and its results are explicitly distinguishable components. Moreover, the result of Bartlett's test of Sphericity is significant (Approx. X² = 4576.843; p = 0.000), suggesting that the variables are related. Both tests lead to the conclusion of the adequacy of Principal Components Analysis. To determine the ideal number of factors we applied both the Eigenvalue and a Scree test. Based on Kaiser Normalization which determines the importance of a component by eigenvalue, we can conclude that the model manifests two significant components in the data. Two factors together explain 59.102% of the variance. The first component has an eigenvalue of 4.263 and accounts for 35.524% of the total variance. The second factor has an eigenvalue of 2.829 and accounts for an additional 23.579% of the variance. The scree plot for these eigenvalues is shown below (Figure 1). Figure 1. Scree Plot As can be seen in the scree chart there is a dramatic break after the third factor which tends to support the existence of three components. However, the third component has an eigenvalue of 0.860 and as long as the number of variables is less than 30 it is appropriate to use Kaiser Normalization [12]. Two-component model in this regard seems to be ideal; when we examined the actual items that were loading on each component we have got the following structure (Table 3). In this case we have applied Promax rotation, because it is designed to help clarify the difference between factors, it provides the optimal balance between simple structure and correlated components result [5]. TABLE III. PRINCIPAL COMPONENT ANALYSIS WITH PROMAX ROTATION METHOD Item 11,867 Item 3,861 Item 7,838 Item 5,827 Item 9,761 Item 1,713 Component 1 2 Item 12,772 Item 4,728 Item 8,721 Item 2,302,679 Item 6,668 Item 10 -,380,514 The factorial distribution presumed by the original CSES questionnaire authors in the two-factor model was not estimated in Slovak translation of the questionnaire. The authors [10] report that locus of control correlates with generalized self-efficacy, self-esteem and neuroticism weaker as these factors are correlated with each other. Therefore, authors assembled a two factorial model of the questionnaire, - 115 -

in which locus of the control represents one factor and three other concepts represent the other factor. As it can be seen in Table 3, items of the questionnaire are distributed in two equivalent components. The items loading to Component 1 would appear to feeling of confidence in life, ability to cope with problems and success in life. Items loading to Component 2 are very similar, they are represented by the feeling of control over one's career, positive thoughts over life and self-worth. It is difficult to distinguish one component from another one, which suggests that to a large extent they are measuring the same construct. A rational analysis of the data can be accepted as a basic confirmation of the findings by authors [10]. As we discovered, it is possible to extract two components from the data, but it does not add much incremental information to the interpretation. Since the informational value of two-factor model is difficult to support, the next step in our analysis should be determining a single-factor model. However, in order to confirm the suitability of the single-factor model, it would be necessary to carry out Confirmatory Factor Analysis, which will be realized in the near future. In the present study, the objective was to verify internal consistency reliability of the Slovak version of the CSES questionnaire and exploring sources of validity in the conditions of Slovak republic. IV. DISCUSSION Quality instrument measuring psychological attribute must meet certain requirements. One of the basic conditions is reliability. However, if a method should be considered as quality and standard, it must reach certain values. Internal consistency can have values ranging from 0 to 1, while the value of empirical reliability index should ideally be as close as possible to the value 1. The aim is not always reach a value of 1, the optimal values range from 0.80 to 0.90 [3]. The minimum value of reliability coefficient should reach 0.70 [17]. In this sense it can be established that the translated version of the CSES questionnaire reached an adequate degree of internal consistency, the evidence of the questionnaire reliability will be verified on larger research samples and under different conditions (such as persons with different professions, age, from different regions, different personality features) in the next period. Another basic requirement of a quality psychological instrument is its validity. Our study s aim is to verify the construct validity of the investigated questionnaire, respectively to verify its construct resources [17]. Construct validation of the present article is verified on one hand by correlation analysis between Core Self-evaluation score and selected personality constructs and on the other hand using Principal Component Analysis. Construct validation pursued on Slovak sample shows positive relations with extraversion and conscientiousness. Those who tend to be more sociable and active tend as well to evaluate themselves and their control over their environment in a more positive manner. Therefore, a moderate correlation between these personality traits and CSE can be expected. Our data provided support for convergent validity of CSE as a part of construct validity. Openness to experience and agreeableness are believed to divergent from CSE. Openness to experience is characterized as a tendency to have an active imagination, intellectual curiosity, flexibility, as well as being autonomous [1]. As such it is distinct from CSE, which we succeeded to achieve since openness to experience doesn t significantly correlate with Core Self-Evaluations. This finding supports discriminant validity of the Slovak version CSES. Agreeableness is defined as a trait with interpersonal tendencies through altruism [15]. Weak correlation between agreeableness and CSE can be broadly interpreted as transfer of gentle approach from oneself to the interpersonal environment. We don t understand this relation as contradictory to discriminant validity of the translated questionnaire since there can be found some common indicators as mentioned and the correlation between them is weak. Construct validity of CSES suggests it is a valid construct. However, further testing the construct validity of the translated questionnaire will be a part of the questionnaire future adaptation. Construct validity will be evaluated through correlation analysis between translated questionnaire CSES and questionnaires oriented to the four original constructs, which are self-esteem, generalized self-efficacy, neuroticism and locus of control. We will proceed to this process because in conditions of Slovak republic there is no instrument that would examine Core Self-Evaluation, in case we understand it as a separate psychological construct. Incremental validity will be verified in the coming period concerning the criteria of job and life satisfaction. CSES should add incremental validity beyond locus of control and personality traits of Five-factor model of personality concerning the criteria of job satisfaction. Executed Principal Component Analysis represents twocomponent model which doesn t have the same item loadings as the original questionnaire. Items load to different components as the original version of the questionnaire; different factor analysis was used and therefore it was not possible to determine in advance which items should be included in which component, but this follows from possibilities of current psychometric testing. Despite different factor analysis, exploratory versus confirmatory factor analysis, similar results are expectable. Different factor loadings could on one hand arise from imprecision in the original meaning of the items or on the other hand understanding of feelings and behaviour described in some items may be in the Slovak population other than abroad. Two-component model includes one component connected to feelings of confidence in life, ability to cope and success in life and another component presented by feeling of control over one's career, positive thoughts over life and self-worth. The difficulty to distinguish one component from another suggests that to a large extent they are measuring the same construct. In order to confirm the suitability of the single-factor model a Confirmatory Factor Analysis will be executed. Preliminary results suggest the Slovak version of CSES to be reliable and there is some empirical evidence about - 116 -

construct validity of the questionnaire. Even though further evaluations are necessary, present study can be an important step towards possibilities in the Core Self-Evaluations research in Slovak environment. ACKNOWLEDGMENT This study was realized as a part of the research project Validation of the Core Self-Evaluations Scale in the conditions of Slovak Republic supported by the University of SS. Cyril and Methodius Research and Development Support Fund number FPPV-07-2014. REFERENCES [1] D. W. Ariani, Personality and learning motivation, European Journal of Bussiness and Management, vol. 5, no. 10, pp. 26-38, 2013. [2] A. Erez, and T. A. Judge, Relationship of Core self-evaluations to goal setting, motivation, and performance, Journal of Applied Psychology, vol. 86, pp. 1270-1279, 2001. [3] J. Gliem, and R. Gliem, Calculating, interpreting, and reporting Cronbach s alpha reliability coefficient for Likert-type scales, 2003 Midwest Research to Practice Conference in Adult, Continuing and Community Eduction. Columbus, Ohia, 2003. [4] T. Heilmann, and K. Jonas, Validation of German-language Core selfevaluations scale, Social Behavior and Personality, vol. 38, no. 2, pp. 209-226, 2010. [5] A. E. Hendrickson, and O. White, Promax: A guick method for rotation to oblique simple structure, The British Journal of Statistical Psychology, vol. 17, no. 1, pp. 65-70, 1964. [6] T. A. Judge, D. Heller, and R. Klinger, The dispositional sources of job satisfaction: A comparative test, Applied Psychology: An International Review, vol. 57, no. 3, pp. 361-372, 2008. [7] T. A. Judge, J. Bono, and E. A. Locke, Personality and job satisfaction: The mediating role of job characteristics, Journal of Applied Psychology, vol. 85, pp. 237-249, 2002. [8] T. A. Judge, A. Erez, and J. Bono, The power of being positive: The relationship between positive Self-concept and job performance, Human Performance, vol. 11, pp. 167-187, 1998. [9] T. A. Judge, E. A. Locke, and C. C. Durham, The dispositional causes of job satisfaction: A Core evaluations approach, Research in Organizational Behavior, vol. 19, pp. 151-188, 1997. [10] T. A. Judge, et al., The Core self-evaluations scale: Development of a measure, Personnel Psychology, vol. 56, pp. 303-331, 2003. [11] T. A. Judge, et al., Dispositional effects on job and life satisfaction: The role of Core evaluations, Journal of Applied Psychology, vol. 83, pp. 17-34, 1998. [12] H. F. Kaiser, The Varimax criterion for analytic rotation in factor analysis, Psychometrika, vol. 23, no. 3, pp. 187-200, 1958. [13] H. F. Kaiser, An index of factorial simplicity, Psychometrika, vol. 39, no. 1, pp. 31-36, 1974. [14] R. F. Piccolo, et al., Core self-evaluations in Japan: Relative effects on job satisfaction, life satisfaction, and happiness, Journal of Organizational Behavior, vol. 26, pp. 965-984, 2005. [15] I. Ruisel, and P. Halama, NEO Five-Factor Inventory, Praha: Hogrefe Testcentrum, 2007. [16] T. Stumpp, et al., Core Self-evaluations in Germany: Validation of a German Measure and its relationships with career success, Applied Psychology: An International Review, vol. 59, no. 4, pp. 674-700, 2010. [17] T. Urbánek, D. Denglerová, and J. Širůček, J. Psychometrika. Praha: Portál, 2011. - 117 -