My Thinking Styles TECHNICAL REPORT. by Judy Chartrand. Copyright 2011 NCS Pearson, Inc. All rights reserved.

Transcription

1 My Thinking Styles TM TECHNICAL REPORT by Judy Chartrand Copyright 2011 NCS Pearson, Inc. All rights reserved.

2 Table of Contents Introduction...1 Purpose...1 Conceptual Origins...1 Scale Development...3 Item Writing...3 Short Form Pilot Study...4 Long Form Pilot Study...8 Interpretation...13 Explaining What My Thinking Styles Scales Measure...13 Interpreting Results...13 Summary...13 References...14 List of Tables 1 Short Form Pilot Sample Demographic Characteristics (N = 1,953) Short Form Scale Properties (N = 1,953) Short Form Mean Scores by Gender Short Form Item Means and Internal Consistency (N = 1,953) Short Form Mean Scores by Self-Rated Critical Thinking Short Form Mean Scores by Education Level Short Form Mean Scores by Occupational Groups Long Form Pilot Sample Demographic Characteristics (N = 300) My Thinking Styles Long Form Item Means Range of Long Form Item Means Final Scale Internal Consistency Final Scale Properties (N = 300) Final Scale Mean Scores by Gender Final Scale Mean Scores by Race/Ethnicity Predicted Relationships Between My Thinking Styles and IPIP Constructs My Thinking Styles Inter-Scale Correlations...12

3 Purpose Introduction My Thinking Styles (MTS) measures seven different thinking styles habits and behaviors that promote effective critical thinking. All seven styles are positive habits that people use with different frequency. Helping people identify their thinking style preferences enables them to become more aware of their styles, how to use those styles to develop better thinking skills, and which styles they could use more frequently. The assessment is ipsative in nature, enabling someone to compare the relative strength of his or her thinking style preferences and identify their most and least preferred thinking styles. This information is meant to increase self-awareness and support development efforts. My Thinking Styles is not intended as a diagnostic or prospective tool, and should not be used for selection, promotion, or classification purposes. Conceptual Origins The conceptual origins of the MTS are drawn from bodies of literature on critical thinking and personality. Critical thinking has a strong tradition in philosophy and education, whereas personality emanates from the field of psychology. Over the years, the critical thinking literature has focused on the components of critical thinking, how to assess it, and how to teach it so that students develop these fundamental skills. The personality literature has focused on defining and measuring traits and dispositions to explain or predict behavior (e.g., workplace performance). Critical Thinking The critical thinking literature was influenced primarily by the work of a panel of critical thinking experts who worked together from to define the skills and dispositions associated with critical thinking. The result was a report (Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction) that came to be known as the Delphi Report (Facione, 1990). The panel of experts characterized good critical thinkers as follows: The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, openminded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit (p. 2). The panel identified the following dispositional characteristics that support effective critical thinking. Affective dispositions of critical thinking approaches to life and living, in general: * Inquisitiveness with regard to a wide range of issues * Concern to become and remain generally well-informed * Alertness to opportunities to use CT * Trust in the processes of reasoned inquiry * Self-confidence in one's own ability to reason * Open-mindedness regarding divergent world views * Flexibility in considering alternatives and opinions * Understanding of the opinions of other people * Fair-mindedness in appraising reasoning 1

4 * Honesty in facing one's own biases, prejudices, stereotypes, egocentric or socio centric tendencies * Prudence in suspending, making, or altering judgments * Willingness to reconsider and revise views where honest reflection suggests that change is warranted Approaches to specific issues, questions, or problems: * Clarity in stating the question or concern * Orderliness in working with complexity * Diligence in seeking relevant information, * Reasonableness in selecting and applying criteria * Care in focusing attention on the concern at hand * Persistence though difficulties are encountered * Precision to the degree permitted by the subject and the circumstance. (Facione, 1990, p. 14) As a follow-up to the Delphi work, Peter Facione and his colleagues (Facione, Facione, & Giancarlo, 2000) measured and analyzed seven dispositions of critical thinking (Truth-Seeking, Open-Minded, Analytical, Systematic, Confident in Reasoning, Inquisitive, and Judicious). This work led to the development of the California Critical Thinking Disposition Inventory (CCTDI), which measures personal mental attributes. The Delphi descriptions outline many qualities that predispose someone to engage in behaviors that support critical thinking skills. The descriptions also clearly differentiate these dispositions from the cognitive skills that comprise critical thinking (e.g., analyses, evaluation, and inference). A review of these dispositional qualities and of the CCTDI led me to look for guidance from the field of personality, a discipline that has traditionally studied and measured emotional, attitudinal and behavioral response tendencies. Personality A quick review of several major personality models suggests that the dispositions described in the Delphi Report and in Facione s subsequent work, can be conceptualized within a personality framework. This conceptualization is somewhat different than Facione s work in that an effort was made to identify personality trait clusters, and in one case, a values cluster. Drawing on the extant personality literature anchors the thinking styles in a literature that has clearly established connections between dispositions and important behavioral outcomes (e.g., specific workplace competencies, workplace performance, occupational success, adjustment to college, etc.). The personality trait clusters link to each of the seven thinking styles identified in MTS as follows: Analytical: The tendencies to notice important details, to anticipate consequences, and to be logical, methodical, and planful cluster together. For example, in the Six Factor Personality Questionnaire (Jackson, Paunonen, & Tremblay, 2000) Methodicalness is identified as a major dimension of personality. In Jungian-based assessments, the preference for Sensing equates to a preference to gather information in an exact and precise manner. Inquisitive: The tendency to be intellectually curious can also be tied to one of the major dimensions of personality, Openness. Although some measure the construct as openness to experience, others (e.g., Jackson et al., 2000) focus more directly on intellectual openness. The MTS focuses more on a curiosity and desire to learn. Insightful: The tendencies to step back reflect, to be steadfast, and prudent are more squarely in the domain of positive character. For example, Peterson and Seligman (2004) measure perspective, prudence, and persistence in their Values in Action assessment. 2

5 Open-Minded: The tendencies to be tolerant of the views of others, empathic, and fair-minded also coalesce. In the California Psychological Inventory (Gough, 1996), the scales of empathy and tolerance are positively correlated with an open-minded approach to the world and others. Systematic: The tendency to place things into context and see the bigger picture and its logical sequence draws from the Jungian construct of Intuition, in which information is organized through patterns and relationships. This basic Jungian dimension can be contrasted with its opposite, Sensing, which is the preference for gathering information through the senses in an exact and precise manner. Broadly, the Systematic style is akin to left brain functions (e.g., conceptual). Timely: The tendencies to be efficient, reliable, and responsive correlate and can be found within the major personality dimension of Conscientiousness. Unlike the other six styles, a Timely style is not clearly described in the Delphi work. However, within the personality literature, behavioral markers of conscientiousness are among the best predictors of effective performance. For the My Thinking Styles, the decision was made to include a scale that represents the ability to proactively move forward and take action when appropriate. This style was viewed as an important addition that captures behaviors that support cognitive processes. Truth-Seeking: This is the tendency to ask tough questions in the pursuit of truth. Within the personality literature, Truth-Seeking is related conceptually to Independence, which has been identified as a major dimension of personality (Jackson, et al., 2000; Gough, 1996). The tendency to distance ones self from others and to be competent enables a person to be frank and push toward the truth, even when pushing causes interpersonal discomfort. These seven definitions share commonalities or positive qualities that often coexist to varying degrees (e.g., the planful and organized aspects of Analytical are related to the reliable and efficient aspects of Timely). The use of multiple concepts to define each construct also suggests that each scale (i.e., thinking style) is multifaceted. Scale Development Item Writing In developing My Thinking Styles, each scale was defined and then marker concepts were identified. For example, for Inquisitive, the marker concepts are curious, inquisitive, and tolerant of ambiguity and complexity. Items for each scale were based on the respective definitions and marker concepts. The pool of items for each scale was approximately three times greater than the number anticipated for final selection. The majority of items were written in the positive direction (i.e., endorsement meant they possessed the positive behavior). The goal was to create a final form consisting of items per scale. Two item formats were used. One format was a statement of preference (e.g. I like to probe deeply into a subject) and the other format was a single-word self-descriptor (e.g., Tough-Minded). A four-point item response rating format was used, ranging from does not describe me to clearly describes me. Research staff at Pearson reviewed the item pool and a series of items were deleted or reworded. The remaining items went through an editorial review and then Short Form was created for the first pilot study. The Short Form included 70 items (35 statements and 35 self-descriptors) with 10 items per scale. Only 70 items were tested because the opportunity to collect data could not accommodate the total number of items in the pool. The items selected had a similar level of social desirability across scales. Thirteen people within Pearson completed The initial version of the Short Form was tested with 13 people 3

6 from Pearson to identify obvious issues before actual Short Form pilot testing. The Long Form was tested with a different sample for the second pilot and included 143 items. Short Form Pilot Study Purpose The goal of the short form study was to evaluate the items by looking at scale means and standard deviations, item distribution (which sheds light on social desirability, as well as item functioning), scale internal consistency, and preliminary evidence of validity. Short Form Pilot Sample Demographic Characteristics The sample consisted of 1,953 participants. Table 1 presents the demographic characteristics of the sample. Of people who reported demographic information, there were more women than men, more people who completed at least a college degree than those who did not, and more people working in professional individual contributor or managerial jobs than others. Table 1 Short Form Pilot Sample Demographic Characteristics (N = 1,953) Sex n % Female Male Not Reported Age n % < > Not Reported Education Level n % 9 12 years of high school years of college years of college Bachelor s Master s Doctorate Not Reported Occupation n % Administrative/Clerical Customer Service/Retail Professional/Individual Contributor Manager/Executive Military Student Other

7 Results The scale ranges, means, standard deviations, and skewness are shown in Table 2. The scales have a reasonable range and standard deviations, but appear slightly skewed (negatively). Table 2 Short Form Scale Properties (N = 1,953) Scale Minimum Maximum Mean SD Skewness Analytical Inquisitive Insightful Open Minded Systematic Timely Truth Seeking There were no significant differences between men and women on the MTS scales. As shown in Table 3, the comparison between males and females revealed only one small gender difference. Timely was the only meaningful difference (Cohen s d effect size was small at.36). Table 3 Short Form Mean Scores by Gender Gender Analytical Inquisitive Insightful Open Minded Systematic Timely Truth Seeking Female (n = 951) Male (n = 547) Effect Size*: Cohen s d * Small Effect Size =.30; Moderate Effect Size =.50; Large Effect Size =.80 Short Form Item Means and Evidence of Internal Consistency Reliability Item means within each scale were examined for skewness and distribution. Because highly skewed items typically do not work well (though there are exceptions to that rule), an effort was made to eliminate items with a mean > 3.0. Item means were reviewed across scales to ensure similar distributions. Because the assessment is ipsative and enables an individual to compare the relative preference of his or her thinking styles, it is important to match social desirability across scales. The item means and the evidence of internal consistency reliability for the short pilot form scales are presented in Table 4. The item means reveal that the Truth Seeking items were not endorsed as frequently as other scale items. The Internal consistency estimates ranged from.68 to.81, which is good for a pilot form, but suggests that the scale items would benefit from further review. Item-to-total score correlations indicated the Truth Seeking scale had several items with low correlations (.24 to.32), which suggested the Truth Seeking scale needed further work. 5

8 Table 4 Short Form Item Means and Internal Consistency (N = 1,953) Scale Mean Alpha Analytical Inquisitive Insightful Open Minded Systematic Timely Truth Seeking Preliminary Evidence of Validity The literature has consistently shown that occupational and educational attainment is associated with higher critical thinking capabilities (Watson & Glaser, 2010). The MTS short-form study used three markers of critical thinking capability: self-rated ability, educational attainment, and occupational attainment. The mean thinking style scores of self-rated critical thinking capability were compared across the scales. The results clearly support the hypothesis that people with better self-rated critical thinking capabilities also score higher on the MTS assessment (Table 5). All differences are moderate to large (i.e., all Cohen s d effect sizes >.50). Table 5 Short Form Mean Scores by Self-Rated Critical Thinking Critical Thinking Self Rating Analytical Inquisitive Insightful Below Average (n = 30) Average (n = 205) Above Average (n = 330) A Clear Strength (n = 363) Exceptionally Good (n = 135) Effect Size*: Cohen s d (Exceptionally Good vs. Below Average) Open- Minded Systematic Timely Truth- Seeking * Small Effect Size =.30; Moderate Effect Size =.50; Large Effect Size =.80 People with doctorate degrees were compared with people who had completed a high school degree. Critical thinking training is often part of advanced education, and as such, it was expected that people with higher educational attainment would have higher thinking style scores. The results, presented in Table 6, confirm this hypothesis. 6

9 Table 6 Short Form Mean Scores by Education Level Education Analytical Inquisitive Insightful High School (n = 138) Doctorate (n = 93) Effect Size*: Cohen s d Open Minded Systematic Timely Truth Seeking * Small Effect Size =.30; Moderate Effect Size =.50; Large Effect Size =.80 The My Thinking Styles mean scores of different occupations were also compared (see Table 7). It was expected that Manager/Executive and Professional Individual Contributors would score higher than Administrative Clerical and Customer Service/Retail. No hypotheses were made about students or military personnel. Table 7 Short Form Mean Scores by Occupational Groups Position/Level Analytical Inquisitive Insightful Administrative/ Clerical (n = 68) Customer Service/Retail (n = 25) Professional/ Individual Contributor (n = 637) Manager/ Executive (n = 501) Military (n = 15) Student (n = 433) Effect Size*: Cohen s d (Manager/Executive vs. Administrative/ Clerical) Open Minded Systematic Timely Truth Seeking * Small Effect Size =.30; Moderate Effect Size =.50; Large Effect Size =.80 7

10 Long Form Pilot Study Purpose The goal of the long form pilot was to make final item selections and to evaluate the evidence of internal consistency reliability and construct validity of the scales. The short form study results indicated that several items needed to be replaced, particularly on the Truth-Seeking scale. It also appeared that the scales could be enhanced by adding 2 3 items to each scale, which would balance out mean item means across scales and increase scale internal consistency reliability. To address construct validity, personality scales conceptually similar to the My Thinking Styles scales were drawn from the International Personality Item Pool ( Significant correlations between the scales and conceptually similar personality constructs would support the theoretical conceptualization of the scales. Long Form Pilot Sample Demographic Characteristics The long form sample consisted of 300 participants. Table 8 presents the demographic characteristics. More women than men were in the sample and the occupations were limited to working professionals. Age and race/ethnicity was reasonably well distributed. Table 8 Long Form Pilot Sample Demographic Characteristics (N = 300) Sex n % Female Male Age n % Not Reported Race/Ethnicity African American/Black Asian/Pacific Islander Hispanic/Latina/Latino White Other Not Reported Education Level High School Diploma or GED Some College Associate s Bachelor s Master s or Doctorate Not Reported Employment Status Professional/Individual Contributor Manager Director

11 Results The Long Form 143 items was tested in the second pilot study. Based on the Short Form results, 10 items were deleted (three Truth Seeking scale items) because of low item total score correlations. The deleted items were replaced with psychometrically sound items from the second item pool. Three items were added to each scale, bringing the number of items per scale from 10 to 13. Similar to the Short Form pilot study, item means were compared to identify highly skewed items and select items with similar mean levels across scales. The mean and range of the item means are presented in Tables 9 and 10. Though there are slight differences between scales, the item means generally are similar and better than in those of the Short Form. Table 9 My Thinking Styles Long Form Item Means Scale Mean Analytical 2.3 Inquisitive 2.2 Insightful 2.3 Open Minded 2.3 Systematic 2.3 Timely 2.2 Truth Seeking 2.0 Table 10 Range of Long Form Item Means Scale Range of Item Means Analytical Inquisitive Insightful Open Minded Systematic Timely Truth Seeking The internal consistency estimates for the revised, final version of the scales are presented in Table 11. Overall, the internal consistency estimates are very good, ranging from.83 to.90. The addition of three items to each scale clearly enhanced the internal consistency reliability estimates of each scale. Table 11 Final Scale Internal Consistency Scale Internal Consistency (r alpha ) Analytical.85 Inquisitive.87 Insightful.87 Open Minded.85 Systematic.90 Timely.86 Truth Seeking.83 9

12 My Thinking Style Scale Means and Standard Deviations After the final items were selected, the scale means and standard deviations were calculated (Table 12). Table 12 Final Scale Properties (N = 300) Scale Minimum Maximum Mean SD Skewness Analytical Inquisitive Insightful Open Minded Systematic Timely Truth Seeking Only small differences were observed between men and women (Table 13) and race/ethnicity (see Table 14) on the MTS scales. Table 13 Final Scale Mean Scores by Gender Gender Analytical Inquisitive Insightful Female (n = 175) Male (n =1 25) Effect Size*: Cohen s d Open Minded Systematic Timely Truth Seeking * Small Effect Size =.30; Moderate Effect Size =.50; Large Effect Size =.80 Table 14 Final Scale Mean Scores by Race/Ethnicity Race/ Ethnicity Analytical Inquisitive Insightful White (n = 233) Other (n = 64) Effect Size*: Cohen s d Open Minded Systematic Timely Truth Seeking * Small Effect Size =.30; Moderate Effect Size =.50; Large Effect Size =.80 10

13 Evidence of Construct Validity Conceptually similar constructs (scales) were identified in the International Personality Item Pool (IPIP) for each MTS scale. The MTS scales were mapped to the IPIP is as follows: My Thinking Styles Analytical Inquisitive Insightful Open-Minded Systematic Timely Truth-Seeking General IPIP Methodical, Planful Inquisitive, Intellectually Open Judgment/Open-Minded, Perspective/Wisdom Empathy, Flexibility Resourceful, Intellect, Judgment* Efficiency, Resourceful Valor/Bravery/Courage** Creativity *Elements of multiple scales are represented in Systematic. No single IPIP construct fully mapped to Systematic. **Valor/Bravery/Courage should also correlate with the dimension of Insightful that measures steadfast behavior. The results in Table 15 provide initial support for the MTS constructs. With the exception of Open Minded, the scales correlated more highly with the IPIP scales than expected, which was higher than with other scales. The relationships between the IPIP scales and the MTS were generally positive. This was expected, given that the MTS scales measure positive thinking dispositions and the selected IPIP scales measure desirable characteristics (e.g., courage, resourcefulness, judgment). Table 15 Predicted Relationships Between My Thinking Styles and IPIP Constructs My Thinking Styles Scale r IPIP Predicted Constructs r IPIP Secondary Constructs Analytical.67 Methodicalness.67 Judgment.62 Planfulness.65 Resourceful.64 Efficiency Inquisitive.73 Inquisitiveness.79 Creativity.76 Intellectual Openness.70 Intellect.65 Resourcefulness Insightful.66 Judgment.68 Resourcefulness.68 Perspective.61 Valor/Bravery/Courage Open Minded.49 Empathy.56 Intellectual Openness.45 Flexibility.52 Perspective/Wisdom Systematic.71 Resourcefulness.49 Intellect.63 Judgment Timely.75 Efficiency.72 Resourcefulness Truth Seeking.66 Valor/Bravery/Courage 11

14 My Thinking Styles Inter-Scale Correlations The correlations among the My Thinking Styles scales are presented in Table 16. As expected, the MTS scales are highly correlated; all represent positive dispositions that support critical thinking. Table 16 My Thinking Styles Inter-Scale Correlations Analytical Inquisitive.56 Analytical Inquisitive Insightful Insightful Open Minded Systematic Open Minded Systematic Timely Timely Truth Seeking

15 Interpretation Explaining What My Thinking Styles Scales Measure An individual may want to complete the MTS for his or her own self development. In educational and work settings, teachers or managers may assign the MTS to students or employees for self-development purposes. In both settings, it is important to stress that the MTS measures behavioral tendencies and that it is in the person s best interest to think about their actual behavior (as opposed to ideal behavior) when answering the questions. The purpose is to get a baseline assessment of their actual thinking style preferences. Interpreting Results When interpreting MTS results, it is important to keep two principles in mind. First, all seven styles are positive, so higher scores suggest a greater tendency to engage in a positive thinking style. Currently, there is no evidence that high scores indicate over use. These styles are positive habits that foster the development of and the use of critical thinking skills. Second, the assessment is ipsative, so people should focus on the relative strength of their MTS scale scores. Their highest scores reflect the styles they use most frequently, and probably, most comfortably. This suggests that these scales are personal strengths. Lower scores on scales reflect the styles they use less frequently. Because the assessment is ipsative, having a lower scale score (relative to other scales), does not necessarily equate to an absolute low score. It is simply lower than the individual s other scores. When sharing results, it is important to keep the interpretation positive and focused on how the person can use this information to leverage or improve his or her critical thinking skill development. In situations where intact teams are reviewing their results as part of a training session, it is sometimes useful to look at the results from a team perspective. Of course, this can be done only if participants are comfortable and have given permission to share results. From a team perspective, it is useful to discuss the benefits of having a number of different styles represented on a team; this diversity creates balance. Similarly, if everyone has the same low score (e.g., Timely) it suggests that this is not a strength within the team. Summary There is psychometric evidence to support the use of My Thinking Style scales in the assessment of critical thinking skills. This tool will benefit from additional research, most notably correlations with critical thinking skills as rated by knowledge observers (rather than self-reported). The use and interpretation of the MTS is solely for self-development. It was not designed for selection or diagnostic purposes. The results are intended to provide people with insight to the relative frequency with which they engage in behaviors that support good critical thinking and to help them develop better critical thinking skills. 13

16 References Facione, P.A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Newark, DE: American Philosophical Association. Facione, P. A., Facione N. C., & Giancarlo, C (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking skills. Journal of Informal Logic, 20(1), Gough, H. G. (1996). California psychological inventory manual. Palo Alto, CA: CPP. International Personality Item Pool: A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences ( Internet Web Site. Jackson, D. N., Paunonen, S. V., & Tremblay, P. F. (2000). Six Factor Personality Questionnaire. Port Huron, MI: Sigma Assessment Systems, Inc. Peterson C., & Seligman, M. E. P. (2004). Character strengths and virtues: A handbook and classification. Oxford: Oxford University Press. Watson, G., & Glaser, E. M. (2010). Watson-Glaser II Critical Thinking Appraisal technical manual and user's guide. Bloomington, MN: Pearson. 14