Brief Report The Sport Psychologist, 2004, 18, 464-468 2004 Human Kinetics Publishers, Inc. Which Questionnaire? Assessment Practices of Sport Psychology Consultants Edmund A. O Connor, Jr. Rehabilitation Professionals, Grand Rapids, MI Sport psychology consultants have numerous choices regarding which questionnaire to use when evaluating an athlete. Ostrow (1996) listed more than 300 psychological tests specific to the sport and exercises sciences alone. Which questionnaire to use, if any, should be based on a clear conceptual definition and good operationalization of the desired construct to be measured. The measure should also demonstrate sufficient validity and reliability (Schutz, 1994). Vealey and Garner-Holman (1998) supported the need for reliable and valid measures but raised the concern that the pursuit of scientific credibility in measurement can often come at the expense of practicality and usefulness. These authors suggested a movement toward practical measurement in applied sport psychology where the validity of a questionnaire is also assessed in relation to how effectively the measure is used in professional practice. Sixty-three to 75% of surveyed sport psychology consultants used some type of questionnaire in their applied work with athletes (Gould, Tammen, Murphy, & May, 1989; Vealey & Garner-Holman, 1998) with much variety their testing choices. The survey used in this brief report was intended to update and replicate the previous research on consultants test choices in applied practice and identify the issues consultants wanted to assess with questionnaires. Identification of both the measures currently used and the issues to assess may serve as a first step to begin exploring the practical validity of sport psychology tests suggested by Vealey and Garner-Holman (1998). Participants Method All 141 United States consultants who were certified through the Association for the Advancement of Applied Sport Psychology (AAASP) or members of the United States Olympic Committee Sport Psychology Registry at the time of this survey The author is with GRSportsCenter.com: A division of Rehabilitation Professionals in Grand Rapids, MI 49503. E-mail: Edmundo@rehabpros.com. 464
Which Questionnaire? Assessment Practices 465 were asked to participate via e-mail (n = 129) or mail (n = 12). Fifty consultants responded (35.5%). Respondents indicated a background in psychology (n = 21), sport psychology (n = 12), both psychology and sport psychology (n = 3), and physical education or sport science (n = 11). Three respondents did not indicate their background. Employment settings included university (n = 29), private practice (n = 20), sport center (n = 2), sports medicine clinic (n = 2), and medical center (n = 2), with 8 respondents indicating more than 1 setting and 3 not reporting any. Respondents spent an average of 15.2 years practicing sport psychology. Questionnaire The consultants completed demographic data and identified whether or not they used questionnaires in the assessment of athletes. If questionnaires were used, the consultant answered the following questions: (a) What issues/areas do you feel should be assessed with questionnaires when evaluating athletes? Then please rank them in order of importance, with 1 = most important, 2 = next most important, etc. All areas were allowed a ranking. (b) What measures do you give your athletes? Please indicate if you give these to all your athletes or just some of them. Vealey and Garner-Holman (1998) cautioned against the use of a predetermined battery of inventories that may not be appropriate for certain athletes if the characteristics measured are not as relevant to their psychobehavioral functioning. This all or some question intended to identify either a broad use of questionnaires or a choice of tests dependent upon other factors (e.g., specific performance issue). Results Thirty-three (66%) respondents reported using questionnaires in their assessments. Respondents who used questionnaires did not differ demographically in academic degree, χ 2 (2, N = 49) = 1.12, p =.570, V =.15; educational background, χ 2 (6, N = 47) = 5.36, p =.498, V =.34; or employment setting, χ 2 (3, N = 46) = 3.70, p =.296, V =.28 from those who did not. Those who used questionnaires had significantly more years experience (M = 16.5) than those who did not (M = 12.1; F = 4.23, p =.046, η 2 =.09). Respondents identified 61 issues to assess and 73 questionnaires used in applied practice. The most frequently cited issues and measures are listed in Table 1. Examination of each respondent s data found that some of the issues identified as important to assess did not have a corresponding test listed. The Self-Created category of 7 different tests is limited to those that are not published or regularly distributed. Eleven respondents reported using 16 different self-created measures when worksheets made available to the public via a book or by contacting the author, but were not known to have published psychometrics, were included. Examples include Competition Reflections from Orlick s (1986) Psyching for Sport and The Nine Mental Skills of Successful Athletes: A Self-Assessment by Jack Lesyk. Availability information for the most frequently cited measures is listed in the Appendix. Discussion Sixty-six percent of the surveyed consultants reported using questionnaires in their applied practice, similar to previous findings of 63% (Gould et al., 1989)
466 O Connor Table 1 Frequency of Issues and Measures Cited f (f given Issues f (M Rank, SD) Measures to all/most) Sport psychology skills 47 (1.5, 1.9) Profile of Mood States 11 (4) Focus/concentration/ Test of Attention & attention 16 (2.9, 2.7) Interpersonal Style 10 (3) Confidence 9 (3.3, 2.7) Self-Created Measures 7 (1) Goals 7 (3.3, 2.0) Competitive State Anxiety Multiple skills 6 (1.7, 1.2) Inventory-2 7 (4) Imagery 5 (5.4, 3.1) Sport Competition Anxiety Arousal regulation/ Test 6 (1) intensity 3 (4.7, 0.6) State-Trait Anxiety Self-talk 1 (4.0) Inventory 6 (1) Emotions 33 (2.3, 1.4) Achievement Motivations Scale Anxiety 19 (2.3, 2.3) for Sporting Environments 4 (1) Emotional states 12 (3.0, 1.8) Trait Sport Confidence Inventory 4 (2) Depression 1 (2.0) Athletic Coping Skills Anger 1 (10.0) Inventory-28 4 (2) Clinical issues 10 (3.0, 2.3) Sport Anxiety Scale 3 (1) Relationships 8 (3.9, 2.1) Psychological Skills Inventory Stress 6 (3.2, 1.5) for Sports 3 (1) Personality 5 (2.2, 2.2) Motivation 5 (3.0, 1.9) Commitment 5 (3.0, 1.6) and 75% (Vealey & Garner-Holman, 1998). Vealey and Garner-Holman (1998) reported that the most frequently cited reason for using inventories/questionnaires in applied work was making an initial assessment but did not indicate the specific issues to evaluate. More than half the respondents in this survey most often cited sport psychology skills (particularly focus/concentration/attention) and emotions (particularly anxiety) as areas to assess with questionnaires, although a total of 61 issues were identified. The number of issues cited makes interpretation of their ranked importance difficult. Among the sport psychology skills identified, however, consultants identified assessing multiple sport psychology skills as most important (M rank = 1.7) and imagery as relatively less important (M rank = 5.4) when other issues are also being assessed. Several issues (e.g., anxiety) were associated with a number of different tests, perhaps due to a preferred operational definition of the measured construct in one test over another. Due to the general description of the issues identified (e.g., clinical issues, stress), it is unclear if the tests identified adequately measure all of the issues raised. Gould et al. (1989) identified 26 instruments used by 28 consultants (.93 unique tests per consultant), while Vealey and Garner-Holman (1998) identified
Which Questionnaire? Assessment Practices 467 94 separate inventories and questionnaires used among 68 consultants (1.4 unique tests per consultant). It appears that consultant testing practices continues to become more diverse as 2.2 unique tests per consultant were identified in this study. The most frequently cited tests used in applied practice in this study were similar to previous research (Gould et al., 1989; Vealey & Garner-Holman, 1998) that found the Profile of Mood States and Test of Attentional and Interpersonal Style to be among the top 3 listed. Almost all of the tests were administered to only some of the athletes. This perhaps illustrates the problem-focused assessment encouraged by Vealey and Garner-Holman (1998) and Beckmann and Kellmann (2003). Self-created measures were cited by 21% of respondents in this survey, similar to 19% of the consultants surveyed by Gould et al. (1989) but more than the 9% reported by Vealey and Garner-Holman (1998). Schutz (1994) discouraged the creation of additional sport psychology tests when he estimated more than 220 tests available, approximately 100 tests less than Ostrow s (1996) 314-test directory. His recommendation may be even stronger today, discouraging new tests unless it can be clearly shown that (1) there is a need for the test, (2) the conceptual definition of the construct is well developed and theoretically sound, and (3) the developer of the questionnaire/test/inventory has followed careful procedures and adhered to rigorous standards in constructing the items and in establishing the reliability and validity of the test. (Schutz, 1994, p. 49) Additional research is needed to clarify if additional tests are necessary, identify what issues are not currently addressed by existing measures, and improve the psychometrics of existing tests. Research efforts that improve the existing measures and interpretation guidelines will benefit the progress of sport psychology assessment. Establishment of norms regarding gender, sport, and participation level could increase the usefulness of the valid tests identified in this study. Researchers may turn to this list as a guide for psychometric examination of tests that are commonly used. References Beckmann, J., & Kellmann, M. (2003). Procedures and principles of sport psychological assessment. The Sport Psychologist, 17, 338-350. Gould, D., Tammen, V., Murphy, S., & May, J. (1989). An examination of U.S. Olympic sport psychology consultants and the services they provide. The Sport Psychologist, 3, 300-312. Orlick, T. (1986) Psyching for sport: Mental training for athletes. Champaign, IL: Leisure Press. Ostrow, A.C. (1996). Directory of psychological tests in the sport and exercise sciences (2nd ed.). Morgantown, VA: Fitness Information Technology, Inc. Schutz, R.W. (1994). Methodological issues and measurement problems in sport psychology. In S. Serpa, J. Alves, & V. Pataco (Eds.), International perspectives on sport and exercise psychology (pp. 35-57). Morgantown, WV: Fitness Information Technology.
468 O Connor Vealey, R.S., & Garner-Holman, M. (1998). Applied sport psychology: Measurement issues. In J.L. Duda (Ed.), Advances in sport and exercise psychology measurement (pp. 433-446). Morgantown, WV: Fitness Information Technology. Manuscript submitted: March 19, 2003 Revision received: September 24, 2003 Appendix Test Availability Achievement Motivation Scale for Sporting Environments. http://members.cox.net/ brushall/ (fee required). Athletic Coping Skills Inventory-28. Ron Smith via e-mail at resmith@u.washington.edu. Competitive State Anxiety Inventory-2. Martens, R., Vealey, R.S. & Burton, D. (1990). Competitive anxiety in sport (pp. 117-213). Champaign, IL: Human Kinetics Publishers. Profile of Mood States. E-mail: customerservice@mhs.com, or by phone in the U.S. 1-800-456-3003, in Canada 416-492-2627, and Internationally at +1-416-492-2627 (fee required). Psychological Skills Inventory for Sports. Michael Mahoney, Dept. of Psychology, 258 Terrill Hall, P.O. Box 311280, University of North Texas, Denton, TX 76203-6587, 940-565-3289; mahoneym@unt.edu. Sport Anxiety Scale. Ron Smith via e-mail at resmith@u.washington.edu. Sport Competition Anxiety Test. Martens, R., Vealey, R.S. & Burton, D. (1990). Competitive anxiety in sport (pp. 3-115). Champaign, IL: Human Kinetics Publishers. State-Trait Anxiety Inventory. www.mindgarden.com. 650-261-3500 (fee required). Test of Attention & Interpersonal Style. www.enhanced-performance.com (fee required). Trait Sport Confidence Inventory. Robin Vealey, Dept. of Physical Education, Health, and Sport Studies, 202 E Phillips Hall, Miami University, Oxford, OH 45056; vealeyrs@muohio.edu; 513-529-6530.