Talent Detection Programs in Sport: The Questionable Use of Psychological Measures

Transcription

1 Talent Detection Programs in Sport: The Questionable Use of Psychological Measures MarkRAnshel Middle Tennessee State University Ronnie Lidor University of Haifa Recent empirical studies have questioned the use of tests assessing physical ability and skill level in talent detection (TD), also called talent identification (TID), and in early phases of sport development, particularly in terms ofpredicting future ability/success of young prospects. The main purpose of this article is to question the efficacy of predicting future talent in sport using psychological measures. It is contended that predicting future high quality performance in sport is based on psychological measures that have low validity, are based on flawed research procedures, lack consistency in defining elite versus non-elite athletes, have inherent bias in the athlete selection process, and contain inaccuracies in measuring sport personality, among other concerns. From a philosophical perspective, issues are addressed that reflect TD as an unethical procedure. Concerns about TD/TID programs include philosophical issues that question the integrity and appropriateness of talent prediction programs are also discussed. Plausible options for replacing TD/TID programs with other approaches (e.g., talent development, performance profiling) are provided Recommendations include improved athlete selection for further skill development, availability of high quality coaching to athletes of all skill levels (particularly in youth sports), and more efficient use of limited financial resources for non-elite community sports programs. Address correspondence to: Mark H. Anshel, Ph.D., Middle Tennessee State University Department of Health and Human Performance, Box 96, Murfreeshoro, Tennessee Mark.Anshel@mtsu.edu, Fax:

2 240/Journal of Sport Behavior, Vol. 35, No. 3 The pressure exerted by sports organizations, sponsors, and, in some countries, by governmental bodies (e.g.. Sport Authorities, Olympic Training Centers) on young athletes to be successful sports competitors is greater than ever. Perhaps not surprisingly, there is also considerable pressure to predict future high quality sport performance in competitive settings by using various physiological, anthropometrical, and psychological tests on young athletes (see Abbott & Collins, 2002,2004; Martindale, Collins, & Daubney, 2005; Tranckle & Cushion, 2006). Vaeyens, Gullich, Warr, and Philippaerts (2009) contend that programs whose aim is to predict future sport success, called talent detection (TD) or talent identification (TID) programs, "are designed to identify young athletes who possess extraordinary potential for success in senior elite sport, and to select and recruit them into talent promotion programs" (p. 1367). The purpose of these programs is, ostensibly, to "increase athletes' potential by means of a variety of institutional measures designed to accelerate talent development" (p. 1367). Another purpose is to provide tests - motor, physiological, anthropological, biomechanical, and psychological - during an athlete's "early" years in order to predict long-term success in competitive sport. The intention of these programs is to allocate scarce resources toward individual athletes whose test scores show "promising" talent, at least in the long-term. Are these objectives met? Are the tests valid? Is this a good idea, at least from a philosophical perspective? The main purposes of this article are: ( 1 ) to critique, both empirically and philosophically, the value of predicting future talent in sport using psychological measures, and (2) to offer suggested directions for enhancing the processes of TD/TID, and talent development (TDV). The existing evidence suggests that the use of psychological inventories to predict future success and achievement in elite-level competitive sport lacks validity and proper ethics. The review will include the following: (1) defining important concepts, such as talent detection, also called talent identification, (2) providing empirical arguments in favor of TD programs, (3) making a case against the use of psychological measures in TD programs, (4) examining the philosophical arguments against the use of psychological measures in TD programs, and finally, (5) providing recommendations and guidelines for initiating talent development programs in sport. Defining Terms and Concepts Brown (2001) and St-Aubin and Sidney (1996) have defined TD as the methodological process of predicting sport performance over various periods of time by obtaining information on the prospects' physical, physiological, and technical abilities, either alone or in combination, with measures of psychological aptitudes. TD has also been described as a process by which children are encouraged to participate in the sports in which they are most likely to

3 TALENT DETECTION IN SPORT.../241 succeed, based on the results of testing selected parameters (Bompa, 1999). Woodman (1985) defined TD programs (in Australia) as "the screening of young athletes to determine those most likely to succeed in sport and directing them towards the sports to which they are most suited" (p. 49). To Hahn and Tumilty (1989), TD programs consist of the selection of individuals who have shown to have the characteristics important for success at the highest levels of a particular sport. Durand-Bush and Salmela (2001) contend that TD programs reflect the attempt to match various performer characteristics - innate, learned, or due to training - with task demands of a given sport, to ensure the highest probability of maximum performance outcome. Williams and Reilly (2000) define TD as the discovery of potential performers who are currently not involved in any sport program. Finally, Lidor, Côté, and Hackfort (2009) use the term talent identification as "the process of recognizing individuals currently involved in sport with the potential to become elite athletes/players" (p. 134). All of these definitions consider TD as any conscious effort that recognizes individuals who have the potential to become elite athletes. Two concepts that have been used interchangeably, but erroneously, with TD are talent selection (TS) and talent development (TDV). TS consists of the ongoing process of identifying athletes/players at various stages of the training program. To Lidor et al. (2009), TS programs refer to specific tasks or tests that target an athlete's capability to demonstrate competence in a particular sport or position within that sport. TDV, on the other hand, "implies that the athletes/players are being provided with the appropriate learning/practice conditions to promote and realize their potential in a specific sport" (Lidor et al, p. 134). TS does not include attempts to predict future success based on identifying the athlete's psychological characteristics. Ostensible Advantages of TD/TID Programs TID programs are often of great importance to select sporting bodies of governments that seek national and international status in competitive sport and apply scarce financial resources toward developing potential champion athletes, so that they may achieve national and international recognition (Vaeyens et al., 2009). Effective use of these financial resources are compromised, however, if these efforts fail to accurately predict future success in competitive sport, particularly among younger competitors (Lidor et al., 2009). Ideally, therefore, the early detection of talent provides the opportunity to obtain the best "retum," or "investment," in giving potential elite level competitors the required resources in coaching expertise, equipment, facilities, practice time, and opportunities to reach their full sport potential. TID programs have been attempted with respect to the testing of sport skills, as well as of physiological and anthropological parameters.

4 242/Journal of Sport Behavior, Vol. 35, No. 3 Proponents of TD programs claim that showing that sport skill tests are, in fact, efficacious in predicting future sport skill performance adds credence to the examination of the psychological dimension (Vaeyens et al., 2009). In fact, the results of selected studies have indicated that measures of motor ability and motor skill proficiency predict future sport performance to a relatively high degree, at least at the elite level (e.g., Falk, Lidor, Lander, & Lang, 2004; Kerr, Booth, Dainty, & Gaborault, 1980). Tennis Canada's First Serve TD program includes measurements of skin-fold, bone diameter, body girth, and reaction times that, ostensibly, accurately predict an athlete's future sport skill level and the athlete's compatibility with the demands of a particular sport (Leone, 1993). The program does not, however, include psychological measures. Another apparent advantage of TD programs is that they maximize the number of gifted individuals participating in a given sport, resulting in stronger domestic competition and likely increasing the number of internationally competitive athletes (Durand-Bush & Salmela, 2001 ; Hahn, 1990). This is because TD programs ostensibly promote competitiveness and direct athletes toward sports in which they are more likely to succeed, increasing the number of athletes aiming for elite levels of sport (Abbott & Collins, 2002,2004; Bompa, 1999). Meeting psychological needs reduces sport attrition, that is, the likelihood of dropping out of a sport at which the athlete is expected to succeed (Petlichkoff, 1993, 1996). As Petlichkoff noted, attempts to determine the sport that best represents a young athlete's skills might provide a more efficient way than traditional trial-and-error approaches. Along these lines, Bompa (1999), Hahn (1990), and Haskell (1983) contend that TD programs profile the athletes' strengths and weaknesses, and provide them with relevant feedback so that they can effectively monitor their progress throughout the entire training program. For example, Pethchkoff ( 1993,1996) contends that children who driftfi-om sport to sport in an attempt to fmd a satisfying and rewarding experience waste an enormous amount of time and resources. As a result, many children with high quality sport talent do not find their niche in sport, consuming considerable time in their search for a sport that is compatible with their skills and goals (Feldman, 1986). TD programs, therefore, can maximize the number of children who have positive sport experiences and a greater likelihood of success, thereby reducing the rate of sport dropout (St-Aubin & Sidney, 1996). The productivity of elite coaches is also enhanced by ensuring that their time, energy, and resources are directed toward the development of younger athletes who have the potential to succeed in elite sport (Bloom, 2002; MacNamara, Button, & Collins, 2010). One additional factor in favor of TD programs is the limited statistical evidence from numerous studies, reviewed by Deaner and Silva (2002), in which discriminant function analysis has detected unique psychological characteristics that predict long-term sport success

5 TALENT DETECTION IN SPORT... /243 among young athletes. For example, athletes are categorized as "elite" and "non-elite" on measures of self-confidence (Andersen, 1976;Vealey, 1985,2002), ambition (Mahoney, 1989), self-motivation (Mahoney, 1989), emotional stability (Missoum & Laforestrie, 1981), and enthusiasm (Missoum & Laforestrie, 1981). When these advantages are considered together, an effective TD program will not only identify (younger) athletes who already possess desirable psychological characteristics that are commensurate with successful sport performance, but will also create a template against which other athletes (and their coaches and parents) can aspire and leam over time (Renger, 1993). Table 1 lists selected studies that discriminate between successful and less-successful athletes. The Case Against the Use of Psychological Measures in TD Programs The case against the use of psychological measures in TD sports programs rests primarily on three factors, failure to take into consideration the performers' physical maturation, the coach's role in the athlete's skill development, and flaws in the scientific process. The Performers ' Physical Maturation TD programs may assist coaches, athletes, and the athletes' parents in identifying the type of sport that is most compatible based on the performer's physical attributes. These programs, however, may not accurately predict future skill development and sport performance. Predicting future successful sport performance using physiological and anthropological measures has received uneven support in the exercise science literature (Lidor et al., 2009). For example, based on their review of 13 studies that were aimed at distinguishing between highly-talented and less-talented athletes, Lidor et al. concluded that "no clear-cut evidence has been found to support the predictive value of physical tests in talent detection and early development in sport" (p. 140). They cite numerous studies indicating "no correlation of physical tests with final selection and ranking of athletes" (p. 140). Along these lines. Till, Cobley, O'Hara, Chapman, and Cooke (2010) found low relationships between anthropométrie, physiological, and selected characteristics in high performance junior rugby league players in the United Kingdom. The authors concluded that these results raise concems about the ability of motor skill testing to identify characteristics of immediate and long-term player selection and development. Similar concems have surfaced conceming the use of psychological testing for prediction purposes.

6 244/Journal of Sport Behavior, Vol. 35, No. 3 Table 1. Psychological characteristics discriminating between successful and less successfid athletes Constructs Study N Sex Age Sport Instruments Used discriminating skill level Andersen (1976) 152 Male Swimming Cattell 16 PF Social skills Self-confidence Hahn (1990) Review of Literature Stubbomness Self-confidence Goal-orientation Self-motivation Anxiety* Haskell (1983) Review of Literature Self-confidence Goal-orientation. Self-control Enthusiasm Self-motivation Ho (1987) Review of Literature Stubbomness Ambition Self-control Intelligence Hogg (1986) * denotes a decrease in the designated trait Review of Literature Stubbomness Self-confidence Goal-orientation Emotional stability Ambition Social skills Self-motivation

7 TALENT DETECTION IN SPORT... /245 Table 1 Continued. Psychological characteristics discriminating between successful and less successfid athletes Constructs Study N Sex Age Sport Instruments Used discriminating skiu level Jerome 273 Female Synchronized (1993) Swimming Cattell HSPQ 16 PF SCAT Rotto-I-E Buss-Durkee Hostility Self-Analysis Test Motivation Analysis Test (MAT) Happy-go-lucky Anxiety* Exta-nalLOC Kalinowski 24 Male & Swimming (1985) Female Komadel (1988) Mahoney (1989") Mahoney, Gabriel, & PeiHfin«(1987) Male Male& Female Weightlifting Various Intaviews Review of Literature SCL-90R Mood (POMS) Psychological Skills in Sport (PSIS-P5) Ambition Self-motivation Emotional stability Intelligence Self-motivation Anxiety* Neuroticism* Self-motivation Neuroticism* Anxiety* Missoum & Laforestrie (1981) 220 Male Various Eysenck Personality Inventory Emotional stability Ambition Enthusiasm * denotes a decrease in the designated trait

8 246 / Journal of Sport Behavior, Vol. 35, No. 3 Along these lines, researchers and practitioners have examined the relationship between TD and TDV in sport. For example, Gulbin, Oldenziel, Weissensteiner, and Gagne (2010) reviewed "key developmental experiences and insights" of 673 high performance Australian athletes (p. 149). They determined that elite athletes possess several selected characteristics that are not found in their non-elite counterparts. All of the identified characteristics, however, were behavioral (e.g., commitment to practice, access to high quality coaching) and not psychological in nature. In addition, no personalify traits were listed. In their review of related literature, Lidor et al. (2009) concluded that assessing physical ability and skill level in order to determine future talent of athletes offers no clear support of the predictive value of these tests, either for individual or for team sports. Thus, while TD programs may help a young athlete decide to which sport he or she is best suited, the capability of these programs to predict future sport success may not be as promising. Coach Expertise and Influence The athlete's coach is almost always the most important external source that influences the development of physical and mental skills (Bloom, 2002). Two issues must be addressed with respect to the coach's role in the use of psychological measures in detecting and predicting an athlete's talent. First, reliance on the use of psychological inventories in TID programs imdermines the coach's role in developing the athlete's talent. Predicting future performance fi-om inventories does not take into account a coach's expertise. Coaches are primarily responsible for each athlete's development and maturation, particularly at the elite level (Salmela & Régnier, 1985). The coach's expertise is far more likely to influence an athlete's performance potential than psychological testing, especially over the long-term (Bloom). The second point related to coaches is that athletes who are designated as having "high," or "good," potential to achieve in sport are likely to receive far superior coaching than their less-skilled peers. This phenomenon, called an "expectancy effect," consists of a person in a subordinate position (e.g., child, student, athlete, experimental participant) responding to an authority figure (e.g., parent, teacher, parent, coach, experimenter) in a manner that is consistent with the authority figure's expectations (Thomas, Nelson, & Silverman, 2011 ). Three fypes of expectancy effects include halo effect, Rosenthal effect, and Hawthorne effect (see Thomas et al., for descriptions). In the current context, this phenomenon might refer to as a coaching bias built into TID programs. Two studies lend credence to this view. Christensen (2009) conducted in-depth interviews with eight elite soccer coaches, who identified the characteristics of highly-skilled soccer players. Christensen found that coaches predicted future success among highly-rated players who "were assumed to be willing to leam" and were "perceived to be hard working and dedicated" by their coaches (p. 379). These

9 TALENT DETECTION IN SPORT.../247 qualities are derived from good coaching rather than being generated from an inventory that ostensibly predicts the level of future sport performance. In another study, Davids and Baker (2007) found that highly-skilled coaches are more likely to be associated with elite athletes due to their excellent teaching and leadership skills. Specifically, better coaches (of elite athletes) offer superior structure and content of practice, maximize training time, and engage in meticulous planning. Thus, the degree of coach expertise is a mediating, but rarely controlled, variable in the attempt to validate the efficacy of TD/TID programs (Reilly, Williams Nevill & Franks, 2000). Flaws in the Scientific Process Flaws in the scientific process, which represent particularly powerful issues in questioning the role of psychological factors in talent TD, include these 12 components: (1) vague definitions of selected constructs, (2) inconsistency in defining an "elite" athlete, (3) invalid inventories/poor predictive validity, (4) poor research methodology and statistical procedures, (5) sample bias, (6) failure to use baseline measures, (7) extensive use of cross-sectional comparisons, (8) paucity of skill level comparisons, (9) poor inventory construction, (10) lhnitations in personality research, (11) inherent problems with self-report, and (12) overreliance on anecdotal evidence. Vague Definitions of Selected Constructs The terms "mental toughness," "competitiveness," and "psychological readiness" are often used when attempting to determine an athlete's potential for future success. Ahnost unknown, however, are their operational defmitions, and the extent to which these characteristics identify or predict sport skill level (Singer & Janelle, 1999). In addition, interpreting and applying such arguments would challenge most sport psychology consultants and coaches. It is not known, for example, whether these measures are stable (i.e., trait) or situational (i.e., state) constructs, or whether they refiect relatively stable, cross-situational dispositions (i.e., traits) and thus are open to change through intervention and experience (i.e., state constructs), as proposed by Anshel (2012). Similar limitations are inherent in examining specific psychological characteristics in TD research (Durand-Bush & Salmela, 2001). Inconsistency in Defining an "Elite" Athlete Examining psychological characteristics of athletes in predicting future success has usually consisted of comparing "elite" and "non-elite" athletes. Operationally defining an "elite" sports competitor, however, has been markedly inconsistent in the literature (Anshel, 2012). Often, researchers have used statistical procedures to discriminate skill level as a fimc-

10 248/Journal of Sport Behavior, Vol. 35, No. 3 tion of his or her current success or achievement (Matsudo, 1996). Traditionally, elite athletes have been defined as individuals "who are eligible for competition at the national, intemational, or Olympic level, or who are professional sports persons" (Van den Auweele, Cuyper, Van Mele, & Rzewnicki, 1993, p. 257). An additional definition of the elite athlete includes individuals who are eligible for such competition, but may not actually compete (e.g., Spamer & Coetzee, 2002), while another definition refers to athletes who are currently involved in sport competition at a particular level (Falk et al., 2004). An additional concem is that the term "elite" is often culturally specific (Gan, Anshel, & Kim, 2009). For example, an elite athlete may be defined as a sports competitor at the national level in some studies, while in other studies the term "elite" is used for college students who played on their high school sports teams. It is unlikely, therefore, that elite athletes, as identified in various studies from different cultures will display the same characteristics in, for example, Africa (Spamer & Coetzee, 2002), Asia (Gan et al., 2009), Europe (Williams & Reilly, 2000), and North America (Brown, 2001). This inconsistency compromises the primary objective of TD programs - to predict the future quality of sport performance. Invalid Inventories: Poor Predictive Validity Perhaps one of the most compelling cases against the use of psychological measures in TD programs is poor predictive validity. Predictive validity reflects "the degree to which a measuring instrument or test yields information allowing prediction of actual behavior or performance" (Myers & Hansen, 2012, p. 592). A plethora of published studies comparing elite and non-elite or high and low-skilled sports competitors on selected psychological variables did not have promising results. For example, Prescott (1996) attempted to identify motivation, goal orientation, attribution, and locus of control as predictors of talent among British gymnasts aged 7-10 yrs, and found a very low prediction rate. In their extensive review, Deaner and Silva (2002) concluded that "while some of these studies do show personality differences based on sport type and gender...many of these studies are old and focus only on a few select sports or a few select characteristics" (p. 61). Poor Research Methodology and Statistical Procedures Researchers and theorists have noted inherent limitations of many studies concemed with identifying current psychological characteristics of athletes, comparing athletes categorized as elite and non-elite or making cross cultural comparisons (Gauvin & Russell, 1993), and predicting athletes' future achievement level in sport. Some of these issues have concemed the use of inventories that were not intended for the current sample (Morgan, 1997), and improper psychometric validation and statistical procedures that render the instrument invalid

11 TALENT DETECTION IN SPORT.../249 (Schutz & Gessaroli, 1993). In his review of related literature concerning methodological research problems, Morgan lists "the absence of randomization, small sample size, inadequate psychological measures, and experimenter expectancy effects, among other flaws" (p. 4). Also problematic in this area is that researchers have labeled constructs interchangeably, such as juxtaposing the athltts's personality traits with his or her orientations, styles, dispositions, and behavioral tendencies (Anshel, 2012). Each of these constructs differ; some are more amenable to change through counseling and treatment (e.g., orientations of mental toughness or competitiveness; behavioral tendencies such as pre-performance routines) than others (e.g., trait anxiefy, neuroticism, trait anger, stimulus-seeking). Failure to control for moderator variables such as gender and culture provides an additional concern. Gauvin and Russell contend, for instance, that "it is widely acknowledged that such cultural factors can potentially produce major distortions and inaccuracies in test interpretation" (p. 892). Based on their thorough review of related literature, Gauvin and Russell concluded that "the selection of sport/exercise-specific tests and scales.,.requires a careful conceptual analysis of the constructs under investigation, an examination of the measurement assumptions of the theoretical framework employed, and in some cases, a consideration of the amount of variance explained in the target variables" (p. 899). Clearly, future study is needed toward the continued development and validation of psychological measures that attempt to predict and identify the potential for future talent in sport. Taken together, a common threat to internal and external validity is the use of a selfreport instrument that was neither constructed nor validated for the intended sample (Thomas et al., 2011). For instance, sample characteristics, or the psychological demands of specific sports in which "desirable" traits are being identified, are not taken into account when developing inventory items (Andersen, 1976; Gauvin & Russell, 2003; Hahn, 1990). Consequently, one inherent limitation in the existing literature is the lack of consistency in determining for whom the inventory was intended and to whom it may be applied (e.g., a universify athlete, a highly talented competitor at the community level, a national or an international level competitor, or an Olympic or professional performer). The use of improper statistical analyses in TD research has been ubiquitous (see Renger, 1993; Schutz, 1998; Schutz & Gessaroli, 1993; and Vealey, 1985, for reviews). While an exhaustive review of these limitations goes beyond the scope of this paper, specific examples abound. For example, one statistical approach by researchers has been to attempt to statistically separate elite from non-elite athletes using multiple regression models and discriminant analyses. However, the use of a regression equation on a different population from the one for which it was developed is inappropriate (Nesselroade & Baltes, 1979). While it is important to test for predictive validity by cross-validating results (Renger, 1993), most studies have not included

12 250/Journal of Sport Behavior. Vol. 35, No. 3 attempts at cross-validation. Another statistical limitation in TD assessment is the frequent use of univariate, not multivariate, statistics resulting in low predictive power, the virtual absence of statistical interactions, and the failure to consider the complex network of factors underlying sport performance (Schutz, 1998). One misuse of multivariate statistics is the violation of acceptable caseto-predictor ratios, resulting in a loss of statistical power. An acceptable ratio is 5:1, and preferably 6:1 (Tabachnick & Fidell, 2007). Instead, Salmela and Régnier (1985) propose using discriminant function analyses (DFA) to determine if the selected variables discriminate among the members of each group, and specifically, to find variables that are appropriate for testing the targeted population. DFA may identify athletes who are highly skilled, however, it does not predict future performance (Régnier, Salmela, & Russell, 1993 ; Schutz & Gessaroli, 1993). Regression analyses more accurately predict outcomes within a targeted population. Yet another limitation of TD research is the incorrect interpretation of correlational data as cause and effect (Schutz & Gessaroli, 1993). Studies of the TD literature report a lack of proper research methods and statistical procedures (see Anshel, 2012; Durand-Bush & Salmela, 2001; Lidor et al., 2009; Schutz & Gessaroli, 1993; St-Aubin & Sidney, 1996; Van den Auweele et al., 1993;Vealey, 1985,2002). According to these authors, the primary issues that have compromised the integrity of attempts to predict high quality sport performance (i.e., TID) among child or adolescent age groups include sample bias, failure to use baseline measures, improper statistical procedures, extensive use of cross-sectional comparisons, a paucity of skill level comparisons, failure to control for coach expertise, poor inventory construction, inherent limitations of self-report, and over-reliance on anecdotal evidence. Sample Bias Selection bias for research purposes occurs in cases where the participants in studies are recruited based on their availability, their personal motivation to engage in the study, investigator coercion (i.e., participation not fully voluntary), or the athletes' current skill level and pre-existing personal characteristics (Thomas et al, 2011). Selection bias may result in statistical regression or spontaneous remission, which may innate positive results. Collectively, these biases may contribute to an expectancy effect (Martinek, & Karper, 1984), also called a self-fulfilling prophecy (Hom, Lox, & Labrador, 1998). This is because these athletes are usually labeled "elite" or "highly skilled," thereby influencing the coaches' (or researchers') attitudes, expectations, and behaviors toward these pre-labeled players. Ostensibly, then, athletes with "superior" scores on selected psychological characteristics may excel because of the high expectations of their coaches or researchers. Coaches with high expectations of