The Use of Testing Technicians: Critical Issues for Professional Psychology

Transcription

1 INTERNATIONAL JOURNAL OF TESTING, 5(4), Copyright 2005, Lawrence Erlbaum Associates, Inc. The Use of Testing Technicians: Critical Issues for Professional Psychology John D. Hall and D. Lynn Howerton Department of Psychology and Counseling Arkansas State University Aaron U. Bolin U.S. Navy Human Performance Center Newport, RI The controversial practice of using unlicensed individuals to administered psychological tests has been questioned by some psychologists, professional organizations, state and provincial boards of psychology, state governments, departments of education, and third-party health care providers. This article provides an overview of the ethical, legal, and technical issues surrounding the controversy. On the basis of a review of these issues, recommendations are offered that are directed toward state and provincial boards of psychology, developers and publishers of psychological tests, and licensed psychologists. Key words: testing, assessment, intelligence test, licensure, technicians Psychological testing has been a defining function for psychologists since the field s inception (Camara, Nathan, & Puente, 2000). In fact, this service has been one of the most important contributions of behavioral science to society (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 1999; Walsh & Betz, 1995). Some also regard psychological testing to be the highest technical achievement of psychology (Swenson, 1997). For several years, there has been an international call to recognize the importance of maintaining high standards in the use of psychological tests (Camara, 1997; Correspondence should be addressed to John D. Hall, Department of Psychology and Counseling, P.O. Box 1560, Arkansas State University, State University, AR jhall@astate.edu

2 358 HALL, HOWERTON, BOLIN Davidson, 1997; Evers, 1996; Fremer, 1996). Specifically, test misuse and user qualifications have become important concerns for many professional groups and practitioners (Fremer, 1996; Merenda, 1997; Moreland, Eyde, Robertson, Primoff, & Most, 1995; Turner, DeMers, Fox, & Reed, 2001; Watkins & Campbell, 1990). Many fear that allowing inadequately trained individuals to administer psychological tests may harm the public and damage the reputation of the psychologists and other trained professionals who use these instruments (Turner et al., 2001). The purpose of this article is to provide an overview of pertinent standards and guidelines regarding proper test use and user qualifications, with specific attention given to the controversial practice of using testing technicians to administer and score restricted psychological tests. Pertinent standards and guidelines are addressed as well as ethical, legal, and technical issues related to the use of testing technicians by professional psychologists. On the basis of a review of these issues, recommendations are offered that are directed toward state and provincial boards of psychology, developers and publishers of psychological tests, and licensed psychologists. BACKGROUND AND HISTORICAL CONTEXT The term testing technician, as discussed in this article, refers to an individual who administers and scores psychological tests who is uncredentialed and not licensed or certified by an appropriate statutory body (e.g., a state or provincial board of psychology, a state department of education). Testing technicians are typically trained and supervised by a licensed psychologist to administer selected psychological tests. The technician generally holds an undergraduate degree in psychology or a closely related field and usually does not hold an advanced degree. As a result, most technicians have not received graduate-level instruction in the administration, scoring, and interpretation of psychological tests. In the professional literature, testing technicians have been referred to by a number of different titles. Examples include (a) psychology or psychological technicians (Cohen, 1974; Gentry, 1974; Guilmette, Faust, Hart, & Arkes, 1990; Musante, 1974; Sloop & Quarrick, 1974; Sweet & Moberg, 1990; Wilson, 1982), (b) psychological associates or aides (APA, 1981a, 1981b), (c) neuropsychology technicians (DeLuca & Putam, 1993), (d) psychometrists (Report of the Division 40 Task Force on Education, Accreditation, and Credentialing, 1989), (e) psychodiagnostic assistants (L abate, 1973), (f) psychological extenders (Arkansas Board of Examiners in Psychology [ABEP], 2000), (g) paraprofessionals (Lowman, 1996), and (h) uncredentialed personnel (Association of State and Provincial Psychology Boards [ASPPB], 2003b). The practice of using testing technicians dates back at least to the 1940s in the United States and is especially prevalent within the context of neuropsychology (DeLuca, 1989). The extent of this practice outside of the United States is unclear.

3 TESTING TECHNICIANS 359 Recent literature searches on the PsycINFO and ERIC databases with the previously mentioned descriptors revealed little published information specific to the use of testing technicians in other countries. In the United States, restricted psychological tests administered and scored by technicians have included the (a) Wechsler intelligence scales, (b) memory scales, (c) Minnesota Multiphasic Personality Inventory, (d) Halstead Reitan test, and (e) Luria Nebraska neuropsychological batteries (Guilmette et al., 1990; Hartlage & Telzrow, 1980). The Wechsler intelligence scales appear to be the tests that are most frequently administered by testing technicians (Camara et al., 2000; Serenty, Dean, Gray, & Hartlage, 1986). Using testing technicians to administer restricted Level C psychological tests is a controversial practice that has generated significant concerns among many psychologists, professional organizations, state and provincial boards of psychology, departments of education, and even third-party health care providers (Lowman, 1996; Wilson, 1982). In fact, in the United States, some state federally supported Medicaid programs clearly prohibit the use of testing technicians by licensed psychologists (National Academy of Neuropsychology, 2003). The major concerns stem from the knowledge that Level C tests require a substantial understanding of testing and psychology in addition to supervised experience in the use of these devices to administer them properly (APA, 1954).The appropriateness of using testing technicians has also been questioned on ethical, legal, and technical grounds. The APA (1988), Hall and Slate (1992), and Anastasi and Urbina (1997) have all indicated that Level C tests should only be used by individuals with a master s degree in psychology and at least 1 year of properly supervised experience provided by other graduate-prepared professionals who possess experience with the tests. A skilled examiner is critical because the assessment outcomes derived from the Level C tests are often used to make high-stakes classification, eligibility, and placement decisions. Despite the persistent questions about the appropriateness of using testing technicians, the practice has a loyal cadre of supporters. For example, DeLuca and Putnam (1993) and L abate (1973) cited several advantages associated with the use of testing technicians. One advantage is objective data collection for blind analysis. However, some have stated that this practice is never appropriate in clinical situations(jarvis& Barth, 1994). Sbordone (1996) stated that a widely held false assumption in the field of neuropsychology is that it is not essential that the neuropsychologist actually test or interview a particular patient if the neuropsychologist has access to patient raw test data (p. 25). Other cited benefits associated with the deployment of testing technicians include the more effective use of the psychologist s time and cost effectiveness. In fact, DeLuca and Putnam found that psychologists who used technicians reported an 89% increase in salary. Frequently used neuropsychology and individual intelligence tests take approximately 2 to 4 hr to administer and score (Camara et al., 2000). This is a substantial amount of assessment time to devote to one case. Time

4 360 HALL, HOWERTON, BOLIN pressure associated with work overload and a desire to engage in other activities may contribute to the delegation of testing to subordinates who may be unqualified (Anastasi & Urbina, 1997). PROFESSIONAL STANDARDS FOR THE USE OF PSYCHOLOGICAL TESTS Several professional and government organizations have addressed concerns regarding test misuse and inadequate examiner qualifications. Prime examples include the most recent revision of the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999) and the work of the Joint Committee on Testing Practices established in Outcomes of this committee include (a) The Code of Fair Testing Practices in Education (Joint Committee on Testing Practices, 1988a), (b) Test User Qualifications: A Data-Based Approach to Promoting Good Test Use (Joint Committee on Testing Practices, 1988b), (c) Responsible Test Use: Case Studies for Assessing Human Behavior(Joint Committee on Testing Practices, 1993), (d) The ABC s of School Testing (Joint Committee on Testing Practices, 1995), and (e) Rights and Responsibilities of Test Takers: Guidelines and Expectations (Test Taker Rights and Responsibilities working Group of the Joint Committee on Testing Practices, 1999). Other professional groups that have recently addressed testing issues include the Committee on Psychological Tests and Assessment established by the APA in 2000 and the International Test Commission (ITC). The ITC (2001) specifically focused on the competence of test users. Relevant excerpts from several of these documents are presented in the following discussion. APA Specialty Guidelines for Clinical, Counseling, Industrial Organizational, and School Psychologists The APA specialty guidelines for clinical, counseling, industrial organizational, and school psychologists (APA, 1981a, 1981b, 1981c, 1981d) were established to provide potential users and other interested groups with essential information about the services available from the different psychology specialties. They represent the best judgment of the profession of the conditions, credentials, and experiences that contribute to competent professional practice. The guidelines also define psychological services associated with each specialty area. For example, the Specialty Guidelines for the Delivery of Services by Clinical Psychologists (APA, 1981a) describe the administration and interpretation of tests of intellectual abilities, attitudes, emotions, motivations, personality characteristics, and psychoneurological status as activities that are commonly practiced by clinical psychologists. On the basis of all of these guidelines, it appears that the professional identity of clinical, counseling, industrial organizational, and school psychologists is partially defined in terms of the assessment role.

5 TESTING TECHNICIANS 361 The specialty guidelines also discuss the use of persons (e.g., testing technicians) who provide psychological services under the supervision of a professional psychologist. The guidelines explicitly address the use of title by stating that such individuals are not psychologists. Instead, they may be referred to as psychological associates, psychological assistants, psychological technicians, psychological aides, school psychological examiners, school psychological technicians, school psychological assistants, school psychometrists, or school psychometric assistants (APA, 1981a, 1981b, 1981d). Standards for Educational and Psychological Testing The Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999) were developed to provide criteria for the evaluation of tests, testing practices, and the effects of test use (p. 2). Part III of the standards focuses on testing applications. These standards include a chapter on The Responsibilities of Test Users that is pertinent to the use of testing technicians. Standard 11.3 notes that the Responsibility for test use should be assumed by or delegated only to those individuals who have the training, professional credentials, and experience necessary to handle this responsibility. Any special qualifications for test administration or interpretation specified in the test manual should be met. (p. 114) The standard also requires professionals who oversee testing and assessment [to be] responsible for ensuring that all persons who administer and score tests have received the appropriate educational training needed to perform these tasks (p. 120). Standard 12.1 states Those who use psychological tests should confine their testing and related assessment activities to their areas of competence, as demonstrated through education, supervised training, experience, and appropriate credentialing (p. 131). Furthermore, Standard 12.8 states that professional[s] should ensure that persons under their supervision, who administer and score tests, are adequately trained in the setting in which the testing occurs and with the populations served (p. 132). ITC International Guidelines for Test Use The ITC, which promotes the proper development, evaluation, and use of psychological and educational instruments worldwide has developed the International Guidelines for Test Use (ITC, 2000). These guidelines encourage the best practices in assessment and focus on the competencies (i.e., knowledge, understanding, and skill) needed internationally by test users. The guidelines contain several sections that are applicable to the use of testing technicians. For example, Section 1.2 calls for ensuring that test users have the competence to use tests. Section 1.3 states that test users take responsibility for their use of tests and only use instruments for which they are qualified. Section 1.4 notes that test materials must be kept se-

6 362 HALL, HOWERTON, BOLIN curely. Section 2.5 calls for the proper administration of the test, whereas Section 2.6 addresses the importance of accurately scoring and analyzing the test results. Finally, Appendix A, entitled Guidelines for an Outline Policy on Testing, addresses guidelines for organizations to consider when developing a policy on testing. Such a policy is needed to ensure that potential test misuse is avoided and to ensure that tests are only used by qualified persons. An organizational policy on testing should address (a) proper test use; (b) security of test materials; (c) who can administer, score, and interpret tests; (d) qualifications of test users, (e) test user training; (f) access to test results and confidentiality issues; (g) feedback of results to test takers; (h) responsibility to test takers, and (i) responsibilities and accountability of test users (ITC, 2000). Use of Nondoctoral Personnel in Clinical Neuropsychological Assessment The Guidelines Regarding the Use of Non-Doctoral Personnel in Clinical Neuropsychological Assessment (Report of the Division 40 Task Force, 1989) present a general position on the use of nondoctoral personnel. These guidelines also delineate the role of these individuals in the context of assessment. The document states that there is no obligation for all psychologists to perform their own assessment (p. 23). Furthermore, it notes that the use of such technicians is a common and accepted practice when the supervising psychologist maintains and monitors high standards of quality assurance (p. 23). According to these guidelines, the technician plays an important role in assessment. However, the role is very narrowly defined. Technicians are responsible only for the administration and scoring of neuropsychological tests under the close supervision of a licensed psychologist. Ethical Principles of Psychologists and the Code of Conduct The code of conduct that most U.S. psychology practitioners are required to follow is the Ethical Principles of Psychologists and Code of Conduct of the American Psychological Association (APA, 1992). This code was first developed in 1953 to ensure the ethical practice of psychology and the protection of the consumer (APA, 1992). A common practice has been for state and provincial boards to adopt these principles and embed them in the local psychology laws. Many of the ethical principles pertain to the use of tests and testing technicians (Walsh & Betz, 1995). One of the most relevant APA ethical principles is General Standard 1.22 (a): Psychologists delegate to their employees, supervisees, and research assistants only those responsibilities that such persons can reasonably be expected to perform com-

7 TESTING TECHNICIANS 363 petently, on the basis of their education, training, or experience, either independently or with the level of supervision being provided. (p. 1602) Furthermore, General Standard 1.22 (b) states that the psychologists provide proper training and supervision to their employees and take reasonable steps to see that such persons perform services responsibly, competently, and ethically (p. 1602). Section 2 of the ethical principles is partly devoted to the proper use of tests in a clear attempt to safeguard the examinee s welfare. Principle 2.02 (b) calls for psychologists to refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide (APA, 1992, p. 1603). Section 2.10 of the principles is titled Maintaining Test Security. This section requires that the psychologist make reasonable efforts to maintain the integrity and security of tests and other assessment techniques (p. 1604). Principles for Professional Ethics The National Association of School Psychologists (NASP) represents approximately 22,000 master s level, specialist, and doctoral-level school psychologists from the United States, Canada, and other countries. The NASP also has a set of Principles for Professional Ethics (NASP, 2000). Several sections of the NASP principles may apply to the use of testing technicians. For example, Section C notes that school psychologists do not condone the use of psychological or educational techniques, or the misuse of the information these techniques provide, by unqualified persons in any way, including teaching, sponsorship, or supervision (p. 626). In Section E ( Use of Materials and Technology 1 ) of the NASP principles, school psychologists are required to maintain test security, preventing the release of underlying principles and specific content that would undermine the use of the device (p. 30). School psychologists are also responsible for the security requirements specific to each instrument used (p. 626). In 1992, the NASP Committee on Ethical and Professional Standards was asked to consider the use of testing technicians by licensed psychologists in light of the Principles for Professional Ethics. The committee responded to the request by stating that the administration and scoring of psychological assessment devices requires rigorous education, preferably within a NASP- or APA-approved graduate programs or, at the very least, through a graduate program at an accredited university. Furthermore, the committee noted the need for instruction in the following areas: (a) test development and standardization; (b) statistics and measurement theory and practice; (c) selection, administration, scoring, and interpretation of a wide range of assessment devices; (d) extensive supervision including direct observation; and (e) education in factors that affect test performance and interpretation

8 364 HALL, HOWERTON, BOLIN (e.g., cultural bias, sensory impairments, specific learning disabilities; J. A. Reinhardt, personal communication, February 29, 1992). The committee concluded that the use of psychological assessment instruments by individuals who do not meet such rigorous standards is unethical professional conduct. Furthermore, the committee noted that the interpretation of assessment results involves more than the sum of scores. The committee stated that it is the responsibility of the practitioner to be part of the interaction from which the scores are derived and to be able to interpret performance and develop conclusions and recommendations. Given the nature of the decisions that are frequently based on the results of psychological assessment, the committee felt that the risk of harm through using unqualified examiners is too great (J. A. Reinhardt, personal communication, February 29, 1992). LEGAL ISSUES SURROUNDING PROPER TEST USE Some state and provincial licensing boards have gone further than issuing professional/ethical guidelines by forbidding licensed psychologists from using technicians for many testing purposes (DeLuca, 1989; DeLuca & Putnam, 1993; Hall & Slate, 1992). For example, in the United States, the ABEP (2001) recently added Section 7.6.B, entitled Unlicensed, Non-Exempt Individuals, to their Rules and Regulations 2001 Revisions. This addition significantly restricted the use of testing technicians by licensed psychologists. According to the ABEP, Psychologist and Psychological Examiners may employ unlicensed, non-exempt individuals only to perform services which do not constitute the practice of psychology (p. 28). These activities must be described in the practitioner s documents, including reports and statements. Technicians are limited to the following duties: (a) secretarial and clerical assignments, (b) nonpsychometric data gathering (e.g., personal histories), and (c) administering, proctoring, or scoring nonprojective assessment instruments that include self-report inventories that do not require psychological education. This ABEP rule or regulation has been the subject of much recent controversy and legislative debate in Arkansas. Proponents for the use of testing technicians in Arkansas have expressed considerable objection to the new rule. Consequently, the regulation was placed on hold until the completion of an interim study on the use of testing technicians by a state legislative committee. The members of the interim study consisted of licensed psychologists, licensed psychological examiners, and academic psychologists with opposing views on the use of testing technicians. The outcome of the study resulted in an agreement on the restricted use of testing technicians by qualified licensed psychologists. The agreement was based in part on the Recommendations for Education and Training of Non-Doctoral Personnel in Clinical Neuropsychology (Report of the Division 40 Task Force, 1991) and the revised ASPPB Supervision Guidelines (ASPPB, 2003b). The state licensing law

9 TESTING TECHNICIANS 365 was most recently amended to allow for the regulated use of testing technicians by qualified licensed psychologists. APA s Model Act for State Licensure of Psychologists The Model Act for State Licensure of Psychologists (APA, 1987) functions as a prototype for the drafting of state and provincial legislation specific to the practice of psychology. In effect, each local law reflects compromises and changes particular to that area. The Model Act is intended to serve as a guide in the drafting process of state practice laws. According to the Model Act, the practice of psychology includes, but is not limited to, psychological testing and the evaluation or assessment of personality characteristics, such as intelligence, personality, abilities, interests, aptitudes, and neuropsychological functioning (p. 697). The document goes on to state that assistants who are not eligible for licensure in some localities are permitted to function under the supervision of a licensed psychologist. The supervising psychologist is responsible for the actions of the assistants. The Model Act strongly encourages local boards of psychology to adopt regulations defining the nature and extent of training for qualified assistants. Furthermore, local boards are encouraged to limit the number of assistants that a psychologist may employ and the conditions for supervision. The Model Act places emphasis on each local board s rules and regulations by noting that a psychologist or anyone under his or her supervision shall conduct professional activities in conformity with ethics and professional standards espoused by the local psychology board through its documents. ASPPB Model Act for Licensure of Psychologists The Model Act for Licensure of Psychologists endorsed by the ASPPB represents a consensus of opinion from U.S. and Canadian psychology licensure boards (ASPPB, 2001). This Model Act, like the APA Model Act, defines the practice of psychology as including but not limited to psychological testing and the evaluation or assessment of personal characteristics, such as intelligence, personality, abilities, interests, aptitudes, and neuropsychological functioning (pp. 2 3). The document also allows for the use of uncredentialed personnel or psychological assistants under the supervision of a licensed psychologist in accordance with the rules and regulations espoused by the state or provincial psychology board. The Model Act calls for each local psychology board to establish in its rules and regulations the qualifications for psychological assistants, the number of assistants that a licensed psychologist may employ, and the conditions under which they will be supervised.

10 366 HALL, HOWERTON, BOLIN ASPPB Model Regulations The Model Regulations of the ASPPB (2002) call for boards to adopt the Guidelines for Supervision of Uncredentialed Personnel Providing Psychological Services. These guidelines are set forth in the revised ASPPB Supervision Guidelines (ASPPB, 2003b). The guidelines specify the (a) setting of the supervised experience (e.g., the roles of the supervisee must be specified in writing, the setting must have a licensed psychologist, the status of the supervisee must be articulated to clients and third-party payers, and work assignments must be commensurate with the skills of the uncredentialed person), (b) qualification of the supervisors (e.g., the supervising psychologist shall be licensed and abide by the appropriate ethical principles and laws; supervisors will possess adequate training and knowledge and the skills necessary to monitor the work of the supervisee, and the supervisor or a qualified designee will be available to the supervisee and clients at all times for emergency consultation and intervention), (c) nature and duration of the supervisory contact (e.g., the supervisor should not be an employee of the supervisee, the supervisory process should address legal, ethical, social, and cultural dimensions), (d) written and oral evaluation (e.g., written and oral evaluations based on the job descriptions of the supervisee are necessary, feedback should be ongoing with written evaluations occurring on a quarterly basis), and (e) regulation and enforcement issues (e.g., both supervisors and supervisees must be registered with the board, important supervisee information includes name, nature of services, academic training and experience, and nature of the supervision). U.S. Special Education Law The U.S. federal law that regulates special education services in the United States is known as the Individuals with Disabilities Education Act Amendments of 1997 (IDEA 97). The IDEA 97 states that any standardized test used with a child must be administered by trained and knowledgeable personnel and in accordance with any instructions provided by the developer or publisher. State and provincial special education laws may also specify that some individual testing can only be conducted by licensed professionals. Local special education supervisors or directors are often charged with the task of ensuring that individual tests are administered by qualified examiners. In these situations, findings obtained from an uncredentialed testing technician would be unacceptable. Such conditions call attention to the need for clearly documenting the credentials of the person who actually conducted a test. This documentation is necessary to determine if the examiner s qualifications are consistent with existing special education rules and regulations.

11 TESTING TECHNICIANS 367 Education Requirements for State Licensure According to the ASPPB (2003a), 34 of the 53 jurisdictions in the United States and Canada that are members have provisions for master s level psychology licensure. In the United States, 28 states allow for licensure at this level. These individuals, like testing technicians, have been referred to by a variety of titles such as psychological examiner, psychological associate, psychological technician, registered or licensed psychological assistant, certified or licensed psychological associate, psychological associate, registered master s psychologist, limited license psychologist, licensed psychologist, and school psychologist. Historically, the major role and function of the master s level psychology practitioner has been assessment and testing. Unlike most bachelor s level testing technicians, these professionals are often licensed by the local psychology board and/or department of education and are bound to a major ethical code and the law. Many of these practitioners may feel that the use of testing technicians by licensed psychologists is encroachment on their professional role and function. TECHNICAL ISSUES AND TEST SCORE VALIDITY Test Security Many psychological tests are classified as proprietary tests, meaning that they are owned by a test developer or publisher. The items on such tests are usually restricted. These instruments are restricted in an attempt to prevent the public from becoming harmedasaresult ofimproperuse.forexample,damagecouldresult toanexaminee if a test was used inappropriately and the results were then called on to assist in making a classification or eligibility decision (Kaplan & Saccuzzo, 2001). Proprietary tests are also controlled in an attempt to maintain test security. Preventing leakage of test content is critical in terms of ensuring the validity of an instrument (APA, 1999). Unfortunately, some major developers and publishers of psychological measures have sold tests to unqualified individuals (Lowman, 1996). A licensed psychologist who uses the services of a testing technician may run the risk of jeopardizing test security. Educating an uncredentialed person in the need to maintain test content is imperative. In fact, keeping test materials secure is regarded as one of the major minimum competencies associated with proper test use (Moreland et al., 1995). Because testing technicians are not typically licensed, they are not professionally or legally bound to the previously discussed ethical codes. Hall and Slate (1992) highlighted the paradox inherent in using testing technicians to administer restricted tests. They note that the testing technicians are charged with administering and protecting the security of a test that they are not qualified to purchase on their own.

12 368 HALL, HOWERTON, BOLIN Informed Consent The licensed psychologist must make a good faith effort to disclose enough information for the client or legal guardian to allow them to make an informed choice (Bersoff & Hofer, 1990). Informed consent is often framed within the context of treatment or intervention. However, it also applies to assessment and testing practices. Licensed psychologists who use testing technicians are required to inform clients that an uncredentialed person will be administering selected instruments. In this situation, it would reasonable for the licensed psychologist to discuss with the client or legal guardian the testing technician s qualifications, role, and function. This type of information should allow the individual to make an informed choice regarding the recommended psychological services. Scoring Errors Avoiding errors in scoring and recording is also regarded as one of the minimum competencies for proper test use (Moreland et al., 1995). Much of the research related to scoring errors on individually administered psychological tests has focused on the Wechsler scales. These findings have revealed that the Vocabulary, Comprehension, and Similarities subtests appear to be especially difficult for in-training graduate student examiners to interpret and score. According to Slate and Chick (1989), incorrect point assignment on these three subtests was a frequent error as was inappropriate questioning. Another common error was a failure to record the examinee s response in a verbatim manner. These researchers noted that errors on two thirds of test protocols that were administered by in-training graduate students resulted in changes in the Full Scale IQ. These types of errors decrease the reliability and validity of the obtained scores and could result in incorrect eligibility, classification, and placement decisions, which could lead to detrimental outcomes for examinees (Hall & Slate, 1992). Unfortunately, few test developers report a measure of examiner error in the form of scorer reliability (Klassen & Kishor, 1994; Ryan, Prifitera, & Powers, 1983; Slate & Jones, 1988). In related research, Slate and Hunnicut (1988) and Whitten, Slate, Jones, Shine, and Raggio (1994) cited inadequate training and instructional design as the major explanation for examiner error. These authors identified a number of variables that may contribute to the problem and identified a number of the key instructional components necessary to adequately prepare graduate students in the administration of intelligence tests. On the basis of their findings, it appears that a quality graduate education in psychological measurement and individual intelligence assessment is essential in preparing examiners to administer, score, and interpret intelligence tests in an appropriate manner. It is important to note that the education of a master s level psychological examiner generally occurs at the graduate-level over a period of several years. No pub-

13 TESTING TECHNICIANS 369 lished evidence could be located to support the notion that persons without this type of education can administer and score these highly complex instruments in a competent manner. As a testimony to this complexity, the Administration Checklist for the Wechsler Intelligence Scale for Children Third Edition and the Wechsler Adult Intelligence Scale Third Edition (Sattler, 2001) lists at least 200 critical behaviors or skills needed to successfully administer and score the instruments. Additionally, some researchers have noted that individual intelligence test data collected by nonpsychologists may be of questionable reliability and validity (e.g., Stone & Reynolds, 2003). Standardization Procedures Another problem associated with the use of testing technicians may rest with the standardization procedures used to develop a particular test. For example, if a specific test was standardized by the deployment of test administrators who were graduate-level prepared, an examination administered by an individual who did not possess this type of educational background may have been a deviation from standardized test administration procedures. Nonadherence or deviation from this condition could theoretically lower the reliability and validity of the test results. Consistent with generalizability theory (Cronbach, 1984), the accuracy of a test not only resides within the instrument but also hinges on the circumstances of its use (Gregory, 2000). Test developers and publishers must carefully consider and describe the minimum examiner qualifications necessary for test administration. These qualifications should mirror those required in the standardization process. For example, the technical manual for the Stanford Binet Intelligence Scale Fourth Edition states that standardization examiners have to (a) have experience in administering the Stanford Binet or Wechsler scales and (b) be certified by the state or other appropriate agency as a school or clinical psychologist or the equivalent, teach graduate courses in individual assessment, or be a graduate student working under the direct supervision of an instructor with appropriate qualifications (Thorndike, Hagen, & Sattler, 1986). The Kauffman Assessment Battery for Children (K ABC) Administration and Scoring Manual (Kauffman & Kauffman, 1983) clearly describes the qualifications of the examiner. The examiner Is expected to have a good understanding of theory and research in areas such as child development, tests and measurement, cognitive psychology, educational psychology, and neuropsychology, as well as supervised experience in clinical observation of behavior and formal graduate level training in individual intellectual assessment. (p. 4) The Wechsler Intelligence Scale for Children Third Edition (WISC III) Manual (Wechsler, 1991) and the Wechsler Adult Intelligence Scale Second Edition

14 370 HALL, HOWERTON, BOLIN (WAIS II) Manual (Wechsler, 1997) state that in a majority of cases, examiners who use the instruments will have formal graduate or professional training in psychological assessment. However, they note that a trained technician can administer and score the test under close supervision. This practice seems inconsistent with the fact that a majority of the examiners who standardized the instrument were certified or licensed professionals. The degree of error introduced by this practice is unknown. Building Rapport With Examinees The need to build and maintain rapport during the administration of a psychological test is regarded as a critical examiner skill and one of the minimum competencies associated with proper test use (Moreland et al., 1995; Sattler, 2001). This skill has been shown to be important with meta-analytical procedures examining the effects of rapport on individual test performance (Fuchs & Fuchs, 1986). The quality of rapport may increase or decrease test performance by several standardized test score points. A difference of several points in an overall score may significantly affect professional decisions involving the examinee including those involving classification, eligibility, and placement. These decisions may, in turn, have pronounced effects on individuals lives. The importance of building and maintaining rapport during individual testing is typically taught and evaluated within the context of graduate-level instruction in psychological and educational assessment. It is unclear whether testing technicians are knowledgeable of this important aspect of test administration or whether they possess the skills necessary for establishing and facilitating a relation and the environment needed to ensure the examinee s optimal performance. Observation Skills Some experts in assessment have stressed the importance of looking beyond test scores. These authors have noted that an examiner should employtheir observational skills to assist in test interpretation (e.g., Kaufman, 1994; Mitrushina, Boone, & D Elia, 1999). Indeed, one of the main advantages of individually administered psychological and educational tests is the opportunity to carefully observe the performance of individuals under standardized conditions. These observations may allow skilled examiners the opportunity to examine an individual s problem-solving processes and their strengths and weaknesses(aera, APA,& NCME, 1999). Observations may also allow the examiner to determine if the derived scores are accurate estimates of the individual s ability (Kaufman, 1994). Kauffman and Kauffman (1983) placed a high value on the observation of behaviors during testing and believed the tests should be interpreted by the examiner. These authors listed key behaviors to observe during testing involving children, which included: (a) attention to tasks, (b) frustration tolerance, (c) problem-solving approaches, (d) fine and gross motor

15 TESTING TECHNICIANS 371 skills, (e) activity level, (f) speech and language, (g) effort or motivation, (h) reflectivity, (i) self-efficacy and self-concept, (j) attitude, and (k) response to praise. The skills needed to observe and understand these types of behaviors are typically not acquired outside of a graduate program of study. CONCLUSIONS AND RECOMMENDATIONS On the basis of our review of the testing literature, professional guidelines, and legal requirements, the argument against the use of testing technicians seems to be very strong. Although a well-trained testing technician may be able to administer psychological tests, the rigorous training requirements that are necessary to ensure minimally competent technicians appear to be unrealistically burdensome. Providing training to testing technicians for a year or more before allowing them to administer assessments would be costly and, in effect, delay any time and cost savings, which are a cited primary advantage associated with technician use. However, if testing technicians are deployed without intensive training, the quality of the information gathered during testing will likely be poor. Because findings obtained from psychological tests are often used to make critical decisions that may have a long-term impact on peoples lives, using testing technicians appears to be ethically questionable. Furthermore, the legal liability created by a reliance on unlicensed technicians may further offset the cost benefits. State and Provincial Boards of Psychology In the United States, the final verdict on the use of testing technicians by licensed psychologists rests with the local psychology boards because control and monitoring emerges from the state legal power of these entities. These boards define practice and outline or delineate offenses and sanctions (Freeman, 2000). State and provincial psychology boards that are contemplating the testing technician issue should examine all of the relevant documents and information discussed in this article. To safeguard the profession and protect the consumer, state credentialing boards that elect to allow the use of testing technicians should consider adopting specific rules based on The Recommendations for Education and Training of Non-Doctoral Personnel in Clinical Neuropsychology (Report of the Division 40 Task Force, 1991) and the Guidelines for Supervision of Uncredentialed Personnel Providing Psychological Services, which appear in the revised ASPPB Supervision Guidelines (ASPPB, 2003b). Test Developers and Publishers According to the Standards for Educational and Psychological Testing, test documents should specify qualifications that are required to administer a test and to in-

16 372 HALL, HOWERTON, BOLIN terpret the test scores accurately (AERA, APA, & NCME, 1999, p. 69). The standards go on to note that statements of user qualifications need to specify the training, certification, competencies, or experience needed to have access to the test (AERA, APA, & NCME, 1999, p. 69). Few, if any, individual psychological or educational tests fully meet this expectation. It seems reasonable to expect examiners at a minimum to be at the level of expertise and education of those who initially administered the test during standardization before they are allowed to purchase, administer, or score the instrument. We believe that test developers and publishers must address this issue in greater detail. Furthermore, we also concur with Klassen and Kishor (1994) that test manuals should report the scorer reliability of tests that are prone to examiner errors. Licensed Psychologists Licensed psychologists are ultimately responsible for using tests correctly and ensuring that others do the same (APA, 1992; Swenson, 1997). Practitioners need to consider the ethical and legal issues associated with the use of testing technicians. This will require one to consult existing ethical and legal guidelines. Licensed psychologists must consider the broad ethical principles and the specific legal mandates (Kitchener, 1986). A major question is whether the use of testing technicians is consistent with local psychology laws. Negative personal and professional consequences for psychologists who inappropriately use testing technicians may be significant. These consequences could include attorney and hearing fees, fines, damage to professional reputation, loss of license, expulsion from professional organizations, difficulties associated with applications for managed care companies, and difficulties associated with renewal of malpractice insurance (Welch, 2001). The rights and welfare of all affected parties also needs to be considered. Licensed psychologists must seriously consider whether the use of testing technicians poses a hazard to others (Wiens & Dorken, 1986). Finally, psychologists should contemplate whether the use of uncredentialed persons to administer and score restricted psychological tests is best for the profession. REFERENCES American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. American Psychological Association. (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin, 51, American Psychological Association. (1981a). Specialty guidelines for the delivery of services by clinical psychologists. American Psychologist, 36, American Psychological Association. (1981b). Specialty guidelines for the delivery of services by counseling psychologists. American Psychologist, 36,

17 TESTING TECHNICIANS 373 American Psychological Association. (1981c). Specialty guidelines for the delivery of services by industrial/organizational psychologists. American Psychologist, 36, American Psychological Association. (1981d). Specialty guidelines for the delivery of services by school psychologists. American Psychologist, 36, American Psychological Association. (1987). Model Act for State Licensure of Psychologists. American Psychologist, 42, American Psychological Association. (1988). Test user qualifications: A data-based approach to promoting good test use. Washington, DC: Author. American Psychological Association. (1992). Ethical principles of psychologists and code of conduct. American Psychologist, 47, American Psychological Association. (1999). Test security: Protecting the integrity of tests. American Psychologist, 54, Anastasi, A., & Urbina, A. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall. Arkansas Board of Examiners in Psychology. (2000). Use of psychological evaluation extenders/technicians. Little Rock, AR: Author. Arkansas Board of Examiners in Psychology. (2001). Rules and regulations 2001 revisions. Little Rock, AR: Author. Association of State and Provincial Psychology Boards. (2001). Model Act for Licensure of Psychologists. Montgomery, AL: Author. Association of State and Provincial Psychology Boards. (2002). Model regulations. Montgomery, AL: Author. Association of State and Provincial Psychology Boards. (2003a, February). Master s level licensure update. Manual presented at the midwinter meeting of the Association of State and Provincial Psychology Boards, San Antonio, TX. Association of State and Provincial Psychology Boards. (2003b). Supervision guidelines. Retrieved April 13, 2005, from Bersoff, D. N., & Hofer, P. T. (1990). The legal regulation of school psychology. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (2nd ed., pp ). New York: Wiley. Camara, W. J. (1997). Use and consequences of assessments in the USA: Professional, ethical, and legal issues. European Journal of Psychological Assessment, 13(2), Camara, W. J., Nathan, J. S, & Puente, A. E. (2000). Psychological test usage: Implications in professional practice. Professional Psychology: Research and Practice, 31(2), Cohen, L. D. (1974). Overview: Past, present, future. Professional Psychology, 5, Cronbach, L. J. (1984). Essentials of psychological testing (4th ed.). New York: Harper & Row. Davidson, G. (1997). The ethical use of psychological tests: Australia. European Journal of Psychological Assessment, 13(2), DeLuca, J. W. (1989). Neuropsychology technicians in clinical practice: Precedents, rationale, and current deployment. The Clinical Neuropsychologist, 3(1), DeLuca, J. W., & Putnam, S. H. (1993). The professional/technician model in clinical neuropsychology: Deployment characteristics and practice issues. Professional Psychology: Research and Practice, 24(1), Evers, A. (1996). Regulations concerning test qualifications and test use in the Netherlands. European Journal of Psychological Assessment, 12(2), Freeman, S. J. (2000). Ethics: An introduction to philosophy and practice. Belmont, CA: Wadsworth/Thomson Learning. Fremer, J. (1996). Promoting high standards for test use: Developments in the United States. European Journal of Psychological Assessment, 12(2), Fuchs, D., & Fuchs, L. S. (1986). Test procedure bias: A meta-analysis of examiner familiarity effects. Review of Educational Research, 56,

18 374 HALL, HOWERTON, BOLIN Gentry, W. D. (1974). Three models of training and utilization. Professional Psychology, 5(2), Gregory, R. J. (2000). Psychological testing: History, principles, and applications (3rd ed.). Needham Heights, MA: Allyn & Bacon. Guilmette, T. J., Faust, D., Hart, K., & Arkes, H. R. (1990). A national survey of psychologists who offer neuropsychological services. Archives of Clinical Neuropsychology, 5, Hall, J. D., & Slate, J. R. (1992, November). Administering and scoring individual intelligence tests: Who is qualified? National Association of School Psychologists Communiqué, 26, 30. Hartlage, L. C., & Telzrow, C. F. (1980). The practice of clinical neuropsychology in the U.S. Clinical Neuropsychology, 2(4), Individuals With Disabilities Education Act Amendments of 1997, 20 U.S.C.A et seq. (West 1997). International Test Commission. (2000). International guidelines for test use. Retrieved April 12, 2005, from International Test Commission. (2001). International Test Commission guidelines. Retrieved April 12, 2005, from Jarvis, P. E., & Barth, J. T. (1994). The Halstead Reitan Neuropsychological Battery: A guide to interpretation and clinical applications. Odessa, FL: Psychological Assessment Resources. Joint Committee on Testing Practices. (1988a). The code of fair testing practices in educaton. Washington, DC: American Psychological Association. Joint Committee on Testing Practices. (1988b). Test user qualifications: A data-based approach to promoting good test use. Washington, DC: American Psychological Association. Joint Committee on Testing Practices. (1993). Responsible test use: Case studies for assessing human behavior. Washington, DC: American Psychological Association. Joint Committee on Testing Practices. (1995). The ABC s of school testing. Washington, DC: American Psychological Association. Kaplan, R. M., & Saccuzzo, D. P. (2001). Psychological testing: Principles, applications, and issues (5th ed.). Belmont, CA: Wadsworth. Kaufman, A. S. (1994). Intelligent testing with the WISC III. New York: Wiley. Kauffman, A. S., & Kauffman, N. L. (1983). Kauffman Assessment Battery for Children (K ABC) administration and scoring manual. Circle Pines, MN: American Guidance Service. Kitchener, K. S. (1986). Teaching applied ethics in counselor education: An integration of psychological processes and philosophical analysis. Journal of Counseling and Development, 64, Klassen, R. M., & Kishor, N. (1994). A comparative analysis of practitioners errors on WISC R and WISC III. Canadian Journal of School Psychology, 12(1), L abate, L. (1973). The laboratory method in clinical child psychology: Three applications. Journal of Clinical Child Psychology, 2, Lowman, R. L. (1996). Introduction to the special section on what every psychologist should know about assessment. Psychological Assessment, 8, Merenda, P. F. (1997). Update on the battle for psychological testing rights, Psychological Reports, 80, Mitrushina, M. N., Boone, K. B., & D Elia, L. F. (1999). Handbook of normative data for neuropsychological assessment. New York: Oxford University Press. Moreland, K. L., Eyde, L. D., Robertson, G. J., Primoff, E. S., & Most, R. B. (1995). Assessment of test user qualifications: A research-based measurement procedure. American Psychologist, 50, Musante, G. J. (1974). Staff evaluations of the technician role. Professional Psychology, 5, National Academy of Neuropsychology. (2003). Random discharges from the editors: A bad moon arisin? National Academy of Neuropsychology Bulletin, 18(1), 2 3. National Association of School Psychologists. (2000). Principles for professional ethics. Bethesda, MD: Author.

19 TESTING TECHNICIANS 375 Report of the Division 40 Task Force on Education, Accreditation, and Credentialing. (1989). Guidelines regarding the use of non-doctoral personnel in clinical neuropsychological assessment. The Clinical Neuropsychologist, 3(1), Report of the Division 40 Task Force on Education, Accreditation, and Credentialing. (1991). Recommendations for education and training of nondoctoral personnel in clinical neuropsychology. The Clinical Neuropsychologist, 5(1), Ryan, J. J., Prifitera, A., & Powers, L. (1983). Scoring reliability on the WAIS R. Journal of Consulting and Clinical Psychology, 51, Sattler, J. M. (2001). Assessment of children: Cognitive applications (4th ed.). San Diego, CA: Author. Sbordone, R. J. (1996). Ecological validity: Some critical issues for the neuropsychologist testing. In R. J. Sbordone & C. L. Long (Eds.), Ecological validity of neuropsychological testing (pp ). Delray Beach, FL: GR/St. Lucie. Serenty, M. L., Dean, R. S., Gray, J. W., & Hartlage, L. C. (1986). The practice of clinical neuropsychology in the United States. Archives of Clinical Neuropsychology, 1, Slate, J. R., & Chick, D. (1989). WISC R examiner errors: Cause for concern. Psychology in the Schools, 26, Slate, J. R., & Hunnicut, L. C. (1988). Examiner errors on the Wechsler scales. Journal of Psychoeducational Assessment, 6, Slate, J. R., & Jones, C. H. (1988). Strategies to reduce examiner errors on the Wechsler scales. Social and Behavioral Sciences Documents: Abstracts and Best Seller List, 18(1), 1 2. Sloop, W. E., & Quarrick, E. (1974). Technician performance: Reliability and validity. Professional Psychology, 5(2), Stone, B. M., & Reynolds, C. R. (2003). Can the National Health and Nutritional Examination Survey III (NHANES III) data help resolve the controversy over low blood lead levels and neuropsychological development in children? Archives of Clinical Neuropsychology, 613, Sweet, J. J., & Moberg, P. J. (1990). A survey of practices and beliefs among ABPP and non-abpp clinical neuropsychologists. The Clinical Neuropsychologist, 4(2), Swenson, L. E. (1997). Psychology and law (2nd ed.). Pacific Grove, CA: Brooks/Cole. Test Taker Rights and Responsibilities Working Group of the Joint Committee on Testing Practices. (1999). Rights and responsibilities of test takers: Guidelines and expectations. Washington, DC: American Psychological Association. Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986). The Stanford Binet Intelligence Scale: Fourth edition technical manual. Chicago: Riverside. Turner, S. M., DeMers, S. T., Fox, H. R., & Reed, G. M. (2001). APA s guidelines for test user qualifications: An executive summary. American Psychologist, 56, Walsh, B. W., & Betz, N. E. (1995). Tests and assessment (3rd ed.). Englewood Cliffs, NJ: Prentice Hall. Watkins, C. E., & Campbell, V. L. (1990). Testing and assessment in counseling psychology: Contemporary developments and issues. The Counseling Psychologist, 18(2), Wechsler, D. (1991) Wechsler Intelligence Scale for Children Third Edition (WISC III) manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1997). Wechsler Adult Intelligence Scale Second Edition (WAIS II) manual. San Antonio, TX: The Psychological Corporation. Welch, B. L. (2001). Caution: State licensing board ahead. Insight: Safeguarding psychologists against liability risks. Amityville, NY: American Professional Agency. Whitten, J., Slate, J. R., Jones, C. H., Shine, A. E., & Raggio, D. (1994). Examiner errors in administering and scoring the WPPSI R. Journal of Psychoeducational Assessment, 12(1), Wiens, A. N., & Dorken, H. (1986). Establishing and enforcing standards to assure professional competency. In H. Dorken and Associates, Professional psychology in transition (pp ). San Francisco: Jossey-Bass. Wilson, S. (1982). Peer review in California: Summary findings in 40 cases. Professional Psychology, 13,