EpicRehab, LLC To re c ogniz e a nd de v e lop the v a l u e in e a c h of us.

Transcription

1 EpicRehab, LLC To re c ogniz e a nd de v e lop the v a l u e in e a c h of us. ABSTRACT Title: Situational Assessment as a Measure of Work-Related Executive Function Principal Investigator: Leonard N. Matheson, PhD Research Questions: 1. Is the POD 1a situational assessment a valid measure of executive function? 2. Does performance on the POD 1a predict return to work? Background of the Study After a person experiences a head injury or a stroke, one of the most difficult challenges is to return to work in a job that requires executive function. Executive function refers to the ability to control and integrate cognitive processes. Executive functions include planning, organization, mental flexibility, working memory, abstract thinking, and other abilities that orchestrate the cognitive processes necessary for problem-solving and to initiate appropriate actions in novel and/or challenging tasks. In brain injury rehabilitation, neuropsychological tests measure some of the executive functions, but their adequacy is not without challenge: A major obstacle to examining the executive functions is the paradoxical need to structure a situation in which clients can show whether and how well they can make structure for themselves most cognitive tests allow the subject little room for discretionary behavior, including many tests thought to be sensitive to executive - or frontal lobe - disorders. (Lezak, Howieson et al. 2004) (pp ) The necessary structure of the neuropsychological examination negates the opportunity to examine most of the executive functions. In the normal course of most neurological or neuropsychological examinations such functions are carried out by the examiner or made unnecessary (or even unwelcome) by the structured nature of the test material and a restricted number of possible responses in most tests of cognitive functions. Thus, these functions typically remain unexamined, although they are absolutely essential to the maintenance of social independence in a complex society. (ibid. p 621) A necessary aspect of a test of executive function is the withholding of structure in ways that allow the client to demonstrate these abilities. Because most neuropsychological tests are highly structured and focus on individual constructs, they do not provide adequate opportunities to use executive functions, which in turn limits the neuropsychologist s ability to observe the presence of executive dysfunction. 188 Woodlands Place Court Saint Charles, MO Phone Fax len@epicrehab.com

2 Page 2 Situational assessments may provide a reasonable alternative to the neuropsychological measurement of executive function. Three that have been developed for neurobehavioral rehabilitation include the Tinker Toy Test (TTT) (Lezak 1982), the Behavioral Assessment of Vocational Skills (BAVS) (Butler, Anderson et al. 1989), and the Complex Task Performance Assessment (CTPA) (Wolf, Morrison et al. 2008). The TTT is a situational assessment that was developed to measure the set of executive functions having to do with purposive action. In the TTT, 50 parts are provided and the client is told, Make whatever you want with these. You will have at least five minutes and as much more time as you wish to make something. In an initial study, the number of parts used and the complexity of the constructed object differentiated between subjects with cerebral pathology at two levels of severity, compared with subjects who were healthy. Research has identified executive functions that contribute to performance in this task, including the ability to form a goal and plan, initiate, and follow through with a complex activity to achieve the goal. Research also demonstrated the ability of the test to discriminate between employed and unemployed persons with brain injury. Employment status was moderately correlated with test performance (r =.44), which in this study was a better predictor than all of the neuropsychological tests aside from the Trail Making Test-B (Bayless, Varney et al. 1989). The BAVS is a situational assessment that encompasses executive function as well as manual dexterity and coordination. Clients are provided with wheelbarrow parts and told, This is a test of your ability to work. To make this as real as possible, imagine that you are working and being paid and that I am your boss. Think out loud as you are assembling the wheelbarrow. After 15 minutes the client is distracted by an alternate activity for 3 to 5 minutes before returning to the assembly task. 15 minutes later, the evaluator interjects a criticism that is confrontational but not hostile. A 16-item behavior rating scale provides a score to reflect impairment in performance. In pilot validity testing to predict return to work, the BAVS score added unique variance. Comparison between BAVS performance and ratings in subsequent work trials was strong. The CTPA is a clerical-work situational assessment based on the Structured Work Activity Group #1 (Matheson, Seaton et al. 2003). These researchers found that the CTPA significantly differentiated between persons with mild stroke and matched controls in terms of task monitoring and performance efficiency, even though there were no significant differences between the groups in terms of performance on a neuropsychological test battery designed to measure executive dysfunction. POD 1 is based on the BAVS and the CTPA, integrating components of each in a suite of situational assessments to implement progressive occupational demand at the "Goldilocks Chair Company", a virtual reality business that has four employees, three products, telephone messages that have been recorded, and a 600-record customer database. Goldilocks manufactures and sells three chairs assembled out of pre-cut PVC pipe; a Baby Bear Chair, Momma Bear Chair, and

3 Page 3 Poppa Bear Chair. Implementation of this virtual reality business allows therapists to enter the role of employer, placing clients in the role of employees who perform various graded-demand simulated work tasks. POD 1a is the set of 12 situational assessments that involves assembly of each chair while transcribing telephone messages presented on an audio CD-ROM. The therapist selects the level of occupational demand appropriate to the client using the matrix presented in Table 1. Table 1. POD 1 Situational Assessment Task Matrix, Level A through Level L. The therapist selects one of twelve situational assessments based on the client s performance in a structured Baby Bear Chair work sample test that uses the same parts as Task A in Table 1. The difference between this work sample test and the situational assessment Task A is that the former structures the task into two steps beginning with organization of parts, while the latter requires the client to work without this structure. The DCAT Professional Manual (Matheson, Matheson et al. 2010) provides guidelines to assign the client to an appropriate level of situational assessment using the matrix in Table 1. The purpose of this research project is to examine the criterion validity of the POD situational assessment in terms of executive function and to determine its validity in terms of whether or not persons seeking rehabilitation subsequently returned to work or prior activity and the level of work or prior activity to which they returned.

4 Page 4 Methods Design: This study uses a concurrent data collection design, comparing performance on the POD 1a Doll Chair Assembly Task (DCAT) situational assessment with scores from the Behavior Rating Inventory of Executive Function (BRIEF), in addition to a prospective data collection design, comparing both of these scores with two occupational status variables reported by the subject in a brief telephone interview six months later. Subjects: Adult volunteers 18 years to 65 years who are receiving services as outpatients after experiencing an acquired brain injury as the result of a traumatic brain injury, various disease processes such as a brain tumor, or a cerebrovascular stroke. When available, information about location and type of brain impairment will be recorded, as will information about whether or not executive dysfunction has been identified, and the method of identification. This will be a convenience sample of 75 males and 75 females, selected as they arrive at the point in their treatment that it is appropriate to administer the DCAT. Each volunteer will routinely receive the DCAT as part of his or her normal rehabilitation program. Each volunteer will be screened by an occupational therapist or vocational evaluator as capable of safely performing the DCAT situational assessment. Procedures for Data Collection: A quiet room at the rehabilitation facility will be used for testing, which will involve only the evaluator and the subject. Before participation, an Informed Consent document that follows established guidelines for persons with a cognitive impairment will be presented by the staff member to the subject and the subject s surrogate or caregiver. It will be thoroughly reviewed with both persons. The signature of the subject will be sought for the original consent document, while the surrogate or caregiver will be asked to sign the duplicate. Once the Informed Consent document is signed, these tests will be administered: Day One 1. Behavior Rating Inventory of Executive Function Self Report (10 minutes) 2. Behavior Rating Inventory of Executive Function Informant Report (10 minutes) 3. DCAT Baby Bear Work Sample Test (8 minutes to 20 minutes) This is used to identify the level of demand (Table 1 above) of the situational assessment to provide a just-right challenge to the subject. Day Two 4. On a subsequent day when the subject is normally scheduled to receive services, the appropriate level of the DCAT Situational Assessment will be administered (refer to Table 1). (12 minutes to 20 minutes) Six Months Later Data concerning two return-to-work (RTW) variables will be collected during a telephone interview 6 months later, initiated by a rehabilitation facility staff person.

5 Page 5 Electronic Data De-Identification Process: After the data are collected, a staff member will transcribe de-identified data onto an Excel spreadsheet and the spreadsheet to EpicRehab on a monthly basis. The staff member will maintain the original data collection sheets and Informed Consent document in the subject s patient record, under HIPAA guidelines as implemented at the rehabilitation facility. In the Excel spreadsheet, each subject will be assigned a code number that is completely unrelated to the subject s identifying data, so that subject identification will be blinded. Only age, gender, primary diagnosis, date of onset, and test data will be included on the spreadsheet transmitted to EpicRehab. The data will be maintained at EpicRehab on password-protected computers in locked offices. EpicRehab staff and the members of the EpicRehab Institutional Review Board will be the only individuals with access to the data, which will be protected according to State and Federal laws. Data Analyses: Data will be entered into an Excel spreadsheet for auditing and arithmetic manipulation prior to porting the data to SPSS and Winsteps, which will be the primary computer packages used to perform the statistical analyses. Basic descriptive analyses will be conducted, as well as analyses of item response characteristics. The executive function variables will be operationally defined by the standardized T-scores from the 9 sub-scales on the BRIEF, with the Self-Report and Informant-Report taken separately. The situational assessment performance variables will be the DCAT Employability Rating total score and the average score and the time to correct completion of the situational assessment. The first RTW variable, concerning whether or not the subject has returned to work or other prior activities, will be recorded as the subject s response to the question, Have you returned to work or to what you were doing before? This response will be recorded as Yes or No. If the subject answers Yes, a question addressing the second variable will be posed: What level of work or activity are you now doing? The subject will be asked to select one response from the 3-response variable (RTW level) described in Table 2. Table 2. Return-to-Work Levels!"#"$ %"&'()*+),-. /0("+1(-"20+,0340*(),(05,60,(07'+)#)+408)+9,1+0 3,2):)'7+),-; < /0("+1(-"20+,0340*(),(05,60,(07'+)#)+408)+903,2):)'7+),-; = /0("+1(-"20+,070-"805,60,(07'+)#)+4;

6 Page 6 Concurrent validity will be studied by analyses of variance of the DCAT Employability Rating score, using a cut-score of 65 on the T-scores from the 9 sub-scales on the BRIEF. This cut-score is identified in the BRIEF examiner s manual as the score at or above which the scale values are clinically significant. The analyses will treat the Self-Report and Informant-Report versions of the BRIEF separately. Predictive validity will be studied by analyses of variance, using the return to work variables for group-wise comparisons of the DCAT Employability Rating score and the BRIEF standardized T-scores. Risks: The only potential risk associated with the research project is the potential for subject identification and embarrassment. Anonymity for subjects will be protected using the routine data handling procedures and the stated electronic data management. No subject names will be part of the analyses, or will be able to be identified once the analyses are initiated, including publication. Benefits: There are no benefits to the subject. The benefit of this study is that it will allow these tasks to be used with persons who have experienced head injuries or stroke to assist with return to work. Thus, the potential benefits to society are important and outweigh the potential risk to individual subjects. Sponsored By: EpicRehab LLC, 188 Woodlands Place Court, Saint Charles, MO References Bayless, J. D., N. R. Varney and R. J. Roberts (1989). "Tinker toy test performance and vocational outcome in patients with closed-head injuries." J Clin Exp Neuropsychol 11(6): Butler, R., L. Anderson, C. Furst and N. Namerow (1989). "Behavioral assessment in neuropsychological rehabilitation:a method for measuring vocational related skills." Clinical Neuropsychologist 3(3): Lezak, M. (1982). "The problem of assessing executive functions." International Journal of Psychology 17( ). Lezak, M., D. Howieson and D. Loring (2004). Neuropsychological Assessment. Oxford, Oxford University Press. Matheson, L., E. Matheson and J. Matheson (2010). Doll Chair Asembly Task Professional Manual. St. Louis, EpicRehab, LLC: 92. Matheson, L., M. Seaton and V. Kaskutas (2003). Structured Work Activity Groups: A new strategy for situational assessment of persons with brain injury. St. Louis, Washington University School of Medicine. Wolf, T., T. Morrison and L. Matheson (2008). "Initial development of a work-related assessment of dysexecutive syndrome: The Complex Task Performance Assessment." WORK 31(2):