Neuropsychology Research Program: Thomas P. Ross, Ph.D. My primary research program is devoted to the psychometric examination of neuropsychological tests of executive functioning, and other abilities involving the prefrontal cortex. In addition to practical testing and measurement issues, I am interested in theory driven models of executive cognition. Most recently, I am examining the reliability and validity of qualitative scoring systems for measures of verbal and figural fluency. Readers who are interested in becoming a research assistant in my laboratory, completing an independent study in this area, or simply learning more about executive functioning may wish to read the detailed information below. My secondary research interests involve the application of social psychological models of preventative health behavior to predict helmet use among bicyclists. I am currently examining the helmet wearing practices, as well as attitudes and beliefs regarding bicycle-helmet usage among College of Charleston students and other groups. In addition to collecting descriptive information about bicycling and helmet wearing practices, I am comparing the relative utility of three social psychological models of health behavior in explaining helmet use. Such models (e.g., Heath Belief Model, Theory of Planned Behavior, and the Motivation Protection Theory) attempt to explain preventative health behavior as a function of several contributing factors including the perception of personal risk or vulnerability, the perceived severity of consequences associated with not performing the health behavior, and the weighted evaluation of perceived barriers (e.g., effort, cost, and negative social impact) versus perceived benefits (e.g., protection, reduced risk) of performing the health behavior. This project aims to address limitations of previous research by developing a better scale (i.e., questionnaire) with which to examine the aforementioned social psychological models. That is, I plan to develop a measure with better psychometric properties (e.g., internal consistency reliability, content and construct validity) as compared to those used in previous studies. General Orientation to Executive Functioning Project(s) This document outlines the historical background and rationale for a research program aimed at developing theory-driven models for the measurement of neuropsychological abilities, known historically, as executive functions. To make information more understandable to students new to the research program, this document includes explanations of many important concepts. In some instances, new concepts are explained as the reader encounters them. In other instances, they are marked in italics and will be explained later. In addition, some common questions of students are presented and answered for the reader in attempts to make this document user friendly. You my also refer to the document entitled "Research Opportunities in the Neuropsychology Research Program" for general information about laboratory practices and the benefits of acquiring research experience prior to graduate studies. What are "neuropsychological" abilities?
Neuropsychological abilities refer to the diverse perceptual and cognitive capabilities of humans (and other animals, but let s stick to humans for now). Some examples include memory, language, sensation and perception, motor dexterity, praxis, attention, and executive functioning. I have an idea what attention, language and memory are, but what is "executive functioning"? As used by neuropsychologists, executive functions refer to a complex group of skills that allow humans to adapt to novel situations through effective problem solving. The term executive functioning is very non-specific, but economical. It is used to represent the many abilities believed to underlie effective problem solving (e.g., hypothesis generation, planning and execution, utilizing an existing knowledge store, entertaining alternative solutions, controlling for potential sources of interference, utilizing environmental feedback to monitor our progress and to modify our approaches when necessary). They are called "executive" functions because they are believed to play a "supervisory" role by allocating old skills to solve new dilemmas. Therefore, executive functions play an important role in allowing humans to adapt to unique or novel situations. We will discuss this in much greater detail, but for now we can think of executive functions as the cognitive abilities that allow us to problem solve effectively. Why are we calling these skills "neuropsychological" abilities? That is, how are they different from the cognitive abilities I learned about in my other psychology classes (e.g., Physiological and Cognitive)? The abilities or processes themselves are not different at all. With regard to memory, for example, both cognitive psychologists and neuropsychologists generally agree that memory involves the learning and retention of information (among other things). What is different is the way various disciplines in psychology go about studying these processes. Historically, neuropsychologists and cognitive psychologists have employed very different paradigms for studying mental functions. Accordingly, the research methods used by these disciplines have differed markedly over the years, although much less so in recent history. What is the traditional neuropsychological approach? Historically, neuropsychologists have been most concerned with understanding brainbehavior relationships. That is, trying to understand what brain regions are responsible for which cognitive abilities and behaviors. Accordingly, a major focus of neuropsychological research and clinical practice is to develop methods that directly assess the functional integrity of specific brain regions. Traditionally, clinical neuropsychologists have sought to identify brain functions by observing the defects that result when specific brain areas are damaged. To appreciate where the field is at today, it is helpful to keep in mind the context in which neuropsychology evolved. Many early figures in clinical neuropsychology were
primarily neurologists working to help patients with diseases of the central nervous system. Early neuropsychological approaches of study were deeply rooted in behavioral neurology (a.k.a. correlative neuroanatomy, a.k.a. lesion analysis). Accordingly, research efforts on humans were very applied (targeting clinical issues) and mainly correlational. That is, early neuropsychologists systematically observed patients with various brain lesions in attempt to understand what brain structures were most necessary for a given cognitive ability or behavior. After all, one cannot randomly assign people into lesion versus non-lesion groups. Most importantly, pioneering figures like Norm Geshwind, Alexander Luria, and Hans Teuber were not well trained in psychometrics. They developed methods to assess cognitive abilities well before considerations about internal consistency, factorial validity, and construct validation were commonplace (these terms will be discussed later). In order to study the functions of specific brain regions, neuropsychologists observed the disabilities that patient s acquired following trauma and disease to the brain. Through the systematic examination of their patients' deficits, they made inferences about which brain structures were responsible for which cognitive skills. At the time, this paradigm for understanding brain-behavior relationships was followed out of necessity. Prior to the 1950s, neuroimaging procedures like CT-scans and nmri were not available. A major function of early neuropsychological testing was to determine the nature and location of a patient s brain lesion. Accordingly, their approach to test development focused on designing measures that could discriminate between groups of patients with and without certain brain lesions. This approach to test development is called criterion-related validity (also explained later). For now, keep in mind that this approach is very useful for making clinical predictions, however, it imposes limitations on theory development (yes, this too is explained later). An example of the classical neuropsychological approach as related to memory: Neuropsychologists noted that lesions to specific brain areas produce certain types of memory problems. For example, lesions to the hippocampal formation are more likely to disrupt the ability to form new memories, while one s knowledge of events that happened long ago remain relatively preserved. In contrast, patients with lesions to sub-cortical brain structures (e.g., basal ganglia) demonstrate a different type of memory problem. These patients retain the ability to encode and remember new information, as they can accurately recognize words they have just seen or heard. However, these patients have great difficulty recalling or "retrieving" these words freely (e.g., without cues to prompt them). In other words, they seem to have gotten the information in their heads, but can t get it back out without help. Neuropsychologists rely on these types of functional patterns (called dissociations) to guide their views about brain function, and to guide the development of new clinical procedures (i.e., neuropsychological tests). With regard to memory, neuropsychologists design tests that allow them to discern impaired performance due to the inability to learn new information, versus impaired performance due to the inability to retrieve old memories. Making such distinctions allowed clinicians to identify what brain structures
are compromised and the disease processes most likely responsible. We ll talk more about patterns of test results as they relate to brain disease in future research meetings. This is also covered in detail in my neuropsychology course. How does the approach of cognitive psychology differ from neuropsychology? Historically, cognitive psychologists (e.g., Anderson, Atkinson, and Shiffrin) worked to develop models that best explain the cognitive operations that characterize thinking processes. For example, models that can explain the nature of memory, the way it works, and its limitations. Cognitive psychologists examined questions like "what is our capacity to register information at what rate does information decay without rehearsal?" Accordingly, cognitive psychologists develop tests to better understand the nature of what they are studying. They place more emphasis on experimental methods. Namely, they carefully manipulate testing procedures to better assess specific abilities as they relate to a model of operation. Why the historical discussion of traditional neuropsychological versus cognitive approaches and what does this have to do with the executive functioning research program? The implications of the adopting a paradigm are profound. Paradigms influence the way you conduct your research, and manner in which your data is conceptualized (i.e., collected, analyzed, and interpreted). Understanding the history of neuropsychological testing will give you a better appreciation for the current limitations of these instruments. Because of neuropsychology s intimate association with behavioral neurology, and the historical emphasis on diagnosis, the refinement of clinical procedures in light of certain psychometric principles (e.g., construct validity - I ll explain this later) is surprising slow. Consider the above example on memory testing. Early neuropsychologists were not developing memory models per se, but instead designing procedures that could discriminate between pathology to the mesial-temporal lobes, versus, other brain structures involved in memory. Accordingly, they began to conceptualize memory processes and memory measurement based on brain organization and function. As mentioned previously, this approach to test development is an example of criterionrelated validation. Although it is very useful for making clinical predictions (i.e., detecting group differences for diagnosis), it provides limited information about the theory or the nature of such differences (e.g., what is memory really and how does it work?). Setting out to study what functions the mesial-temporal lobes perform is quite different from setting out to understand the nature of memory, as in the cognitive operations underlying human memory processes. I ve used memory as an example here, but we ll see that a similar state of affairs exist for the concept of executive functioning. Do not get the impression that early neuropsychologists weren t interested in developing theories about cognitive operations, they were. The models they formulated were a
function of their understanding of neuroanatomical pathways and the behavioral effects of lesions that interrupt neuroanatomical circuits. A classic example of this type of theory building is the Wernicke-Geschwind model aphasia. If that was then, what s going on now? The historical gap between the "functional neuroanatomy" approach of neuropsychology and the "information-processing" approach of cognitive psychology has narrowed drastically over the years. More recent terms like "cognitive neuropsychology" and "cognitive neuroscience" reflect a more integrated approach that has deeply enriched both disciplines. Cognitive psychology has attempted to integrate their hypothetical models and information-processing diagrams to reflect brain functioning. In turn, neuropsychologists place much less emphasis on localizing cognitive skills to brain areas, and direct more efforts to understanding the nature of cognitive operations that underlie the neuropsychological tools used in clinical practice. So is everything cool now? Nope. Developing good models about cognition is very difficult, and human problem solving is an especially complex behavior. Additionally, the use of terminology like "executive functioning, divided attention, and procedural memory" to describe cognitive abilities implies several assumptions. These words are psychological constructs, and their validity must be demonstrated empirically. What exactly is a "construct" and what does demonstrating its validity involve? Constructs are hypothetical concepts developed by scientists to explain complex phenomena. They cannot be measured directly like height and weight, but rather we make inferences about their meaning based on their ability to organize and predict relationships among variables. Some common examples include intelligence, anxiety, job satisfaction, self-esteem, and introversion. We often take these terms for granted and sometimes we mistakenly reify them, but their validity is dependent on assembling a strong body of empirical evidence. Consider for example, the construct "intelligence." Although psychologists cannot measure intelligence directly, we can estimate levels in persons in order to make farreaching predictions beyond the IQ tests themselves. Based on peoples ability to adapt and reason their way through several tasks (e.g., defining words, solving puzzles) we make inferences about the ability to excel in other areas. So if the construct of intelligence has any scientific value, we better be able to make predictions in a manner consistent with the theory or rationale that guided its development. For example, if we are arguing that intelligence is a capacity to reason efficiently and adapt readily, then people with high levels of this construct should excel academically, occupationally, and financially to a greater extent than people with lower levels (and generally they do). Constructs are therefore the hypothetical building blocks of models.
There is a great deal more to the enterprise of construct validation, and there are several ways to demonstrate the validity of a test (I typically go over them during lab meetings). Furthermore, one must demonstrate that a test can generate reliable or consistent measurements prior to any assertions about its meaning. Psychometrics is an area of study concerned with the systematic investigation of the reliability and validity of psychological tests. What does psychometrics have to do with research on executive functioning? A great deal of our agenda for the executive functioning research program focuses on improving the psychometric properties (reliability and validity) of test procedures. Some of them need considerable refinement before they can be used to test models about specific cognitive abilities. As mentioned, early figures in neuropsychology were neurologists or experimental physiologists that did not know how to subject their tests to psychometric scrutiny. Surprisingly, many tests designed in the 1940s, 50s and 60s to assess executive functioning are still used today in their original form. Even more surprising is that they do not yet demonstrate strong enough psychometric properties to employ them in construct validation efforts. How can this be? Are today s clinical neuropsychologists practicing outdated or inadequate methods? Absolutely not. One important point to keep in mind is that ascertaining the validity of a test (assuming it s reliable) is dependent on what you re using it for. In fact, this is why a test can be very reliable, but totally invalid. This point can be confusing for students so I will use a very concrete example you can relate to. Some of you reading this are probably interested in gaining research experience because you are also considering graduate school, right? Well, what if I used your height and weight (instead of GPA and GRE scores) to decide on whether you gain admission to a graduate program or not? I could use a scale to measure your weight and get the same value each and every time you step on it. In this sense my test is very reliable, but using your weight to determine graduate school admission would be totally invalid. That is, there is no evidence to support its use for this purpose; weight has not been shown to predict success in graduate studies with accuracy. Neuropsychologists use their tests to make determinations about the presence or absence of brain damage. They do so to make treatment recommendations based on the type of cognitive deficits people exhibit, and to assess the degree to which their mental conditions are improving or deteriorating. Most neuropsychological tests are well suited for these clinical considerations. In fact, they were developed precisely for those reasons. With regard to executive measures, there is ample evidence that these procedures can discriminate between brain damaged and non-brain damaged patients with great accuracy. Additionally, executive functioning tests scores can be used to make valid predictions about who may require a legal guardian to help them make legal and financial decisions. Other valid applications of executive test scores include making predictions about safety risks, such as the ability to drive a car.
Well then, what is the field s current dissatisfaction with executive functioning measurement? Remember that pioneering figures in neuropsychology were not seeking to develop a theory about the nature of executive functioning, but rather a model or framework to conceptualize what mental functions were mediated by the activity of the frontal lobes. They ve done a pretty good job at this. That is, neuropsychologists have a pretty good understanding about what type of deficits a patient is likely to exhibit following trauma or disease to various regions of the frontal lobes. But, (am I sounding like a broken record yet) understanding what mental operations underlie performance on these tasks (i.e., what executive functions really are) is another matter entirely. In other words, executive measures lack evidence for their construct validity. At present, our understanding of these measures as representing a cognitive construct is quite limited. A common approach by neuropsychologists thus far has been to review many of the tests considered to be measures of executive functioning, describe each of them, and then attempt to surmise their common elements. For example, based on her review of the literature, Lezak (1983; 1995) contends that the common elements of executive functioning tasks are (1) goal formation, (2) planning, (3) carrying out goal directed plans and (4) monitoring effective performance. Attempts to understand the nature of operations the frontal lobes mediate by looking for common elements among frontal lobe sensitive tests is entirely reasonable. However, engaging in speculation after the fact does not have much explanatory power. For one thing, everyone that performs such a review seems to form a different opinion about what executive tests are measuring. Models generated from these reviews are considered rationally, as opposed to empirically, derived. That is, Lezak (1983) has determined that the executive tasks she reviewed have four common elements, based on her own reasoning. Other peoples reasoning has lead them to propose five components, while some propose only three. How do we solve this dilemma? That is, how do we know who is actually right? For starters, we can take a more objective path to investigating executive functioning, namely, the empirical approach. We can use data to help organize and guide our understanding of executive measures, rather than speculation. There are several statistical techniques available (e.g., factor analysis) to assist with test construction and the development of models. We will discuss these methods in detail during lab meetings. What are some of the specific problems with measures of executive functioning as they exist currently?
There are several current limitations and questions to be answered by psychometric research. As we turn our research efforts towards the nature of executive functioning, we will examine the following issues: First and foremost, how reliable are these measures? Do individual items generally assess the same thing? Are certain items sound, while others are poor? Do test takers receive the same score when it s administered on different occasions? These are questions pertaining to a test s internal consistency and stability or test-retest reliability. Are these measures, too easy or hard? Are the instructions clear enough? In what manner can the administration procedures be modified to produce more reliable results, and to better measure specific aspects of executive abilities? To what extent are these measures confounded by other demands like reading ability, motor speed, visual scanning requirements, etc.? There are many tests that are quite sensitive to frontal lobe lesions and are therefore considered "executive" tests, but are they really assessing the same thing? What are the common elements among executive tasks, and what features are unique to individual measures? With so many terms and labels to describe the demands posed by executive tasks (e.g., planning and initiation, self-regulation, set maintenance, cognitive flexibility, representation memory, interference control) which one is right? Alternatively, can all of these fancy descriptions be accounted for more parsimoniously by a single common underlying mechanism or at least fewer mechanisms than descriptors? Exactly how many components are necessary to have a reasonably complete understanding of this construct? In order words, is executive functioning a unidimensional (primarily one) or multidimensional (composed of several) construct(s)? Can we conceptualize the construct of executive functioning more clearly and organize its components according to a theory? Once we have a way to conceptualize these measures and how they should relate to each other, which ones bests assess the abilities they were intended to measure? Is executive functioning really something unique that deserves its own place in the kingdom of cognitive constructs, or should we simply consider it just another facet of intelligence, working memory, or some other psychological construct? During students experiences in the neuropsychological laboratory, they learn about how these questions are investigated. They also learn of several theories about brain function, cognition, and even animal physiology so that we can develop and refine these neuropsychological measures in light of theory.