Research in Physical Medicine and Rehabilitation. I. How to Ask the Question

Transcription

1 Research in Physical Medicine and Rehabilitation I. How to Ask the Question THOMAS W. FINDLEY, MD, PhD A predictable pattern in research efforts is seen in 28 academic centers in physical Medicine and rehabilitation. They consist of at least one technician and one small research room (220 square feet), with an additional small room (280 square feet), and 1/4 technician for every MD or PhD in the department (r = 0.72). Persons who are not active researchers with external funding must first define the research question in order to develop fundable research proposals. Clinical research is a process that starts from the clinical situation, leads to formulation and answering of the question, and eventually results in integration back into clinical practice. There are many reasons for posing a clinical research question and each results in a different type of question. Asking the right question is the most important part of research, as how the problem is stated determines what data is to be collected, the analysis to be done, and what kind of conclusions can be drawn. A strong research study addresses questions that are clearly spelled out and leads to conclusions that are within the limits of the experimental design and the availability, reliability, and validity of the data. Complex design and analysis do not make a study better if the question itself is not well formulated. It is not as important to know how to answer the question as it is to know how to ask the question. If the initial question is incomplete or incorrect, the rest of the research is at best irrelevant. If the type of question (descriptive, normative, or cause and effect) and the desired use of the research (generalized or not) are identified for each potential research question, one can begin to decide which questions are feasible to answer, given available resources. The resources available that must be considered in selecting the question include time, cost, staff expertise, facilities, and potential subjects. The research format includes not only the questions, but also the methods employed to answer those questions, a data collection plan, and an analysis plan. Posing these in advance, in addition to being good research, substantially reduces costs by decreasing unnecessary data collection and analysis and decreasing the time necessary to complete the project. Exercises and readings are provided to assist development of the entire research format. This is the first in a series of articles on research prepared specifically to assist the physiatrist who is not an active researcher to develop an understanding of the research process. These articles will cover in a nontechnical way the formulation of the research question and design, critical evaluation of research journal articles, use of computers and statistical consultants, informed consent, and preparation of grant applications and manuscripts. By the end of the series the reader should know how to identify the clinical implications of research articles and should be able to make an informed decision about whether they want to undertake research themselves. This is not a course in statistics or research design and is not designed to duplicate written materials which already exist. It will not make you an instant researcher, although with time, effort, and eventual funding you may become one. What it will do is develop an understanding of research as a thought process, allow you to develop an idea, and to see how research may impact on your clinical care. And that, ultimately, is our goal: to assist persons with disabilities to reach their fullest potential by providing the best care we possibly can. Although there are many definitions of a researcher, for the purposes of this series, I will use the factors most closely identified with academic researchers in internal medicine: external research funding and at least one publication in the preceding 2 years. These are stringent criteria, as less than 20% of faculty with an MD degree only in an academic department of internal medicine fit this definition of active researcher. 1 For physiatrists the proportion may well be less as physiatrists were principle investigators in only 22 of 217 projects funded by the National Institute on Disability and Rehabilitation Research during Research effort by physiatrists can be put in perspective by examining information collected by the Association of Academic Physiatrists. There were 28 academic departments that provided information on the number of technicians, professional staff

2 (MD or PhD), square feet of research space, and sources of funding. The number of technicians per professional staff member or per square foot is quite predictable, with the notable exception of two centers that had a great number of technicians (and a large number of outside grants); these two centers are excluded from the regression equations included with these two graphs (Fig. 1). The research space available per professional staff is quite comparable for all centers. Although the equations in Figure 1 may seem complicated, academic centers do follow a predictable pattern. They consist of at least one technician and one small research room (220 square feet), with an additional small room (280 square feet), and 1/4 technician for every MD or PhD in the department. The sources of federal funding for these efforts were primarily from the National Institute on Disability and Rehabilitation Research although a number of other federal agencies were also represented: National Institutes of Health (24 grants), Veterans Administration (9), National Science Foundation (4), and single grants from five other agencies. For those of you with above average research space and technical support and diversified funding sources, my congratulations. For the rest of you, read on. Who should read these articles? Persons who never intend to do research but who think that perhaps they can improve their clinical practice if they understand the research process a little better will find that they will become better consumers of research. Persons who think they might want to do research but are not sure will have a better idea of what is involved and how to go about finding out if they are really interested. Persons who do some clinical research but do not currently have any outside funding will find it easier to prepare an above average proposal and to get valuable review comments, which will make it all the more likely that they will get funded the next time around. And finally those advanced researchers who are frequently called upon to teach their skills either in formal classes or to individual students will find these articles to be useful references. This series is based on a course designed for residents in physical medicine and rehabilitation at the UMDNJ-New Jersey Medical School. Although it is geared specifically for the physician, this course has been given successfully to physical therapists and to occupational therapists at Kessler Institute and does not contain language or concepts that require technical training to understand. It is customary for statistical articles to start with the required mathematical background in order to understand the concepts presented. Most people are too scared to read any further even though "only" high school algebra or some other basic level is specified. For these articles, no such sophistication is assumed; in fact, you will do better with the open, curious mind of the 10-year-old child who constantly asks "Why?" This series will consist of outside readings and exercises in addition to the material presented here in the Journal of Physical Medicine and Rehabilitation. The suggested readings will help you understand the material and serve as reference material. The exercises will help you clarify your thoughts and identify those areas which you understand well. Answering the questions in your mind, however, will only give you about 25% of the benefit as compared to actually writing them out and giving them to someone else in your facility to read. Readers are encouraged to write their exercises and to send them with questions to the author. Specific questions will be addressed in this series, although the individual writer will not be identified. Lest you feel embarrassed, be reassured that no question is so simple or so specific that it has not occurred to other readers.

3 ASKING THE QUESTION Why should we ask questions? There are a number of possible reasons, many of which are overlapping. For those of you who are primarily in "basic" research, my apologies as my bias as a clinician first and researcher second is quite evident (despite having started my first experiment at the age of 3). 1. To improve overall care of our patients 2. To prove the effectiveness of a specific type of intervention 3. To help us record our daily notes in an objective measurable fashion rather than in subjective terms. (In behavior modification psychology it is well known that defining and observing the behavior will by itself result in change in the desired direction, without any direct intervention.) 4. To make our interventions timely and effective 5. To know if what we do really makes a difference 6. To figure out how to achieve a certain result 7. To know when maximal treatment effect has been achieved 8. To motivate staff and avoid burnout when improvement is not evident 9. To allow reasoned choice among therapeutic options 10. To produce accountable, responsible clinicians 11. To make each treatment part of an orderly process toward the final goal rather than an aimless stumbling along 12. To maintain and expand programs by being effective, efficient 13. To enable staff to develop professionally and not become stagnant technicians 14. To add credibility to the profession 15. To satisfy our curiosity 16. To develop an understanding of some basic process 17. To build upon previous research 18. Because we like to ask questions WHAT IS A RESEARCH QUESTION? When most of us think of research, we think of the process of answering a question-research design, analysis, etc. All of this seems quite formidable to the novice, but these topics will be covered later in this series because it is not as important to know how to answer the question as it is to know how to ask the question. If the initial question is incomplete or incorrect, the rest of the research is at best irrelevant. Clinical research is a process which starts from the clinical situation, leads to formulation and answering of the question, and eventually results in integration back into clinical practice. More basic research may at first glance seem to follow a different pattern and have less relationship to any outside events, but if you look at the laboratory environment as the "clinical context" the same interplay is seen. In other words, research is not a mad scientist working in isolation in a basement; it is an orderly process of scientific thought occurring in a larger social context. Without a purpose and a road map, one may wander aimlessly among research projects. Without the larger context, your results may never be incorporated by others. Therefore, generating a research question means more than just asking "what if?". It means taking into account the whole process. Defining the Question to Study The question must be posed in a way which is answerable, and any terms used must be well defined, preferably in ways which have been used by others. The type of question posed will determine much of the methods and analysis required. A research question may be descriptive, normative, or cause and effect. Identifying the Methods-Do They Exist or Can They Be Developed? To a large extent, research is limited by the methods available and makes advances proportionate to the development of better measures. This is particularly clear in basic physical sciences where measures have become very sophisticated and accurate, but is no less true in the "softer" social sciences. A great deal of progress has been made in categorizing and modifying information that does not immediately seem quantitative by anthropologists, sociologists, and psychologists. If you can't measure it, you can't study it.

4 Can the Necessary Data be Collected? Many good projects falter because the researcher cannot find enough subjects. It is always helpful (and for most grants essential) to collect pilot data on several subjects to ensure that the measures as proposed can indeed be collected and at what rate subjects can be entered into the study. Will the Planned Analysis Answer the Question? Before collecting data, one should outline the analysis to be sure that the data collected can really answer the question posed. Actually drawing the key graphs or tables you want to use to present your results will show you clearly whether you are collecting the right type of data, and helps a statistical consultant determine the appropriate statistical tests. If any data you are collecting does not appear in these key charts, do you really need to collect it? Who Will Use the Research Results? How will the results be disseminated to those who will use them? Will they be presented in an easily understood format? Are there particular items of information a certain user group may want? WHY ASK QUESTIONS IN A RESEARCH FORMAT? The research format characterizes not only the question, but includes also the methods employed to answer those questions, a data collection plan, and an analysis plan so that the question can be answered in the best possible fashion. All of these elements are inherent parts of the research question. Posing them in advance, in addition to being good research, substantially reduces costs by decreasing unnecessary data collection and analysis and decreasing the time necessary to complete the project. TYPES OF RESEARCH QUESTIONS The type of question posed will determine much of the methods and analysis required. A research question may be descriptive, normative, or cause and effect. 1. Descriptive (what is the situation?). Example: How we rehabilitated a person with spinal cord injury who had severe mental retardation. 2. Normative or comparing an observed to an expected outcome (what should be the situation?). Example: Is nursing home placement after stroke lower in persons discharged from rehabilitation centers than in persons discharged from acute care facilities? 3. Cause and Effect (are particular observations caused by certain factors?). The major distinction is between two things occurring simultaneously in time (correlated), such as decreased bladder control and leg weakness in persons with spinal cord injury, and one factor which truly causes another (e.g., level of spinal cord injury and leg strength) Example: Does a regular aerobic exercise program for persons with peripheral vascular disease increase their exercise capacity? GENERALIZATION Do the results need to be generalized to other groups of persons? Studies often describe patients in a particular clinic (descriptive) who are assumed but not known to represent the entire population of similar patients. Although one can describe cognitive deficits in spinal cord injured patients in a particular rehabilitation center, this is not the same as stating that among spinal cord injured patients a certain amount of cognitive deficit occurs. What about the elderly person with spinal cord injury and head trauma from a fall who never is sent to a rehabilitation center? Given the type of research question and the need for generalization, you can determine the basic study design necessary (see Table 1). Note that the "randomized controlled trial," traditionally thought of as the standard research design, only forms a very small part of the total possibilities here. The major point is that you do not need to be able to do randomized studies in order to do research. I hope with this series of articles to remove most of the excuses clinicians use for not doing research and to show you how you can collect very valuable information which will be eagerly used by others. TABLE 1

5 Study Design Descriptive Normative Cause and Effect Questions No comparison With compari- Randomly Group son group assigned groups No need to gen- Case study Criteria ref- Before and after Nonequivalent Ramdomized eralize erenced design comparison controlled Case Study trial Generalization Cross-sectional Criteria ref- Sample survey Sample survey Sought Survey erenced and before and compari- Survey and after or son causal analysis ASKING THE RIGHT QUESTION Asking the right question is the most important part of research, as how the problem is stated determines what data is to be collected, the analysis to be done, and what kind of conclusions can be drawn. Although eventually the question will be turned into "hypotheses," it is easier to start by formulating the question in at least four different ways. For example, consider the effects of electrical stimulation of the quadriceps muscle on functional outcome in paraplegia. Does electrical stimulation strengthen the muscle? Does it result in improved ambulation? Does it result in improved independence in activities in daily living? Should it be routinely incorporated into treatment of all persons with paraplegia? Do people adjust better to their disability after electrical stimulation? Is bone density increased and are fractures decreased? Are there particular groups for whom electrical stimulation works better than other groups? A number of questions should be proposed, even if they do not seem especially plausible or testable. This process will help determine which questions are the most important to the investigator, although these will not necessarily be the final questions addressed by the research. If the type of question (descriptive, normative, or cause and effect) and the desired use of the research (generalized or not) are identified for each potential research question, one can begin to decide which questions are feasible to answer given the available resources. The resources available that must be considered in selecting the question include time, cost, staff expertise, facilities, potential subjects. For example, if treatment x and treatment y are relatively equivalent in expense and risk, a research design to determine "is outcome after x better than outcome after y" would be sufficient for clinical program decisions. However, if one treatment is quite a bit more expensive or risky than the other, one must ask the question "how much better is x than y?" The second question requires a much more extensive design than the first. If treatment x results in an increase in Barthel independence score of 2 out of 100 possible points, even if this is statistically significant such a small result may not warrant the effort of changing current treatment programs. REFINING THE QUESTION INTO TESTABLE HYPOTHESES Once the basic question has been settled, a list of testable hypotheses must be developed. This involves writing some very specific questions which will be judged true or false as a result of your research. Each hypothesis should correspond to one table, chart, or statistical analysis. For descriptive studies, rather than stating "We will study the long term use of orthoses in our spinal cord injury clinic," you might specify "The use pattern (hours used per day) of reciprocal gait orthoses in persons with spinal cord injury 6 months after discharge from therapy will vary according to level of spinal injury." The success of the hypothesis is demonstrated by a table or chart of hours per day v level of spinal injury; or if you think it more important, tabulate use v length of physical therapy. The more general statement of long term use (hours per day, distance traveled, attitude toward self, independence in activities of daily living) v spinal cord injury (level of injury, length of therapy) actually requires eight separate tables to examine the relationships among each of the important components (hours v level, distance v level, attitude v level, independence v level, hours v therapy, distance v therapy, etc.). By specifying all the hypotheses in advance, one avoids becoming bogged down in the data analysis.

6 SUMMARY A strong research study addresses questions which are clearly spelled out, and leads to conclusions which are within the limits of the experimental design and the availability, reliability, and validity of the data. Complex design and analysis do not make a study better if the question itself is not well formulated. READING ASSIGNMENT 1. Designing Evaluations. Methodology Transfer Paper 4, The Design Process, Program Evaluation and Methodology Division, GAO, U. S. General Accounting Office, July 1984, pp Babbs CF, Tacker MM: Writing a scientific paper prior to the research. Am J Emerg Med 1985;3: EXERCISES In the following exercises, you may either choose a question which particularly interests you or you may choose the topic of rehabilitation of the amputee as discussed by P. H. Stern in the American Journal of Physical Medicine and Rehabilitation 1988;67: A specific example of this topic will be presented in the next article. 1. Generate a research question and pose it in at least four different ways. Categorize the questions as descriptive, normative, or cause and effect. 2. Ask at least two other persons how they would use your research results, for each of the four potential questions. 3. Reformulate your questions based on the further clinical and research implications elicited in Step Identify methods to answer each of the four questions, including the kind of data, type of subject, method of selecting and recruiting subjects, method of collection of data and timing of data collection. Which of your methods are already existing and which do you need to develop? How long will it take you to collect the data? 5. Specify your hypotheses and outline the analysis (graph of time v level of injury, table, chart, correlation, etc.) for each one. It is not necessary at this point to specify the statistical test (that is what statistical consultants are for) but only you can specify what it is that you want to test. 6. Who will use your research results? How will you reach your intended audience (in which journal will you try to publish?)? Will your audience understand your analysis? Will you answer all of their questions? Did you include a representative selection of potential readers in those who gave you comments in Step 2 above? 7. Choose the one question which seems most appropriate, given available resources of time, staff, subjects, and cost. Justify your decision by specifying which options were or were not chosen and why. ACKNOWLEDGMENT The substantive contribution of Deborah Darrow, RPT, in identifying clinical reasons to ask research questions is gratefully acknowledged. REFERENCES 1. Levey GS, Sherman CR, Gentile NO, Hough LJ, Dial TH, Jolly P: Postdoctoral research training of full time faculty in academic departments of medicine. Ann Intern Med 1988;109: Fowler WM: Departments of physical medicine and rehabilitation with organized research programs. Research Report No. 6, AAPM&R/AAP joint Research Committee.