Back to the Basics: Crafting the Research Question TAA Teleconference Sonja K. Foss and William Waters February 14, 2013 Criteria for a Good Research Question 1. It clearly identifies the theoretical construct you are studying transmission of political perspectives students motivation branding 2. It contains some suggestion of recognizability of the theoretical construct the experience of nontraditional women in college (not recognizable) nontraditional women s experiences of discrimination in the classroom nontraditional women s use of support services on campus group purpose in therapeutic settings (not recognizable) participants methods of sabotaging group purpose 3. It (usually) transcends your data What is the nature of the resistance strategies used by subordinate groups to challenge hegemonic institutions? accounting practices used in children s theatres in Detroit accounting practices used in nonprofit arts organizations What strategies are used by Alcoholics Anonymous to attract members? How was a counter-culture identity sustained in Humboldt County, California, in the 1980s and 1990s? How do troll images function in the narratives of Scandinavian writers between 1960 and 1990? 4. It identifies your study s contribution to an understanding of the theoretical construct An explanation of the communication processes by which political beliefs are transmitted within families An explanation of how stay-at-home mothers legitimize the role in an era when most women work 5. It has a capacity to surprise How do traumatic events produce long-term negative effects on individuals? 6. It can produce robust results. What is the nature of What are the functions of What are the mechanisms by which How do... perceive What factors affect What strategies are used What are the effects of What is the relationship between Crafting Research Questions 1. Research Question: Are minority mentoring programs effective in mentoring minority undergraduate students? This question doesn t meet the criterion of robustness because it s a yes-no question that will not produce complex and insightful findings. A better question is: What factors characterize successful mentoring relationships for minority undergraduate students?
2. Research Question: What is the history of public education in Washington, D.C.? This question for a study in the discipline of history is inappropriate because it contains no identification of the contribution the study will make to an understanding of the theoretical construct education in Washington, D.C. Because no contribution the history will make is specified in the question, anything can count as data to include in the study (history of minority relations within the schools, history of school finances, history of assessment of students, history of pedagogical practices used in the schools), thus violating the criterion of recognizability. A better question is: How did the relationship between teachers and unions in Washington, D.C., affect pedagogical practices within the schools in the decade of the 1960s? 3. Research Question: How do climate-driven changes in the biophysical environment of the Great Lakes region affect the sustain ability of wetlands? This question violates the criterion of transcendence of data. It names the particular data that will be collected to answer the question the biophysical environment of the Great Lakes region. A better question that meets the criterion of transcendence of data is: How do climate-driven changes in the biophysical environment affect the sustainability of wetlands? 4. Research Question: How do Amish parents insure that their children actively contribute to the survival of the Amish community? This question does not have a recognizable theoretical construct. Virtually any practice in which Amish parents engage could be seen as a possible mechanism for insuring active contributions by their children, and collecting and coding data to answer this question would be very difficult. A better question is: What disciplinary practices do Amish parents use to facilitate their children s active participation in contributing to the survival of the Amish community? Disciplinary practices focuses the theoretical construct and enables the researcher to be clear about what will answer the question. Notice that the data are named in this question. That s because the researcher is interested in the Amish community in particular, so lack of transcendence of data is not a problem here. 5. Research Question: How does the Starbucks chain engage in oppressive practices toward consumers? - 2 - Let s assume the researcher really wants to study only Starbucks for some reason, so this question doesn t violate the criterion of transcendence of data. But it does violate the capacity to surprise. The researcher is already assuming that Starbucks performs a particular function. A question that asks a genuine question is: What are the impacts of Starbucks on the consuming patterns of its patrons? 6. Research Question: Are the learning strategies used by law students at St. Louis University s School of Law effective? There are three problems with this question. It includes the data in the question, violating the criterion of transcendence of data. It asks a yes-no question, violating the criterion of robustness. And it has a vague theoretical construct strategies to learn what? A better question is: What learning strategies do first-year law students use to develop their case-analysis skills? 7. Research Question: What happens when motivational techniques from the business world are applied to non-profit arts organizations? This question lacks specificity in identification of the contribution to an understanding of the theoretical construct. What happens does not provide a clear and specific understanding about what interests the researcher about non-profit arts organizations. A better question is: What motivational techniques are reported as effective by the staff of non-profit arts organizations? Multiple Research Questions 1. Proposed study of online chat rooms and whether they have the capacity for deep culture A. Original research questions What artifacts do chat rooms use as the basis for developing culture? What norms characterize chat rooms? What processes are used to socialize new members into chat rooms? What mechanisms are used in chat rooms to repair breaches of organizational norms? B. Improved research questions Can chat rooms develop deep culture? What mechanisms do participants in chat rooms use to create deep culture?
2. Proposed study of differences for students between online and face-to-face courses Are there differences in students retention of subject matter between online and face-to-face courses? If so, what are the differences? Do the differences correlate with students learning styles? Aligning the Research Question with Other Key Pieces of a Study 1. The other key pieces with which you want to align your research question are: Research design Data Method(s) for collecting data Method(s) for analyzing data Categories of literature to cover in the literature review Reasons why the study is significant Alignment between Research Question and Data 1. Research question: How do successful female university professors account for their success? Data: Stories by female university professors about success in their careers (alignment) 2. Research question: What strategies are used by founder-centered arts organizations to insure viability following the departure of the founder? Data: Minutes of board meetings, publicity materials, and interviews with the board members and staff of a ballet company where the founder had died (alignment) 4. Research question: What is the nature of the process used in small governmental agencies to decide whether to privatize public service? Data: Case studies of two cities in Nebraska of comparable size, one that has privatized a particular service and one that has not, using minutes of meetings; newspaper articles; press releases around the decision; and interviews with mayors, city managers, and others involved in the privatization decision (alignment) 5. Research question: How is identity constructed through social interaction in conflict situations? Data: Individual interviews with participants in a conflict (lack of alignment) 6. Research question: How do independent dance companies bal ance economic viability with commitment to an artistic vision? Data: Participant observations of the creation and daily practices a dance troupe uses to choreograph its dances (lack of alignment) A better question: How is leadership enacted among members of a dance company to create an artistic product? Different data: Budgets and financial records of the dance company and interviews with directors and other staff members about how they reconcile financial viability with their artistic visions Alignment between Research Question and Method of Data Collection 1. How you collect the data is not as likely to be affected by your research question as other elements of your research design. 2. We don t usually see issues with a lack of alignment between the research question and the method of data collection. 3. Research question: What is the ideology concerning leadership implicit in canonical literature for young adults? Data: Children s books that won the Newbury Award (alignment) - 3 -
Alignment between Research Question and Method of Data Analysis 1. Research question: Why is protest music effective for mobilizing people to join social movements? Method of data analysis: Discursive or rhetorical analysis (alignment) 2. Research question: How does religious discourse function in political contexts? Method of data analysis: Feminist analysis focused on the construction of gender (lack of alignment) Alignment between Research Question and Categories of the Literature Review 1. Research question: Are the arguments that female and male legis lators make in relation to children s issues different? If so, how? Categories of literature to cover: (1) Gender differences in decision making; (2) Gender differences in legislative actions; (3) Gender differences in argumentation; and (4) Kinds of arguments used in relation to children s issues. The terms female and male suggest literature reviewed regarding gender, legislator suggests decision and legislative actions, arguments suggest argumentation, and children s issues suggests argumentation strategies in terms of policies related to children. 2. Research question: What factors characterize a successful men toring relationship for minority students? Categories of literature to cover: (1) Characteristics of successful mentoring; (2) Characteristics of successful academic mentoring of minority students; (3) Characteristics of successful academic mentoring of university students; and (4) Factors that affect the completion of graduate degrees (you would include this body of literature because you are defining successful in the question as completion of degree). - 4 - Alignment between Research Question and Significance of the Study 1. Common ways of talking about significance: The study constitutes a starting place from which to understand a phenomenon. Understanding this phenomenon may help restore something. How a phenomenon is understood comes from one perspective, and the study provides a different perspective that will expand understanding. A complex relationship exists between two phenomena, and this study helps explain it. The phenomenon being studied does not fit existing theories, and the study will explore the nature of the phenomenon to see if the theories need to be expanded or revised to account for it. The phenomenon being studied produces or affects something important, so understanding what that is requires an understanding of the phenomenon that generates it. The study provides a missing piece in the understanding of a phenomenon. The study theorizes a phenomenon that has been under theorized but that is key to understanding a larger picture. The study helps solve a problem. Individuals are engaged in a particular practice that seems not to be working. The study provides ideas for practices that work. Sample Plans for Testing Alignment with the Research Question Sample Plan #1 Research question: What factors facilitate or constrain the institutionalization of reproductive health-care policy in Nigeria? Areas of literature to review: Literature concerning: (1) The process of insti tutionalization in developing countries; (2) Reproductive health-care policies in Africa, particularly the factors that affect such policies; and (3) Reproductive health-care policies in Nigeria, focusing on those factors that facilitate or constrain institutionalization Data: Documents related to health-care policy produced by the government of Nigeria and interviews with health-care policymakers and educators employed by the government
Data collection: (1) Unstructured interviews; and (2) Request of documents from the government of Nigeria Data analysis: Grounded-theory analysis of documents and interviews to discover factors Significance: The study will inform those who are contributing monetary assistance to Nigeria to improve health care. Currently, those funds do not seem to be having any postive effect on health care. Sample Plan #2 Research questions: (1) What meaning do individuals attribute to a feeling of loneliness if they feel it within the context of what they perceive to be an intimate relationship that is healthy and strong? (2) How does communicating with one s partner (or choosing not to communicate) about this feeling affect the intimate relationship? Areas of literature to review: Literature concerning: (1) Typologies of loneliness; (2) Characteristics of good intimate relationships; and (3) Loneliness within relationships Data: Information from people in healthy, committed, romantic relationships about their perceptions of loneliness Data collection: Face-to-face interviews with individuals, both heterosexual and homosexual, who do not have children living with them, using a snowball sample to locate participants Data analysis: Grounded-theory analysis of interviews Significance: The study might assure people who have good relationships that their relationships are not necessarily lacking if they experience a sense of loneliness within them. It might help reassure them that feeling some thing like loneliness and articulating it may be perfectly natural in the process of human relating. Outline of the study: Chapter 1: Introduction, including research question, literature review, research design, significance, and outline of the study Chapter 2: Thematic representations of loneliness (findings related to the first research question) Chapter 3: Loneliness and communication (findings related to the second research question) Chapter 4: A theory of relational loneliness Sample Plan #3 Research question: What practices by arts advocates discourage individuals from arts participation? Areas of literature to review: Studies about reasons for individuals lack of participation in the arts Data: Information from individuals who do not participate in the arts about what arts advocates do to discourage them from participating Data collection: Focus groups composed of individuals who do not attend arts events Data analysis: Grounded-theory analysis of focus groups Significance: Attendance at many arts events is declining, and the reasons may have to do with practices in which arts advocates unintentionally engage that exclude potential participants. Discovery and identification of some of these practices will help arts advocates avoid them and build better cases for arts participation. Final Assessment 1. Does everything align with the research question? 2. Are the data a good example of the phenomenon you want to study? 3. Can your question be answered with the method you have proposed? 4. Are the categories of the literature review derived from the key terms of the research question? 5. Do the reasons for doing the study relate to the research question and the categories of literature covered in the literature review? - 5 -