Research Methods in Psychology Josh and friends were discussing recent changes in the DUI laws. Jasmin stated that anyone can drink a few beers with no impact on their driving. Ben said a few beers would impact Jasmin but not a big guy like him. The discussion raised questions in Josh s mind regarding gjust how much beer it takes to impact driving. How would one answer Josh s question? What kinds of experiments would you create to find answers? Alicia experienced a traumatic loss, and reacted with shock and grief. Over time, she recovered, but she noticed that she was experiencing one physical illness after another. A medical doctor suggested she see a psychologist regarding the traumatic loss she had experienced. After doing so, she was much healthier than before she talked with someone. Why would talking about a traumatic experience lead to better health than not talking about these experiences? How would you design an experiment to test your idea? 1
What is Good Research? Good research Uses a theoretical framework Uses standardized procedures Is generalizable Uses objective measurements Theory A theory systematically organizes and explains observations includes a set of statements about the relations among various phenomena Theories are abstract statements of relationships that are designed to describe behavior in a very broad manner A theory will lead the researcher to make a specific hypothesis Theory What theory would Josh make? He might make a theory that there is a relationship between alcohol consumption and performance. What specific questions would you want to answer? 2
Hypotheses A hypothesis is a tentative belief that tries to predict or explain the relationship between two or more variables Hypotheses are specific tests t of theories Hypotheses Josh s theory about alcohol and performance might lead to some of these hypotheses: Hypothesis 1: the more beer subjects drink the more likely they will run into an orange cones in a driving experiment - Hypothesis 2: those served near beer, will run into fewer orange cones than those who drank real beer - Hypothesis 3: after consuming alcohol, those with lower body weight will make more mistakes in the driving experiment than those with greater body weight - Hypothesis 4: there is no relationship between body weight, alcohol consumed and the number of cones hit Hypotheses and Variables A variable is a phenomenon that changes across circumstances or varies among individuals Hypotheses measure some form of behavior The things being measured are called variables 3
Hypotheses and Variables Continuous variables involve varying amounts Temperature Blood alcohol level Body weight Categorical variables involve groups, categories, or classifications Male/female Real beer/near beer Standardized Procedures Standardized procedures are applied uniformly to participants to minimize unintended variation For research to be effective subjects need to be treated in the same way as much as possible Same model of car in the driving experiment Same kind of alcohol (beer/wine) Same distance between orange cones Population Population: a group of people or animals of interest to a researcher from which a sample is drawn Researchers usually study a small subset of the people to whom the findings might be applied. It is impossible to include everyone in a given experiment, so researchers use samples 4
Sample Sample: a subgroup of a population likely to be representative of the population as a whole A representative sample reflects the characteristics of the population as a whole. A small group of students can be a sample of all the students from a university Professors could not be in a sample of students as they are not from the appropriate population Internal Validity Internal Validity is the extent to which a study is methodologically adequate The study should be constructed to test the hypothesis in question The researcher should record how much alcohol students consumed, not merely that they consumed alcohol in some amount External Validity External validity refers to the extent to which findings can be generalized beyond the sample Researchers should try to simulate life outside the laboratory Often see the experimenter s dilemma: the trade-off between internal and external validity The more tightly researchers controls what participants experience, the less the situation resembles life outside the laboratory 5
Objective Measures Objective measures are designed to control for the researcher s inherent bias Which of the following was submitted for a survey by the NRA, and which h by Handgun Control?: Do you support our Constitution s Bill of Rights, including the 2 nd Amendment s Right to Bear Arms? Do you support anyone having access to guns, including convicted criminals, drug addicts, and child molesters? Reliability of Measures Reliability refers to how consistently a measure produces consistent results A measure is unreliable if it does not produce similar results over repeated trials There are three important types of reliability: Test-retest reliability Internal consistency Interrater reliabilty Test-Retest Reliability Tendency of a test to yield relatively similar scores for the same individual over time Administer a test this week Administer the exact same test a week later The scores should be reasonably similar 6
Internal Consistency A type of reliability that assesses whether items in a test all measure the same construct also known as interitem reliability Internal consistency involves consistency of participants answers across similar items Do you like mushrooms? Do you like mushrooms with ketchup? Do you like mushrooms on pizza? (If you don t like mushrooms, then you should answer no to all questions) Interrater Reliability A measure of the similarity with which different raters apply a measure Do two thermometers measure the same temperature? Do two police offers agree on estimated speed of driving? Do two readers agree on how much optimism is expressed in a diary entry? Validity Validity addresses whether a test measures the construct it attempts to assess A study should adequately address the hypothesis it attempts to assess Once you establish than a measure is reliable then you determine if it is valid A study cannot be valid unless it is reliable. 7
Validity first, then Reliability Suppose when you applied for a driver s license you were measured on the following: Drive around the block and not hit anything Park the car between two orange cones Take a paper and pencil test on street signs Take a shorthand test Validity first, then Reliability Reliability Very good reliability as everyone consistently fails the shorthand test Validity Poor validity as shorthand has nothing to do with driving a vehicle Types of Validity Face Validity: the degree to which a measure appears to measure what it purports to measure Does a question look like it will help measure the construct in question? Shorthand test to drive a car has very poor face validity 8
Forms of Validity Construct Validity: the degree to which a measure actually assesses what it claims to measure Convergent Validity A measure correlates with similar tools designed to measure the same phenomena under investigation E.g., driving test and paper and pencil measure of driving rules Discriminant Validity The measure correlates poorly with tools of an unrelated nature E.g., driving test is not correlated with cookie baking skill Forms of Validity Criterion Validity the degree to which a measure allows distinguishing between groups on basis of certain behaviors or responses A measure s ability to accurately differentiate among those in a sample A difficult but fair final examination has good criterion validity It differentiates capable from less capable students A measure that predicts behaviors years later has good criterion validity How to accurately measure? Multiple measures of same variable increases accuracy There is error in every measure Multiple l measures reduces the influence of those errors. 9
Error The part of a subject s score on a test that is unrelated to the true score Studies involving humans are never 100% accurate The difference between the actual outcome and 100% accuracy is referred to as error Message and Meaning Self disclosure leads to better health outcomes: Disclosure of traumatic events leads to better physical and psychological health over next months and years Word choice can predict other psychological states: E.g., cognitive complexity, femininity, age, depression, honesty, anger, insecurty, social distance Descriptive Research Descriptive research describes phenomena as it exists in the real world Cannot demonstrate cause and effect Types of Descriptive Research include: Case Study Naturalistic observation Survey research Correlational methods Case Study 10
Case Study In-depth observation of one subject or a small group of subjects Interpretive approach: trying to understand complex meanings underlying behavior why did this one person do what they did (e.g., hit their partner) Uses and Advantages: Learning about new complex phenomena that are not well understood Exploration that is hard to reproduce experimentally Few participants available to study Case Study Example of a Case Study Interested in effects of alcohol on performance: Study single individual who drove drunk, and generalize findings to broad population. Case Study In-depth observation of one subject or a small group of subjects Potential Limitations: Difficult to generalize to broader population In-depth interviewing is time consuming Reliability: study may not be replicable Very susceptible to researcher bias Cannot establish causation 11
Naturalistic Observation In-depth observation of a phenomena in its natural setting Uses and Advantages: Findings clearly applicable outside the laboratory Can study complex phenomena that cannot be replicated in the lab Data can be used to formulate specific hypotheses for the lab Naturalistic Observation Example of a Naturalistic Observation Interested in effects of alcohol on performance: Drive along with police officers, observing performance of those who drive drunk. Naturalistic Observation In-depth observation of a phenomena in its natural setting Can only record what is and cannot ask the subjects why Researcher try to not be noticed by those being observed 12
Naturalistic Observation Potential Limitations: Observer effects: Awareness of being watched may alter natural behavior May not be able to generalize to population May not be able to replicate Susceptible to researcher bias Cannot establish causation Survey Research Asking participants questions using questionnaires or interviews Often asking about attitudes or behaviors Key issue is honesty of responses Uses and Advantages Learn about attitudes and behaviors of large sample Allows quantification of attitudes or behaviors Survey Research Example of Survey Research Interested in effects of alcohol on performance: Interview or give surveys to participants regarding their ability to maneuver well while drunk. 13
Survey Research Surveys often use samples from a given population Random Sample: a sample of participants p selected from the population in arbitrary manner Stratified Random Sample Breaks up sample by some criteria 40% of sample to be male 20% males over 50 and 20% males under 20 Survey Research Asking participants questions using questionnaires or interviews Often asking about attitudes or behaviors Potential Limitations: Honesty of responses: self-report bias Wording and order of questions can change results Did intended person actually complete the survey Sample should be random (selected from population in arbitrary manner) Cannot establish causation Mean, Median, Mode Mean Statistical average of scores of all participants on a measure Median The score that falls in the middle of the distribution of scores; half are below it, half are above it Mode Most common or most frequent score or value Scores: 10,10,10,9,9,11,11=70 Mean 70/7=10 Mdi Median 10 2 scores over 10 2 scores under 10 Mode 10 (Three 10s) In big enough sample, mean, median, mode will be same. 14
Variability of Scores The extent to which participants tend to vary from each other in their scores on a measure Range The difference between the lowest and highest scores in a sample Standard Deviation The amount that the average participant deviates from the mean of the sample on a measure Experimental Research A research design in which investigators manipulate some aspect of a situation and examine the impact of this manipulation on the way participants respond Only way to establish causation Directly by proving that manipulating one variable causes predicted changes in another Independent Variables The variables an experimenter manipulates, or whose effects the experimenter assesses Things the researcher manipulates (controls) as part of a study Conditions in the study Amount of alcohol consumed 15
Independent Variables For our experiment the researcher would control Amount of alcohol each participant consumed Spacing of the orange cones Type of vehicle used Weight of participants Dependent Variables Participants responses in a study, hypothesized to depend on the influence of the independent variables The behavior being investigated by the researcher Not under the direct control of the researcher. This is the response the experimenter measures to see if the experimental manipulation had an effect. Th Dependent Variables Participants responses in a study, hypothesized to depend on the influence of the independent variables The behavior being investigated by the researcher Not under the direct control of the researcher. This is the response the experimenter measures to see if the experimental manipulation had an effect. 16
Steps in an Experiment Form Hypothesis Operationalize Variables Develop Standardized Procedure Select and Assign Participants Apply Statistical Analyses Draw Conclusions Steps in an Experiment Form a hypothesis Design to test theory Operationalize Variables turning an abstract concept or variable into a concrete form that can be defined Steps in an Experiment Develop a standard procedure The only think that varies are independent variables and participants performance Experimenter should avoid Demand Characteristics: cues in the experimental situation that reveal the experimenter s purpose Ideal to run Blind Studies: participants and researchers are kept unaware of important aspects of the research 17
Steps in an Experiment Standardized procedure includes Experimental Group: participants who receive the treatment of interest As compared to Control Group: participants who receive neutral condition to serve as a comparison group Steps in an Experiment Select and assign participants Select a representative sample Use random assignment to place into conditions Experimental Research Apply Statistical Analyses of Data Summarize findings through Descriptive Statistics Draw conclusions about population through Inferential Statistics What can you say about differences between groups Draw Conclusions See if hypothesis was supported Make conclusions about broader theory See if you can generalize to population 18
Experimental Research Advantages of Experimental Research Can test cause and effect Can be replicated If same results with different sample, strong conclusions can be drawn Maximizes control over variables of interest Experimental Research Potential Limitations Results may not generalize outside the lab setting Complex phenomena may not be easily controlled in the lab How might a researcher test marriage longevity without random assignment? Quasi-experimental methods use logic of experimental method but lack absolute control over variables Methodology: Inferential Statistics How likely is it that the study results reflect the population as a whole? Significance tests show if differences between groups are likely to have occurred by chance Probability value: the probability that obtained findings were accidental or just a matter of chance 19
Correlational Research Explores the degree to which two variables are related Goal is that knowing value of one variable will allow prediction of other variable Correlational Coefficient is the statistic used to compare two variables A strong correlation (high absolute value) means that knowing one variable will allow a good prediction of the other variable Range: -1.0 to +1.0 Correlational Research Positive Correlation Two variables move in same direction Negative Correlation Two variables move in opposite directions No correlation Two variables demonstrate no pattern of predictable movement Correlational Research Uses and Advantages Reveals relationships between variables as they exist outside the lab Allows quantification of relationships among variables ibl Can test cross-cultural nature of phenomena 20
Correlational Research Potential Limitation Results describe data, they do not explain why there is a connection Correlation does not equal Causation Even if two variables have a causative relationship, correlational research doesn t check for it 21