Reliability and Validity. Temilade Adefioye

Size: px

Start display at page:

Download "Reliability and Validity. Temilade Adefioye"

Bennett Shields
7 years ago
Views:

1 Reliability and Validity Temilade Adefioye

2 Content Overview of Reliability and Validity Reliability and Validity in Quantitative Research Reliability and Validity in Qualitative Research Reliability and Validity in Mixed methods

3 Brief overview Reliability and Validity are important concepts in research as they are used for enhancing the accuracy of the assessment and evaluation of a research work (Tavakol and Dennick, 2011, p.53) They have different meanings under the different types of research i.e. quantitative and qualitative research (Creswell, 2014, p. 201).

4 Quantitative Research - Reliability and Validity RELIABILITY refers to the consistency, stability and repeatability of results i.e. the result of a researcher is considered reliable if consistent results have been obtained in identical situations but different circumstances (Twycross and Shields, 2004, p.36) VALIDITY is the extent to which any measuring instrument measures what it is intended to measure (Thatcher, 2010, p.125). It is possible for a measurement to be reliable but invalid; however, if a measurement is unreliable, then it cannot be valid (Thatcher, 2010, p.125; Twycross and Shields, 2004, p.36).

5 Quantitative Research - Reliability and Validity Example A table clock that is always five minutes fast It is reliable because it is always five minutes fast, however, it is not valid because when compared to a standard format such as the GMT, it is not correct.

6 Quantitative Research Types of Reliability Stability - This is when a researcher obtains the same result in repeated administrations or when the same test tools are used on the same sample size more than once, and when there is a reliability co-efficient that provides an indication of how reliable the tool is Homogeneity - This is a measure of the internal consistency of the scales. Cronbach s alpha is used to measure the reliability of a tool. Equivalence - This is level of agreement among researchers using the same data collection tool. The ratings of two or more researchers are compared by calculating a correlation co-efficient. (Twycross and Shields, 2004, p.36)

7 Quantitative Research Types of Validity Criterion-related validity It is also called Predictive Validity. According to Nunnally (1978) cited in Thatcher (2010, p.125, 145), predictive validity is when the purpose is to use an instrument to estimate some important form of behaviour that is external to the measuring instrument itself. For instance, a written driver s license test can be validated by hypothesizing that the test will accurately predict driving competence amongst a group of drivers. If this test fails to accurately predict this competence, then it is not valid. Content validity It is the extent to which an empirical measurement reflects a specific domain of content. For instance, in arithmetic operations, the test problem will be content valid if the researcher focuses on addition, subtraction, multiplication and division but will be content invalid if the researcher focuses on one aspect of arithmetic alone say, subtraction. It is also called Face validity (Thatcher, 2010, p.125, 141). Construct validity It is concerned with the validity of empirical measures and hypothesis testing of theoretical concepts. It is the extent to which a particular measure relates to other measures consistent with theoretically derived hypotheses concerning the concepts that are being measured (Carmines and Zeller, 1979, cited in Thatcher, 2010, p.147).

8 Qualitative Research Reliability and Validity RELIABILITY is referred to as when a researcher s approach is consistent across different researchers and different projects VALIDITY is when a researcher uses certain procedures to check for the accuracy of the research findings (Gibbs, 2007 cited in Creswell, 2014, p.201).

9 Qualitative Research Reliability Strategies Document procedures and steps taken in the research work (Yin 2009, cited in Creswell, 2014, p.204). Setting up a detailed protocol and database so that others can follow the procedure (Gibbs, 2007 cited in Creswell, 2014, p.204). Check transcripts for errors and obvious mistakes (Creswell, 2014, p.204). When using codes, check to see that there is consistency in code definition (Creswell, 2014, p.204). In team research, there should be coordinated communication through regular documented meetings and analysis sharing (Creswell, 2014, p.204). Cross-check codes developed by different researchers by comparing results that are independently derived (Creswell, 2014, p.204).

10 Qualitative Research Validity Strategies Triangulate Examining evidence of information from different sources of data Member checking This is when the final report or specific descriptions from a research work is taken back to the participants in order to determine if the participant feel that the results are accurate. Comprehensive narration This is the use of a rich, thick description to convey findings Bias Clarification This is when the researcher clarifies the bias he/she brings to the study Presenting negative or discrepant information that runs contrary to the information in the findings Time Spending prolonged time in the field in order to gain an in-depth understanding of the problem under study. Peer debriefing Locating a peer debriefer to ask and review questions about the study so that it will not be only the researcher asking questions about the problem External auditor Using an external auditor to review the entire project. This individual is not familiar with the research problem and can be obejective in the assessment of the research work. (Creswell, 2014, p , )

11 Mixed Methods Research When using the mixed methods approach, validity can be carried out by using a convergent approach i.e. a strategy from the quantitative method e.g. content validity and another from the qualitative method e.g. triangulation (Creswell, 2014, p.223, 225)

12 End Comments? Questions? Observations?

13 References CRESWELL, R. (2014). Research design: qualitative, quantitative, and mixed methods approaches. USA: Sage Publications TAVAKOL, M. & DENNICK, R. (2011). Making sense of Cronbach s alpha. International journal of Medical Education, 2, pp THATCHER, R. (2010). Validity and reliability of quantitative electroencephalography (qeeg). Journal of Neurotherapy, 14, pp TWYCROSS, A. & SHIELDS, L. (2004). Validity and reliability - What's it all about? Part 2 Reliability in quantitative studies. Paediatric Nursing, 16 (10) p. 36

Qualitative Research. A primer. Developed by: Vicki L. Wise, Ph.D. Portland State University

Qualitative Research. A primer. Developed by: Vicki L. Wise, Ph.D. Portland State University Qualitative Research A primer Developed by: Vicki L. Wise, Ph.D. Portland State University Overview In this session, we will investigate qualitative research methods. At the end, I am hopeful that you