CAQDAS past, present and future: A comparison of reporting practices in the use of ATLAS.ti and NVivo Megan Woods Tasmanian School of Business & Economics, University of Tasmania Trena Paulus Educational Psychology & Counseling, University of Tennessee David P. Atkins University of Tennessee Libraries Rob Macklin Tasmanian School of Business & Economics, University of Tasmania
Purpose & Scope Who is using CAQDAS? How is CAQDAS being used? How is that use being articulated? 2
Methods Scopus database (Web of Science, Academic Search Premier) 763 Peer-Reviewed, English only ATLAS.ti: 349 NVivo/Nud*ist: 414 Empirical, not methodological, articles Qualitative content analysis & discourse analysis 3
Methods Started out using Excel Moved to NVivo (with Endnote) Now also using ATLAS.ti 4
Methods Number of articles published by year Journal discipline Country of first author Methodologies used Data types used Phase of research process Degree of reporting detail How they described their CAQDAS use 5
Findings: Articles by publication year 160 140 120 100 80 NVivo Atlas.ti 60 40 20 0 1994 99 2000 01 02 03 04 05 06 07 08 09 10 11 12 13 6
Findings: Subject disciplines by journal Maths and statistics Arts, languages, music and humanities Agricultural sciences Physical and natural sciences Engineering and applied sciences Exercise Science, Sports, and Kinesiology Multidiscplinary Social work Health policy/systems Nutrition Communication & information sciences Mental health/psychology/addiction Health education Education Allied health Business Social sciences Nursing Public health/epidemiology General health care (other) Medicine Atlas.ti Nvivo 0 20 40 60 80 100 120 140 160 7
Findings: Country of author, Top 10 Country ATLAS.ti NVIVO Total US 109 158 267 UK 108 50 158 Australia 74 2 76 Canada 31 15 46 Netherlands 9 20 29 South Africa 4 9 13 Italy 1 10 11 New Zealand 8 2 10 Germany 0 8 8 Ireland 7 1 8 8
Findings: Methodologies, Top 13 Narrative inquiry or analysis Phenomenology Other Mixed methods Case study or analysis Ethnography or observation study Content analysis Atlas.ti NVivo Thematic analysis or qualitative content analysis Grounded theory Focus group Interview study Generic qualitative 0 20 40 60 80 100 120 140 160 180 200 9
Findings: Data types used ATLAS.ti NVivo Total Data Type Articles % Articles % Articles % Interview data 233 66.8% 326 78.7% 559 73.3% Focus groups 115 33.0% 64 15.5% 179 23.5% Documents 37 10.6% 55 13.3% 92 12.1% Observational field notes Survey or questionnaire 47 13.5% 40 9.7% 87 11.4% 27 7.7% 51 12.3% 78 10.2% Video or image data 16 4.6% 11 2.7% 27 3.5% Naturally-occurring conversation 11 3.2% 15 3.6% 26 3.4% Online data 5 1.4% 8 1.9% 13 1.7% Other 6 1.7% 5 1.2% 11 1.4% Websites 1 0.3% 6 1.4% 7 0.9% 10
Findings: Phase of research process ATLAS.ti NVivo Total CAQDAS for Articles % Articles % Articles % Data analysis/ management Data display/outputs/ representation of findings 349 100% 411 99.3% 760 99.6% 30 8.6% 49 11.8% 79 10.4% Team management 39 11.2% 15 3.6% 54 7.1% Data collection/ creation 3 0.9% 3 0.7% 6 0.8% Literature review 0 0.0% 1 0.2% 1 0.1% 11
Findings: Degree of reporting detail ATLAS.ti NVivo Total Degree of Detail Articles % Articles % Articles % Minimal to None 307 88.0% 361 87.0% 668 87.5% Moderate 34 9.7% 43 10.4% 77 10.1% Substantial 8 2.3% 10 2.4% 18 2.4% 12
Findings: Minimal detail Coding was entered into ATLAS.ti software for further analysis The resulting data were analysed using the qualitative computer package NVivo, as well as the grounded theory method The transcripts were analyzed using ATLAS-ti 5.0, a software program designed to analyze textual and other types of qualitative data (Muhr and Friese, 2004) 13
Findings: Moderate detail Interviews were taped, transcribed verbatim, and processed with ATLAS.ti. The use of such a specialized computer-assisted qualitative data analysis software package is considered to be a useful instrument to improve not only the pace and flexibility of textual data management in specific but also the consistency and internal reliability of qualitative research in general (Maso & Smaling, 1998; Seale, 1999), at least as long as basic prerequisites of qualitative theory building are taken into account (Kelle, 1997). 14
Findings: Substantial detail 15
Findings: Substantial detail All articles imported into NVivo in this research are called Sources. The sources were analyzed by using the Node function in NVivo. A node is a collection of references regarding a specific theme. The references were gathered when reading through the sources, and references about the same theme were categorized into the corresponding node. This process is called coding. 16
Findings: Substantial detail 17
Findings: Substantial detail To what extent did NVivo assist in the qualitative evaluation of the intervention? NVivo greatly facilitated the process of examining participants various responses to the interview questions. Since the purpose of the qualitative analysis was pragmatic rather than theoretical or empirical to assist in improving and refining the MM intervention prior to using it in a large multi-site study NVivo provided an efficient means for managing and categorizing participants responses to the interview questions, and facilitated subsequent use of the data to further develop and improve the intervention. 18
Findings: Ambiguities in reporting Interviews were taped, transcribed, and coded using qualitative analysis software NVivo. Where did transcription take place? 19
Findings: Ambiguities in reporting What is the source of data displays? 20
Findings: Ambiguities in reporting "Coding was conducted by three independent analysts and checked for consistency using ATLAS.ti qualitative software" How did the software support your teamwork? 21
Findings: Levels of reporting Simple mention (name dropping) Description of what the program generally is used for Details of analysis for transparency Advocacy/justification via perceived advantages (rigor, large datasets, supplement to manual coding, teamwork) 22
Conclusions Increasing use, esp. in health sciences Generic qualitative methods Interviews, focus groups, some documents Data management & analysis Minimal levels of detail which may perpetuate CAQDAS misconceptions 23
Next steps Expand dataset to include other CAQDAS programs Review of qualitative research reporting standards (e.g. NSF, COREQ) Interviews with journal editors 24
Thank you Questions? tpaulus@utk.edu 25