Publishing mixed-method research: A case example Leila Hurmerinta & Niina Nummela. Turku School of Economics

Transcription

1 Publishing mixed-method research: A case example Leila Hurmerinta & Niina Nummela Turku School of Economics

2 Aim of this presentation To discuss the challenges of publishing mixedmethod research with the help of an illustrative case example

3 Mixed-method research Controversy of terms Blended/multiple/combined/integrated methods Methodological mix Multi-method research Triangulation Mixed-method study is a study which involves the collection and/or analysis of both quantitative and/or qualitative data in a single study in which the data are collected concurrently or sequentially and are combined at one or more stages in the research process (Hurmerinta-Peltomäki & Nummela 2006)

4 Mixed-method vs. QN & QL tradition Source: Johnson et al. (2007, 124)

5 The case example a mixed-method study in International Business (1) Opening and closing doors: The role of language in international opportunity recognition and exploitation RQ: What role does language play in company internationalisation? Four key findings: the decision-maker s linguistic knowledge is related to how s/he recognises and exploits international opportunities the linguistic knowledge may create a knowledge corridor, which either encourages or prevents the decision-maker from seeing a greater number of international opportunities a construct (i.e. matching linguistic knowledge) was developed to analyse the relationship between linguistic knowledge and opportunity recognition and exploitation matching linguistic knowledge particularly helps managers recognise international opportunities, especially if it is accompanied by language diversity and culture-specific knowledge, but the importance of the knowledge corridor has diluted with the emergence of Business English Target journal: International Business Review JUFO2, ABS 3, publisher: Elsevier

6 The case example a mixed-method study in International Business (2) Mixed-method research design in the study Choice: to answer both confirmatory and exploratory research questions in a single study (cf. Teddlie & Tashakkori 2013) Sequential data collection: first quantitative data, then qualitative Web-based survey among Finnish exporters in food industry (n= 160) to investigate whether the propositions found support Telephone interviews among Finnish exporters in food industry (n=16) to obtain a deeper understanding of the underlying microprocesses Equal role of both types of data First analysis of both types of data separately, then the findings were integrated Cyclical iteration rounds between the different data sets followed End result: merged findings of both data sets

7 Publishing process for the case Submitted to the journal June, 2013 Revise & resubmit January, st revision submitted April, 2014 Revise & resubmit August, nd revision submitted November, 2014 Revise & resubmit December, rd revision submitted March, 2015 Accepted April, 2015

8 1st reviews R1: Please continue with this line of research but in my opinion your study should be redone. As it stands the low response rate, the questionable measures will make this a study that could have limited acceptance if published as is and given the importance of the topic doing the study right should be first and foremost in your mind. Even your analysis methods are problematic. Simple ANOVA? Surely there are causal mapping and testing approaches that would have been more appropriate given the nature of your hypothesis. No R Squares? R2: Regarding the methodological part of the paper, one must observe an interesting proposal of combining qualitative and quantitative research. I was a bit surprised as the sequence of this research was quantitative and afterwards qualitative, whilst the most common thing is the reverse order in search of a generalization of a relationship. R3: In section 3.3 the author(s) specify (rather implicitly) an empirical model, i.e. a relationship between their 'target' variables ('international opportunity recognition' and 'international opportunity exploitation') and a series of independent variables. Unfortunately, they do not proceed to the next logical step, a multivariate analysis of this model (either an econometric analysis, probit or ordered probit estimation, or structural modelling, if the author(s) prefer less rigorous methods). It remains unclear to me the relevance of the qualitative part of the study (section 3.2.2) and the contribution of the 16 interviews, for which at least some more information in needed with respect to characteristics of the interviewed persons' firms.

9 1st response to reviewers Constructive education of the reviewers, length of response letter 26 pages! Separate section explaining the nature and setting of the study for all reviewers In terms of research design, the combination of quantitative and qualitative data was found as an interesting approach. Indeed, we deliberately chose mixed-method research design which is a distinct research tradition from quantitative or qualitative research design. The main reason for this important decision was rooted in our research questions. One of the advantages of mixed-method approach is that it enables addressing both confirmatory and exploratory research questions in one study (Teddlie & Tashakkori 2003), which makes it very suitable for our study. In a mixed-method study quantitative (QN) and qualitative (QL) data and methods may take different roles: either QN or QL data may dominate or they may take equal roles in solving the research questions. In our study the two types of data and methods are equal, thus our study could be labelled as a pure mixed-method study (cf. Johnson et al. 2007). Combination of methods improved not only the validity and reliability of the findings, but it also allowed us to capture a more holistic and contextual view (cf. Jick 1979), and thus allowed us to create novel knowledge and deeper understanding of the phenomenon under study (Hurmerinta-Peltomäki & Nummela 2006). However, although the complementarities of the combination of methods are noticeable, there are also some challenges. In particular, it is typical for mixed-method studies that the findings of the quantitative and qualitative parts of the study are sometimes contradictory (Jick 1979, Brannen 1996). In our study we have tried our best to clarify how we sought explanations for the partly complementary, partly divergent results, thus making the research process as transparent and justified as possible. The methods section on MM needed to be shortened considerably

10 QUANTITATIVE DATA FROM THE WEB-BASED SURVEY QUALITATIVE DATA FROM THE TELEPHONE INTERVIEWS Language diversity Language diversity Knowledge corridor Language as means of communication Matching linguistic knowledge in minority Language as means of communication Language as a source of cultural knowledge Swedish as a mother tongue The importance of mother tongue Hurmerinta & Nummela 2015

11 2nd reviews R1: Overall some great things in the revision. Responding to reviewers concerns. This one is very mixed, you have provided a lot of explanation and added an appendix of responses to reviewers but you need to put more of this in the article. We ask questions because we feel that the reader (not just us) needs explanation. In fact there are many elements in what you have said in the responses that are at odds with the manuscript. A few comments as well on analysis and methodology. Choice of country Non-response analysis Measures Selection of interviewees Go more into the 44 company observations (that really is not a lot) and see if you can make more sense out of the country and corridor concept. R2:? R3: The authors have tried to improve their paper. They achieved this with respect to conceptual issues but not in terms of empirical analysis because the data restrictions do not allow them to pursu a satisfactory empirical strategy.

12 2nd response to reviewers 2nd response letter 6 pages Thank you for this comment. After re-reading the paper we realize that we did not include all the arguments which we pointed out in the response letter and we have now tried to include them in the paper as well as remove the inconsistencies. We are very sorry of not communicating the choice of country and industry better in the paper. In this version we have tried to improve our argumentation concerning the strategic choices. We have carefully examined all the 44 observations and checked all the variables as well as other information related to them. This careful examination did not provide any novel insights, although through this we found a good number of examples of language-related corridor behaviour in these companies. In the revised version we have tried to highlight these as much as possible.

13 3rd reviews R1: Significantly improved, well explained and definitely an important study. One that I hope is published soon in the Journal.. As you can see, compared to my other reviews of your paper I have far fewer comments and most are designed to further amplify the contribution of the paper. Thank you for doing such a good job on my earlier comments. R2:? R3: I have read carefully the answer of the author(s) to the first round of comments and recommendations of the referees. The author (s) provide now in the revised version of the paper an extract of interview answers (not the questions) and a correlation matrix of some variables. So the reader still does not know what questions were posed, i.e. which information is available. Only on the basis of the entire available information gained by the survey it is possible to assess the quantitative part of the paper. In this sense, my critique with respect to the use of multivariate methods to investigate the relationship between the relevant variables remains valid. As a consequence, I still find the quantitative part of the paper not satisfactory.

14 Challenges in publishing Strong legacy of single-method research design, particularly qualitative Research training Experience and competence of reviewers (and editors) Length restrictions

15 The use of mixed-method in IB research (Hurmerinta-Peltomäki & Nummela 2006) most common type features mixed data collection, both quantitative and qualitative both types of data are analysed within methods of their own tradition, i.e. quantitative with quantitative methods and qualitative with qualitative methods the data are collected sequentially first the qualitative and then the quantitative qualitative data most often used during the initiation phase in formulating the research problem, finding a suitable sample, selecting the most appropriate data-collection method and operationalising the key constructs

16 Recommended literature Caracelli, V.J. & Greene, J.C. (1993) Data analysis strategies for mixed-method evaluation designs, Educational Evaluation and Policy Analysis, Vol.15 No.2, Creswell, J.W. & Plano Clark, V.L. (2007) Designing and conducting mixed methods research, Sage Publications: Thousand Oaks. Edmonson, A.C. & McManus, S.E. (2007) Methodological fit in management field research, Academy of Management Review, Vol. 32 No.4, Handbook of mixed methods in social and behavioral research (2003) Tashakkori, A. and Teddlie, C. (eds.), Thousand Oaks: Sage Publications. Hurmerinta-Peltomäki, Leila Nummela, Niina (2006) Mixed methods in international business research. A value added perspective, Management International Review, Vol.46 No.4, Hurmerinta-Peltomäki, Leila Nummela, Niina (2004) First put in the sugar, and then add the eggs or is it the other way round? Mixing qualitative and quantitative methods in international business research, In: Handbook of Qualitative Research Methods in International Business, eds. Marschan-Piekkari, R. Welch, C., Edward Elgar: Cheltenham, UK,

17 Thanks!