Application of multiplecase study method in doctoral dissertation. Dr Piotr Zaborek

Transcription

1 Application of multiplecase study method in doctoral dissertation Dr Piotr Zaborek

2 Presentation structure: 1. Outline of a real-life research problem faced while working on the doctoral thesis 2. Dilemmas over research method for the project: analysis of the choice options and decisions made 3. Characteristics of the case study method applied in the doctoral thesis 4. Description of the techniques used to increase quality of the study results 5. Concise analysis of the research process and its findings 2

3 Research problem of the doctoral thesis Is application of the Internet technologies increasing efficiency of manufacturing companies? which was subdivided into following subproblems: Are the Internet technologies causing noticeable changes in efficiency as measured by operating profit or are they only visible at the level of specific cost and income sources? What costs and incomes are being affected by different kinds of Internet technologies? What internal and external factors determine relationships between the Internet technologies and efficiency of manufacturing companies 3

4 Basic constructs of the research problem Internet technologies 25 different functions divided into three groups including (1) publicly available Web sites, (2) intranet and (3) extranet Efficiency Defined as a ratio of value of outputs from production processes to the costs of inputs used for production. For comparability sake both outputs and inputs were measured in the same monetary units. 4

5 Contemplated alternative research approaches Quantitative method (survey of managers at Polish manufacturing companies) Research problem Quantitative method supplemented Qualitative by qualitative method inquiry (survey (multiple case with in-depth study research) and focus group interviews) 5

6 Major differences between qualitative and quantitative research methods Objective Number of research units Method of selecting research units Data collection Data analysis Qualitative research Usually theory development based on inductive approach One to several Purposeful sampling Flexible, usually multiple sources Rarely applying statistical techniques Quantitative research Usually theory verification based on deductive approach Several dozen to several hundred Reprezentative sampling Rigid, usually one source Relying on statistical techniques Generalization Analitical Statistical 6

7 Case study Advantages of case study and survey method: Is especially useful in studying contemporary phenomenon within its real-life context especially when the boundaries between phenomenon and context are not clearly evident Survey Is preferable to other methods when the research goal is to describe the incidence or prevalence of a phenomenon or to formulate predictions about certain outcomes It is a valuable technique when complexity of research problem makes it difficult to design standardized inquiry instruments and when answers are contained in multiple data sources Is widely accepted among scholars of economics and management as a valid research approach Is adequate for research projects which attempt to explain mechanisms and outcomes of certain phenomena The fundamental method of data analysis are highly standardized statistical techniques, which makes the analysis easier and faster 7

8 Main factors taken into account in deciding on the research method Some scholars have maintained that survey method may not uncover the whole range of benefits from Internet technologies Evidence from American and European scientific literature of potential usefulness of case study method in analyzing information systems The most advanced Internet technologies (extranet) were used only by a small fractions of the Polish manufacturing companies (3-4%): randomized sampling technique might result in too small number of these companies to draw statistically significant conclusions Pilot study done by the author involving a questionnaire filled in by managers have shown different results from the ones obtained from the same companies by direct analysis of other sources of data 8

9 Based on the research problem characteristics and contextual conditions decision was made to use multiple-case study approach as a research method. The choice between single- and multiple case study is thought to be the single most important decision in research design. It strongly affects all elements of case study design 9

10 Single- or multiple-case design? The single case design can be selected when: 1) it represents the critical test of a significant theory, 2) it pertains to so rare phenomenon that any single case is worth documenting and analyzing, 3) it represents phenomenon which is hardly accesible to scientific investigation 4) the same case is studied at two or more points in time 5) the single case serves as a pilot case for a more complex research design In all other situations the better choice seems to be the multiple-case study design 10

11 A complete case study design should contain 5 elements: 1. Research problem General statement of your research scope. It helps to decide if case study is the best research strategy 2. Case study hypotheses They direct attention to all important elements that should be examined within the scope of the study. Hypotheses also indicate what evidence is needed and where to look for it. 3. Units of analysis Definition of the primary objects of study together with the time and space boundaries. Will it be individuals, groups, organizations, policies, programmes. It determines the limits to data collection and analysis. 4. Analysis techniques Contain detailed guidelines for linking empirical data with research hypotheses. They may include: pattern matching, explanation building and time-series comparison. 5. Criteria for interpreting findings 11 Should specify when it is possible to accept hypothesis and when it should rather be discarded.

12 Research design of the dissertation: case study hypotheses H.1 Application of the internet technologies in the informational area of customer service decreases costs of customer service. H.2 Application of the internet technologies in the transactional area of customer service decreases costs of sales transactions. H.3 Application of the internet technologies in the informational and transactional area of customer service increases sales incomes. H.4 Application of the internet technologies in the informational area of cooperation with suppliers decreases costs of supplier service. H.5 Application of the internet technologies in the transactional area of cooperation with suppliers decreases costs of supply purchases. H.6 Application of the internet technologies for the internal communication needs of a manufacturing company decreases costs of internal communication. H.7 Application of the internet technologies in a manufacturing company increases costs of amortization and the purchase of services from external providers. H.8 Application of the internet technologies in a manufacturing company decreases costs of human resources C o s t s S a l e s C o n d i t i o n s H.9: The strength of impact of the internet technologies on a manufacturing company s efficiency is dependent on the set of factors related to internal features of the company and characteristics of micro- and macroenvironment in which it operates. 12

13 Research design of the dissertation: case study hypotheses H.1 Application of the internet technologies in the informational area of customer service decreases costs of customer service. H.2 Application of the internet technologies in the transactional area of customer service decreases costs of sales transactions. H.3 Application of the internet technologies in the informational and transactional area of customer service increases sales incomes. H.4 Application of the internet technologies in the informational area of cooperation with suppliers decreases costs of supplier service. H.5 Application of the internet technologies in the transactional area of cooperation with suppliers decreases costs of supply purchases. H.6 Application of the internet technologies for the internal communication needs of a manufacturing company decreases costs of internal communication. H.7 Application of the internet technologies in a manufacturing company increases costs of amortization and the purchase of services from external providers. C u s t o m e r s S u p p l i e r s I n t e r n a l o p e r a t i o n s H.8 Application of the internet technologies in a manufacturing company decreases costs of human resources H.9: The strength of impact of the internet technologies on a manufacturing company s efficiency is dependent on the set of factors related to internal features of the company and characteristics of micro- and macroenvironment in which it operates. All areas 13

14 Research design of the dissertation: units of analysis Characteristic of the unit of analysis Manufacturing companies of middle or large size (employing in excess of 50 people), operating in B2B area, diversified with respect to utilized internet technologies Number of units of analysis 6 companies, among them 2 using publicly available web sites only, 2 with intranet implementation, and 2 utilizing elements of extranet technology Selection criteria Meeting above-specified characteristiscs of the unit of analysis Allowing unrestrained access to data sources within the company 14

15 Each company was studied using multiple information sources from several embedded subunits of analysis as shown in a chart: Data collection process within each company Observation and analysis of the company s information system A series of indepth interviews with employees Internal documentation analysis Additional questions sent to informants by e- mail 15

16 Research design of the dissertation: analysis techniques Analysis strategy Deductive approach with strong reliance on theory (used to propose the set of hypotheses) and close attention given to rival explanations of discovered relationships Analysis techniques Pattern matching for dependent variables Dependent variables were contained in research hypotheses which predicted certain outcomes of applying specific internet technologies. Comparison was made with empirical results 16

17 Research design of the dissertation: criteria for interpreting findings Every hypothesis was verified based on measurable variables It was checked if literal replications occured It was checked if theoretical replication took place Simple statistical techniques were used to provide objective confirmation of relationships 17

18 Verification based on measurable variables Decision to accept or reject hypothesis was based on financial results of applying respective internet technologies. Quantifications were made by comparison of current performance measures to historical levels of the same measures from before implementation of the internet technology When historical data were unavailable current performance measures were compared to a hypothetical situation in which a given technology was not used at all (scenario method). Hypothetical scenarios were built using insights from the company s employees and pertinent documentation and later verified by the most competent employees 18

19 A fragment of the table presenting financial benefits of internet technology application for one of the studied companies (in original form) 19

20 A fragment of the table presenting costs of internet technology application for one of the studied companies (in original form) 20

21 Replication logic in multiplecase studies Each case in research design must be carefully selected so it either predicts similar results or predicts contrasting results but for known reasons When empirical analysis shows that two cases provide similar findings a literal replication can be claimed If two cases give different results, which was predicted by the theory pertaining to the research, it can be said that a theoretical replication was established 21

22 Stages in analysis process within the dissertation case study research Analyzing validity of the research hypotheses for all studied cases Comparative analysis across three levels of Internet technology application to see if theoretical replication took place Comparison of case 1 and 2 to check for literal replication Comparison of case 3 and 4 to check for literal replication Comparison of case 5 and 6 to check for literal replication Individual analysis of case nr. 1. Individual analysis of case nr. 2 Individual analysis of case nr. 3 Individual analysis of case nr. 4 Individual analysis of case nr. 5. Use of web pages Use of intranet Use of extranet Individual analysis of case nr

23 Design quality of a case study research is determined by four conditions: Construct validity Internal validity External validity Establishing correct operational measures to lower the risk of subjective judgments Establishing causal relationship, whereby certain conditions are shown to lead to other conditions as distinguished from spurious relationships Establishing a domain to which a study s findings can be generalized Reliability Demonstrating that the operations of a study such as the data collection procedures can be repeated with the same results 23

24 Tactics for increasing case study quality by Robert Yin: Quality aspect Construct validity Internal validity External validity Reliablility Case Study Tactic Use multiple sources of evidence Establish chain of evidence Have key informants review draft case study report Do pattern-matching Do explanation-building Adress rival explanations Use logic models Use theory in single-case studies Use replication logic in multiple-case studies Use case study protocol Develop case study database Phase in which tactic occurs Data collection Data collection Composition Data analysis Research design Data collection 24

25 Tactics used in the dissertation to increase construct validity: triangulation Multiple sources of evidence were used to establish convergent lines of inquiry in the process of triangulation. Triangulation of sources of data takes place when at least two sources independently confirm a finding In the dissertation a finding was deemed valid when it was confirmed by at least two sources of evidence and no source presented contradictory information 25

26 Information on triangulation instances was presented in tables in the dissertation appendix (an original sample below) 26

27 Tactics used in the dissertation to increase construct validity: establish chain of evidence To establish chain of evidence is to give the case study reader a possibility to follow the derivation of any evidence, ranging from initial research questions to ultimate case study conclusions In the dissertation every step of conducting research study was amply documented. A whole, 271-pages-long appendix was dedicated to the detailed presentation of results from every company. The author assumed that for every finding such an amount of information on application of Internet technologies and its context must be presented that the reader should be able to draw its own conclusions and see if they matched those of the author. 27

28 Tactics used in the dissertation to increase construct validity: have key informants review draft case study report At the end of writing-up case-study report from each company a management-board member was consulted to check the most important findings of the study. When justifiable objections were raised the case study report was corrected accordingly. 28

29 Tactics used in the dissertation to increase internal validity Data in each case study were analyzed using earlier described pattern matching approach for dependent variables, whereby each hypothesis contained a single, measurable dependent variable To increase contrast between solutions only two dichotomous patterns were used: when there is an increase in benefits or decrease in costs as measured by a variable or lack thereof Attention was paid to definitely rule out this benefits which might have been brought about by other causes then internet technologies Every more complex chain of cause and effect was illustrated by a chart, which form was inspired by graphs presented by M. Miles and A. Huberman 29

30 An example of a chart used to illustrate relationships pertaining to one hypothesis for a company 30

31 Tactics used in the dissertation to increase external validity As a vehicle for increasing external validity a design with both literal and theoretical replications was used It included three literal replications and three pairs of theoretical replications based on three different levels of advancement in using internet technologies Obtained results have shown that both types of replication took place which increased strength of the theoretical proposals of the study 31

32 Tactics used in the dissertation to increase reliability A case study protocol contains the instrument as well as the procedures and the general rules to be followed in conducting case study. In this sense most of the third chapter of the dissertation contained elements of case study protocol, including general and probing questions, procedure for collecting data, methods of analysis and templates of tables and charts to be used in displaying information The information contained in the third chapter appeared to be enough to repeat the investigation, thus the result was increased reliability of the study 32

33 Changes in operating income margins in studied companies attributed to the Internet technologies Chart summarizing the results of the dissertation multiple-case study Relationship between the number of Internet technolgy applications and changes in operating income margins in studied companies 6,0% 5,0% Firm F 5,396% 4,0% Firm E 4,278% only permanent cost and income benefits 3,0% 2,0% 1,0% 0,0% Firm A 0,003% Firm B 0,900% Firm D 2,197% Firm D 1,006% Firm C 0,388% Firm E 1,789% Firm F 2,176% Number of Internet technology applications including one-time cost and income benefits 33

34 Two major conclusions of the dissertation multiple-case study (as illustrated by the graph) 1. There is a positive relationship between the number of used Internet technologies and increase of efficiency. In case of five out of six companies efficiency gains have resulted in noticeable improvement of financial standings. 2. The relationship seems to be linear in nature, meaning the more there were applications of Internet technologies the stronger increases in efficiency were observed. 34

35 Recommended case study literature 1. Robert Yin.: Case Study Research: Design and Methods, SAGE Publications, Inc., Thousand Oaks, California, USA, 2003 Possibly the best primer of case study method for economics and management sciences; reasonably short, extremely useful, readable and filled with illustrative examples 2. Perry C.: Processes of a Case Study Methodology for Postgraduate Research in Marketing, European Journal of Marketing, vol. 32, nr. 9/10, 1998, str Lots of useful guidelines for conducting case studies for doctoral thesis 3. Miles M., Huberman A.: Qualitative Data Analysis: An Expanded Sourcebook, SAGE Publications, Thousand Oaks, California, USA, 1994 A comprehensive collection of analysis techniques applicable in case study research 4. Piotr Zaborek.: Studium przypadku jako metoda badawcza, Rozdział XVI publikacji zwartej pt.: Doktoranci o metodologii nauk ekonomicznych, red. Kazimierz Kuciński, Szkoła Główna Handlowa w Warszawie, Warszawa 2007, s An article in Polish on case study research for dissertation purposes 35

36 Thank you for your attention and good luck in your research! 36