Is information infrastructure design theory consistent with participatory design?

Transcription

1 Is information infrastructure design theory consistent with participatory design? Petter Øgland Department of Informatics, University of Oslo, Norway Abstract. Application of information infrastructure theory has resulted in rich descriptions of politically and technically complex and usually quite large information systems. Although such insights have been used for developing and testing prescriptive information infrastructure design theory, action research using information infrastructure design theory seems less victorious that one might perhaps have expected. This paper speculates some reasons for why this might be so, in particular asking the question when information infrastructure design theory might be in conflict with participatory design. To shed light on the question, a case study of the design a reading group among PhD students in the IS field is discussed, using challenges in the design of the group as basis for analysis. The case analysis suggests that information infrastructure design theory and participatory design may not necessarily be inconsistent, but they seem to reflect political views that are opposing extremes; information infrastructure design theory values pragmatic opportunism while participatory design is more explicitly focused on social values such as democracy and giving voice to the underprivileged. Key words: Information Infrastructure Design, Participatory Design

2 1 Introduction Information infrastructure design theory is a theory for designing information infrastructures. As information infrastructure theory is a theory that aims to describe a class of politically and technically complex and often quality large information systems, such as the internet, mobile technologies, health information systems in developing countries, electronic patient journals, global management systems etc., one might expect the information infrastructure design theory giving prescriptive advice on how to assemble such systems. However, despite decades of action research in various contexts, such as the Health Information Systems Project (HISP) and the Electronic Patient Records Project (ERP), the success of information infrastructure design theory seems somewhat limited. What could be the reason for this? Is the theory presented in a language that makes it difficult to understand for practitioners of design? Are there conflicting opinions on what information infrastructure design is? Or could it be that the values inherent in the design theory are in conflict with the values in the cultures where the information infrastructure design approach is being used? In order to provide some insights on these questions, an action research study has been carried out in the style of design science research (Järvinen 2006), in order to establish a community of practice for PhD students who want to share their knowledge of information infrastructure theory. In the next section follows a presentation of the elements of information infrastructure theory and participatory design that have been used in this study. The design of the study is presented in the following section, and is then followed by case report with analysis. The papers ends with a discussion of the lifecycle of the reading group, arguing that the case is valid for understanding aspects of both information infrastructure design theory and participatory design, and then gives some suggestions on how the two approaches seem to be inspired by different political agendas. 2 Designing information infrastructures Information infrastructure theory can perhaps be described as a framework for doing sociological analysis of large socio-technical systems where there is an interest in describing issues related to power, standards and complexity. Concepts and vocabulary used by information infrastructure researchers is often influenced by actor-network theory [ANT] (Latour, xx; Callon; ) mixed with elements of concepts and vocabulary from complex adaptive systems [CAS] theory (Arthur, xxx; Holland, xxx). Consistent with this intellectual foundation, the design theory described by information infrastructure researchers (e.g. Hanseth & Aanestad, 2003;

3 Hanseth & Lyytinen, 2004) is partially political (as would be expected from the ANT heritage) and partially algorithmic, as would be expected from the CAS heritage. For instance, in a key paper describing a flexible standards strategy for systems development in developing countries, the design issue is vaguely described by a reference to Holland s genetic algorithms, although the algorithm is not explicitly explained (Braa et al 2005). On some of the final slides from the information infrastructure course given at the University of Oslo, the design theory is summarised in terms of a bootstrap algorithm (Hanseth, 2006): Start with o Simple, cheap, flexible solution o Small network of users that may benefit significantly from improved communication with each other only o Simple practices o Non-critical practice o Motivated users o Knowledgeable users 1. Repeat as long as possible; enrol more users 2. Find and implement more innovative use, go to 1 3. Use solution in more critical cases, go to 1 4. Use solution in more complex case, go to 1 5. Improve the solution so new tasks can be supported Presenting a design theory in the shape of an algorithm is consistent with the ideas of Design Science, as originally defined by Simon (1969) and more recently being a hot topic in information systems research (e.g. March & Smith, 1995; Hevner et al, 2004; Iivari, 2008). There are several interesting aspects to be seen from the bootstrap algorithm when trying to see whether it might work in the context of software quality assurance. Firstly, in order to bootstrap an information infrastructure, the algorithm says, for instance, that one should start with motivated and knowledgeable users. As mentioned in the introduction, one of the reasons the SQA practitioner often ends up as a scapegoat has to do with management neither being motivated by quality nor having knowledge about quality management principles. However, as was pointed out in the case of HISP, lack of motivation and knowledge among users may be compensated by redefining the users to be the action research network, which may often be a network of highly motivated and highly knowledgeable researchers. A second interesting aspect of the algorithm is that the first four command lines all focus on positive feedback; doing more of what already works. There is no explicit mention of control (negative feedback) among the list of commands, although there is an implicit reference to control in the fifth statement that says one should improve the solution so new tasks can be

4 supported. Improving something implies so sort of comparison between the unimproved state and the improved state of the information infrastructure, so there is an implication of evaluation and control in the design method, as in most design methods (e.g. Cross, xxx). Further, in order to apply and test the design theory in the context of SQA, it is necessary to be explicit about what is meant by an information infrastructure. The term is defined differently by different authors, and even in the case of the Scandinavian tradition, the term information infrastructure does not yet seem to have a stable definition. As the design theory focuses on exponential growth, one might expect that information systems evolve from information systems, that once information systems grow to pass a certain treshold of size and complexity, they become information infrastructures, like how the internet in its infancy started as a communications network between a handful of computers. This was of understanding information infrastructures might lead to the parallel of trying to develop an SQA action research network by starting with the isolated SQA action researcher within a given organization who gradually manages to enrol more researchers within a socio-technical communications network that eventually evolves into an action research infrastructure. However, one of the basic premises of information infrastructure theory, not made explicit in the bootstrap algorithm, is that there are always infrastructures in existence (one of the leading Scandinavian information infrastructure researchers view knowledge as information infrastructure (Hanseth, 2004)), and rather than building from scratch, the design idea is to use what is already in existence and influence the development of certain aspects of that network in order to evolve the type of infrastructure that is needed. In other words, the perspective of the SQA practitioner should be to identify something that already looks like an action research network and then see to which extent this action research network could be used as a SQA action research network or as a part of a SQA action research network. As there are strong similarities in methodologies in working as a practitioner within the field of quality management (organizational development) and action research (e.g. French & Bell, 1978), one approach for the SQA practitioner might be to try to view his work as action research and enrol into an academic network of research. Besides getting into an expert community where best practices, challenges and pitfalls may be mapped and discussed, being a part of such a network may also have political consequences for practical work in the organization. In the context of health information systems development (HISP) in developing countries, the networks of action approach has reported as useful and successful (Brå et al, 2004).

5 The theoretical foundation for HISP is partly based on Information Infrastructure Theory (Hanseth, 1998; Ciborra et al, 2001; Hanseth et al, 2007). 3 Methodology One way of investigating the application of the current information infrastructure design theory in the context of an information infrastructure discuss forum, would be to implement an infrastructure and then use the evaluation of the infrastructure as a basis for research. This way of conducting research is sometimes called design science (Hevner et al, 2004; Sein et al, 2007; Iivari, 1998). The diagram in figure 1 illustrates the design research approach used in this paper. The research process is defined as a process that is reverse of the engineering process, contributing to design theory were as engineering design consists of applying design theory (Juran, 1964; Krick, 1969). The research process is seen as an integrated part of the design cycle, more or less equivalent to system evaluation in the case where evaluation of a particular system can be generalized to produce insights about a class of systems of similar kind (March & Simon, 1995). Practical problem Engineering Design Specified solution Decision to redesign Design Theory Implement Evaluation of effectiveness Design Research Implemented solution Figure 1: Design research as an embedded process within the design lifecycle As an information infrastructure can be seen as a socio-technical system, the design of an artefact in such a system can also be seen as the design of an intervention, meaning that design science in a social context can also be seen as a type of action research (Sein et al 2007).

6 The purpose of the remaining parts of the methodology section is to explain the contents of the four boxes in figure 1. 4 Case study with analysis While the action research framework is designed to produce improvent and knowledge for several iterations, in this case the project only lasted one cycle. The results for the various stages in the cycle will be presented below. 4.1 Practical problem From the perspective of the SQA practitioner in industry, wanting to enrol in an action research network, the practical problem consists of identifying potential networks, joining such networks and to influence the networks in a way that makes it possible to discuss SQA related issues that might provide insights useful for solving the practical SQA problems currently being investigated. 4.2 Specifications for a solution For the case study reported in this paper, the idea of establishing a reading group for the PhD students to share ideas and keeping each other up to date on recent developments in information infrastructure theory was an idea that came from the faculty. When asked to assemble a meeting to see whether such an idea would make sense, the PhD students agreed that it seemed like a very good idea indeed, and it was decided that a reading group should assemble on a weekly basis. One of the students volunteered for selecting a first paper and chairing the first meeting. Another student volunteered for setting up a mailing list and a wiki to be used for planning meeting and using as a link for the papers to be discussed. A third student volunteered to do administrative work in terms of practical arrangements for time and place and to send s to remind the others of the next meeting. A part of the requirement specification from faculty was that the group should be run by the PhD students themselves without participation from faculty. This had partly to do with the fact some of the PhD students (especially those from third world countries) were hesitant to discuss and challenge ideas in the presence of faculty members, and partly to do with the fact that faculty members felt that involvement in theoretical discussions with students would detract from other types of obligations. As one of the faculty members wanted to know, on a regular basis, how the group was working, one of the students (the writer of this paper) made a checklist of attendants and also updated weekly quality control statistics (SPC

7 diagrams and Pareto charts) to help the group monitoring it s own success in terms of people showing up to discuss Planning and establishing the forum The process started in August 2006 with one of the Professors asking one of the students whether she would like to discuss with the other PhD students whether it might be a good idea to establish a discussion forum where students could present interesting research they had read about, and thus help each other gaining better overview of the IS research domain in a quicker way than by just spending time in their individual cells. 4.3 Implemented solution At the time when the discussion forum was implemented, the internet web pages indicated a total of 26 students were enrolled in the PhD programme. Together with the technology ( list, wiki, measurements, office buildings), this made up the installed base for the discussion forum. The 26 students were roughly in the age span between 25 and 45 (most PhD students being in their late twenties or early thirties), a gender balance of 9 female students (34%) and a cultural balance of 14 students being associated with foreign universities (54%). A total of 15 students were researching health information systems in a third world context (57%), 4 students were researching mobile technology (15%), 2 students were researching health information systems in a Norwegian context (7%), while the remaining 4 students focused on other issues related to technology and social change (15%). During the 51 weeks the discussion forum existed, a few students completed their PhD research while some new members joined the community. At the end of the experiment, the total PhD student population somehow related to the discussion forum at one time or another consisted of about 30 people. The implemented solution (discussion forum) could consequently be seen as a heterogenious mix of people and technology, interacting in a complex manner in order to solve different problems that were slightly related by the fact that they were to a large extent using the same theoretical framework Weekly meetings and monitoring During the 51 week period from August 23 rd 2006 to August 1 st 2007 the discussion forum existed, a total of 38 meetings were held. The diagram below was used for monitoring the number of weeks between each meeting. On the average, meetings were held every week, although holidays and special events sometimes resulted in two or three weeks between consecutive meetings. Due to statistical calculations, a warning line (upper control limit) was drawn for three weeks, meaning that the process of having meetings was

8 stable as long as there was a meeting once a week on average and never more than three weeks between two consecutive meetings. 4,5 4 3,5 3 2,5 2 1,5 1 0, Weeks between meetings LCL = 0 UCL = 3 AVG = 1 Figure 2: Frequency of meetings Data on the first diagram seem to indicate that the PhD discussion meetings are being successfully implemented on a weekly basis, as originally planned, and seldom is there more than three weeks between two meetings. The second diagram indicates that there was a general interest in discussions as the discussion forum was set up, and there was a general attendance of about nine people. More recently, the number of average participants in the discussions has stabilized on about four people.

9 Attendance for each meeting LCL = 1 UCL = 7 AVG = 4 Figure 3: Attendance for each meeting Data on the first diagram seem to indicate that the PhD discussion meetings are being successfully implemented on a weekly basis, as originally planned, and seldom is there more than three weeks between two meetings. The second diagram indicates that there was a general interest in discussions as the discussion forum was set up, and there was a general attendance of about nine people. More recently, the number of average participants in the discussions has stabilized on about four people. On the 14 th of February 2007 the people present at the group decided to rearrange readings for the discussions to fit with the curriculum for the INF9200 course. After three meetings, however, the attendance dropped to the old level again.

10 Number of meetings attended Accumulated % Figure 4: Number of meetings attended for each participant The diagram shows a frequency distribution describing how many meetings each of the forum members participated in. The member coded as #1 in the diagram, attending 36 out of the 38 meeting, is the action research (writer of this document), who was also responsible for keeping track of the people attending the meetings and distributing attendance lists with statistics and graphs similar to those shown in this section. The person attending the second most meetings (17 meetings) was a PhD student focusing his research on the application of information infrastructure in mobile technologies. The person attending the third most meetings (16 meetings) was a PhD student researching the applications of information infrastructure theory in developing health information systems in developing countries. The two next people on the list were two African PhD students (one female, one male) who were both investigating how aspects of information infrastructure theory could be applied in problems related to IS development in developing countries. No pattern were to be found among those attending or not attending the meetings due to age, gender, ethnic background or practical research domain for applying the information infrastructure theory. 4.4 Evaluation The research approach is related to the evaluation of the solution. While measurable and non-measurable patterns were identified and discussed along

11 the way as the group was in existence, the main focus of the research is to reflect on the discussion forum in retrospect, looking at the statistics and documentation produced by the system (e.g. the wiki) and discussion with people who were involved, trying to figure out why it ultimately failed and whether insights from the experience could be used for designing a new forum, or whether the experience showed that creating a discussion forum was not a good idea in the first place. The final piece of analysis was a self-reflection from the perspective of to which degree the solution solved the practical problem of enrolling the SQA practitioner into an action research network that could be used efficiently for reflecting relevant SQA problems in the context of HISP, mobile technology development and other issues that were natural subjects for discussion within the reading group Trying to find out why it didn t work out The motivation for the study in this paper was the fact that software quality assurance practitioners might take advantage of being enrolled into an action research network in order to prevent ending up as a scapegoat in an industrial software production setting. Based on historical analysis of success and failure in building information infrastructure, a Scandinavian community of information systems researchers have developed design guidelines ( the bootstrap algorithm ) for building such information infrastructures. The case study reported a case where all necessary boundary conditions for designing an infrastructure seemed present but where the design neverheless failed. Through a critical examination of the case, two aspects of the algorithm seem particularly interesting. Firstly it was the question of whether it was a good idea or not to have a forum among PhD students for discussing papers and sharing insights. Perhaps the students were already part of external networks and were better served by trying to enrol themselves into these networks and make themselves important in these networks rather than by establishing a local network of learning and knowledge sharing. Secondly, while the use of numerical monitoring and feedback seemed to have a significant effect in sustaining the forum, it also seemed to have a significant effect on the breakdown of the forum as soon as the measurements accidentally were made available to people not being a part of the forum and thus quickly turned into a topic for ridicule and controversy. In conclusion, the case study does not seem to provide insights that contradict the bootstrap algorithm or identify way to improve the algorithm, but it provides a contribution in terms of emphasizing the risk of failure when trying to establish new practices rather than building on practices already in existence. As a practical guideline for the software quality assurance practitioner that needs to enrol in academia in order to get necessary power to avoid becoming a scapegoat, focus should be kept on trying to be a free

12 rider rather than a leader until the person in question have sufficient academic capital to become a leader by academic authority. 5 Discussion The aim of this paper is to investigate and contribute insights to the current information infrastructure design theory. The case study illustrates a practical problem of were there is an explicit need (at least from the point of the view of QA action researcher) to enrol himself into a social network of fellow action researchers, investigating different topics but using the same theoretical perspective. The reason this research is described as design research (or design action research) rather than just action research is the fact that PhD mailing list, discussions by and the use and distribution of statistics from the meetings played an important role ( IT artefact ) in the design of the information infrastructure. What kind of insights does the case study implicate in the context of design theory? Looking at how the information infrastructure was designed as compared with the bootstrapping algorithm, most statements in the algorithm seems to have been met. The use of mailing lists, wiki technology, simple practices, knowledgeable users etc all seem to be present. While discussing attendance problems with people regularly attending the meetings didn t seem to provide much insight on why people were skipping the forum, discussing the forum with those who attended it on a more irregular basis was more useful. 5.1 Information infrastructure design? One of the challenges of the information infrastructure theory is that it is primarily developed as a framework for doing qualitative research, meaning that the theory is mostly metaphorical and used for grasping issues of politics and complexity that are difficult to describe with a more quantitative approach. This aspect becomes a challenge when it comes to producing normative advice in terms of how to design. The problem is partly solved through the formulation of the bootstrap algorithm, but as the algorithm uses concepts like motivated users, knowledgeable users, simple practices etc., it is very much a question of interpretation of whather on has been following the algorithm or not. To which extent could one say that selecting academic papers, reading the papers and discussing the papers with fellow students would be a simple practice? To which extent could one say that this was a non-critical practice? For a PhD student with limited time resources, time might perhaps be better spent reading and discussing papers that are more specifically relevant to research than something relevant for the neigbouring student.

13 However, following the hermeneutic tradition of interpreting texts in a manner that makes them meaningful rather than meaningless, it should be possible to argue that the information infrastructure design theory (bootstrap algorithm) was followed The paradox of control There is no mention of control in the bootstrapping algorithm. The focus in the algorithm is on positive feedback in terms of enrolling useres and expand infrastructure by innovative use, critical cases and complex cases. However, the final statement in the algorithm says improve the solution so new tasks can be supported. Improving a solution indicates that the previous solution can be measured against some kind of quality standard, and measuring against standards is a way of exercising control. The sociology of knowledge might provide some insights on the application of measurements and the relationship between the application of statistics in discourse and social power (e.g. Porter, 1995). 5.2 The breakdown of the group Why did the reading group break down? Various people say various things. They all seem to be right in one way or another. Perhaps one might say that the group failed as participatory design (PD) but succeeded as information infrastructure in terms of people understanding that they could learn more from aligning with other networks The blind leading the blind The statement the blind leading the blind from one of the senior PhD students combinded with faculty not being allowed to participate in the discussions (as the professor leading the group believed PhD students were more free to express themselves without faculty present). One interpretation of why the forum didn t work out could be due to forum members not understanding information infrastructure theory properly ( blind leading the blind ) and thus not being able to help each other in selecting the right papers to read and discuss. Another interpretation could be that information infrastructure design theory is poorly developed, and the blind leading the blind being more a consequence of the theory being blind in terms of providing insights on how to design information infrastructures. 5.3 Did the group break down? Given the internet as the primary model for explaining what an information infrastructure is, one might say that it si unikely that an information infrastructure should break down, althoug it is not impossible.

14 The complexity of the information system, partly described through theory of complex adaptive systems (CAS), serves the purpose of making the system adapt and thus less likely to break down. Another way of interpreting the breakdown of the reading group could perhaps be to say that it bifurcated from one network into serveral networks as members of the group got involved in other networks. 5.4 Does information infrastructure design theory conflict with participatory design? From an ideological point of view, information infrastructure design theory seems to have something in common with market liberlism while participatory design seems more focused on the welfare of the community as a whole. As the Scandinavian contries have a political tradition of blending market economy with social responsibility, the design philosophies seem more or less like right-wing and left-wing approaches within the same social context, without necessarily being in conflict with each other. Although it was the people more concerned with individual agendas that won over the people more concerned with the community that won in the case of the readings group, being a researcher is consistent with having own individual agendas, and social value should not interfare with intellectual integrity. In conclusion, one could perhaps say that there are strong elements of machiavellian opportunism in the information infrastructure design theory, while the participatory design philosophy has a more naïve belief in the good in others. But, on the whole, the scientific community may perhaps be better understood as an endeavour maintained by the values implied by participatory design, such as encouraging democracy, rational discussion and giving voice to the underpriviledged. Perhaps information infrastrucutre and participatory design are consistent in a wider context where one goes beyond the focus on the individual. Acknowledgment Thanks to friends and colleagues who participated in the study. References Avison, D., Rigour in Action Research: Some Observations and a Plea, Scandinavian Journal of Information Systems, (10:1&2), 1998, pp

15 Braa,, J. Monteiro, E., and Sahay, S. Networks of Action: Sustainable Health Information Systems Across Developing Countries, MIS Quarterly (28:3) 2004, pp Bjerknes, G., Ehn, P., and Kyng, M. (eds.), Computers and Democracy - A Scandinavian Challenge, Avebury, Aldershot, Checkland, P., Systems Thinking, Systems Practice, John Wiley & Sons, Chichester, Chenhall, R. H., and Romano, C. A., Formal Planning and Control Presence and Impact on the Growth of Small Manufacturing Firms, in Job Generation by the Small Business Sector in Australia, W. C. Dunlop and A. J. Williams (eds.), Institute of Industrial Economics, Newcastle, 1989, pp Ciborra, C.U., From Control to Drift Oxford University Press, Oxford, French, W. L. and Bell, C. H., Organizational development, Prentice-Hall, New Jersey, Payne, J. E., Quality Meets the CEO, Software Testing & Quality Engineering, (1:3), 1999, pp Pettigrew, A. M., On Studying Organizational Cultures, Administrative Science Quarterly, (24), 1979, pp