Distributed Pair Programming in Global Software Development

Transcription

1 Distributed Pair Programming in Global Software Development Jingchun Hao E H U N I V E R S I T Y T O H F G R E D I N B U Master of Science School of Informatics University of Edinburgh 2011

2 Abstract Global software development has become a widespread business practice over the course of recent years. Organizations are seeking to apply agile methodologies to the globally distributed environment. However, the principle of close collaboration as applied in agile methodologies remains in conflict with the present trends favouring the development of software in geographically distributed teams. Distributed pair programming (DPP) is one of the most important and commonplace practices in distributed agile software development. It supports two developers working on the same task in different locations. To resolve the collaboration and other issues related to DPP, researchers have proposed varied approaches to increase the flexibility of pair programming in the distributed environment to make it more widely available to the increasingly large numbers of disparate users. This dissertation first analyses and evaluates the current situation as related to DPP, including analysing social practices associated with DPP and evaluating several existing DPP tools. The final analysis of results shows that learning or knowledge sharing has been recognized as the biggest issue in regards to DPP and this is a new research area in the area of DPP. According to the unique features of the learning process associated with DPP, two high level design principles have been created for the purpose of designing new learning functions, to be applied to the existing tools: Active learning design principle; Pair formation pattern design principle. The Active Learning design principle is intended to improve the learning from active learning theory and the pair formation pattern design principle is intend to improve the learning from practical pair formation pattern in relation to DPP. XPairtise, a widely DPP tool, has been used as the basis of a framework upon which to base new learning functions. Finally, the results of the experiment show that pairs with different backgrounds can achieve better learning results by using the new tools, than if they were using the traditional tools. i

3 Acknowledgements Behind every effort is a network of people and situations which unconsciously mark you attempts. The most important thing that happened to me is my supervisor, Professor Stuart Anderson, who offers me constant support during every stage of my project. Words cannot convey how entirely grateful I am for his infinite patience. I would like to thank my parents for their loving care and for always being there for me. Without them, studying in Edinbrugh would not have been possible. I would also like to thank all my friends who have supported me throughout this year and special thanks goes to Jimmy Piao and Howard Lin. I will never forget the time that we have been together. Finally, an especially big thanks goes to my lovely girlfriend Linda who literally stood by me during this whole year. ii

4 Declaration I declare that this thesis was composed by myself, that the work contained herein is my own except where explicitly stated otherwise in the text, and that this work has not been submitted for any other degree or professional qualification except as specified. (Jingchun Hao) iii

5 To my family... iv

6 Contents 1 Introduction Motivation Objective Thesis Outline Background And Related Work Agile Software Development Globally/Distributed Software Development Distributed Agile Software Development extreme Programming Distributed extreme Programming Pair Programming Distributed Pair Programming Tools for Distributed Pair Programming Application Sharing Tools Standalone Customised Tools Analysis Distributed Pair Programming Analysis Social Practices of DPP Analysis One DPP Tool Collaborative Session User Gallery Shared Editor Role Switching Chatting Remote Selection Collaborative Testing v

7 3.2.8 Spectator Whiteboard Summary The General Requirement of DPP Tools Evaluation of Existing DPP Tools Motivation Evaluation Plan Evaluation Result Final Analysis Analysis Approach Summary Final Analysis Result Design Design Principles Active Learning Design Principle Recording Design Principle Reviewing Design Principle Pair Formation Pattern Design Principle Shared Learning Design Principle Personal Learning Design Principle Design Solutions Overall Design Solution Recording Design Solution Reviewing Design Solution Shared Learning Design Solution Personal Learning Design Solution Implementation Architecture Eclipse Plug-in XPairtise Development Methodology User Story Framework Test Driven Development Software Modules Eclipse Version vi

8 5.3.2 Client Side Implementation Model-View-Controller Design Pattern Graphical User Interface Implementation Server Side Implementation Replicated Element Replication Process Evaluation Evaluation Design Overview Pilot Study Target User Groups And Evaluation Sample Size Pairing And Randomisation Pairing Method Randomisation Method Tasks General Programming Task Project Related Task Results And Discussion Pre-Evaluation Survey Usability Evaluation Functional Evaluation General Programming Task Section One Project Related Task Section One Other Sections Final Evaluation Results Analysis Conclusions and Future Work Conclusion Future Work A DPP Tool Requirement Document 72 B DPP Tool Survey 74 C Pre-Evaluation Survey 75 D Post-Evaluation Questionnaires 76 vii

9 Bibliography 79 viii

10 List of Figures 2.1 Obstacles to adopting agile to distributed development; source: [20] Practices to address challenges in distributed agile development; source: [20] extreme Programming process model; source: [26] Cross-Workspace information infrastructure; source: [8] Sangam Eclipe plug-in; source: [13] Awareness features used in Saros; source: [22] COPPER system architecture; source: [18] XP Session Gallery XP User Gallery XPairtise Role Switch Button XPairtse Whiteboard XPairtise Evaluation result for questions 1 and Evaluation result for question Evaluation result for question Evaluation Matrix result for Saros and Sangam Systematic approach to analyze DPP Cone of Learning; source: [6] Personal learning panel and Shared learning panel An example of new functions workflow in XPairtise Recording design solution Adding records to table example Shared learning panel example Personal learning panel example ix

11 5.1 XPairtise MVC architecture XPairtise Client-Server architecture; source: [23] Test driven development state chart diagram Related test package in XPairtise MVC client side class diagram Two views in XPairtise plug-in file Shared learning element class diagram Shared learning replication process class diagram Sequence diagram for shared learning process in XPairtise Student and Software developer group s Java programming experience Usability evaluation result Completion task time General programming task questions 2, 3 and 4 results Project related task questions 2, 3, 4 and 5 results x

12 List of Tables 2.1 Additional support required in DXP Pair formation pattern Learning type and category User story creation steps An user story example XPairtise source files structure Number of problems found by number of users; source: [3] Pair formation steps General programming task question Project related task question General programming task question 1 result Project related task question 1 result xi

13 Chapter 1 Introduction 1.1 Motivation Over the recent years, global software development has become a common business practice. With the rapid changes affecting the current business environment, organisations need to develop software fast to service the demands of the Internet. Therefore, software development companies are trying to introduce the agile software development processes, including Extreme Programming, into the globally distributed environment. The idea is to seek to reap the benefits of both. However, global software development and agile software development process approaches differ significantly in their key attributes [20]. The most vital attribute of agile programming is that it emphasises the importance of collaboration between participants. The agile approach mainly relies on informal communication to coordinate the software development process. Therefore, it requires the whole project team to work together, ideally with regular face-to-face communication. In contrast, globally distributed software development cannot meet such conditions, especially with regards to the important principles and practices that arise in relation to agile software methodologies [11]. extreme Programming (XP) is considered to be one of the most important agile methodologies. It states 12 general principles or techniques that highlight the importance of flexibility and collaboration [23]. One of these principles, which is being commonly in use by all major organisations is known as pair programming. Pair programming requires two developers to work one by one, on the same computer, to design and code software together. This can provide a much more high quality of code than when a single developer is working over the same time period. However, it is impossible for two developers to work in face-to-face in a global software development 1

14 Chapter 1. Introduction 2 setting. If we taking this principle seriously, it implies that Distributed Pair Programming (DPP) is a paradox predetermined to fail. However, global economic pressures are forcing companies and organisations to employ more distributed software teams across the world; software developers are likely to be strangers located in different parts of the world, connected only via computer networks. A popular question concerning the future of the IT industry is: Can DPP be successful in global software development? Many researchers have proposed using computer tools to help achieve the same goal as that associated with pair programming practice; to allow two developers in different locations to collaborate to solve the same tasks [8]. In recent years many tools have been developed to support DPP, but to date, there is still a need for additional research into analysing the process of DPP and for enhancing the functionality of these tools [23][11]. 1.2 Objective In the current dissertation we are interested in addressing the following questions sufficiently: What are the social practice issues in DPP? What are advantages and disadvantages of existing DPP tools? What are the general requirements for DPP tools? Which social practice or area needs to be improved or further supported under the current situation of DPP? How can we improve on the areas discovered above? What kind of design principle and implementation technology is required? Can software developers really benefit from either the enhanced or new functions associated with DPP tools? The main objective of this project is to research and investigate DPP - specifically, to analyse and evaluate previous work in relation to DPP and then enhance the functionalities of existing DPP tools. Therefore, this paper will begin by analysing the social practices of DPP and evaluating several existing DPP tools. According to the final result, one area of DPP will

15 Chapter 1. Introduction 3 be identified as the biggest issue in regards to the current situation. Once identified the researcher will seek to implement new functions to improve or further support this area in relation to an existing DPP tool, XPairtise. Finally, this new functionality will be evaluated by different groups of users under different conditions. 1.3 Thesis Outline The dissertation is divided in to following chapters: Chapter 2 equips the reader with some background information to better understand the project. It provides an overview of agile software development, distributed software development and DPP. Chapter 3 analyses and evaluates the current situation with regards to DPP from social practices and existing DPP tools. Chapter 4 illustrates the two major design principles (Active Learning Design and Pair Formation Design) used to implement the new functions in XPairtise. Chapter 5 introduces the architecture of the Eclipse-plugin and XPairtise and then explains the development methodology and implementation technologies used to develop new functions for both the client and server side of XPairtise. Chapter 6 presents the experiments conducted to compare the learning results between pairs using the new tool and pairs with the old tool and concludes by discussing the results. Chapter 7 concludes the work by evaluating the findings and suggesting some future work associated with the project.

16 Chapter 2 Background And Related Work 2.1 Agile Software Development Today there are several different software development methodologies in use. These are typically grouped into two major categories: heavyweight and lightweight. Agile software development has been recognised as a representative of a lightweight development methodology and it has received significant attention in the last few years. The majority of companies worldwide are currently using agile software development methodology or planning to adopt it in the future as their main software development methodology. Agile software development methodology has had a huge impact on how software has been constructed and developed during the last decade. In traditional software development, team is of the view that by defining as high a specification as possible in the analysis stage could save considerable of time and resources for each project; however, if there are modifications that occurred in the later stages of the development process, this would result in setbacks and increase in cost, time and scope. One such example is Waterfall software development, which does not allow changes later in the implementation and verifications phases. As a result, many software projects have failed to meet their objectives. The aim of the agile software development approach is to overcome the limitations of traditional software development by considering rapid changes to the system s requirements [14]. It employs a short iterative cycle, and relies on frequent informal face-to-face communication and collaboration between teams and customers so that softwares can be delivered within budget and within time constraints. The overall procedure associated with the agile software development process is repetitive, adaptive, and minimally defined due to reliance on communication and collaboration between developers and customers rather 4

17 Chapter 2. Background And Related Work 5 than providing lengthy documentation. Agile software development methodology has the following important features and principles: 1. continuous gathering of requirements 2. frequent communication and collaboration in the team or between teams 3. early feedback from both customers and experts 4. minimal documentation extreme Programming and Scrum are the most widely used methodologies based on the above agile principles. There are also several other agile methods that have been used in the industry, such as crystal clear and feature-driven development [1]. 2.2 Globally/Distributed Software Development It is relatively normal for large software projects to involve teams working in separate locations, sometimes even more than one continent. Many factors have been forced to bring about this situation, such as the need to employ appropriate highly skilled resources for the particular software project, satisfying certain regulations imposed by local governments in foreign markets and especially the concerns for labour costs [12]. With the repaid development of global markets, these factors are not expected to diminish in the future. Rather, they are expected to increase the opportunities for globally distributed projects and this is generally referred to as globally/distributed software development. In a co-located software project, team members have frequent formal and informal interactions so that they all have a shared view of how the project will proceed. Information about projects flows freely through the network during the face-to-face interactions between team members. Miscommunication does not usually occur as the teams normally share a common language and culture. Even in the case of projects that involve understanding or communication problems, the long-standing professional and social relationships produced before prior to collaborations provide a context and history to solve these issues. However, global software development is different from the collocated software development. Global software development lacks mechanisms to coordinate work in the distributed environment compared with collocated setting. Even a small distances in the case of a distributed project can have major effects. Distributed

18 Chapter 2. Background And Related Work 6 software projects are subject to various disruptions and the following is a summary of the common issues in global software development [12]: Less effective communication: temporal distance, socio-cultural distance and geographic distance are all the reasons that make the communication less effective. Lack of awareness: people at one location tend to have little context and knowledge of what other people at other locations are doing day to day. This lack of information can lead to misunderstandings between teams. Incompatibilities: teams at different locations often use different development tools, processes and practices. These differences can cause some problems and incompatibilities. Despite the fact that global software development encompasses several issues, companies still want to achieve the same goals as those achieved by traditionally connected team. Many people have introduced the option of applying agile methodology to the global software development. The aim is to reap the benefits of both; however, agile methodology relies heavily on face-to-face interactions and global software development cannot normally satisfy such conditions. Can such differences be reconciled to support distributed agile software development? What are some challenges that arise from blending agility with global software development? The following section provides more information about distributed agile software development. 2.3 Distributed Agile Software Development The previous sections, have introduced the key concepts associated with agile methodology and global software development. Form this information it would appear to be counterintuitive to apply agile methods in global software development. However, several software companies have reported their successful experiences of incorporating agile method into global software development [20]. They have expended extra energies to overcome challenges associated with this combination in order to make agile work effectively in the distributed environment. Figure 2.1 shows the obstacles to adopting agile methods to distributed environment. The major issues can be summarised as related to differences in culture, personnel, time zones, communication and knowledge management [12]. Different software organisations have proposed various

19 Chapter 2. Background And Related Work 7 Figure 2.1: Obstacles to adopting agile to distributed development; source: [20] solutions to handle these issues. Figure 2.2 shows the general practices in place to address these issues. In general, these challenges can be successfully resolved by using appropriate processes. Figure 2.2: Practices to address challenges in distributed agile development; source: [20]

20 Chapter 2. Background And Related Work extreme Programming extreme Programming (XP) is one of the most popular agile software development methodologies. Like all other agile methodologies, XP attempts to reduce the time and cost associated with building software by having short development cycles. Today many companies from different industries have chosen XP as the basis for their project development processes and this has been proven to be a very successful endeavour. Unlike other processes, XP places more emphasis on software engineering principles and practices, especially in reference to how it addresses communication and collaboration between team members. All of these practices and principles are intended to improve software quality and responsiveness to changing requirements within a limited time period at low cost. XP provides a more advanced and efficient process for developing softwares. The following figure depicts overall process of XP. Figure 2.3: extreme Programming process model; source: [26] 2.5 Distributed extreme Programming Distributed extreme Programming (DXP) is one of the main distributed agile software development methodologies. It is an extension of traditional co-located XP; it inherits the benefits of XP and applies them in a distributed environment. However, if we consider the principles associates with XP, they would imply that DXP is a paradox predetermined to failure due to the close collaboration and communication requirements of XP. Therefore, DXP needs to relax the requirements of close physical collaboration between team members to match the distributed environment. The following table lists principles in XP that require additional support in DXP.

21 Chapter 2. Background And Related Work 9 XP Practice Planning Game Pair Programming Continuous Integration On-Site Customer Require additional support in DXP Yes, these principles in XP requires close communication and collaboration between either stakeholders or technical people. Therefore, these principles requires additional support in DXP. Table 2.1: Additional support required in DXP The table above has shown the four important practices that require additional support in DXP and this dissertation focuses on pair programming practice in the distributed environment. 2.6 Pair Programming Pair programming is not a new concept in the world of software engineering and is a widely applied practice [18]. It is also one of the most popular and important practices in relation to XP. As the name indicates, it is a style of programming that involves two programmers working in front of one computer, continuously producing an artifact (creating code, analysing requirements or developing design) [16]. In a typical pari programming session, one of the programmers acts as the driver and the other programmer acts as the navigator. Both of them share a common working environment, one mouse, one keyboard and one monitor. The driver has control of the keyboard and mouse, so the major responsibility of driver is in typing codes at the computer or writing down a design document. The navigator has many jobs but the primary role is to observe the work of the driver and prevent any tactical or strategic defects in code. Whenever the navigator thinks there are errors present or alternative solutions, two programmers will exchange their opinions or ideas immediately. Both programmers actively participate continually in the development and every line of code is written with both partner s presence and agreement. The relationship in pair programming is very flexible and active. Both programmers may choose to exchange their role at any time during the development [2]. Pair programming is not only used during coding phases, but throughout all the stages of development process, including the debugging and testing. It emphasises real-time problem solving and quality control. Pair Programming is particular useful for complex tasks. The more complex a task is, the greater the need for pair programming. The key attribute of this technique is that the

22 Chapter 2. Background And Related Work 10 two programmers should sit side by side and share the computer screen, keyboard and the mouse to produce the artifact. Previous controlled studies have shown that pair programming has many benefits over solo programming. The following list is a preview of the benefits of pair programming that most developers have seen, experienced [28]. Quality: Two developers produce code with less defects. The navigator continuously reviews the code and design, which enables early defect detection and removal during the pairing session. Time: Two developers use less time to produce the same quantity of code when compared with a single developer. Teamwork: Pair programming can improve the relationship between developers, which builds trust and improves teamwork. Knowledge transfer & Learning: Pair programmers can share and learn knowledge from each other during the pairing session. Job satisfaction: The intensive collaboration helps job retention for employees. As well as its popularity in the business world, pair programming has also been applied within the field of education. Studies have shown that it is an effective method in teaching software engineering courses at university. Researchers have found that it provides a more active learning and socially interactive environment for students, which leads to increased interested and confidence in computer science. The necessity that both members of the pair must be co-located is a major drawback and the principle limitation of pair programming [10]. With the increasing demand for developing software globally, pair programming remotely is becoming a trend. This trend towards DPP conflicts with the co-located requirements of pair programming. However, I believe with the correct process and additional support and tools, DPP can achieve the same goals and produce the same advantages as co-located pair programming. 2.7 Distributed Pair Programming Pair programming in DXP is referred to as DPP. In this programming style two geographicallydistributed programmers collaborate on a given implementation task. In other words,

23 Chapter 2. Background And Related Work 11 DPP enables developers to collaborate from different locations to work on the same project. Compared with co-located pair programming, DPP has raised a new set of issues, especially in the area of communication and collaboration. Therefore, DPP requires additional support to overcome these challenges. One of the dominant solutions is to use powerful tools. Figure 2.4 illustrates proposed computer software that could be used to support DPP. Figure 2.4: Cross-Workspace information infrastructure; source: [8] The figure shows a cross-workspace visual, audio, and manual information infrastructure, which could enable programmers located in different places to collaborate over the Internet. The most common cross-workspace infrastructure nowadays is a shared common Internet-based editor [8]. 2.8 Tools for Distributed Pair Programming Many contemporary researchers have proposed different style of tools to support DPP in order to achieve the same benefits as co-located pair programming. These existing tools either use an application sharing approach to enhance an existing development environment or integrate various groupware features, such as shared editors, to a customised tools. The application sharing approach lacks process support and is therefore not collaboration aware. Some features of pair programming are unsupported by customised tools, which limit pairs ability to accomplish their work successfully [16] Application Sharing Tools The support for DPP by using an application sharing approach can be classified into two groups. The first group relates to general purpose tools, which are not dedicated solutions for DPP, such as conferencing applications (e.g. Microsoft NewMeeting) and desktop sharing solutions. The two examples from the early days of this group are JAZZ and MILOS [5] [17]. Both of these are application sharing systems that make

24 Chapter 2. Background And Related Work 12 use of existing IDEs (e.g. Eclipse) to support the whole XP life cycle. JAZZ can support synchronous pair programming and allow users to stay aware of co-workers; it can also be used to invite users to join a pair programming session. MILOS has a similar functionality to JAZZ. However, the workflow of pair programming has not been integrated into these tools, and thus both of them lacks the capability to support flexible pairing and role switching. TUKAN and Moomba were produced in order to bridge this gap [24] [21]. TUKAN not only provides a shared editor for manipulating code together, but also allow developers to highlight critical code using remote selection. Moomba extends TUKAN to support developers to use a shared Java editor inside specific Java IDEs. However, both tools did not gain high popularity due to the special ENVY environment used in these tools. The second group is dedicated solutions for DPP. As Eclipse is becoming one of the most popular and common development environment; as a result, more and more researchers have proposed DPP tools based on the Eclipse environment. Furthermore, the Eclipse community provides an Eclipse Communication Framework (ECF) which is particularly aimed at the integration of a collaboration infrastructure with IDE. This special framework allows developers to work on the same file remotely in real time. As a result, most of tools in this group are Eclipse plug-ins. One of the earliest Eclipse plug-ins for DPP was the Sangam system [13]. It provides a user interface specifically designed for DPP and allows two developers to synchronise their development environments. The Sangam plug-in is designed based on an event-driven principle. When the driver undertakes some tasks in Eclipse (e.g. writing more codes into one file), then the plug-in at the navigator s end will be notified to perform the same task. Therefore, both developers alwyas see the same result at all the times. Figure 2.5 shows an Eclipse workspace after installation of Sangam. Figure 2.5: Sangam Eclipe plug-in; source: [13]

25 Chapter 2. Background And Related Work 13 The fundamental idea behind this plug-in is that the commands are replicated between the connected developer s editor when they are participating in the same pair programming session. Subsequently, the majority of DPP tools follows the same idea. Saros, another Eclipse plug-in, developed additional functionality to support the drivernavigator interaction in a DPP session [22]. It provides more support awareness because the Saros team thinks awareness support in the context of software reflects how an application is able to convey such information regarding a collaborative partner who is not co-located; it especially emphasises awareness from the navigator s perspective, such as allowing the navigator to point at relevant pieces of code. The following figure highlights the features necessary to create awareness of participants in Saros. Figure 2.6: Awareness features used in Saros; source: [22] XecliP and XPairtise are the other two Eclipse plugins used for supporting DPP. These two plugins are developed in competition by separate team within one research group. This project uses XPairtise as the base framework for implementing new functions, because it provides an efficient and simpler solution Standalone Customised Tools One of the DPP tools in this category is the COPPER system [18]. It is a typical groupware tool developed to address the issues raised in DPP. COPPER has three main components: Collaborative editor, User and Document Presence (U&DP) service, and

26 Chapter 2. Background And Related Work 14 Audio systems. The whole COPPER system is based on Client-Server architecture. The architecture of the system is presented in the following figure. Figure 2.7: COPPER system architecture; source: [18] The IM&P (Instant Messaging and Presence) module is used to distribute messages among users and maintains user and document presence. The module is implemented by the open source Jabber IM&P server. An audio module is used to establish an audio communication channel so that the two developers can have a conversation while collaborating. Editor module is part of the collaborative editor which implements a turn-taking synchronous editor so that two developers can work on the same artifact in a distributed environment. However, the requirement of the same resolution for two developers monitors and the low display refresh rate of the COPPER system has made it hard to use in a distributed environment. COLLECE is another groupware system used to support DPP [7]. It was developed using Java technology. In addition to the set of tools provided by the COPPER system, it adds a new functionality called the session panel. The session panel shows more information about the partner, including name and photo. In conclusion, the two approaches have their both advantages and disadvantages. In the recent years, there is a trend towards integrating DPP tool in professional IDEs, and Eclipse IDE dominates such development. The following chapter explores more details about DPP in reference to the currently available tools.

27 Chapter 3 Analysis Distributed Pair Programming 3.1 Analysis Social Practices of DPP The social practices of traditional pair programming can normally be divided into the following [23]: 1. Pair formation: This is the first step in pair programing. Before a pair programming session starts, one developer needs to first pick up a task and looks for another developer to form a pair. 2. Working together: After the pair has been formed, the two developers can start to produce an artifact together. One developer acts as the driver while the other developer acts as the navigator. Both can communicate their opinions either by talking through the shared display or drawing the solutions on a sheet of paper. During the session, the two developer can switch their roles at any time they want. 3. Knowledge sharing and pair learning: Learning could happen either during pair programming session or at the end of session. It is particularly useful when the pair involves an experienced developer and a novice developer. The novice developer could learn best practices from the experienced developer and the expert will learn by making his knowledge explicit. Furthermore, this social practice has been extended further to allow more than two developers to participate in the session to share or learn knowledge. 15

28 Chapter 3. Analysis Distributed Pair Programming Debug and testing: Pair programming is not only the practice for coding, two developers could also use tools (e.g. JUnit) to debug a software program. DPP wants to achieve the same goal and benefits as co-located pair programming and thus has the same social practices. However, DPP has raised new issues for the interaction between two developers due to the pair being located in different locations. Some social practices that can be easily achieved in co-located pair programming could require additional effort in distributed pair programing. The following list explains various issues in social practices of DPP: 1. Pair formation: In the distributed environment, formation of the pair is much more challenging than in the co-located environment. It is hard to detect the current status of remote users and time shifts may mean developers can be unaware of each other. Moreover, user story, task cards and project artifacts need to be available as shared objects so that two separate developers can both access them. 2. Working together: In a co-located environment, two developers typically share one computer screen, a keyboard and a mouse which allows them to produce an artifact in front of a computer. In a distributed environment, it is impossible for two developers to share one computer screen. Two developers must user their own computers to work together. Therefore, inconsistency issues may arise during the pairing session, especially in case where two developers modify in the same place simultaneously. Synchronisation is another important issue in DPP, when the driver writes or changes some of the contents in his editor, the navigator should see the same result on his editor. In addition, it is difficult for developers to switch their roles explicitly during the DPP session. 3. Knowledge sharing and pair learning: In a co-located environment, two developers communicate their opinions face-to-face. It is extremely easy to share or learn knowledge from other developers in a co-located environment. This situation does not occur in DPP. Even basic communication between developers can not achieve easily without a support tool. So, knowledge sharing and pair learning will be minimised in a distributed environment. Moreover, learning knowledge is not only limited by verbal communication, some knowledge is better illustrated through the drawing of diagrams on sheets of paper and this is even harder to manage in a distributed environment.

29 Chapter 3. Analysis Distributed Pair Programming Debug and testing: This has similar issues as working together social practice has. When the driver starts using tools to debug the program, the same effect should happen on the navigator s side. The concern with such an approach is how best to keep both driver and navigator devices synchronised. According to the above analysis, DPP issues can be summarised into five areas: (1) Coordination, (2) Communication, (3) Learning, (4) Coding Together and (5) Testing. 3.2 Analysis One DPP Tool XPairtise is an open source Eclipse plugin that developed on a university course. It is one of the most popular DPP tools in the industry. XPairtise provides 6 main functionalities: (1) Shared editor, (2) Project synchronisation, (3) Shared test execution, (4) Chatting tool, (5) Shared whiteboard and (6) User management [23]. This section analyses the various approaches of XPairtise to solving the issues associated with DPP Collaborative Session Collaborative session is used to refer to a DPP session in XPairtise. All the interactions in XPairtise are based on this concept. When one developer feels that there is a need to implement pair programming for a task, then the developer needs to first create a new collaborative session in the XP Session Gallery (see Figure 3.1). Once the session has been created, it will be visible to all online users who are currently connected to the XPairtise server. So, online users can join the session as either navigators or drivers if that role is available (no other users have selected that role). Thereby, it provides an opportunity to start a session in the role which from user s perspective is best suited to the task requiring completion. Collaborative sessions solves the coordination and communication issue in regards to distributed pair formation to some extent. Figure 3.1: XP Session Gallery

30 Chapter 3. Analysis Distributed Pair Programming User Gallery User gallery is the place used in XPairtise to show all the currently registered developers (see Figure 3.2). It allows developers to view other developers who are connected to the XPairtise server. It shows the users current status which helps developers to form new collaborative sessions in XPairtise. Moreover, the user gallery provides an invitation function which allows users to invite other online users to join the session. This function allows developers to find the most suitable expert for assisting in the task. Figure 3.2: XP User Gallery Shared Editor Shared editor is the core component or function in DPP. After the pair has been formed, shared editor allows the driver and navigator to work cooperatively. All driver s actions can also be performed on the navigator s side (e.g. opening and closing file, editing and highlighting text). So the driver s site is always consistent with the navigator s site. Shared editor not only solves the coordination issues in social practices, but also improves the communication issues in DPP Role Switching The role change button in XPairtise allows two developers to switch their roles during the development process in DPP providing a clear notification to each other (see Figure 3.3). The driver or navigator requests a change of role by pressing the role change button and if the other side agrees, then the role is switched between the developers. This role switch cannot be forced by one side. Figure 3.3: XPairtise Role Switch Button

31 Chapter 3. Analysis Distributed Pair Programming Chatting XPairtise employes two types of chatting: textual communication and audio communication. Textual communication can be used within a collaborative session or with all users who are currently connected to the XPairtise server. Therefore, XPairtise provides two different chat tabs. One is used to chat within a session and the other tab is used to chat with online users. Skype has been integrated into the XPairtise to establish audio communication Remote Selection Remote selection is used for the navigator to select or highlight specific code. However, the navigator can not modify the codes. This allows the navigator to focus the driver s attention on specific parts of the source code and thereby begin a communication Collaborative Testing XPairtise supports basic collaborative testing. When the driver starts tests or runs some test scripts, the same test begins on the navigator s site. Collaborative testing has been implemented in the same way as that used on shared editors. However, this function was not included in the official release as there were still some issues regarding adequately synchronising the effects of external factors on the execution of the programs being tested Spectator XPairtise creates a new type of role referred to as the Spectator. The spectator can join an ongoing pair programming session and watch the interactions between driver and navigator. This application is mainly used to widen the audience of a DPP session. Spectators can not change the code. The main responsibility of this role is to participate in the session and acquire knowledge. Therefore, it is particularly useful for novices joining a DPP session between two experts. This type of DPP is called one-to-many pair programming pattern or multiple-developer pair.

32 Chapter 3. Analysis Distributed Pair Programming Whiteboard White board is a unique feature of XPairtise (see Figure 3.4). It allows two developers to communicate by drawing diagrams through a shared editor. This is especially useful when there are complicated tasks which are not easily explained by either textual or audio chat. It improves the learning and knowledge sharing between developers in a distributed environment. Figure 3.4: XPairtse Whiteboard Summary This section introduces and explains the various approaches that XPairtise uses to handle issues in DPP. Figure 3.5 presents a combination of all the functions mentioned above in relation to XPairtise. Figure 3.5: XPairtise

33 Chapter 3. Analysis Distributed Pair Programming The General Requirement of DPP Tools Based on the above analysis, we have identified that a DPP tool needs to satisfy the following basic requirements or provide the following functionalities: 1. Workspace awareness: A DPP tool should allow developers to be aware of the presence and state of other developers and documents. 2. Shared editor: A source code editor not only provides the common editor functionalities, but can also synchronise projects between developers in their own development environment. The shared editor can only be modified by the driver but not by the navigator. 3. Shared file repository: All project related documents or files should be shared between developers during a DPP session. A central repository should contain all these files. It should provide mechanisms to allow developers to request and obtain shared files. 4. Supporting of role changes: A DPP tool should provide the function to allow two developers to easily switch their role without any confusion. 5. Supporting gesture from navigator: A DPP tool should provide a second pointer for the navigator so that navigator can highlight certain parts in the editor which can improve the communication between two developers in distributed environment. 6. Supporting either textual, audio or video communication: Either textual, audio or video communication should be provided via DPP so that developers can easily communicate with each other in a distributed environment. 7. Supporting learning and knowledge sharing: Learning and knowledge sharing can be difficult in DPP even with additional support from communication. Therefore, the DPP tool should provide certain functionality to support learning and knowledge sharing. The above list only shows the most important and basic requirements for a DPP tool. The complete requirement document for a DPP tool is shown in Appendix A.

34 Chapter 3. Analysis Distributed Pair Programming Evaluation of Existing DPP Tools Motivation Many software engineering principles and ideas have been criticised for claiming benefits without adequately meeting the requirements for analysis or evaluation. For example, advocates of object-orientation have made claims regarding improved quality and productivity. However, there is no systematic analysis or evaluation to illustrate whether these claims are true [25]. Nowadays, software development suggests conducting evaluation early in the analysis stage which could improve the quality and accuracy of the final analysis results. These criticisms and suggestions are valid, and provide a strong motivation by which to evaluate existing DPP tools in my analysis. Perhaps a more important reason for this evaluation is to obtain a broader view of DPP and can also reflect my analysis to some extent Evaluation Plan The evaluation of existing DPP tools involves two steps. The first step in the evaluation is to identify two popular DPP tools that used by software engineers and the second step is to find out issues in DPP from the developers perspective. The survey was conducted by 10 experienced software engineers. Moreover, all of them have completed some tasks by using DPP tools for global software projects. The complete survey is shown in the Appendix B. In the second step, 4 software engineers are randomly selected from among the 10 software engineers. The 4 participants were grouped randomly into two pairs and then asked to evaluate one of two DPP tools identified during the first step. Each pair was then to produce an artifact by using the tool that they were evaluating. The evaluation criteria they use is the complete requirement document for DPP tool described in section Evaluation Result The following figure summarises the result for questions 1 and 2 of the survey.

35 Chapter 3. Analysis Distributed Pair Programming 23 Figure 3.6: Evaluation result for questions 1 and 2 According to the above figure, we can see that the majority of engineers have lots of experience in using different DPP tools. 70% of the software engineers have used DPP tools for more than twice. More than 50% of the software engineers have used different DPP tools in projects. The next figure shows the result for question 3. Figure 3.7: Evaluation result for question 3 XPairtise, Saros and Sangam are the top three popular DPP tools in this evaluation. Because XPairtise was already analysed by the researcher in section 3.2, the participants were asked to evaluate Saros and Sangam. Figure 3.8 shows the result for the

36 Chapter 3. Analysis Distributed Pair Programming 24 last question in the survey. Figure 3.8: Evaluation result for question 4 Most developers believe that learning needs to be improved in the current DPP tools. The following are some of replies that I received: Current tools do not support anything related with knowledge sharing or learning process in DPP and in fact this has lots of problems. Without face-to-face communication, it is quite easy to miss or forget some important knowledge learned during the DPP session and it is hard to talk to the paired developer again for the same question due to the location issue. In distributed environment, it is hard to share knowledge on complicated tasks. An easy and friendly function in the tools needs to support this. The final result from the second step of the evaluation is presented in figure 3.9. The main focus of the second step is to explore more about the weaknesses associated with the DPP tools currently available regarding the analysis in the previous sections. Two pairs solved the real task by using different tools. The evaluation matrix is mainly based on a subjective estimation values of a tool s usefulness as provided by the two pairs. The matrix indicates that learning is also a big issue with Saros and Sangam. Both pairs were dissatisfied with the learning functionality of both tools.

37 Chapter 3. Analysis Distributed Pair Programming 25 Figure 3.9: Evaluation Matrix result for Saros and Sangam 3.5 Final Analysis Analysis Approach Summary The main goal of this chapter is to analyse the current situation in relation to DPP. The following figure summarises the systematic approach applied to perform the analysis in this dissertation. Firstly, DPP was analysed from both a theoretical and practical perspective. Then, based on this analysis, a complete requirement document for DPP tool was generated. Finally, two existing DPP tools are evaluated based on this requirement document. Each step in this systematic approach explores DPP in depth. Figure 3.10: Systematic approach to analyze DPP

38 Chapter 3. Analysis Distributed Pair Programming Final Analysis Result According to the first steps as applied for the analytical approach, knowledge sharing or learning is one of the main social practices applied in DPP. The second step has shown that XPairtise, a DPP tool, supported learning using Spectator and Whiteboard. Lastly, the final evaluation results showed that learning is currently the most important issue related to DPP. The software engineers that took part in the evaluation believed that learning is one of the area requiring improvement in regards to existing tools. Although DPP tools has been shown to have several mechanisms that support learning, it is still inadequate in the opinions of the software engineers consulted for the evaluation. Most of researchers working in this area focus on providing tool functionalities on supporting communication and coordination but not learning. Some researchers have argued that improving communication can enhance learning in DPP. In fact, this is not a widely accepted opinion. Learning and communication are two different concepts; even we have perfect communication in the distributed environment, two developers could also experience learning issues. Hence, improving communication does not mean that learning is automatically improved. DPP allows organizations to locate the most suitable expert to solve complicated problems relating to software projects worldwide. Therefore, companies apply these to complete critical tasks when carrying out projects in the past. At the present time, most companies are using it to share knowledge between developers, especially in cases involving cooperation between an expert and a novice. Furthermore, this approach has also been applied to the field of education. Several universities use DPP in their software engineering courses. Most students complete group work by using DPP tools; meaning they do not need to meet each other at the university in order to complete group work [10]. As a result of the above analysis and from understanding the current trends in DPP, the following chapters of this dissertation focus on improving learning area in DPP. XPairtise is used as framework to implement this new design to support learning.

39 Chapter 4 Design The previous chapter revealed that learning is the area that needs to be improved on most in regards to the currently available DPP tools. This chapter explains design principles and solutions that can be used to improve or enhance learning based on XPairtise. 4.1 Design Principles This sections introduces two design principles to improve learning in DPP: (1) Active learning design principle and (2) Pair formation pattern design principle. The active learning design principle considers how best to improve learning based on active learning theory and the pair formation pattern design principle is concerned with how best to enhance learning from a practical pair formation pattern. The overall design is a combination of active learning theory and practical pair formation patterns Active Learning Design Principle There are two major learning models: Active learning and Passive learning. This design principle is based on the active learning model. So, what is active learning? In simple terms, it involves learners actively and directly engaging in the learning process itself. Active learners not only receive knowledge or information verbally and visually, but also participate in discussions and become involved in actual tasks. In general, receiving, participating and doing are a group of actions taken in relation to the active learning model. The opposite of active learning is passive learning. One example of passive learning is students listening to the instructor and looking at the occasional 27

40 Chapter 4. Design 28 slides in a classroom. Several experiments show that passive learning generally leads to a limited retention of knowledge by learners [27]. The following figure shows the difference between active and passive learning for remembering knowledge. Figure 4.1: Cone of Learning; source: [6] The learning process in DPP involves active learning. Two developers receive each other s opinions and then both of them participate in the discussion to finalise a solution. If we consider, based on the above definition of active learning, receiving and participating action are already included in current DPP learning. DPP tools have already provided several communication mechanisms to support receiving and participating. Actively doing action has not yet been implemented in distributed pair programming. Figure 4.1 shows three doing actions: (1) giving a dramatic presentation, (2) simulating the real experience and (3) doing the real thing. However, these are not applicable to DPP. DPP requires real-time learning, and these actions do not support real-time learning and so it is not reasonable to do these actions in a distributed environment. Therefore, I decided to create unique design principles which involve lots of learning contributions and are based on active learning theory. The two design principles are: recording design principle and reviewing design principle.

41 Chapter 4. Design Recording Design Principle Recording design principle does not mean recording a DPP session. As shown in the figure 4.1, watching a demonstration, movie or videotape qualifies as passive learning. Passive learning is not encouraged in pair programming and this is why the majority of DPP tools do not serve this function. In addition, pair programming requires realtime learning. However, recording a session and replaying it later does not function as real-time learning. So what does this recording design principle mean? It basically means to record all valuable knowledge learnt during DPP session by users. Simply put, learners (driver and navigator) actively record (kind of doing action) knowledge using DPP tools. Recording action in this design can improve the understanding and impression of knowledge, which can then enhance the learning process and result in DPP Reviewing Design Principle Recording design principle focuses on the learning process during a DPP session. In contrast the reviewing design principle focuses on improving learning at the end of a DPP session or even after sessions. In fact, the reviewing design principle is an extension of the recording design principle. It encourages drivers and navigators to review recorded knowledge at the end of DPP session. Reviewing action can not only enhance understanding of knowledge acquired from the DPP session, but also consolidate that knowledge. Moreover, it provides another opportunity for both drivers and navigators to resolve uncertain issues in this session. Reviewing action in this design principle can achieve the same benefit as simulating real experiences action, as shown in Figure 4.1. When developers review knowledge, they typically think about related issues and questions, which lead them to act as redo the tasks. Therefore, the reviewing design principle can improve the learning further in DPP Pair Formation Pattern Design Principle There are different pair formation patterns in pair programming. Each pattern has advantages and disadvantages, so it is hard to measure which pattern is the best. The normal method is to match the makeup of the pair to with the task at hand. Table 4.1 shows the four general pair formation patterns.

42 Chapter 4. Design 30 Driver Navigator Pair Pattern 1 Expert Expert Pair Pattern 2 Novice Novice Pair Pattern 3 Expert Novice Pair Pattern 4 Novice Expert Table 4.1: Pair formation pattern One question that might be asked is: Why do we need to consider pair formation patterns to improve learning in DPP? The answer is that different pair formation patterns have a distinct impact on learning in DPP. Learning is a process of communication between people. People are a major factor in learning. Different pair formation patterns involve people with different levels of experience. Hence, learning processes and learning methodology between different pair formation patterns are considered to be distinct and unique. This is especially important for DPP because developers cannot communicate with each other face-to-face. As a result, this design principle is considered to improve learning baed on different learning styles in regards to pair formation patterns. Based on the pattern, this design principle can be further divided into two design principles: the shared learning design principle and personal learning design principle Shared Learning Design Principle As the name indicates, this design principle focuses on providing a shared learning platform for both drivers and navigators. Both of whom are studying knowledge simultaneously during the pair programming session. The pair may need to invite another expert to join the session and to answer some questions occasionally. A shared learning platform could also be very useful when an expert wants to share knowledge between developers. For example, it can be used as a platform for experts to make additional comments or suggestions for the novice. In addition, when developers join a session late and they still want to check knowledge that has already been discussed, then this design principle can help them to check and study the knowledge shared by other developers in DPP session.

43 Chapter 4. Design Personal Learning Design Principle Compared with the shared learning design principle, the personal learning design principle focuses on personal learning during DPP session. It can be regarded as the opposite of shared learning. One developer acquires knowledge or skills from another developer who has already mastered the knowledge. For instance, the novice can always adopt the best programming practices from the expert in the DPP session. This design principle is aimed at helping developers to identify gaps in personal knowledge. 4.2 Design Solutions This section firstly illustrates the overall design solution based on the design principles mentioned above and then explains individual solutions respectively Overall Design Solution The overall design solutions are based on XPairtise. The new functions are added as several parts to XPairtise. The Figure below presents the two core components of the overall design solution: Figure 4.2: Personal learning panel and Shared learning panel XP Personal Learning and XP Shared Learning are two panels and they are added in the same place as the XP Session Gallery and XP User Gallery in XPairtise. Each

44 Chapter 4. Design 32 panel contains one recording table and two buttons. In the overall design solutions, the two panels play the core functionality in improve learning. All developers can use these two panels simultaneously. Figure 4.3 displays new function s workflow in XPairtise. Figure 4.3: An example of new functions workflow in XPairtise

45 Chapter 4. Design 33 The following sections explain each component in terms of each design principle and its functionality Recording Design Solution The table contained in the two panels is the solution derived from the recording design principle. The main function of the table is to record knowledge learnt during the DPP session. Figure 4.4 shows the table. Five columns in the table are used for recording purpose. Figure 4.4: Recording design solution 1. Type and Category: The two columns are used to record types and categories of learning. In the solution, there are two types of learning: general programming learning and project related learning. General programming learning involves all types of general programming knowledge. Project related learning only refers to that knowledge related to one particular project. Each category further divides each learning type into several groups. The following table displays different combinations of type and category. Type Category Type Category General Concept Project Information General Performance Project Dependency General Coding Standard Project Architecture General Data Structure Project Debug General Algorithm Project Other General Testing General Programming General Software Design General Other Table 4.2: Learning type and category

46 Chapter 4. Design File Name, Start Line and End Line: These three columns are used to record files and codes related with learning so that users can easily find and refer to them later. It connects the general knowledge and the project together. To ensure this recording design solution does not interrupt workflow between developers in distributed environment. The solution for adding records to table is convenient and automatic. Users only need to highlight codes which related with learning and then right-click. In context menu, users can select either shared learning or personal learning and finally choose learning type and category (see Figure 4.5). After that, record will be added to table automatically with corresponding column values. Figure 4.5: Adding records to table example

47 Chapter 4. Design Reviewing Design Solution As shown in Figure 4.4, the other three columns in the table are designed by the reviewing design principle; the main functionality of this solution is to allow developers to review and consolidate knowledge: 1. Reviewed: This column is used to show whether the learning recorded in the table has been reviewed or not. The reviewing design principle encourages developers to review knowledge at the end of DPP session. When developers first add their learning record to the recording table, the reviewed column shows false. After developers have reviewed the record, they can then change the column to true. In addition, developers can sort the column so that they can check unreviewed knowledge quickly. 2. Further Learning: Developers may require further study in order to master and consolidate knowledge learnt during DPP session and this column is used for this purpose. It identifies whether the learning record requires further study. 3. Notes: The last column in the recording table allows developers to enter additional information. For example, if a learning record requires further study, then developers can enter related references in this column. Another important solution based on the reviewing design principle is the PDF button at the bottom of each panel. After developers finish reviewing knowledge, they can generate a PDF that contains a summary of records and corresponding codes in a recording table for the session. This reviewing solution provides opportunity for developers to review and consolidate knowledge. Furthermore, a PDF document can be used as a summary for a code review meeting to share knowledge with other team members. It enables the connection of DPP with other activities in the software development methodology Shared Learning Design Solution The XP Shared learning panel is a solution based on shared learning design principle. The recording table is contained inside a shared learning panel. The unique function of this solution is that whenever one developer adds a learning record to the table, the same learning record will be added to the tables of all the other developers table currently working in the same DPP session. In other words, the learning record is

48 Chapter 4. Design 36 shared amongst all the developers sharing the same session on the panel. Furthermore, whenever a developer modifies or deletes records in a table, the same effect will show on other developers tables. A shared learning design solution allows developers to record uncertain knowledge during a DPP session and discuss the information later in order not to interrupt the other developer s workflow. This is especially important in a distributed environment as people may have different cultures and habits. In addition, this unique sharing ability allows that any latecomers to see any knowledge that has already been discussed in the session. The following figure shows an example of how when one developer adds a record, the same record is then shown in another developer s record table. Figure 4.6: Shared learning panel example Personal Learning Design Solution XP Personal Learning panel is a solution based on personal learning design principle. It has similar functionality to a shared learning panel. The major difference is that learning records can not be shared among developers during the same session. The main functionality of this solution is to build a personal learning list (see Figure

49 Chapter 4. Design ). The common scenario uses this functionality is between expert and novice pairs. The novice can add knowledge learnt from the expert into the personal learning panel during the DPP session and the expert then does not need to record the knowledge. Figure 4.7: Personal learning panel example

50 Chapter 5 Implementation The intended outcome for the above design is the development of new functions in XPairtise. Upon implementation, most of these affirmations remain true. This chapter describes the development methodology used to implement the new functions, discussing the software modules encountered during implementation. 5.1 Architecture XPairtise is implemented as a plug-in for Eclipse. The Java programming language provides all the necessary functionalities to implement DPP tools. The major advantage of integrating XPairtise with Eclipse is that developers will be able to easily produce artifacts in distributed environment as Eclipse is becoming the most popular development environment. Therefore, details about Eclipse plug-in architecture and XPairtise architecture are outlined in this chapter Eclipse Plug-in Eclipse is an open source platform which is primarily known as a development environment for many software projects. One unique feature of Eclipse is that it is designed so as to be easily and infinitely extended by third parties. Eclipse is a small extendable kernel containing a plug-loader surrounded by hundreds of plug-ins [4]. In reality, Eclipse is not a single program, but rather a Java program combining the functionalities of various plug-ins [9]. Each plug-in does a different number of tasks: compiling, debugging, or supporting distributed pair programming. Therefore, Eclipse is a platform which is comprised of hundreds of plug-ins and its functionality is readily 38

51 Chapter 5. Implementation 39 extendable by using new plug-ins. A plug-in has been recognised as the smallest unit in Eclipse and it can therefore be developed and deployed separately. Eclipse plug-in consists of all codes and resources in a Java Archive (JAR) library. There are two important files in each Eclipse plug-in: a manifest file (manifest.mf) and a plug-in file (plugin.xml). The manifest file is used to describe what resources are needed during the Eclipse runtime in order to active plugin and the plug-in file describes all the information about this plug-in itself. There are two vital considerations when developing an Eclipse plug-in: extension and extension points. These two concepts are closely related with each other. Extensions in plug-ins extend the functionality of the platform. For each extension, several extension points have been defined to support the system s behaviour. In essence, extension points are declared as a contract that only accepts those extensions which abide by that contract. For instance, Editor is an extension point in Eclipse and developers can make their own plug-ins to extend to the editor. Extension and extension points are all declared in the plug-in file (plugin.xml). In summary, each plug-in provides a new functionality or service by using an extension mechanism in Eclipse based on other available plug-ins. Plug-in Development Environment (PDE) provides tools to create, develop, build and deploy Eclipse plug-ins XPairtise XPairtise employs two types of architectures: Model-View-Controller (MVC) architecture and Client-Server architecture. MVC architecture is used for all user interactions and Client-Server architecture is applied in the network model. MVC is the main design pattern enforced by Eclipse architecture. By adopting this design pattern, XPairtise has low coupling between modules. MVC architecture is particularly useful in GUI-type applications. Its main objective is to separate data logic from presentation logic so that modification to one part does not influence the overall application. In MVC, the view presents the result or data to users, the model deals with data storing and the controller connects the model and view, receiving user input and initiating actions on model objects. XPairtise has a collection of model/view/controller triplets at different levels, each responsible for different tasks. The MVC architecture mapped on XPairtise is depicted below:

52 Chapter 5. Implementation 40 Figure 5.1: XPairtise MVC architecture If we consider XPairtise from the network perspective, this plug-in makes use of Client-Server architecture. Clients connect to a single central sever to exchange data. XPairtise does not support client to client connection without a server in the middle. The Java Messaging Service (JMS) has been chosen as the underlying infrastructure to support communication between XPairtise clients and the XPairtise server. However, JMS is only a specification, the concrete implementation is the ActiveMQ library. It is the most popular and stable open source messaging service implementation. Figure 5.2 shows the conceptual client server architecture of XPairtise. Figure 5.2: XPairtise Client-Server architecture; source: [23]

53 Chapter 5. Implementation Development Methodology As introduced at the beginning of this dissertation, a software development methodology refers to the method that is used to plan and control the process of developing a software. A number of different software development methodologies have evolved over the time, each has its own strengths and weaknesses. Whether the methodology is suitable depends on project information, such as technical, organisational and team considerations. Compared with other software projects, this project is quite unique. No software development methodology is really suitable for this project. Therefore test driven development (software engineering principle) was taken from XP and user story (project management principle) was taken from Scrum to form the unique development methodology for this project. This section describes how these principles assisted in the successful completion of this project User Story Framework Unlike traditional software development methodology, agile methodology expresses software requirements as user stories; this is one of the most widely used techniques in the case of agile processes, especially in Scrum. User story has been recognised as an effective approach to apply to all time constrained projects and is a great way to begin introducing agility to projects. User story is not a highly documented requirement but rather a narrative text that describes interactions between users and software. The reason that the decision was taken to use this technique was largely because it provides clearer requirements than use cases. Moreover, user story has priority so that it is possible to start working from core components and this is important for this project due to the pressure of the deadline. Table 5.1 shows steps to create user stories in this project: Step 1 List all learning issues from design stage Step 2 Convert these learning issues into narrative text based on design solutions Step 3 Create user story for each learning issues from step 2 Step 4 Divide high level user story into low level separate tasks Step 5 Generate acceptance tests for each user story, including test criteria Table 5.1: User story creation steps Often people think that step 3 is the final step for completing the user story. How-

54 Chapter 5. Implementation 42 ever, a complete user story should also contain low level tasks and corresponding test cases. Table 5.2 shows a complete user story example. On the front of the card, it shows the main description of the user story. The description clearly mentions the roles, actions and benefits of the user story. Another important part of the front card is that it shows the user story s priority and the estimated time to finish the user story. These two pieces of information can help me to track the project s progress. Front of User Story Card 1 As a driver or navigator, I want to record knowledge occurred during the distributed pair programming session so that either driver or navigator can check or review these knowledge later. Priority: High (Must) Estimate: 13 Back of User Story Card 1 Task 1: create the recording table Task 2: create the new view in XPairtise Task 3: connect new view with recording table Task 4: create the adding record functionality Acceptance Test: Description: check that both developers can record knowledge during the pair programming session. Setup: both developers join the same distributed pair programming session. Instruction: One of the developer adds a learning recording to the table. Expected Result: the table should contain the corresponding learning record. Table 5.2: An user story example The back of the card displays low level tasks and acceptance tests. By writing the user story in this format, it is possible to quickly match the design with the implementation in case requirement details have been forgotten when the time comes to implement it. In total, there are six user stories and all of them are displayed on a user

55 Chapter 5. Implementation 43 story wall. There are four stages on user story wall: Ready for development, Developing, Testing and Finishing. When the researcher starts working on a user story, it is moved from ready for development to developing. The testing and finishing stage follows the same process. In conclusion, the user story framework helps clarify the requirement and track the project s progress more accurately Test Driven Development The main procedure for implementation of this project is to add new functions to the existing XPairtise project while preserving the rest of the system s behavior. These new functions reuse existing XPairtise architecture and codes. The original XPairtise project has already contained 16 test packages to test each of the functions implemented. Based on these facts, the test driven development approach was found to be suitable for developing new functions for this project. The following state chart diagram illustrates test driven development steps in this project: Figure 5.3: Test driven development state chart diagram Compared with traditional test driven development, this project has modified the original approach. The test driven development cycle is based on user story level, not on task level. This is because all the tasks in the user story for this project are closely related to each other, thus it is better to code and refactor them together. The tools used in this project to support TDD are JUnit and EasyMock. Figure 5.4 shows the test packages that were used for the XPairtise project:

56 Chapter 5. Implementation 44 Figure 5.4: Related test package in XPairtise 5.3 Software Modules The last section of this chapter details on each newly implemented functionality for this project. The XPairtise plug-in source files are grouped in the de.fuh.xpairtise main package and 3 sub-package groups (see Table 5.3). The black line in the table splits the source files into two parts: the source files above the black line implement the XPairtise server and the source files below the black line implement the XPairtise client (Eclipse Plug-in). More details regarding implementation as related to each part will be described below Eclipse Version For developing this project, the tool chosen was Eclipse 3.7 (Indigo). It is the newest Eclipse release, and is claimed to be more stable and dynamic than other versions. Eclipse Indigo is the annual release of Eclipse projects in It added a new level of functionality in editors, wizards, code merging and refactoring. More importantly, it was also found to supports existing XPairtise project.

57 Chapter 5. Implementation 45 Table 5.3: XPairtise source files structure Client Side Implementation In this project, most of client side implementations were related to Eclipse Plug-in development. A shared learning panel, personal learning panel and recording table were three of the most important components on the client s side. Hence, this section describes the implementation of these components Model-View-Controller Design Pattern The researcher as experienced the benefits of the MVC design pattern previously and so this design pattern was applied on the client side to implement the recording table and two panels. The model involved the learning record object containing informa-

58 Chapter 5. Implementation 46 tion or attributes about the record. This is usually referred as the domain object. The table layout and two panels (including two buttons at the bottom of the panel) are the views. The controller uses to connect the table or panel views with the learning record object. Figure 5.5 shows the class diagram of the MVC design pattern on the client side. All controller and view classes are contained in a package named de.fuh.xpairtise.plugin.ui.xpviews.learning. The two model objects (LearningObject and LearningObjectList) are contained in the package de.fuh.xpairtise.plugin.core.resources. The two interfaces on top of the controller and view components are: ILearning- Controller and ILearningView. The two interfaces express the main functionality of this part in relation to the rest of world and the XPairtise project. The interfaces hide internal implementations and thus they can be easily replaced. PersonalLearningView is responsible for creating GUI components for the personal learning panel and SharedLearningView is responsible for creating GUI components for the shared learning panel. Both panels contain same GUI components. Therefore, both of them extend LearningView superclass. This superclass includes all the operations required to create GUI components for both panels. The major difference between these two view classes is that they are both associated with different controllers. The reason that they use different controllers is that the shared learning panel needs to interact with the XPairtise server in order to send or receive learning records. In contrast, the personal learning view only relates to personal learning records locally, so there is no need to interact with the server. However, the class diagram below reveals that SharedLearningController extends LearningController and there is no PersonalLearningController in the class diagram. This is because it was decided to use the LearningController to deal with the PersonalLearningView directly. The SharedLearningController only involves operations for interacting with XPairtise Server and the rest of the operations are inherited from LearningController. This approach produces less duplicate codes which makes the whole structure clearer. In general, the controller is declared regarding compile time in views. However, XPairtise creates instances of controllers and views classes whenever the plug-in has been activated. Therefore, the instance of controller needs to be injected or mapped to the corresponding instance of view during the run-time Graphical User Interface Implementation Graphical user interface development plays an important role in this project. The majority of the developments on client side are related to GUI implementation. This

59 Chapter 5. Implementation 47 Figure 5.5: MVC client side class diagram

60 Chapter 5. Implementation 48 section describes two major GUI implementations. 1. Building Recording Table: The recording table is not only used to display learning records, but also allows developers to edit, modify and delete records. Traditional Eclipse table does not support these features. Therefore, for this project it was decided to use JFace technology to build the table. JFace is a technology based upon SWT in Eclipse. It provides many helper classes and frameworks to choose from, which simplifies many complicated tasks in SWT. JFace provides several Viewers, which are model based widgets sets. These Viewers simplify the mapping of data models to a visual representation. Among different Viewers, TableViewer is used to build editable table in plug-in. A single TableViewer class is not sufficient, there are other two important classes associated with TableViewer: ContentProvider and LabelProvider. ContentProvider is responsible for creating the required structure of the domain model for the TableViewer and LabelProvider determines how best to display individual domain model element. The org.elcipse.swt.widgets.table instance was created in the first and then its parent, style and layout attributes were set. Subsequently, org.eclipse.jface.viewers.tableviewer object was created, passing along the Table instance created in the previous step since a TableViewer must operate on a Table instance. After the Table and TableViewer instance have been created, a setcolumnproperties method must be called on the TableViewer instance in order to use it in callbacks to recognise which column the users want to operate. It was determined to create an editable table with different columns, thus an array of CellEditor objects was used, each CellEditor object could contain a various of different editors, such as CheckboxCellEditor, ComboBoxCellEditor. The table now contained all the required parts to display learning records but it still needs support for cell editing. For this, the author implemented an org.eclipse.jface.viewers.icellmodifier. This class defines canmodify(), get- Value() and modify() methods. The canmodify() method identifies whether certain columns can be modified. The getvalue() method returns column name and corresponding value. The last modify() method updates domain objects with the new values and displays the latest updates in table. Finally, the content provider, label provider and input for the TableViewer were set. The content provider is implemented as an inner class (LearningObjectContentProvider) inside the LearningView class. The reason for implementing a content provider as an inner class is it allows ease of implementation without affecting our mod-

61 Chapter 5. Implementation 49 els with JFace dependencies. For the label provider, the author chose to implement it as a top-level class (LearningLabelProvider), this class implementes org.eclipse.jface.viewers.itablelabelprovider. The getcolumntext() and get- ColumnImage() methods in LearningLabelProvider label the text and image for a given column in table. 2. Building Two Panels: Personal learning panel and shared learning panel are two views presented on the Eclipse platform. The Eclipse view is typically used to navigate or display information for components inside Eclipse. Any modifications made in a view are automatically saved [15]. The layout of views is normally controlled by the active perspective. Four steps were used to add the two views to XPairtise. In step one, the author first identified the plugin.xml file from the XPairtise project and then located the extension point for extending views in this xml file. As introduced at the beginning of this chapter, plugin.xml is one of the most important manifest files. It describes how a plug-in extends a platform and how it implements its functionality. Eclipse plug-in defines a single extension point for view contribution: org.eclipse.ui.views. Therefore, this will be put in place with regards to plugin.xml and two views need to be added. In step two, the two views were added in plugin.xml. (see Figure 5.6) Figure 5.6: Two views in XPairtise plug-in file Several attributes for the views were defined in this file. The id attributed is a unique name that identifies the view. The name attribute is used in user interface for this view. Most importantly, the class attribute is the name of the class that implements this view. In step three, I defined the view class. To make the developers life easier, Eclipse provides an abstract class called

62 Chapter 5. Implementation 50 org.eclipse.ui.part.viewpart, which implements the majority of the operations in org.eclipse.ui.iviewpart. By subclassing the ViewPart class, LearningView class inherits all behaviours. The final step is to add the two views as a shortcut to Eclipse page layout (Perspective class) which then can be shown in the view menu to allow users to rapidly navigate to the appropriate views Server Side Implementation XPairtise uses the Java Messaging Service as the underlying transport mechanism. Broker is the core concept in JMS; it acts as a message-relaying hub. However, the XPairtise proejct has not taken advantage of the broker function at all. Whenever clients want to send messages to the server, the server opens a central message queue to receive messages from clients. The server then sends a unique session token to the client in reply to connection request. This special session token allows both server and clients to identify the unique server-client queue. Clients also use this session token to tag each messages sent to the server and then the server can recognise incoming messages on a user s session. This section, describes the server side implementation of new functions in XPairtise Replicated Element In XPairtise project, operations or objects that need to be shared between developers in DPP sessions are replicated elements. Simply put, replicated elements need to be transferred between distributed developers by using a Java Messaging Service in XPairtise. Replicated elements are stored in a list data structure. Whenever the element changes in a list, the whole object can then be retransmitted. For example, when a client changes selection in Eclipse editor, the plug-in sends a selection command to the server. Then the server adds the command to the corresponding session queue according to the token. Next, clients who are listening on this queue will receive this command and executes it locally. There are several replicated elements in XPairtise and one session may involve lots of clients at one time. XPairtise uses two attributes in replicated element to identify different replicated elements and clients: elementid and sequenceid. ElementId identifies replicated element and sequenceid represents a counter for a timeline of editing operations. The two attributes are defined in de.fuh.xpairtise.common.replication.replicatedobject class. ReplicatedObject only stands for a single replicated element, so a ReplicatedListElement has been created

63 Chapter 5. Implementation 51 to handle a list of replicated elements. All replicated elements extend this class in XPairtise. In the design presented here, shared learning records need to be shared between distributed developers in one session, so ReplicatedSharedLearningElement has been created in the de.fuh.xpairtise.common.replication.elements package. As with all other replicated elements in XPairtise, ReplicatedSharedLearningElement also extends ReplicatedListElement (see Figure 5.7). Figure 5.7: Shared learning element class diagram

64 Chapter 5. Implementation Replication Process The previous section illustrated, replicated shared learning element that was created successfully. The next stage is to modify the replication process to transfer this replicated elements from one developer to other developers in XPairtise. The core component of the replication process is the replicator. The replicator receives messages from the client, which either deletes or adds elements in the replicated list by using elementid and sequenceid attributes. Meanwhile, clients in the same session listen to the replicator. If there are any changes to the replicators internal list, then it will send the altered elements to the clients. SequenceId not only exists on the server side, the listener on the client side also has a lastsequenceid attribute. Whenever the listener receives a new message from the server, it sets this attribute to the highest value it has received. This is the general replication process in XPairtise. The INetworkFactory interface in the de.fuh.xpairtise.common.network package generates all the network related objects; for example, replicated master list, replicated receiver list, client command and server command. The replicated master list sends updates or commands to the XPairtise server and the replicated receiver list acquires operations with corresponding objects from the XPairtise server. This means that, the receiver list consumes updates on the developer side. IReplicatedListMaster and AbstractReplicatedListReceiver are the master list and receiver list in XPairtise. Therefore, this project involves the creation of a ReplicatedEventListener inner class inside a SharedLearningController to listen to any updates or changes in the replication list for ReplicatedSharedLearningElement. Also, a sharedlearninglist HashMap field insider ServerApplication (implementation of ServerCommandListenerInerface) was created to act as a replication master list for ReplicatedSharedLearningElement. Figure 5.8 shows a simplified class diagram detailing the shared learning replication process. After adding the necessary replicated master list and receiver list, new methods were added for sending the shared learning replicated element in ActiveMQServerCommandInterface and LoopbackServerCommandInterface class. The two classes are subclasses of ServerCommandInterface. Now the views (SharedLearningController is recognised as a component on the View side of XPairtise architecture) can connect with replication process on server side. Figure 5.9 shows the complete sequence diagram for shared learning replication process implementation in XPairtise.

65 Chapter 5. Implementation 53 Figure 5.8: Shared learning replication process class diagram Figure 5.9: Sequence diagram for shared learning process in XPairtise

66 Chapter 6 Evaluation As mentioned in the chapter 2, the software engineering community has been criticized for claiming new functionality provides benefits without performing critical evaluations. Therefore, evaluation is seen as a critical element of this project and this section describes evaluation design and results in detail. 6.1 Evaluation Design Overview An evaluation plan was constructed during the project s design phase in order to evaluate new functionalities in XPairtise that were accurately based on design requirements. The aim of this evaluation design was to maximise the quality and effectiveness of new funcitons in XPairtise by defining the criteria they were evaluated by. As XPairtise is an interactive plug-in used to support DPP in the Eclipse envrionment, usability analysis forms an integral part of its evaluation. This evaluation is designed specifically to answer the following questions regarding the new functions added in XPairtise: Does the new functions added in XPairtise improve learning for users in DPP session? Is the user able to understand the new function s GUI and use it with ease (Usability Analysis)? To answer the two questions and evaluate the effectiveness of the tool functionality, a controlled experiment was conducted using two groups of people. The experiment was based on the two following methodologies: 1. Task-oriented evaluation methodology: In the experiment, each group of people completed two different tasks. These tasks were designed to evaluate certain 54

67 Chapter 6. Evaluation 55 behaviour of the system. After users have completed their tasks, they should have a better understanding and feeling about system; allowing them to give more accurate feedback. 2. A randomised, single-factor with two treatments controlled experimental design methodology: In this form of experimental design, each group was randomly assigned to one of two groups: a control group and a treatment group. The treatment group contained independent variables only and an experiment was used to collect data to ascertain whether it influences the dependent variables. However, this type of experiment is only valid if there are no conflating factors that would also influence the dependent variables. For example, it was important to the evaluation that people in each group should have similar background in my evaluation. Based on the two evaluation methodologies, people were randomly assigned to one of two groups: a tool with new functionality group (the treatment) and a tool without new functionality group (the control). The treatment group can use new functions in XPairtse to complete tasks in a distributed environment. The control group has to use original XPairtise to complete tasks in a distributed environment. Before the experiment, both groups were required to complete a survey. At the end of each task, the experimental subjects needed to complete a task based questionnaire and a usability questionnaire. The following sections of this chapter describes: (1) an initial pilot study that was conducted to evaluate the tool before the experiment, (2) two main target user groups and evaluation sample size for this experiment, (3) pairing and randomisation methods in this experiment, (4) two tasks and questionnaires for the experiment and (5) experiment results. 6.2 Pilot Study Previous to evaluating this tool in the experiment, the researcher conducted a pilot study using a pair of students and a pair of software developers. The pair of students were master students at the University of Edinburgh. Both of them were studying for a Master of Computer Science degree. The pair of software developers had worked in the industry for more than two years. Both individuals had previous experience with DPP. At the beginning of the pilot study, the two pairs were given a quick introduction

68 Chapter 6. Evaluation 56 to XPairtise, and a careful explanation of the new functionality to improve learning in this tool. Both of the pairs completed a simple programming task using the new tool and both provided valuable feedback on the new functionality of the tool. Firstly, the two pairs felt that the new functionality in XPairtise had improved learning in DPP. Perceived benefits of the new functionality of the tool includes the ability to check and review knowledge learnt during the DPP and an easy method by which to locate codes for the particular knowledge associated with the project. The pairs also made one excellent suggestions for improving the shared learning process of this tool. The suggestion was to only allow developers to add records in the shared learning panel if they have joined the session successfully, otherwise the shared learning panel should be disabled. In the pilot version of the tool, developers were able to use the shared learning panel regardless of whether they had joined the session or not. This suggestion was found to be very valuable as the main aim of the shared learning panel is mainly used to share knowledge in a distributed session.this issue was resolved by ensuring the enabling of a shared learning panel GUI component only occurred if developers had joined the session successfully. 6.3 Target User Groups And Evaluation Sample Size When the author analysed the current DPP situation, it was apparent that it has not only been used widely in the software industry, but that is also applied in software engineering courses at universities. Therefore target users are likely to be either individuals from the academic community or professional software developers from various industries. As a result it was decided to include two target user groups for this evaluation: (1) Student group and (2) Software developer group. The student target group only contains students currently at universities and software developer group only contains those who are currently working in the industry. The major type of evaluation or experiment that is conducted for this project has a close relationship with a usability evaluation on learning in XPairtise. The number of users in each target group able to evaluate the new functions in XPairtise is highly variable. Taking into consideration of several factors in this project, such as time and scale of the project, I aimed to evaluate functionality and usability in this project on a minimum of 8 individuals in each target group. According to the Nielsen s analysis, no more than 5 individuals are required to perform usability evaluation adequately [19].

69 Chapter 6. Evaluation 57 Number of issues found in usability evaluation with n users = N(1 (1 L) n ) (L = 31% based on experience) N = total number of usability issues L = the proportion of usability issues found while evaluated by a single user Table 6.1: Number of problems found by number of users; source: [3] According to the Table 6.1, 85% of the usability problems could be identified by only 5 users. Nielsen has shown that evaluation carried out by 5 users would give a 70% probability of getting within ±15% of the true mean and that a 90% confidence interval would be ±24%. For most of the software projects, this level of evaluation accuracy is enough [3]. According to the above results, 10 users should be adequate to carry out the evaluation required for this project. In additional to the usability of the evaluation, this evaluation involves functional evaluation, which is to evaluate the new functionality so as to improve learning in XPairtise. Because an extra functional evaluation is involved in the whole evaluation process, the evaluation sample size was increased to 10 users in each user group. 6.4 Pairing And Randomisation DPP requires at least two developers to produce an artifact in a session. How to form the pair is a critical factor in this experiment. In addition, randomization is one of the key requirement according to the above methodologies. This section describes the

70 Chapter 6. Evaluation 58 pairing and randomization method used for this evaluation Pairing Method Before the pair formed in this evaluation, a presentation and basic training was given to the groups regarding DPP and some general principles for the experiment. After listening to the presentation, each group was asked to fill in a simplified form so that they could be assigned to a compatible partner. The form asked each person to list two other persons in the group whom they would like to partner and some additional personal information. Table 6.2 shows the general process applied to assign distributed partners in this experiment. Once the pair has been formed, it was expected the same partner would be maintained for the duration of the experiment. This experiment involved not only a two-developer pair, but also created a three-developer pair. The method for forming the three-developer pairs was the same as that used for forming the two-developer pair. Step 1 Step 2 Step 3 If two persons listed each other as desired partners, they are paired. People with similar distributed pair programming experience were partnered. If there are still people who have not formed the pair, an attempt is made to assign pairs such that both partners have similar experience in Java programming. Table 6.2: Pair formation steps Randomisation Method After the pairs had been formed in each group, a slightly modified version of completely randomised design method was used to assign the pairs to either the treatment or control group. This is a basic randomisation method and is particularly useful when experimental units are homogeneous. The homogeneity does not mean the experimental units are exactly alike. Rather, it emphasises that there is no important source of variation in the experimental units. According to the forms filled in by the people in each group, all of them were found to have a similar background. Therefore, is the participants meant that it was appropriate to use this modified version of a completely randomised design. The basic step is to label each pair, and then use a table of random

71 Chapter 6. Evaluation 59 numbers to select from the labelled pairs. The only manner in which the original completely randomised design method was modified was when the treatment group had two pairs, then the other two pairs were automatically assigned to control group. The same method applies to the generation of a three-developer pair, but this time only one pair was assigned to each group. As a result of this randomisation method, both student and software developer groups have three pairs in the treatment group and three pairs in control group. One of the pairs in each group has three developers. 6.5 Tasks The new functions in the system can be used to record two main types of learning: general programming knowledge and project related knowledge. Therefore, each pair will do two tasks in this evaluation. One task is a general programming task and the other is a project related task. Before they start working on any of the tasks, all the pairs were required to fill in a survey. The survey investigated each person s programming skills and experience. After the pairs completes the tasks, they then answered a questionnaire. Each task was associated with a particular questionnaire and all the questions were designed specifically for that task in order to evaluate its new functions. In addition, the treatment group also needed to complete an usability questionnaire. All questionnaires are shown in Appendix D General Programming Task General programming task evaluates the new functionality in XPairtise so as to improve the learning of general programming knowledge in the DPP. The task mainly involved knowledge related to how to manipulate String in Java. String is one of the most common and fundamental type in Java. Most of programming tasks involve string manipulation problems. Table 6.3 shows the general programming related task question.

72 Chapter 6. Evaluation 60 Question Technical Description An Example Reversing words in a sentence but the characters in each word should keep as the same sequence as before. Write a Java program that reads sentence from command line and then reverses the words in the sentence which are separated by an empty space in the input. Input: I am a student. Output: student. a am I Table 6.3: General programming task question Project Related Task Project related task evaluates the new functionality in XPairtise to improve the learning of project related knowledge in DPP. The difference between project related task and general programming task is that it should involve project related knowledge. For example, a model-view-controller is a well known design pattern but a project related task should focus on how this design pattern has been implemented in an individual project. As XPairtise is the tool used to enhance learning functionality and has been introduced in previous chapters, it was decided to use the XPairtise project for this task. Table?? shows the XPairtise related task. Project Question Technical Skills XPairtise (Distributed Pair Programming Tool) Investigate shared learning replication process in XPairtise and finally show the process in a diagram Java Messaging Service (ActiveMQ), Eclipse Plug-in Development Table 6.4: Project related task question 6.6 Results And Discussion The whole of the experimental evaluation phase is performed by two groups: student group and software developer group. In total, there are sixteen users involved in the evaluation; the sixteen users formed eight pairs. Four pairs completed the tasks with the new functions (treatment group) and the other four pairs finished the task without the new functions. Furthermore, six people were randomly selected from each group to form two three-developer pairs to execute the experiment again so as to evaluate the new functions when there are multiple developers in a DPP session. I joined every

73 Chapter 6. Evaluation 61 DPP session as a spectator during the experiment in order to observe the learning interactions between developers Pre-Evaluation Survey The pre-evaluation survey collected data regarding programming experience and the skill base of each group. Figure 6.1 shows the programming experience for each pair in two groups. Figure 6.1: Student and Software developer group s Java programming experience In Figure 6.1, it is shown that the first three pairs were assigned to the treatment group and the other three pairs were assigned to control group. All the users in the

74 Chapter 6. Evaluation 62 two groups came across String manipulation problems. None of the users in the student group has learnt about the Java Messaging Service before and only 3 software developers had touched on ActiveMQ technology before Usability Evaluation The usability evaluation is aimed at understanding if users were able to understand new functions easily and correctly interpret and utilise them. Therefore, a usability evaluation only needs to be completed by pairs in the treatment group once. The result of this is shown in the Figure 6.2 Figure 6.2: Usability evaluation result According to the above results, the overall feedback for the usability of the new functions in XPairtise is positive. The following suggestions are replies that were received regarding improving the usability of the new function: When users click the line number of the learning record in the table, the editor should open to the corresponding place. (instead of users looking up the location by themselves) After users add learning records to the table, the corresponding codes should be highlighted or marked in Eclipse editor. The icons for shared learning panel and shared learning panel should be different.

75 Chapter 6. Evaluation Functional Evaluation The functional evaluation process was particularly designed to evaluate new functions to improve the developer s learning during DPP. All the users need to complete questionnaires after they finish their tasks. The treatment group is allowed to use the new functions and I joined every DPP session as spectator in this experiment in order to observe the interactions between the developers. When each pair completed their task, the total time they took to finish the task was recorded. The results are shown below: Figure 6.3: Completion task time The first three pairs in each group were assigned to the treatment group and the other three pairs were assigned to the control group. The above figure shows that it

76 Chapter 6. Evaluation 64 almost takes the same amount of time to complete a task in each group. The average time to complete general programming task in student group is 23.6 minutes. The longest and shortest completion time for general programming task in student group is 27 minutes and 20 minutes, which only has a small amount of different from the average time. Both treatment group and control group take almost the same amount of the time to complete same task in the experiment. In other words, the experiment shows that new functions in XPairtise has not resulted a longer time to complete tasks. Then, we need to find out whether learning has been further improved in new tools compared with old tools within almost the same amount of time. The following questionnaires result reflects this General Programming Task Section One Question 1: Please list all the String related knowledge you learned in this task from this session. Results and Analysis: Student Group Software Developer Group Treatment Control Treatment Control String.split(), StringUtils.reverse, StringUtils.reverse, StringBuffer StringUtils.reverse, StringUtils.reverseDelim StringUtils.reve se StringUtils.reverseDeli ited, StringTokenizer, StringBuilder ited Table 6.5: General programming task question 1 result The results given in Table 6.5 clearly show that the pairs in the treatment group achieved a better learning result than the pairs in the control group. The recording and reviewing functions helped users to remember the knowledge more effectively. According to own notes during the session, the pairs in the control group had forgotten some of the knowledge that they had discussed during the session. For example, one of the student pairs in the control group had discussed String.split() during the DPP session. In fact, this pair first proposed using the String.split() method after studying

77 Chapter 6. Evaluation 65 Java String API; however, they find that StringUtils has already provided a method for such kinds of problems during their discussion, so they decided to use reverse() method in StringUtils. As a result, they only remembered the final solution and forgot some of the knowledge they had acquired during the session. Questions 2, 3 and 4: If I have a String containing a list of number, for example , then how can you split them into an array of String in Java? Apart from the Java API, do you know any other methods in other libraries that can solve the above issue in question 8? What is a more efficient way to processing a large amount of String? Results and Analysis: Questions 2, 3 and 4 are used to further evaluate the knowledge that the pairs learnt in reference to question 1. The result is shown in Figure 6.4. Figure 6.4: General programming task questions 2, 3 and 4 results According to the result, the treatment group answered questions more accurately than the control group. This reflects the fact that the new function has improved general programming knowledge learning in relation to DPP Project Related Task Section One Question 1: Please list all relevant classes to the shared learning replication process you learn in this task from this session.

78 Chapter 6. Evaluation 66 Results and Analysis: Compared with a general programming task, the project related task is more complicated, so this task can evaluate the effectiveness of the new functions more accurately. The result shown in Table 6.6 reflects that the pairs in the treatment group has answered this question much better than the pairs in the control group. The pairs in treatment group had filled in most of the classes related to the shared learning replication process with the help of new functions in XPairtise. In contrast, the pairs in the control group only remembered several classes, some important classes were omitted. Later, when checking the notes made during observation of the session, I found that the pairs in control group had discussed information about the missed classes, so the pairs in the control group had not fill in the missed classes is not because they had not seen or discussed the them. Student Group Software Developer Group Treatment Control Treatment Control ReplicatedObject, ReplicatedListElement, IReplicatedMaster, IReplicatedReceiver, ReplicatedShared LearningElement, IRemoteCommand, IServerCommandInterface, IServerCommandListener, ServerApplication, ILearningController, LearningController, LearningView, SharedLearningController, SharedLearningView, ActiveMQServerCommandInterfa ce, LoopbackServerCommandInterfa ce ReplicatedSharedLearnin gelement, SharedLearningController, SharedLearningView, ServerApplication ReplicatedObject, ReplicatedListElement, IReplicatedMaster, IReplicatedReceiver, ReplicatedShared LearningElement, IRemoteCommand, IServerCommandInterface, IServerCommandListener, ServerApplication, LearningController, LearningView, SharedLearningController, SharedLearningView, ActiveMQServerCommandInterfa ce, LoopbackServerCommandInterfa ce, ReplicatedEventListener, IReplicatedListEventListener, ClientUserChangeListener, ReplicatedSharedLearnin gelement, SharedLearningController, SharedLearningView, ServerApplication, LearningView, LearningController ActiveMQServerCommandInterface, LoopbackServerCommandInterfa ce, IClientUserChangeListener Table 6.6: Project related task question 1 result Questions 2, 3, 4 and 5: Which class implements IServerCommandListener? What is the relationship between IServerCommandInterface and IServer- CommandListener?

79 Chapter 6. Evaluation 67 Please draw a simplified sequence diagram for shared learning replication process. Please draw class diagram for shared learning record view part. Results and Analysis: Questions 2, 3, 4 and 5 are used further evaluate the project knowledge that developers had acquired during the session in reference to question 1. The results in Figure 6.5 show that the treatment group understands and answers questions much better than the control group again. With the assistance of new functions, the treatment group can easily identify the relationship between classes by looking at the knowledge and codes that been recorded in the table. Figure 6.5: Project related task questions 2, 3, 4 and 5 results Other Sections Question 7 in General Programming Task Questionnaire and Question 8 in Project Related Task Questionnaire: If you join the distributed pair programming session late, can you see the general programming knowledge already recorded in the session? Results and Analysis: All the pairs in the treatment group had selected yes as the answer. Users agreed that this was useful I like this idea and find it is useful for learning purpose, especially when there are more than two developers and I joined the session as spectator late. By using the shared learning recording table, I will not miss some important knowledge.

80 Chapter 6. Evaluation 68 Question 9 in General Programing Task Questionnaire and Question 10 in Project Related Task Questionnaire: When you act as either driver or navigator, have you raised any general programming questions by using the recording table and discussing them later in order not to interrupt the other developers? If yes, do you feel it is better to learn knowledge in this way? Results and Analysis: This is controversial questions in the questionnaire. The 2 users did not make any comments about this question. Another 2 users think this is not useful and they prefer to discuss the questions immediately. The rest of the users find this to be useful: Again, this function helps to improve the learning process in distributed pair programing. The code quality can benefit from this as well, developers can concentrate on producing artifact and leave questions at the end. This is particular useful when the pair has time pressure to share knowledge. Last Question in both Questionnaires: Does the new function really improve the general programming learning? Does the new function really improve the project related learning? Results and Analysis: The final question on the questionnaires asks users to conclude whether the new functions have improved the learning in DPP. All the replies received were positive: Very impressed. Obviously a small amount of additional work may be needed to make it better!, but the concepts and design shows the opportunities for improving the learning in distributed pair programming effectively. To some extent, I feel that learning knowledge is much easier by using XPairtise in distributed environment. I had no difficulty in using the new functions, it is pretty straightforward, which is a always a plus point. The learning process is much improved and enhanced. It is much better than the current tool.

81 Chapter 6. Evaluation Final Evaluation Results Analysis In the previous sections of this chapter, individual evaluation results were presented and analysed. This final section further analyses this evaluation based on a combination of several of the results presented previously. As shown in the beginning of section 6.6.3, both the treatment group and the control group took the same amount of time to complete the tasks for this evaluation and then treatment group answered the questions more effectively than the control group in both the general programming task questionnaire and the project related task questionnaire. If we consider the two results of the evaluation together, we can conclude that pairs with new tools understand and use the new knowledge learnt from DPP better than pairs using the old tools. In other words, new functions can improve the learning during the DPP. In addition, preevaluation survey showed that both treatment and control group contained pairs with different levels of experience (different pair formation patterns) and if we consider the pre-evaluation result with the questionnaire results, we can also conclude that the new functions can improve learning in different pair formation patterns. Therefore, this evaluation proves that the new functions can improve and further support learning to some extent.

82 Chapter 7 Conclusions and Future Work 7.1 Conclusion The current project has focused on improving and increasing learning support and knowledge sharing between developers in DPP. Pair learning or knowledge sharing is one of the most important goals in DPP. However, according to the researcher s analysis and evaluation of the current situation in DPP, learning is the biggest issue of concern. The decision was therefore taken to improve learning by adding new functionality using an existing DPP tool; XPairtise. The implementation of the new functions was based on two major design principles: The active learning design principle and the pair formation pattern design principle. The active learning design principle considers improvements to learning based on active learning theory. Active learning is considered to be the most effective approach to learning. However the pair formation design principle, considers improvement of learning using a distributed pair programming formation pattern. Thus, the overall design aims to improve learning from both a theoretical and a practical perspective. Finally, the new functions have been successfully implemented. All planned user stories were completed on time. The evaluation results proved that developers with the option to use the new functions achieve better learning results than those using the old tool. 7.2 Future Work In spite of the relatively good results achieved, we must be realistic: this project only involves evaluating one existing DPP tool in detail and a relatively small number of users opinions have been gathered. Given the time frame and the fact that researching 70

83 Chapter 7. Conclusions and Future Work 71 on learning activity or knowledge sharing in DPP is a new area, the results are reasonably good; but we must see them only as a starting point for more studies into a wider variety of DPP tools including more users. Learning activity is a complicated interactive process, and so if more users can be involved in both the analysis and evaluation stages, we can achieve a more accurate result. In addition, several suggestions proposed by users in the evaluation could be implemented in the future to further improve the usability of the new functions included in the tool.

84 72

85 Appendix A. DPP Tool Requirement Document 73 Appendix A DPP Tool Requirement Document Requirement Document [29] General Tool Requirements Support pair formation Support workspace awareness Project synchronization Editor synchronization Support of role changes Support of floor control Easy to switch role Show the current role of developer Textual chat Audio chat Video chat White Board Support Support second pointer Support navigator highlighting Support Learning Other Tool Related Properties Cross-Platform Open source tool Commercial tool Special IDE required Tool Usability Easiness to install and configure Support real-time behavior Easy to use and control Stability Priority High High High High High Medium Medium Medium High High Medium Low High High High Low Low Low Low Medium High High High

86 Appendix B DPP Tool Survey PLEASE ANSWER EACH OF THE QUESTIONS HONESTLY YOUR ANSWERS WILL NOT INFLUENCE ANYTHING. The information answered in this survey will only be used for research purpose. Your information will be held in the strictest confidence. Tool Survey Questions: 1. How many times have you use distributed pair programming tools? 2. How many different distributed pair programming tools have you ever use? 3. Which distributed pair programming tool do you prefer to use? 4. For the following areas, which area do you think need further support or improve in current distributed pair programming tools? and Why? (Please choose only one and give the reasons) Answer Pair formation: the steps to form the pair in distributed environment Working together: when two developers produce artifact together Learning & Knowledge sharing: To learn knowledge from pair session Testing: collaborative testing 74

87 Appendix C Pre-Evaluation Survey The table below contains a series of questions about your personal information and your working experience, PLEASE ANSWER EACH OF THE QUESTIONS HON- ESTLY YOUR ANSWERS WILL NOT INFLUENCE ANYTHING. The information answered in this survey will only be used for research purpose. Your information will be held in the strictest confidence. Survey Questions: 1. What is your current employment type? Student or Software developer 2. Have you programming in Java before? (if Yes or No yes, please answer question 3) 3. How many years have you programming in Less than 2, 2 to 4 years, Java? More than 4 years 4. Have you come across any String Yes or No manipulation problems before? 5. Have you learn anything about Java Yes or No Messaging Service, especially using ActiveMQ as the implementation? 75

88 Appendix D Post-Evaluation Questionnaires Now you have completed the task, I would like to ask you a few of the questions regarding the knowledge you learnt from the task. PLEASE ANSWER EACH OF THE QUESTIONS HONESTLY YOUR ANSWERS WILL NOT INFLUENCE ANY- THING. The information answered in this questionnaire will only be used for research purpose. Your information will be held in the strictest confidence. Usability Evaluation Questionnaire (This is only for treatment group) 1. Are you satisfied with the overall use of the software? Yes or No 2. Is it easy to understand new functions in XPairtise? Yes or No 3. Is it easy to use new functions in XPairtise Yes or No 4. Can you easily add/find learning panel in Eclipse? Yes or No 5. Do you think adding learning records to table is easy? Yes or No 6. Can you easily understand different categories in each learning type? Yes or No 7. Can you easily modify learning record in table? Yes or No 8. Can you easily find code location where knowledge occurred by using Yes or No recording table? 9. Do you satisfy the usability of new functions in improving learning in Yes or No distributed pair programming? 10. If you answer NO for the above questions, please tell us your suggestions on improving the usability? 76

89 Appendix D. Post-Evaluation Questionnaires 77 General Programming Task Questionnaire Section 1 Knowledge Testing 1. Please list all the String related knowledge you learn in this task from this session. 2. If I have a String containing a list of number, for example , then how can you split them into an array of String in Java? 3. Apart from the Java API, do you know any other methods in other libraries that can solve the above issue in question 8? 4. What is a more efficient way to processing a large amount of String? Section 2 Other Functions Evaluation 5. How many users have join this distributed pair programming session? (if there are more than 2 users, please answer question 6) 6. Have you join the distributed pair programming session late? (If yes, please answer question 7) 7. If you join the distributed pair programming session late, can you find and understand the general programming knowledge already recorded in the session? 8. Have you switch the role during the task? (if yes, please answer question 9) 9. When you act as either driver or navigator, have you raise any general programming questions by using the recording table and discuss them later in order not to interrupt the other developers? If yes, do you feel it is better for learning the knowledge in this way? Section 3 Overall Evaluation 10. Does the new function really improve the general programming learning? 2 or More than 2 Yes or No Yes or No Yes or No

90 Appendix D. Post-Evaluation Questionnaires 78 Project Related Task Questionnaire Section 1 Knowledge Testing 1. Please list all relevant classes to the shared learning replication process you learn in this task from this session. 2. Which class implements IServerCommandListener? 3. What is the relationship between IServerCommandInterface and IServerCommandListener? 4. Please draw a simplified sequence diagram for shared learning replication process 5. Please draw class diagram for shared learning record view part. Section 2 Other Functions Evaluation 6. How many users have join this distributed pair programming session? (if there are more than 2 users, please answer question 2) 7. Have you join the distributed pair programming session late? (If yes, please answer question 3) 8. If you join the distributed pair programming session late, can you find and understand the project related knowledge already recorded in the session? 9. Have you switch the role during the task? (if yes, please answer question 5) 10. When you act as either driver or navigator, have you raise any general programming questions by using the recording table and discuss them later in order not to interrupt the other developers? If yes, do you feel it is better for learning the knowledge in this way? Section 3 Overall Evaluation 11. Does the new function really improve the project related learning? 2 or More than 2 Yes or No Yes or No Yes or No10.

91 Bibliography [1] A. Ahmed, S. Ahmad, N. Ehsan, E. Mirza, and S.Z. Sarwar. Agile software development: Impact on productivity and quality. In Management of Innovation and Technology (ICMIT), 2010 IEEE International Conference on, pages , june [2] Kristy Elizabeth Boyer, August A. Dwight, R. Taylor Fondren, Mladen A. Vouk, and James C. Lester. A development environment for distributed synchronous collaborative programming. SIGCSE Bull., 40: , June [3] Priya Chandran. Updating and extending the open source java coverage tool, coverlipse. Master s thesis, School of Informatics, August [4] Haoyang Che. Review of "eclipse: Building commercial quality plug-ins by eric clayberg and dan rubel". addison wesley, 2004, (paperback). ACM SIGSOFT Software Engineering Notes, 30(1):36, [5] Li-Te Cheng, Susanne Hupfer, Steven Ross, John Patterson, Bryan Clark, and Cleidson de Souza. Jazz: a collaborative application development environment. In Companion of the 18th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, OOPSLA 03, pages , New York, NY, USA, ACM. [6] Edgar Dale. Audiovisual Methods in Teaching. Third Edition. Holt, Rinehart and Winston, Inc., [7] Rafael Duque and Crescencio Bravo. Supporting distributed pair programming with the collece groupware system: An empirical study. In Pekka Abrahamsson, Richard Baskerville, Kieran Conboy, Brian Fitzgerald, Lorraine Morgan, Xiaofeng Wang, Will Aalst, John Mylopoulos, Norman M. Sadeh, Michael J. Shaw, and Clemens Szyperski, editors, Agile Processes in Software Engineering 79

92 Bibliography 80 and Extreme Programming, volume 9 of Lecture Notes in Business Information Processing, pages Springer Berlin Heidelberg, [8] Nick V. Flor. Globally distributed software development and pair programming. Commun. ACM, 49:57 58, October [9] Erich Gamma and Kent Beck. Contributing to Eclipse: Principles, Patterns, and Plugins. Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, USA, [10] Brian F. Hanks. Empirical studies of distributed pair programming. PhD thesis, Santa Cruz, CA, USA, AAI [11] Jingchun Hao. Informatics research proposal (distributed pair programming). March [12] James D. Herbsleb. Global software engineering: The future of socio-technical coordination. In 2007 Future of Software Engineering, FOSE 07, pages , Washington, DC, USA, IEEE Computer Society. [13] Chih-Wei Ho, Somik Raha, Edward Gehringer, and Laurie Williams. Sangam: a distributed pair programming plug-in for eclipse. In Proceedings of the 2004 OOPSLA workshop on eclipse technology exchange, eclipse 04, pages 73 77, New York, NY, USA, ACM. [14] K. Siau J. Erickson, K. Lyytinen. Agile modeling, agile software development, and extreme programming: the state of reserach. Journal of Database Management 16 (4), pages , [15] Mirasol Op nworks Laurent Gauthier. Building and delivering a table editor with swt/jface. July [16] Wanfeng Dou;Yifeng Wang; Sen Luo. Analysis and design of distributed pair programming system. Intelligent Information Management, 02: , [17] Frank Maurer. Supporting distributed extreme programming. In In Extreme Programming and Agile Methods - XP/Agile Universe 2002, number 2418 in LNCS, pages Springer, 2002.

93 Bibliography 81 [18] Hiroshi Natsu, Jesus Favela, Alberto L. Mor?, Dominique Decouchant, and Ana M. Martinez-Enriquez. Distributed pair programming on the web. Mexican International Conference on Computer Science, 0:81, [19] Jakob Nielsen. Usability engineering, volume Chapter 6. Academic Press Inc., [20] Balasubramaniam Ramesh, Lan Cao, Kannan Mohan, and Peng Xu. Can distributed software development be agile? Commun. ACM, 49:41 46, October [21] Michael Reeves and Jihan Zhu. Moomba - a collaborative environment for supporting distributed extreme programming in global software development. In XP, pages 38 50, [22] Stephan Salinger, Christopher Oezbek, Karl Beecher, and Julia Schenk. Saros: an eclipse plug-in for distributed party programming. In Proceedings of the 2010 ICSE Workshop on Cooperative and Human Aspects of Software Engineering, CHASE 10, pages 48 55, New York, NY, USA, ACM. [23] Till Schuemmer and Stephan Lukosch. Understanding tools and practices for distributed pair programming. 15(16): , oct [24] Till Schuemmer and Jan Schuemmer. Support for distributed teams in extreme programming. In Proceedings of extreme Programming and Flexible Processes Software Engineering - XP2000, pages Addison Wesley, [25] Walter F. Tichy. Should computer scientists experiment more? IEEE Computer, 31(5):32 40, [26] Don Wells. Extreme programming: A gentle introduction. September [27] Houghton-Mifflin Wilbert J. McKeachie. Active learning. [28] Laurie Williams and Robert Kessler. Pair Programming Illuminated. Addison- Wesley Longman Publishing Co., Inc., Boston, MA, USA, [29] Dietmar Winkler, Stefan Biffl, and Andreas Kaltenbach. Evaluating tools that support pair programming in a distributed engineering environment. In Proc. Conference on Evaluation and Assessment in Software Engineering (EASE),

94 Bibliography 82 pages 1 10, Vortrag: Proc. Conference on Evaluation and Assessment in Software Engineering (EASE), Keele, Great Britain;