Socio-Technical Support for Knowledge Collaboration in Software Development Tools

Transcription

1 Socio-Technical Support for Knowledge Collaboration in Software Development Tools Yunwen Ye 1,2 1 SRA Key Technology Laboratory, 3-12 Yotsuya, Shinjuku Tokyo , Japan 2 Department of Computer Science, University of Colorado Boulder, CO80309, USA yunwen@cs.colorado.edu Abstract. Software development is inherently a knowledge-intensive and distributed cognitive activity. From this perspective, one of the major issues in user-centered software development tools is to provide support for knowledge collaboration between the developer and the external knowledge resources, which include both information in repositories and knowledgeable peers. This paper describes two prototype systems CodeBroker and STeP_IN that we are developing. The two systems combine to provide continuous support for the acquisition and application of external knowledge resources, designed to follow a design principle we call layered information-on-demand. 1. Introduction Software development is inherently a knowledge-intensive and distributed cognitive activity [16]. It requires knowledge from several different domains. Few developers, if any, have all the knowledge needed in their own heads. The knowledge is distributed between the developer and the external world [9], and the development of a software system requires learning from and integrating the knowledge of various external knowledge sources in the world. Knowledge in the world comes from cognitive tools that support programming and knowledgeable collaborating peers. The development of software is therefore no longer confined to an individual software developer but has to rely on distributed cognition by reaching into a complex networked world of information and computer mediated collaboration. Most existing software development tools have not paid enough attention to the above viewpoints: they are either product-oriented or process-oriented. Earlier CASE tools have mainly focused on the management of products generated in the process of software development, and those later process-oriented tools have mainly focused on the management of the process that dictates how software development should be conducted. Both groups of tools are methodology-oriented because those tools are developed based on an adopted software development methodology to support and manage activities of software development sanctioned by the methodology. Consequentially, such tools view methodology as the most important thing and thus often

2 require that programmers adapt their practice to incorporate the new methodology. Needs and concerns of tool users (software developers) are not fully considered. In contrast, the user-centered perspective--in this case, the developer-centered perspective, focuses on the behavior of software developers and aims at providing support for the cognitive process of software development [10]. Since software development is a process of progressive crystallization of software developers knowledge into a system, lack of needed knowledge is one of the major reasons that cause poor quality and productivity [8]. To create convivial tools that are consistent with the cognitive process of software developers requires the provision of means for software developers to acquire the needed external knowledge as easily as possible. Usability of software development tools in this vein goes beyond the careful layout and design of interfaces, but also include considerations of the way when the external information is presented, how the information is accessed and presented. Another major shortcoming of existing software development tools is that most of them are designed to support individual software developer, although the design process now is inherently distributed. Although there are tools that support the coordination and cooperation of developing software systems, such as concurrent version control systems and groupware systems, but those tools mostly considers coordination and cooperation through the mediation of managing products or processes. They are not meant for the collaborative knowledge construction among software developers, through which knowledge gets transferred from one developer to another developer during the production of software systems and new knowledge gets created. Most existing software development tools do not have integrated support for collaborative knowledge construction. Software developers have resort to other tools and communication media for knowledge collaboration. This paper describes our efforts in creating developer-centered software development tools from the perspective of framing software development as knowledgeintensive and distributed cognitive activities. We use reusable software components as the example domain to illustrate how external knowledge acquisition can be seamlessly integrated with current software development tools. With the advent of objected-oriented technology, reusable software components has now become an indispensable part of the knowledge required for software development. Supporting easy access to needed external information, reusable components in particular, to complement the insufficient knowledge of software developers is thus critical to the improvement of the quality and productivity of software development [5]. 2. Problems with Knowledge Acquisition in Software Development We use the distributed cognition as the theoretical foundation to support our claim that easy acquisition of external knowledge, either from an information repository or a knowledgeable peer is a critical issue in creating usable and convivial software development tools. In particular, we want to develop software development tools that support the two kinds of distribution of cognitive process identified by Hollan et. al. [9]:

3 cognitive processes that involve the coordination between internal and external structure, and cognitive processes that are distributed across the members of a social group. Our previous research has revealed the following cognitive challenges in utilizing external information and knowledge [22]: software developers may not know the existence of useful external information and create suboptimal solution; software developers may not be able to find the right external information; software developers may not be able to understand and apply the new information; and different software developers have different knowledge needs. When perusing the external information itself is not enough for learning, software developers often turn to knowledgeable peers for help, as one empirical study has found that one important factor that differentiates expert developers from novices is that expert developers are able to use other experts faster to help them solve problems collaboratively because they often know the expertise of other experts and have a more reciprocal relationship with one another [2]. Therefore, utilizing external knowledge sources also poses the following social challenges: software developers may not know to whom they can turn for help on this particular problem [1]; and experts who are able to help may not be willing to due to the interruption to their own work and other various reasons [4]. Using one of the typical software development activity software reuse as an example, the following sections briefly introduce two systems that we have been developing: CodeBroker and STeP_IN (standing for Socio-Technical Platform for in situ Networking) that combines to provide a continuous support for timely access to external information and expertise of knowledgeable peers. The CodeBroker system addresses the cognitive challenges in software reuse: being aware of the existence of reusable components as well as being able to locate the needed component from a large repository; and the STeP_IN system helps a software developer find peer developers who have the expertise on the reusable component of interest and who are likely willing to offer timely help. The STeP_IN system is based on a new conceptual framework Dynamic Community that we have proposed to support situated knowledge collaboration. 3. The CodeBroker System Successful reuse requires that a software developer know the reusable component, when to reuse it, and how to reuse it in his or her task. If the developer has learned about the reusable component in advance and the reusable component is a part of his

4 or her working knowledge, reuse is a fairly straightforward activity. However, the vast amount of modern reuse repository, like the API libraries of Java that consists of thousands of reusable components (both classes and methods), makes it hardly possible for software developers to know everything about the reuse repository. The notknown-yet part of the reuse repository thus becomes the external resources that software developers have to utilize for developing software systems efficiently. If software developers realize that some reusable components they do not know yet can be reused in their current task, they might start browsing or searching the reuse repository. However, in many situations, they may not even be aware of the existence of applicable reusable components, therefore they would not make an attempt of locating the reusable components at all and reuse fails in the first place. CodeBroker aims to make software developers aware of unknown reusable components with the information delivery mechanism. It supports Java developers by delivering task-relevant and personalized components components that can potentially be reused in the current development task and that are not yet known to the developers (see [22] for details). CodeBroker is seamlessly integrated with the programming environment Emacs. It consists of an interface agent and a backend search engine. Running continuously as a background process in Emacs, the interface agent infers and extracts reuse queries by analyzing the partially written program in the normal editing space of Emacs (Fig. 1(a)). Inferred queries are passed to the search engine, which retrieves matching components from the repository. The list of retrieved components is then delivered by the interface agent back into the programming environment (Fig. 1(b)) after it removes those components that the developer has known already by using user models for filtering. An adaptive and adaptable user model is used for each developer so that each software developer will get a different set of delivered components based on what they have known already [6]. Fig. 1. The CodeBroker System in Use CodeBroker makes it possible for software developers to reuse those reusable components that they do not even know exist by creating an awareness of those unknown components that are potentially reusable in the current task. If the developer finds one of the delivered components interesting, a click will take him or her to the full document of the reusable components in the external documentation system for Java libraries (Fig. 1(d)).

5 4. The STeP_IN System When reading the document is not enough for software developers to understand how to use the reusable components, software developers need further help. The STeP_IN system that we are currently developing provides the social support for understanding the reusable component of interest. It creates a social platform for knowledge collaboration by superimposing the current Java documentation system with three kinds of support, finding example programs, browsing the archives of past discussion, and sending questions to selected experts. In the new documentation system, in addition to the conventional document created by the Javadoc program, each method in the reuse Java repository is enhanced with three clickable buttons: Examples, Discussion Archive, and Ask Experts (Fig. 2). Fig 2: Enhanced Java Documentation Clicking the Examples button of a method brings example programs that use the method. Those example programs are written by other software developers who are the participants of the knowledge collaboration space of the STeP_IN system. Clicking the Discussion Archive button of a method brings the archived previous discussions about the usage of the method. The archive stores messages that participants of STeP_IN have exchanged in the past using the following Ask Experts support. Clicking the Ask Experts button of a method sends an requesting for help on the method to a small group of experts on the method who are selected from the STeP_IN community. The small group of experts is chosen based on the principles of the new conceptual framework Dynamic Community that we have proposed to support situated knowledge collaboration (see the next section for a brief introduction, for more details see [4]). When the selected experts exchange messages with the developer who is seeking for help, those messages are stored in the discussion archive associated with the method, so that other developers who have similar problems later can benefit by clicking the Discussion Archive button.

6 4.1 Dynamic Community A dynamic community, or DynC, is dynamically formed in a knowledge work space. A knowledge work place consists of a group of knowledge workers, and the knowledge that the workers hold. The group can either be a formal organization such as a company or an informal online community. A DynC is a subgroup of knowledge workers that forms ad hoc in support of a particular user and a particular task, and dissembles as the task is finished. Unlike a static community that forms around a particular domain and exists for a long time [20], a dynamic community forms for a particular task and a particular user, and exists only for a short period. The members of a DynC are selected using the following two criteria: they have expertise on the particular task, and they already have social contacts with the particular user. The first criterion is grounded in the observation that in today s highly specialized world expertise is no more an absolute attribute of person but a relative function of a person and a task. In other words, omniscient experts for many domains do not exist any more, and experts can be identified only after the task is known. The second criterion is grounded in the assumption that existing social contacts between the helpers and the helpee could provide extra motivation to engagement in knowledge collaboration based on the social norm of generalized reciprocity [7]. In summary, a DynC is task-specific: for different tasks, different DynCs will form even for the same user because different groups of experts will be identified; a DynC is user-specific: for different users, different DynCs will form even for the same task because users has different social contacts; and a DynC is in-situ and ephemeral: it is created dynamically when a need for collaboration arose in a real task and dissembles when the task is solved. 4.2 Creating Dynamic Communities in STeP_IN The STeP_IN system can be used to support either an online community or an organization, and the system can be set up to support any Java reuse library. For simplicity of discussion, let s say for the standard Java API library. The system maintains, in its central server, a database that stores three kinds of relation: (1) the relation from a library component to example programs, (2) the relation from a library component to experts, and (3) the relation from a member developer to other members. Upon joining the collaboration space supported by STeP_IN, a member has to perform the following task to initialize his or her profile in the database. First, he or she has to use one of the STeP_IN s subsystems, in his or her own computer, to analyze all the Java programs that he or she has written in the past. The analysis examines what reusable components of the repository have been used by the member to identify his or her expertise on those reusable components.

7 Second, he or she is required to provide a set of Java programs that he or she agrees to share with other members. Those programs will be uploaded into the STeP_IN s central server and will e used as example programs. Third, he or she has to use another STeP_IN s subsystem to analyze his or her s. The analysis mines the s to create an initial social network for him or her based on the exchanges. To protect privacy, the analysis is conducted in the member s own computer and only the resulted relation (nothing about the contents) will be uploaded into the central database after he or she has confirmed and edited with a provided editing interface. The social network is updated whenever knowledge collaboration activities take place in the STeP_IN space. With the database that has all three kinds of relations, STeP_IN is able to locate appropriate examples when the Examples button is clicked. At first, when multiple examples exist, the system will list them in the order of their length based on the rather simplistic assumption that shorter programs are easier to understand. Each example will be associated with a voting mechanism for members to vote on its helpfulness for understanding the reused component. As votes accumulate, examples with more votes will advance to the head of the list of the examples and the number of votes is also shown to assist selection of an appropriate example. When a member (i.e. a helpee) clicks the Asking Experts button, according to the principles of DynC, STeP_IN goes through two steps expert identification and expert selection to create the list of experts who should receive the help request. The expert identification process examines the relation from the component to the experts in the database to find all the members who have used the component before in their programs, and create a list of Candidate Helpers. The expert selection process examines the social relation from the helpee to other members in the database to select a small group of people from the list of Candidate Helpers. Principles for selection are discussed further in the next subsection. The selected people become the DynC Members. An is then sent to DynC Members to ask for their help. The DynC Members who receive the requesting for help can send back their help through s and the s are captured by the system and stored in the database in association with the component. Such messages are then displayed for later members who click the Discussion Archive button. 4.3 Social Factors Considered in STeP_IN To initiate and sustain successful knowledge collaboration requires that experts be motivated to engage in sharing their valuable time and knowledge. In the design of the STeP_IN system, we have taken into consideration of the following social factors as a way of increasing motivation for knowledge collaboration during the process of selecting experts. Economic use of expert attention. Experts have their own work, and sharing their knowledge with others costs extra time and attention. Sending a request to all possible experts demands the time and attention of all of them and creates a huge waste when some of them are not experts on this particular task. To better utilize experts time, attention and good will, we use the expert identification and selection process to limit

8 the number of experts to be contacted to a much smaller one and thus reducing the collective waste of expert attention and time. Utilizing existing social networks. Collaboration happens more spontaneously and smoothly when strong ties exist between participating members [18]. To motivate software developers to participate in knowledge collaboration in STeP_IN, we seek to utilize existing personal social networks of each member to seek those experts that have strong social ties with the helpee. The social network is initially mined from their s, and gets updated when collaboration takes place. The personal social network is also editable and updatable by its owner who can specifically specify whom he or she would like to help and also whom he or she does not want to help. If a person X specifies he or she would not like to help Y, then any request from Y would not go to X even if X is identified as an expert for a component that Y wants to learn about. Promoting reciprocity. Generalized reciprocity is the foundation for long-term collaboration [12]. To promote reciprocity, in the expert selection process, we first seek to enlist the help from those experts who have been helped by the helpee in previous occasions. Avoiding forced collaboration. When a helpee requests help from experts he or she knows well, the social pressure is often shift on the experts because they might fear potential social backlash if they do not provide help and violate the rule of reciprocity. In such cases, the experts are often forced into collaboration even if they might not in a condition or a mood for collaboration at this particular moment. This would be detrimental to the long-term sustainability of the collaboration platform and some experts might simply withdraw from it. We believe avoiding such forced collaboration is essential for creating a sustainable voluntary collaboration platform that relies on the good will of benefactors. In STeP_IN, when the request for help is sent out to the DynC members, the helpee does not know to whom the request is sent, so that even if a DynC member does not offer help due to various reasons, the helpee would not notice. However, any help message from a DynC member to the DynC shows its origin so that the DynC member s contribution to the group is publicly recognized and properly acknowledged. 5. Individualized and Layered Presentation of Information Locating, comprehending, and using information involves at least three different phases: information discernment, detailed evaluation, and information application [3]. In the information discernment phase, users quickly determine whether a piece of information is relevant to their current task. The detailed evaluation phase involves a relatively thorough study of the information. In the information application phase, users need not only to understand the information but also to apply it in their current situation. Each phase requires different levels of details of the information. For the information discernment phase, an overview of the information is enough. This phase also requires the simultaneous presentation of several candidates for users to compare and find the information that is most relevant. For the detailed evaluation phase, users

9 need to focus on the details of a chosen piece of information. Examples that use the information are very effective in helping users understand, adapt, and integrate the information by providing context and enabling users to draw an analogy between their task and examples [13, 15]. The details required for information comprehension and use depend greatly on the knowledge of users. A continuum of demands for different levels of details exists. On one extreme, if the user already knows the presented information vaguely, an overview of the information may be enough. On the other extreme, if the user has never encountered the information before, he or she may need to go through all the three phases, and that iteratively because he or she may not be able to make the right choice at first. Because of the tacit nature of knowledge [14], information repository cannot capture all the knowledge. In such cases, finding an expert on the topic becomes necessary for the user to understand and apply the information [11, 19]. In the design of CodeBroker and STeP_IN that combine to become an integrated socio-technical platform in support of knowledge collaboration in software development, we have followed the principle of layered information-on-demand to guide the design of interaction and coordination between users and external knowledge resources. The design principle of layered information-on-demand aims to provide a natural link from the task at hand to a variety of external information and knowledge resources, which are presented to users at different levels of abstraction in response to their different needs. Depending on the working context and background knowledge of a user, a unique network of information and knowledge is dynamically constructed and presented to provide a seamless transition from users interaction with tools to interaction with external information and other knowledgeable peers. Structuring information into different layers and presenting each layer on demand to accommodate the different needs of users has its root in the concepts of progressive disclosure and details on demand [17]. Progressive disclosure is a design principle to reduce the complexity of user interfaces by presenting the most common choices to users while hiding more complex choices or additional information, which is displayed when needed. Details-on-demand is widely applied in information visualization systems that allow users to obtain details about a selected element from the visualized overview of an information set. Layered information-on-demand supports a new interaction style between information systems and users that focuses on the economical utilization of human attention by presenting information in different levels of details and on demand from overviews to details to examples and to experts. This design principle accommodate the differing needs of each individual user by putting control at the hand of users, who can stop at any level whenever they deem the presented information enough. For experts, the system remains brief and simple, without taking too much attention from their real task; for novices, the system has a complete socio-technical support, using human experts as the social infrastructure to back up the inadequacy that is inherent in knowledge repository systems due to the tacit nature of knowledge. The following subsections describe the layers of information that are provided in the two systems.

10 5.1 Autonomous Delivery of Task-Relevant and Personalized Information Overviews In CodeBroker, software developers do not need to explicitly input queries for external information. As software developers enter document comments and method signatures in the editing space (Fig. 1(a)), the CodeBroker system extracts the comments and signatures and uses them as a query. A list of task-relevant components that match the query is autonomously retrieved from the Java API library. Different users have different knowledge and needs different set of external information even for the same task. CodeBroker, therefore, uses user models to personalize the retrieval results before they are delivered. The retrieved task-relevant components are compared against the user s user model that contains the components he or she already knows. The known components are removed because the developer would be able to reuse those known components by himself or herself if they are reusable in the current situation. The personalized task-relevant components are delivered into the delivery buffer (Fig. 1(b)). Each delivered component has the rank of the relevance between the component and the current task, the relevance value, the name, and the synopsis of its functionality. This is a context-aware list of reusable components for the developer to browse and serves as the first layer of information presented on demand. Not all the components are useful for the current task, but many of them are. In fact, traditional recall and precision measurement shows that for achieving 80% recall rate, the precision rate is 33.71%. In other words, about 1/3 components returned could be used to implement the task in one way or another [21]. 5.2 Mouse Movement-Triggered Information Another important piece of information for a Java API method is its signature. We decide not show it in the delivery buffer because signatures are usually long and takes too much of the scarce screen resource. Furthermore, signature becomes relevant only when the developer finds the reusable component has the potential to be reused. In this sense, the signature of a component is secondary information needed to determine the reusability of the component. Therefore, only when the developer moves the mouse pointer over the component names in the delivery buffer (Fig. 1(b)), which triggers the display of the component s full signature in the mini-buffer (Fig. 1(c)), the second layer of information is presented on demand to assist the developer in determining further the relevance of the component. The full signature shows the package, the class, and the types of input and output data. To note that all the interaction with CodeBroker by far was conducted within the programming environment. This makes the location of reusable components a natural extension to the developer s current programming practice because he or she does not need to make a conscientious decision to shift from programming mode to reusing mode.

11 5.3 Jumping To The Documentation When the developer finds one promising reusable component and wants to know more about the component, a click on the component name brings up a web browser that shows its full documentation (Fig. 1(d)), and with the integration of the STeP_IN system that are still under development, the developer will see Fig. 2, with three new buttons added to the standard Java documentation. 5.4 Finding Examples If the developer still does not know how to use the component, an example of its use might be helpful. He or she can then press the Examples button (Fig. 2), which has been added to the standard Java documentation system in STeP_IN, to get an example program that illustrates the use of the component. 5.5 Reading the Discussion Archive When an example is still not enough for understanding or using the component, the developer can, by pressing the Discussion Archive button, to the archive of previous discussions that are saved from previous information exchange about the component between helpers and helpees. 5.6 Finding Experts If the developer s question does not have an existing answer in the discussion archive, he or she can press the Ask Experts button to formulate a dynamic community for knowledge collaboration. As we have mentioned in Sections 4.1 and 4.2, this dynamic community is user-specific, it is different for different users because each user has different inter-personal relationship network. The discussion that takes place afterwards is saved in the Discussion Archive for the benefit of later developers. 6. Summary To improve the usability of software development tools, we have to focus on the needs and concerns of software developers. Since software development is inherently a knowledge intensive and distributed cognitive activity, in addition to methodological support, user-centered software development tools need to provide support for knowledge collaboration with external knowledge sources that include both information repository and knowledgeable peers. Successful knowledge collaboration requires the awareness of available external information, the identification of knowledgeable peers, and the willingness of the peers to participate in collaboration, as well as the obtaining of such information and expertise in a timely manner. This paper

12 described our two research prototypes CodeBroker and STeP_IN that combine to provide continuous support for knowledge collaboration in software development. The former creates the awareness of and the access to unknown reusable components, and the latter creates and facilitates collaboration among software developers as a social backup to complement the insufficiency of reuse repositories in support of software reuse. The two systems are designed under the guidance of the principle of layered information-on-demand that can satisfy the wide variety of individual needs for external knowledge. Acknowledgments. This research is supported by the Ministry of Education, Culture, Sports, Science and Technology of Japan, Grant of the Open Competition for the Development of Innovative Technology program. References 1. Becks, A., Reichling, T., and Wulf, V.: Expertise finding: Approaches to foster social capital, in Huysman, M. and Wulf, V., (eds.): Social capital and information technology , The MIT Press, Cambridge, MA, Berlin, L.M.: Beyond program understanding: A look at programming expertise in industry, in Cook, C.R., Scholtz, J.C., and Spohrer, J.C., (eds.): Empirical studies of programmers: Fifth workshop. 6-25, Ablex Publishing Corporation, Palo Alto, CA, Carey, T. and Rusli, M.: Usage representations for reuse of design insights: A case study of access to on-line books, in Carroll, J.M., (ed.) Scenariobased design: Envisioning work and technology in system development , Wiley, New York, Cross, R. and Borgatti, S.P.: The ties that share: Relational characteristics that facilitate information seeking, in Huysman, M. and Wulf, V., (eds.): Social capital and information technology , The MIT Press, Cambridge, MA, Detienne, F.: Design strategies and knowledge in object-oriented programming: Effects of expertise. Human-Computer Interaction, 10(2/3): , Fischer, G.: User modeling: The long and winding road, in Kay, J., (ed.) Proceedings of user modelling , Springer Verlag, Banff, Canada, Fischer, G., Scharff, E., and Ye, Y.: Fostering social creativity by increasing social capital, in Huysman, M. and Wulf, V., (eds.): Social capital. (in press), Green, T.R.G.: The nature of programming, in Hoc, J.-M., et al., (eds.): Psychology of programming , Academic Press, New York, Hollan, J., Hutchins, E., and Kirsch, D.: Distributed cognition: Toward a new foundation for human-computer interaction research, in Carroll, J.M., (ed.)

13 Human-computer interaction in the new millennium , ACM Press, New York, Jarzabek, S. and Huang, R.: The case for user-centered case tools. Communications of the ACM, 41(8):93-99, Mockus, A. and Herbsleb, J.: Expertise browser: A quantitative approach to identifying expertise., in 2002 International Conference on Software Engineering , Orlando, Florida, Nahapiet, J. and Ghoshal, S.: Social capital, intellectual capital, and the organizational advantage. Academy of Management Review, 23: , Neal, L.R.: A system for example-based programming, in Human factors in computing systems (chi'89) , ACM Press, Austin, TX, Polanyi, M.: The tacit dimension, Doubleday, Garden City, NY, Repenning, A. and Perrone, C.: Programming by analogous examples. Communications of the ACM, 43(3):90-97, Robillard, P.N.: The role of knowledge in software development. Communications of the ACM, 42(1):87-92, Shneiderman, B.: Designing the user interface: Strategies for effective human-computer-interaction, 3rd edition, Addison-Wesley, Reading, MA, Taylor, J., Sumner, T., and Law, A.: Talking about multimedia: A layered design framework. Journal of Educational Media, 23(2/3): , Vivacqua, A.: Agents for expertise location, in Proceedings of 1999 AAAI Spring Symposium on Intelligent Agents in Cyberspace. 9-13, Stanford, CA, Wenger, E.: Communities of practice learning, meaning, and identity, Cambridge University Press, Cambridge, England, Ye, Y.: Supporting component-based software development with active component repository systems. Department of Computer Science, University of Colorado, Boulder, CO, Ye, Y. and Fischer, G.: Supporting reuse by delivering task-relevant and personalized information, in Proceedings of 2002 International Conference on Software Engineering (ICSE'02) , Orlando, FL., 2002.