Open Source Bug Tracking Characteristics In Two Open Source Projects, Apache and Mozilla Mikko Koivusaari, Jouko Kokko, Lasse Annola Abstract 1. Introduction 2. Bug tracking in open source projects 2.1 Bug tracking 2.2 Apache Project 2.2.1 Bug process in Apache 2.3 Mozilla Project 2.3.1 Mozilla bug process 3. Conclusions References Abstract Bug reporting systems are very important part of projects and nowadays their importance is rising as we speak. What is the right way to do bug reporting and are there tools that do it better than others. These are very important questions to be answered and in this article we will take part of analyzing two big bug reporting systems and look objectively at the problems the users face and what advantages they can gain from usage of these particular systems.
1. Introduction The research question in this article revolves around bug reporting systems in open source software development. We are taking two quite popular tracking tools and processes into consideration. These tools are Bugzilla developed by Mozilla and Jira by Atlassian that is used for bug reporting by Apache Foundation, one of the biggest IT projects ever. The main point is to find out why are bug tracking software s useful and what kind of differences are found in the processes when using either one of these tools compared to traditional ways. Where the bug reporting is managed only by mailing lists or rests solely on the shoulders of developers to find the bugs inside the software. How easy and efficient bug reporting, tracking and fixing can be with these tools. Bug tracking itself is not a new revolution and is very crucial especially in open source projects where the developer base can be wide and spread out in different continents. In companies who provide software for proprietary use, bug tracking is often done by a tool called Jira. Jira will be covered in this article and represented as one of the very big systems in bug tracking. Jira can be used in more widely than just as a bug tracking element in a project, as it has capabilities to expand into a full project management tool. It has strong roots and is a widely used system for project management in IT sector. Bugzilla has a very wide user base and is developed by one of the biggest open source communities ever, called Mozilla. Originally it was developed and started by Netscape, which later changed name into Mozilla. It is described as a very light, easy to use bug tracking tool that is mainly being used in web development bug tracking. It can be found in almost every bigger web project. Despite the popularity of these tools, it is important to take an objective look into these systems and give a full coverage why and when they are best to use, process wise. An important point is to know if these tools bring advantages to open source software projects through usage, in all aspects of a project. There has to be clear separation for these tools and the way that bugs are handled in them as most of the bug reporting systems are quite similar and work as a database interface. Bug reporting is one of the most important aspects of open source software development, when wanting to keep the project long lasting and the user base happy. Bug management processes tend follow the following pattern as described by Yonchang et al (2011). IEEE software maintenance process starts by developer, user or manager s need to change something in the software. This request is then classified under the corresponding feature. After classification the request gets analysed. Analysis determines possible solutions and effects the requested change may cause. Design phase summarizes all available solutions and tries to provide the best possible solution for request. Implementation phase involves the actual programming and testing the modifications. After implementation the logical step is system wide testing, when the whole system is brought up to test the changes. This is where the overall effect of the change should be identified. Version building and release management takes care of acceptance testing before releasing new versions of program. The last part is to deliver the version to users.
2. Bug tracking in open source projects 2.1 Bug tracking Bug tracking is based on a very simple idea which is no different from reporting a crime at the police station. It s based on the fact that the finder of a certain problem can articulate the bug forward in such way that someone can make use of the report and use it to improve the end product. Often bug reports are considered as the key elements in open source project version development. Bugs are the necessary evil in software development and the bigger the product the more bugs it will have during its lifespan. Anvik, Hiew & Murphy (2002) states that the bug fixing workload is remarkable requiring two person hour per day for bug fixing activity. It is up to the community to manage the coordination and make sure that the future challenges are reported in good fashion and someone, even in the far away in the future can use the bug reports for fixing the problems. Bug tracking tools are used as project management and coordination systems. What is especially a good thing in these tools is that they automatically produce documentation about the ideas behind modifications and provide version history of the project. Which would not otherwise often exist. This makes release management and version controlling easier. Bug tracking enables easy task sharing and gives potential developers information about what could be problematic with the project (s)he is going to participate in. Tracking bugs is usually made by dividing the actual problem into smaller pieces. There the user will fill information about the event that led into the actual crash or malfunction. There can be many ways to achieve the best result and to get the most information from users that make these reports. The usual workflow for a bug report is the following, stated by Zimmerman et al. (2009), first the user finds the actual portal where to submit the bug, for example Apache projects JIRA portal. The workflow is quite simple as the user who takes part in the project or uses the software will make open a bug report in the following order: I tried to make an archive and the program crashed with error code 777. I have an ios application using only local libraries from ios core and I tried to archive my app into a deliverable and Xcode programming environment crashed after a long wait. I am using version X.X with OSX version X.Y. I am experiencing this bug every few weeks as I archive applications. Here is a quite simple explanation that is usually put in the system which then moves it into the database as a NON VERIFIED OPEN bug. From the system other developers can try to recreate the bug and maybe find a solution where the bug exists. People including the user who submitted the bug can follow the lifespan of the bug report and see if its being checked or
commented. Usually there is a way to change the state of the bug into critical or such to keep the bug in a different order on the list or keep it as low state bug as it is not critical and will not be included in the next release. There are different things considered as good information by the developers and stated by Zimmermann et al. (2009) is the following Items such as stack traces, steps to reproduce, observed and expected behavior, test cases, and screenshots ranked high on the list of preferred information by developers. Here one can see the most important information that can be put into a bug system, stated by the people working in and with Apache and Bugzilla. The bug systems and information for reporting are in the end up to the human being using it and its best when input to the system is quite strictly organized as its easier to follow and there are not bugs that can t be articulated or reproduced as they give too less information about the actual problem. These are the bug reports seen often in proprietary software projects, given by the customer as they feel like they shouldn t be forced to give a huge time effort for the bug report. This is understandable as the customers paid for the product. As stated already above software bugs are very prominent in software development and while hardware is evolving very vastly, software seems to have the same problems when talking about bug reporting and so on. Bug reporting systems are made to help but there are bugs that run behind the scenes and can cause dramatic damages if they are not noticed soon enough. Usually these bugs are not found right away since they might be the cause of something made before and can drift with the software for a very long period of time. Usually bugs are logical errors in the code that no one just never had the time to clear, or no one had the time to debug them away. Bugs themselves are quite abstract and can affect the product in many different ways (Hangal & Lam, n.d.). Bugs can bring a new dimension to the project as a taker of workforce. In open source projects it s very common that bugs are fixed as they would be their own little entities. People tend to take care of an area where the bug is found and then, they might spend lots of time trying to fix the bug from where it was found. Bug reporting is one of the key elements in a open source project and usually there is a tool behind the scenes, that is used to manage the whole bug database. Often bug reporting tools are just database to webpage interfaces, which allow you to make entries and modify entries via the web interface to the database. This is however not the case for example Bugzilla developed by Mozilla and used widely for Mozilla Firefox development. With Bugzilla it has been noticed that bugs are often fixed by core persons that take part in 80% of the programming happening in a project, and so they are the most qualified also for bug fixing. The support community for bug fixing are usually the ones reporting bugs and it has been noticed that the core developers often need additional information about the bugs and so they keep close contact to the reporters via the portal of BugZilla for the specified bug (Masmoudi & Boughzala, 2013).
2.2 Apache Project Apache projects and products use multiple different bug or issue tracking tools. The project itself is one of the largest open source software projects that exists. Apache provides the most used Web server platform plus some additional side projects and interesting products. Thats why its reasonable to have multiple issue management and bug tracking tools available. The two main issue tracking tools that are: Bugzilla and Jira tools. Apache also has its own Bloodhound tool. Jira is a platform provided by Atlassian, that is available for free to qualified open source projects. Other groups have to pay a monthly fee in order to use the Jira management tool. Mockus et al. (2002) have studied project Apache before these tools and back then there was BUGDB, where all the issues were stored. The issues had attributes like: person responsible, affected module, status, priority, submitter, dates and comments. The biggest issue with BUGDB used to be that very few developers actually took a look at its content. The Apache Group is in the charge of pointing out the most important bugs, that actually require fixing. Here lays the problem about centralizing the bugs into a more typified system base, as Atlassian JIRA offers. 2.2.1 Bug process in Apache The usual case for a bug report process is the following: First someone encounters an issue with one of the Apache products. Then (s)he writes a bug report about it. If the quality of report and the issue found is severe enough someone from Apache Group reviews it and tries to find a volunteer person to write a fix for it. The core developers usually like to work with things that they are most interested or have most experience in. The modular approach allows this kind of task distribution and often leads to better results when the developers are considered as the managers of their modules. This way the original creators or the active developers may have more knowledge and more respect among the community and specially among the other core developers. So even if someone finds a solution to a existing problem they may be rejected for various reasons. (Mockus, 2002) It s easily forgotten that many peoples brains just aren t wired the same as ours, and they just don t understand what we re asking for in bug reports, and why! (Just et al.,2008). Bug reports often provide insufficient amount of information to actually do something about the issue. Bug reporters usually are not really developers, so that must be taken into account when analyzing the quality of reports. (Mockus, 2002) Once a solution is found it goes through a pre release testing, where testing is done on locally and on developer controlled server. This approach relies heavily on the expertise of the developer. Developer has to know what kind of behavioral changes or logical changes the fixes may have caused in order to identify possible new problems. This is a problem that has
been studied largely by different groups, how to create a classified structure that helps to maintain the actual software. This is very crucial so that even new developers can easily check where a change might break something else. In software projects as old as Apache there can easily be regression when changing something, and there might be a reason why the bug is left there. Since there is no other easy way out from the problem. (Mockus, 2002) This nature of producing new issues by fixing older ones, can cause more workload than just finding a way around the existing issue. After the testing phase the changes go through inspection. For a bug to pass to stable releases bin, they have to be reviewed. However the development releases are reviewed after committing the bug. Each commit gets documented and generated as a change in the summary of the whole project. (Mockus, 2002) This feature is one of the most valued from the perspective of the project management and release managers. When the reviews and all the other steps have passed the release, managers take over and try to find out anything that developers might have missed according to the changes made. Release managers role is to handle packaging the release and test the whole version. In the end if a bug is passed into the production stream the release manager is responsible, more than the actual developer. The developer mission was to fix a bug and the release manager is there to check that anything new that was changed did not break something else in the code. (Mockus, 2002) 2.3 Mozilla Project Mozilla open source project started in March 1998. Starting codebase for Mozilla was Netscape Navigator 5 codebase, which was donated by Netscape Communications. Because the original codebase based on commercial one, the requirements and documents were difficult to reconcile with open source practices as well as some components were to remove because of licensing and legal issues. All these issues caused needs for re coding and changing actions for some components in the open source codebase. The Mozilla development process took several years and in this open source development phase was found several needs for tools and common practices to coordinate the concurrent and distributed development work. (Reis, C. R., & de Mattos Fortes, R. P. 2002) In Mozilla project was developed following tools: Concurrent Version Tool (CVS), Bonzai, Tinderbox, LXR and Bugzilla. CVS is a version controlling tool working in a client server fashion. Bonzai and Tinderbox are tools which provide access to code changes. Bonzai is a query interface to CVS repository. Tinderbox is an automated system that tracks compilations and tests. LXR is a hypertext tool that that indexes source code and generates HTML pages. Bugzilla is a bug tracking tool and was originally developed by Netscape, Terry Weissman using a programming language called TCL. Weissman ported Bugzilla code from TCL to PERL, and it still remains under PERL today. Bugzilla is an open source program after it was spun off by Netscape in 1998 as a part of an Mozilla open source project.
Barnson (2001) discusses that Bugzilla is a very popular bug tracking system, which is used by many open source projects and hundreds companies around world. Success is based on its usage in Mozilla project and also its license fee free usage. (Reis, C. R., & de Mattos Fortes, R. P. 2002) 2.3.1 Mozilla bug process Figure 1: Bugzilla bug tracking process. Bugzilla process found on (http://www.bugzilla.org/docs/3.0/html/). Delugach (2007) has studied that the bug tracking process in Mozilla follows ISO/IEC 12207 standard and even adds to it. According to Remillard (2005) Bugzilla is a Web application meaning that interacting take place with HTML pages. Bugzilla uses MySQL Database and it is the only supported
database at the moment. Remillard (2005) discusses that in Bugzilla bug of a code or an enhancement of a code can be processed basically on the same way. Typical information of a bug record in Bugzilla consist of description of identified bug, related components, developers and current status of bug fixes and also verification. Bugzilla also provides a link to modified code for each bug fix. (Shin,Y., Williams, L. (2008)) Bugs are found by users or developers and then reported to a bug database. Then bugs are either confirmed or rejected. Typical reason for rejection is duplicate or badly written report. After acceptance the state of bug is set to new state. Then bug gets assigned and then the person responsible tries to solve the issue or figure out the possible solutions for it and then ask for other developers opinions.this is approach to bug fixing is common when the bug affects many areas of the overall product. In Mozilla bug owner has no privileges to change state of a bug. After developers have come up with a solution The solution gets tested and verified and maybe included in upcoming packages (Delugach, 2007). 3. Conclusions While studying the we found that there is an issue about how to actually compare if one is more effective than the other. What are the metrics to take into account while comparing. Is it bug lifetime, size of community around the project, amount of active developers or combination of these and few other possible metrics. Does the tools used in these projects have effect on the processes. Both of these bug tracking tools seem to provide certain set of functions and features. Every bug has a title, relative part of project or feature, date reported, state, reporter and usually there is person or team responsible. The bug management process in both of these projects follow closely the IEEE model provided by Yongchang et al (2011). All the steps in this model can be identified from Apaches and Mozilla s maintenance processes. Although there is a bit different approaches and variations in the way they are done. And majority of open source requests gets rejected, because of the inability to reproduce the situation or badly written reports. There could be more automation involved inside the bug management. Something like automated analysis based on certain keywords. Automated setup to make bug recreation easier. Many bugs get rejected because of developers are unable to reproduce the bug. Another interesting point of improvement could be to make the decision making less core developer centric. This would naturally bring up new issues, like how to keep the project on its course and who has the responsibility of making decisions according to projects. But if the lifetime of bugs start to exceed the lifetime of product version, that is something that users don t want to deal with (Lotufo, T., et al, 2012). Reducing the required work of core developers
might give them extra time to focus on some new features or improvements, which could make the overall product better. The existing literature about Apache s modern tools was limited. Analyzing these processes, available tools and figuring out if there is some fundamental reasons why other projects do better with their issue management, could result in as improved process models and new features to the existing systems. With Apache as with Mozilla the core users are very much part of the changes happening and so there might be some room to give more freedom for the low level developers / users. The bigger differences that we can see are the way that each tool is being constructed against the database. As JIRA was developed for a bit different purpose before and later modified to fit a more specific problem scheme, it is not as flexible as Bugzilla is. However, also the contender Bugzilla has been evolving for the whole time span of its life and has been previously developed under a different core. What comes in the future is up to the actual community that develops these programs as only the key professionals can change what happens. They are the ones that can shift one product into a direction that actually makes sense for the actual core users. Processes in both projects are very similar, but the key difference seems to be the portal look and feel of JIRA. This is understandable as JIRA is based on top of a bigger system and handles a larger base of users. However look and feel can affect tremendously on the actual process and how people handle these processes (GNOME, n.d.). This also means that even if it is not considered as being a straight difference in the system it certainly is an indirect one. This can be overlooked easily but in order to get broader look into the look and feel of an system made for productive use one must understand more how the human brain functions with colors and patterns. This is more about user interface design than straightforward software development, yet still it has to be considered well enough when developing and using bug report systems. BugZilla is quite hard looking system, yet it is extremely easy to understand. This is a good point to taken into account when dividing these two systems and the way they work overall in their process flow. In more research to do it has to be noted that there needs to be more information shed around the user interface ideas of an open source or proprietary bug tracking system. All in all its important to notice that bug reporting per se is not the problem that lies before us but rather the way its being implemented into a software in front of us. There has to be ways to create different software for different purposes and there might be some room in this to check the whole abstract framework and make it into a more specific grid that it can be implemented in a new more user friendly way. When choosing which tools to use you have understand the nature of your project and does the project involve other management tools. References Anvik, J., Hiew, L., & Murphy, G. C. (2006). Who should fix this bug?. In Proceedings of the 28th international conference on Software engineering (pp. 361 370). ACM.
Barnson, M. (2001). The Bugzilla Guide. Retrieved October 29, 2014 from http://ftp.task.gda.pl/mirror/sunsite.unc.edu/pub/linux/docs/linux doc project/bugzilla guide/bu gzilla Guide.pdf Delugach, H., (2007). An evaluation of the pragmatics of web based bug tracking tools. ICPW 07: Proceedings of the second international conference on Pragmatic web. ACM. GNOME. (n.d.). No Title. Retrieved from https://developer.gnome.org/hig/stable/ Hangal, S., & Lam, S. M. (2002). Tracking Down Software Bugs Using Automatic Anomaly Detection. In ICSE 02 Proceedings of the 24th International Conference on Software Engineering (pp. 291 301). Just, S., Premraj, R., & Zimmermann, T. (2008). Towards the next generation of bug tracking systems. Visual Languages and Human Centric Computing, 2008. VL/HCC 2008. IEEE Symposium on, pp. 82 85. Lotufo, T., Passos, L., Czarnecki, Krzystof. (2012). Towards improving bug tracking systems with game mechanisms. Proceedings of the 9th IEEE Working Conference on Mining Software Repositories (pp. 2 11). Masmoudi, H., & Boughzala, I. (2013). Open Source development: Examining bugs reporting activity among mozilla community participants. Communications and Information Technology (ICCIT), 2013 Third International Conference, 126 130. doi:10.1109/iccitechnology.2013.6579535 Mockus, A., Fielding, R. T., & Herbsleb, J. D. (2002). Two case studies of open source software development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology (TOSEM), 11(3), 309 346. Reis, C. R., & de Mattos Fortes, R. P. (2002). An overview of the software engineering process and tools in the Mozilla project. Retrieved from http://ww1w.ics.uci.edu/~wscacchi/software Process/Readings/Mozilla study.pdf Ren Yongchang, Liu Zhongjing, Xing Tao, & Chen Xiaoji. (2011). Software maintenance process model and contrastive analysis. Information Management, Innovation Management and Industrial Engineering (ICIII), 2011 International Conference on,, 3. pp. 169 172. Shin, Y., Williams, L. (2008) Is Complexity Really the Enemy of Software Security? Proceeding of the 4 th ACM Workshop of Quality of protection. New York USA. 47 50. doi 10.1145/1456362.1456372
Zimmermann, T., Premraj, R., Sillito, J., Breu, S. (2009). Improving bug tracking systems. Software Engineering Companion Volume, 2009. ICSE Companion 2009. 31st International Conference (pp. 16 24).