Successful Product-based Agile Software Development without Onsite Customer: An Industrial Case Study

Transcription

1 Successful Product-based Agile Software Development without Onsite Customer: An Industrial Case Study Irum Inayat *1, Muhammad Asim Noor 2 and Zubaria Inayat 3 1 Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia 2,3 Department of Computer Science,COMSATS Institute of Information Technology, Islamabad, Pakistan 1* irum@siswa.um.edu.my, 2 asim_noor@comsats.edu.pk, 3 z.inayat@gmail.com Abstract Software industry is increasingly using Agile Methods due to its benefits. However the way industry practices agile methods is some time divergent to the basic Agile Principles. For example customer involvement in the entire software development process is considered essential; but under certain conditions this requirement is not so critical. We have studied in detail such an adaptation by a large software company using Agile Method i.e. SCRUM. The organization develops software products using SCRUM; however, after initial development; they customize the product for their potential customer. At no time (during initial development or customization stage), real customer is present at the development site; but still their products are immensely successful in terms of number of products sold, and customer satisfaction. In the real world case study we have analyzed reasons for the success of the organization and have studied in detail the circumstances which obviate the need of readily available customer. The results show that, extensive domain knowledge, prior experience of the product and experienced workforce mitigate the need of onsite customer. Keywords: Agile Software Development, Off-Site Customer, Product-based Agile Software Development, SCRUM, Case Study 1. Introduction Agile Software Development is becoming increasingly popular in software industry [1, 2] XP [3] and SCRUM [4] are formalized methods which embody agile values and principles, espoused by the agile manifesto. Resident customer is considered essential in these methods. However the case study reported in this paper shows that under certain circumstances, customer involvement thorough agile software development life cycle is not that essential. The paper aims to study what factors obviate the need of resident customer for the organization using agile method (SCRUM) and how absence of the customer impacts the business success of the developed product. In order to study the problem we conducted an industrial case study with a large software company. For secrecy purposes we rename the company to Omega. It is an international software company which has tens of successful products. The company has its clients in 70 countries with employees strength over The case study focuses on Product which is a rapid application development and production tool for high-volume, personalized document publishing. It describes the impact of an off-site customer on development cycle particularly its effects on understanding software requirements. This purpose is accomplished by tracking the life cycle of a customized version 1

2 of the product mentioned above i.e. application development and production tool for high volume personalized document publishing. The domain of Omega, the organization under discussion is developing Content Management Systems (CMS). Omega produces CMS and launches them to market. Later on customized versions of already launched CMS are developed in order to meet customer s needs and cater upcoming market challenges. The customized product we considered in this paper is also a CMS dealing with documents more precisely it is a Document Management System. This DMS is studied in detail through its development lifecycle in order to prune impact of customer s absence while following Agile Method. Information is gathered through observation by onsite presence and (semi structured) interviews of concerned team members. To scrutinize the success or failure of this product based organization several perspectives are chosen such as: customer involvement in product development lifecycle, requirements gathering with an Off-site Customer and empirical evidence. Issues raised due to absence of customer like communication issue, misinterpreted requirements giving vent to rework and more iterations are also discussed in detail The result of this case study is the success analysis of product based organization with an off-site customer. This success is analyzed by taking into account the customer satisfaction, amount of rework done, stability in market and by reviewing maintenance data. The composition of this paper is in such a way that the second section shows a brief overview of literature review related to customer involvement in Agile software development methods and. The third section shows the case study design. The fourth section comprises of empirical findings. The fifth section focuses on the analysis of these findings. The sixth section concludes the findings. 2. Literature Review Agile software development methods are an established practice in software industry [1, 2]. The increasing use of Agile Methods is due to their being people centric, adaptive, and incremental/iterative [6]. This is all we mean by being Agile. It shows that organization perform best when they refine their processes instead of throwing them while adopting agile methods [7]. In literature its felt that many scholars focus more on the quantity of methods used in the industry, few emphasize quality of the Agile practices [5]. The quality of Agile practices refers to find the match between industry practice of Agile and basic principles. The basic Agile Principle says that customer involvement is a critical success factor insofar definition of the project scope is concerned [10]. But in real world software industry is not following it as it is. With context to focus of our research it is found that having fulltime resident customer is rarity instead of common practice [7] in present day software industry. Studies show that it also not absolutely essential to have onsite customer 100% of the time [8] as (s)he is needed at most 21% of the software development time. Customer absence is compensated in several cases by user representative or Project Manager acting as a proxy customer. 3. Research Questions The research questions investigated in this case study are: Q1: Is product based software development successful without On-site Customer? It is established that Agile methods stress on customer co-location which is found contrary to present day industry practices. Most of the customers being abroad and geographically dispersed cannot manage to be on site throughout the process. Especially product based 2

3 software development organizations have very limited customer interaction. Being the originator and financer of their own products Product based software development companies do not possess any outside customer. Yet customer is there when it comes to customized versions of launched products. This is the topic of discussion in this paper. This case study studies the impact of customer s absence in customized product based software development. This research question has other sub questions like: q1: How absence of customer is compensated? q2: How success of organization regarding certain product can be analyzed? Q2: what are the circumstances which obivate the need of readity available onsite customer? This case study identifies the factors which help Omega being successful in the market apart of its deviation from Agile Principle of having no onsite customer. The factors are deduced from this case study by studying the development and post delivery scenario in detail. 4. Case Study We conducted a real world case study in software company Omega to assess the impact of the nonresident customer on the success of the product with the perspective of customer satisfaction. Omega is primarily a product base company i.e. developing products of its own and launching to the potential market. However they offer customized products for their clients with respect to client s organization demands. So there is two kind of product development 1. Initial product development and 2. Customized Product Development or Product Customization. Under both scenarios (initial product development and product customization) there is no real (external) customer present on site throughout the development life cycle. In rest of the section we introduce the, company and product, structure of the development team, research methodology and work flow. 4.1 Research Design of the Case Study We designed some measures for carrying out this research such as: Productivity is measured by considering story points per sprint, number of cycles per sprint, function points per sprint and time to market. Observation of two teams working on modules of Product under discussion for 6-8 hours per week. Semi-structured interviews of the team members. Questionnaires administered to team members including customers or customization indenters to access their satisfaction. We observed the working scenario of the team by being on site. In addition to on-site observations semi-structured interviews with the team; was the major source of data collection in this case study. We had multiple rounds of discussion with key persons over the period of 8 months. In first round we collected information regarding product, its history, development methodology, experiences of the development team and market success of the product. The team shared some key documents such as work backlog, requirement specifications in the form of story cards, work schedule, effort distribution charts and product burn down chart with us. This gave us a clear insight of details involved in product development and its working phenomena. Later round of discussion was focused on clarification and validation of already gathered information. 3

4 The details of interviewees along with their experience and qualification are show in Table 2. Team members contacted were well aware of the work flow due to their extensive experience with the company for past several years. They have witnessed several initial and customized products life cycle from scratch till the launch. Table 1. Interviewees from Omega along with their Experience and Qualifications No Role Years of Qualification Experience 1 Project Manager 5 MS Computer Science TEAM 1 2 Team Lead 5 BCS 3 Marketing Manager 3 MBA (Marketing) 4 Senior Software developer 4.5 BS(CS) 5 Software developer 3 BS(CS) TEAM 2 2 Team Lead 5 BCS 3 Marketing Manager 3 MBA (Marketing) 4 Senior Software developer 4.5 BS(CS) 5 Software developer 3 BS(CS) 4.2 Introduction of Company and Product Omega is an international software development company started in 1985 and has presence is 70 countries. Its employee strength is over It focuses on development of content management systems especially dynamic document publishing. The product X, subject of the case study is a comprehensive document publishing solution. It has four customized versions launched since its initial development in Development of one of these customized versions named X-1 is the focus of this study. The original size of X and its subsequent customized increments is given in the Table 2. The size information is important for the company in order to keep record of finances and efforts. Project Manager mentioned about keeping track of this information as: It helps us define the productivity rate, defect rate and helps us define the product customization cost. Table 2. Size of the Original Product and its Customization Increments Product Lines of code Functional points Product X Customized Version Customized Version 2 (X-1)

5 The product X-1 took 32 weeks and 3 days for completion other details regarding number of sprints, sprint sizes, story points and story points per sprint are mentioned below in Table 3. As mentioned above this information is kept by company to keep track of their finances, efforts and productivity. 4.3 Workflow at Omega Table 3. Details of Customized Version 2 X-1 Development Time 32 weeks 3 days Sprints 16 Sprint Size 2 weeks Story points 370 Story Points/ Sprint 11 approx The work flow followed by Omega during development of X-1 is described as below (taken from [11]) Product Innovation: Concept and Management teams decide the specifications of new product. Both the teams organize sessions with potential customers and established clientele to gather the in demand features for new product. Technical experts are on board for suggesting the best suitable and advanced platforms and tools to be used. The customized versions of launched products depend upon customer s indentation, feedback after product demonstrations and to cope with challenging upcoming technology. Technical and business people collaborate with the customer at this stage Requirement Gathering: It is observed while being at Omega that requirements come through Marketing Department to Professional Services Department and Project Manager which are then communicated to Development Manager trickling down finally to Developer through , face to face interactions and Skype Approval of Proof of Concept: The Proof of Concept is a materialization of proposal at prototype level. First it is discussed among Project Managers and Marketing and later on presented to the customer for final approval Development: Project Manager owns the customization on behalf of customer and communicates the requirements to developers. The code is later on discussed in daily and weekly sprints. Prototypes are developed by Concept Team, responsible for developing Proof of Concept earlier Testing: The code is tested by Software Testing Group (STG). The interviewee mentioned that they often use Stress testing, and combination of white box and black box testing. The tests are conducted as and when coders hand over a module to testing team. The testing team performs the tests and report to the test team lead. The test reports are shared with the development team in daily or weekly sprint meetings Demonstrations: For internal demonstrations the product or module of the product is presented to Project Manager and Professional Services (PS) in order to be presented to prospect customer later on. This shows that developers never get to meet the customer. 5

6 Maintenance: After the customized version of product is sold to customers, maintenance calls are received and responded by Professional Services (PS) or Sales and Technical Support. Figure 1. Work Flow at Omega Table 4 Working Flow during Development of Product X-1 Task Outcome Actors Product Customization s Innovation Product Initiation Requirements from customer become the motivation of initiation Project Manager, Marketing Team, CEO Infra Structure development Requirement Gathering Requirements come from the customer who indents customization in already developed Marketing department Project Manager product of Omega. Specification Requirements from customer are Professional Services Department Document Proof of concept (PoC) formalized. Materialization of proposal Professional services department, Project Manager, Marketing team, Senior Executives Product customization s Evolution Development Coded modules Software developers Product customization s Release Testing Tested software Software testing Group Documentation User Manual Technical Writer 6

7 4.4 Development Team The team strength working on modules is intentionally kept smaller in order to maintain high degree of communication and avoid confusions. One interviewee mentioned during interview. We have 4-5 men teams, usually every person works on a separate module with little rotation within team and Team Lead keeps initial requirements and code reviews with him. Software testing group and Quality Assurance are separate departments at Omega; however they sometime lend their personnel to participate in development activities. This coincides with the basic concept of self organizing teams in Agile Software Development. The team worked for about 8 months to come up with product X Software Development Methodology at Omega Omega has started using SCRUM because it ensures incremental timely delivery of well tested software. In words of team lead: because Agile Software Development, stresses more on in time delivery of Project and similarly SCRUM offers small sprints thus delivering some highly tested functionalities in less time. (Team Lead at Omega) The team comprises of 4-5 members including a Scrum Master, Project Manager/Owner and Developers/Testers. Sometimes Scrum Master is replaced by Project Manager himself. The teams are self organizing at Omega as mentioned by an interviewee: Different scrum levels have different scrum owners it is not always necessarily the Project Manager. (Interview at Omega) Project Manager manages the development of original product; therefore he takes part in customization process as well. Project Manager and Marketing team is responsible for meeting the customer and gathering requirements and specifications. The teams meet up daily for 15 to 20 minutes in an informal way and discuss their achievements/ problems and daily plans (sprint backlog). The detailed workflow of the team worked at Omega for development of Product X-1 is shown in Fig. 1. Omega team maintains a product burn down chart in order to display overall progress. Fig. 2 shows a sample burn down chart for the product X customization work. Figure. 2 Product Burn down chart 7

8 Product backlog maintains the priority and record of every task or user story. Table 5 shows the sample product backlog with priority of tasks, estimated hours required for that task and task of remaining hours. Figure 3. Product Backlog The product backlog shows the task or user stories to be carried out in each sprint. In this product backlog chart 14 th and 15 th sprints are explained details are changed due to organization s secrecy issue. Furthermore task boards are used to have a larger view of work progress. It includes the work to be done, in progress, completed and to be verified tasks. It is displayed on a white board in the meeting room where daily and weekly sprints are conducted. The updates are made with board marker or sticky notes. 5. Discussion/Findings In this case our focus was on customized product developed by Omega. It is observed that in case of product enhancement or customization in compliance with customer s demands the involvement of Customer becomes a must, which is not expected in a product based company. In this case the requirements and specifications of the customer are followed by the developers. Thus like all the other agile development technologies here SCRUM also requires full time customer involvement during Sprints. It is observed while conducting this case study that Omega even in customization cases does not stress much on customer s physical presence in daily sprints. This is observed that Project Manager still remains the person to be looked upon for requirement gathering. According to the interviewee. The Project Manager communicates with the customer and gives us demos, I don t think so we ever needed to see the customer ourselves. (Interviewee at Omega) The Project Manager meets the customer who indents customization in the already developed product. The Project Manager being the owner of product is always in a better position absorb customer s viewpoint about customization. Thus even in customized development we have not observed any Customer representative from business or technical side taking part in sprints. The developers are found satisfied with absence of customer. This aspect is explained by the interviewee as:- 8

9 The project Manager being the owner of original product never asks for something out of the blue even in customization. He understands the product scope very well and thus directs customer s customization demands to optimal edges as well. (Interviewee at Omega) This provides answer to sub question q1 that Project Manager takes up the role of Proxy Customer and owns the product in addition to his previous knowledge about the product makes him the well suited information provider and project owner. This fact explains the lack of negative impacts of absence of real customer. The real customer and Project Manager hold face-to-face conversations sometimes as the clientele is dispersed over continents so they frequently use video conferencing and video chat. We found real customer s absence in many of the ongoing product customizations within the company. The Customers are missing in the sense that they are never present physically. Most often their wishes and their feedback are communicated to developers through Project Managers/team lead. This should be seen as a broken link in the communication between development team and real customer. But the success with which Omega is holding its clients and maintaining its business internationally makes this broken communication link no hurdle! It is observed that the development team shared good relations and friendly atmosphere. In addition to the scheduled meetings or sprints they also had off and on informal discussions. The collocation of whole team increases the communication ratio between them. The project Managers in customization cases are not always collocated with the rest of the team. In this case Project Manager was also at the same site. In other cases the Project Managers are even from other international offices indenting their requirements and details through video conferencing and frequent visits to the country where rest of the team is residing. The interviewee explained about meeting the customers. We sort of never felt the need of meeting the customer ourselves. The demonstrations given to us by Team Lead or Project Manage rare enough for us to deal with our queries. (Interviewee at Omega ) At one other point the interviewee once again emphasized that I have never heard that every developer needs to meet customer, PM (Project Manager) or someone responsible supposed for demonstrating requirements is enough for us. (Interviewee at Omega ) The Project Managers being the high rank personnel of the company are the ones who remain busy Thus they are rarely available for face-to-face meetings i.e. daily, weekly sprints. The developers and other team members contact them through usually. With reference to sub question 2 success of company is measured through customer feedback, post delivery maintenance data, error logs and market report about product sale. 6. Success Analysis This section provides answer to first research question Q1 that product based software development at Omega is successful without onsite customer proved by customer comments and post delivery maintenance data Success Case 1 Product X-1 was developed for a Business and Technology solution provider organization. The company needed a solution that would simplify and accelerate the process of creating and maintaining flexible healthcare enrollment booklets for their customer. They needed to reengineer the way booklets were produced so as to make the process easier, quicker and more cost effective. Omega s solution, Product X-1, cut application development time in half 9

10 and document customization has improved dramatically. The company has accelerated the entire process of building, changing and enhancing applications saving both time and money. Development time for the booklets is cut 50% and management effort is improved by 78% Booklets previously represented by millions of lines of code now managed and edited in the Product X s graphical design interface. (Customer Comments about Product X-1) 6.2. Success Case 2 The second success case is of Product X-1 implemented in a large US Insurance Provider. The company demanded an efficient work flow management system from Omega with slight customizations regarding their own environment. This product was delivered in eight months span as promised. To reduce the cost and time required for generating a benefit booklet and ID card for new customers. This required a complete workflow for the efficient management and production of customized benefit booklets. The production of ID cards needed to be integrated into this process. With Omega the company now has a custom workflow solution that dramatically accelerated the production process. The time from enrollment to delivery was reduced from 10 weeks to one day. Lower costs, elimination of errors, and the consolidation of products in one mailing were also achieved. Time from enrollment to booklet delivery reduced from 10 weeks to 1 day. Reduced mailing costs by consolidating all related documents into one package. Customer questions reduced because all information is delivered together. (Customer Comments about Product X-1) 6.3. Success factors The case discussed above is a real life example of a well established company having many recognized customers internationally. The clients having repeated business with the organization establish the fact that development techniques in practice are adding to its success. This section answers the second research question Q2 discussing the success factors of Omega based on the detailed study of the success stories of the company. It shows which factors are responsible for subsiding the onsite customer demand in product based software development. The products of Omega are open to international market and it is gaining more and more customers with the passage of time. Omega maintains its customers in almost 70 countries around the globe. This number remained increasing in past few years and is deemed to expand further too in coming years. Maintaining such a big clientele is a fact related to quality and productivity. The success factors observed are described as under: Same domain development: Omega deals in document composition, resource management, print stream conversion, document archival and web-based support for document production and workflow collaboration. It produces variety of products with respect to upcoming changes in technologies and requirements but within the same domain. This provides an ease and experience to the developers in carrying the work out successfully. The challenges of understanding the new domain specifications are subsided here. Even the intricate details become a routine matter after producing variety of products of same genre. This also equips the company to excel in its domain and provide the workforce experience with skill and expertise. 10

11 Another important factor in working under the domain is that it makes you well aware with every new change in technology and market demand. This in turns helps the company react to it in an efficient manner to gain more clients Technical background of Acting Customer- Project Manager: As discussed in organizational workflow, project Manager is the one who acts as customer when he indents the requirements for a new product. The project Manager s technical background and experience in organization helps him demonstrate the requirements efficiently and effectively to the agile teams. Moreover in case of customized product modules Project Manager being the focal person also makes the customers ask for applicable requirements. The technical background of Project Manager provides an edge in contrast to a non technical customer Rich means of communication: The customer and Project Managers residing outside the Country or at remote locations are accessed through Video conferencing and sometimes frequent visits to the site for demonstrations and specification prioritization. The service is also used sometimes but above all video conferencing makes the communication link faster and reliable. Furthermore working in the same domain for years raise the expertise of developers and they find a rare need to bother Project manager and Customers Experienced workforce :If there is a need for developing a new product or solution, the company puts a lot of faith in the skills and existing knowledge of their developers. The experienced developers generate concerned ideas for the company. The developers spent almost have a dozen years working with the same company have seen many successful products as well. This gives them confidence and excellence in the domain. An experienced developer is always in a better position to understand and respond Customer requirements Focus on deliverables: Particular focus of scrum is on delivering deliverable as a product of each iteration. The development team concentrates on handing noticeable product within limited time. In this case the developers are supposed to show their performance in every Sprint. This maintains the development of new sprint log and goals for the next sprint Less Iterations: The Scrum culture followed at Omega makes the developers show their output and future work plan regularly. This not only keeps a check on the work done but also determines future goals. The modules developed are presented in daily sprints or weekly sprints depending upon the frequency of sprints and are handed over to Software Testing Group. This scrutinizes the error occurrence rate and keeps follow up of what is to be built next. As every release is demonstrated to the Project Manager hence it lessens the chances of wasting efforts in wrong direction. The Table 5-1 shows release 6 after sprint 14 and sprint 15, both these sprints outcomes are demonstrated to the concerned Project Manager and this it reduces number of iterations which would have been caused otherwise Less documentation : Less or minimal documentation in Agile focuses more on real implementation of the product. in this particular case the documentation is minimized to only backlogs and burn-down charts, rest of the documentation is prepared either by Concept Team as Proof of Concept or User Manuals after the product completion. 7. Validation This section describes the validation of success claims mentioned in previous section. The success factors are checked and validated so as to proof the hypothesis, same validation approach is used in [11]. 11

12 Table 5. Validation Approach 8. Conclusion The case study explains in detail the development culture of a product based software development organization with absence of real customer. Omega being a product based company completely subsides the role of real customer in initial product development. But for customized versions requirements and finances come from customer so in this scenario tailoring of an already launched product according to customer s needs is required. Focusing on customer absence in such customization product development cases it is considerable that in spite of customer s absence the products are working well in market as proved by customer s views and post delivery maintenance data. The Project Manager takes up Proxy Customer s role for customization ventures; communicates the requirement to team lead and developers and participates in sprints. In this particular case considered the constant non availability of real customer on site has not put any remarkable difference on company s clientele and product sale under certain circumstances. These circumstances are discussed in detail as success factor in this paper. Thus it can be concluded that in this particular case and others having similarities, deviation from basic Agile Principle of having on site customer for face to face discussions throughout the development life cycle followed is not affecting the business. Customer experience, comments and post delivery maintenance data after 6 months of product launch show that customer satisfaction level is met. This shows that Omega (Product based software development organization) is following a hybrid approach emerged from this deviation according to company s own workflow is working for it. In this case it is contradictory to theoretical principals but giving output practically. References [1] A. Begel and N. Nagappan, "Usage and Perceptions of Agile Software Development in an Industrial Context: An Exploratory Study", First International Symposium on Empirical Software Engineering and Measurement (ESEM), (2007), pp

13 [2] S. Nerur, R. K. Mahapatra and G. Mangalaraj, "Challenges of migrating to agile methodologies", Communications of the ACM, (2005), Vol. 48, pp [3] K. Beck and C. Andres, "Extreme programming explained: embrace change", Addison-Wesley Professional, (2004). [4] K. Schwaber and M. Beedle, "Agile software development with Scrum", Prentice Hall PTR Upper Saddle River, NJ, USA, (2001). [5] P. Abrahamsson, J. Warsta, M.T. Siponen and J. Ronkainen, "New directions on agile methods: a comparative analysis", International Conference on Software Engineering, IEEE Computer Society, Portland, Oregon, (2003), pp [6] N. Abbas, A. M. Gravell and G. B. Wills, "Historical Roots of Agile Methods: Where did Agile Thinking Come from?", Agile Processes in Software Engineering and Extreme Programming, (2008), pp [7] D. Reifer, "How to get the most out of extreme programming/agile methods", Extreme Programming and Agile Methods XP/Agile Universe, (2002), pp [8] J. Koskela and P. Abrahamsson, "On-site customer in an xp project: Empirical results from a case study", Software Process Improvement, (2004), pp [9] P. S. Thiyagarajan and S. Verma, "A Closer Look at Extreme Programming (XP) with an Onsite-Offshore Model to Develop Software Projects Using XP Methodology", Software Engineering Approaches for Offshore and Outsourced Development, (2009), pp [10] T. Chow and D. B. Cao, "A survey study of critical success factors in agile software projects", Journal of Systems and Software 81, (2008), pp [11] I. Inayat, M.A. Noor and Z. Inayat, "Facilitating Off-site Customer in Agile Product based development: an industrial case study", Proceedings of IMTIC 2012 conference To be published in Communications in Computer and Information Science (CCIS) series by Springer, (2012). Irum Inayat Authors Ph.D scholar at Faculty of Computer Science and Information Technology, University of Malaya, Kuala lumpur, Malaysia. Her research focus is on Requirements Engineering specifically with respect to customer oriented Agile software development methods. Dr. Muhammad Asim Noor Assistant Professor, Department of Computer Science, COMSATS University of Science and Technology, Islamabad, Pakistan. His particular research focus in Software Engineering is on collaborative product line software development, Agile software develpoment with a number of publications in academic journal and conferences. Zubaria Inayat Graduate Research Student at Department of Computer Science, Comsats University of Science and Technology, Islamabad, Pakistan. Her research interest lies in combining Agile methods with product line software development and risk management. 13

14 14