A Study of RE Across Different Software Development Lifecycle Models Afiya Nusrat and Navreet Ghag CS 846 Spring 2015
Motivation In-depth look at the SDL process and requirements gathering in two companies Experiental details, personal observations, not a comparative study! Discuss differences in : SDL process Product Team organization
Case Study A
Case Studies Company A o Fortune 500 MNC headquartered in NY o Manufactures and markets computer hardware and software, and offers infrastructure, hosting and consulting services o Market cap of 162.20B
The Team Worked in the L3 Continuing Engineering team in the Software Group for leading RDBMS The team owns code bases for the database server products over a variety of platforms Responsible for product maintenance and bugs Identify and drive new product features to improve functionality, performance and customer satisfaction
The Team
Operations Overview Level 3 Global Support Lab Advocate Customer Level 2 Global Support Technical Sales Level 1 Local Support
Development Lifecycle SDL - loosely based on classic V model Each phase corresponded to a stage in the V model Customer Req. Elicitation System Req. Development Architectural Development Operation Acceptance Test System Test At each stage o Qualification planning o Requirement collection o Testing in parallel Subsystem Req. Dev. Component Development Integration Test Component Test
Customer Requirement All code changes are customer driven Customer issues are usually abstract and performance related CE takes the abstract problem descriptions and formalize requirements Issues could be categorized into: o Code Bugs Customer Req. Elicitation APAR - Authorized Program Analysis Report Documented, fixed, tested and delivered o Request For Enhancement Line Items - New feature development Go through the entire software cycle System Req. Development Architectural Development Subsystem Req. Dev. Component Development
Planning and System Req. Formation of Planning Team Stakeholders: Architects, Legal team, Project managers, Sales Negotiations to decide on best course of action Customer requirements are elicited Planning phase consists of commiting customer specifications to development plan Customer Req. Elicitation System Req. Development Architectural Development Subsystem Req. Dev. Component Development
Architectural Development All architectural, system and subsystem requirements are gathered through regular meetings with the architects and developers Functional, Non-functional and Domain requirements charted out The team comes up with two main documents: o High Level Document (HLD) - functional design o Low Level Document (LLD) - code level design Customer Req. Elicitation System Req. Development Architectural Development Subsystem Req. Dev. Component Development
Component Development Coding Stage Based on type of work, developer specialization and availability, the project was assigned to one or more developers At each stage of development, code is reviewed for efficiency and serviceability Developer is also responsible for code unit testing Customer Req. Elicitation System Req. Development Architectural Development Subsystem Req. Dev. Component Development
Testing Testing Along with the system requirement collection phase, testing requirements are collected in parallel QA members of the planning team develop two different plans: o Functional Verification Testing (FVT) o System Verification Testing (SVT) o PQA Goal was to test early and trace to the requirements Requirements driven testing Operation Acceptance Test System Test Integration Test Component Test
Observations Every stage of the process involved reviews and feedbacks Requirements were modified based on these reviews It is to be noted that the customer only gave functional requirements The developers come up with all other requirements Inflexible and linear nature of the entire process led to problems on encountering changes!
Observations (contd) Lot of work was left for future extensions o Fragmentation and reduced modularity of code o Technical debt Insufficient testing - lack of resources, skills and looming deadlines Test plans did not account for integration testing During unit testing, I found static SQL testing to be very slow. Felt the need for alternate testing models V-model process cycle was generally slow, APARs took 3-6 months to release and line items took around 1 year Organization wide shift towards a cloud based model...
The New Approach
DevOps DevOps = Developers + Operations Extends the goals of agile movement to operations Promotes cross-functionality, shared responsibilities and trust Encourages automation of change, configuration and release process
What Changed.. Incorporated agile model in both development and product release odivide large items into short stories ocreate squads to handle stories in each sprint Opted for shorter delivery times owork with test and production team Frequent revisit of initial requirements o Consideration for both development and operations o Faster failure recovery times
The Squad
How did this new model effect the requirements? In DevOps, requirements are not only shaped by development feedback but also by operations monitoring. This means requirements affect development, development affects operations and operations affect requirements. Requirements Operations Development
Case Study B
Case Studies Company B o MNC headquartered in Vancouver o Provides a cloud-based customer intelligence platform o One of the Top 500 Fastest Growing Companies in Canada
The Team
The Team Agile Methodology Daily Standups Code pushed to production every week or sometimes twice a week Weekly scrums
The Team Feature development - Analysis - divide feature into smaller stories - analyse + requirements gathering + time estimates + rounds of design proposals + change in requirements + meetings + requirements finalized + start development - changes in requirements can still pop in
My Team s Job Community Management
My Team s Job Community Management Each community has its own manager
My Team s Job Community Management Each community has its own manager Create recruitment/research surveys, Manage invitation/reminder/confirmation emails, create participation stats, export member/participation data, upload member information, etc.,
The New Feature: Custom URLs The ability to replace system generated anonymous survey + recruitment links with cleaner labels, i.e. replace www.cocacolacommunity.com/c/aspxlksj12hjsh o12 with www.cocacolacommunity.com/c/r/join
Feature Analysis Design and implement a restful API: 8 story points (3 sprints) Design and implement a redirect controller: 2 story points (1 sprint ) Design and implement a UI: 2 story points (1 sprint) Defined an acceptance Criteria for each
Back end Work Analysis Created a design proposal - sent in email to dev manager, team lead, software architect and a senior team member Presented the design proposal in a meeting
Back end Work Analysis Created a design proposal - sent in email to dev manager, team lead, software architect and a senior team member Presented the design proposal in a meeting REJECTED
Back end Work Analysis Created a design proposal - sent in email to dev manager, team lead, software architect and a senior team member Presented the design proposal in a meeting REJECTED Missed requirements, lack of requirement understanding, identified better implementation ways, compromised on some requirements, etc.
Back end Work Analysis A lot of emails were exchanged back and forth
Back end Work Analysis A lot of emails were exchanged back and forth That brought us: clarity + better vision of future requirements of the same feature
Back end Work Analysis Created second design proposal - sent in email - some emails exchanged suggesting changes - changes made - another meeting - another presentation - a healthy discussion
Back end Work Analysis Created second design proposal - sent in email - some emails exchanged suggesting changes - changes made - another meeting - another presentation - a healthy discussion ACCEPTED
Back end Work Analysis Created second design proposal - sent in email - some emails exchanged suggesting changes - changes made - another meeting - another presentation - a healthy discussion ACCEPTED requirements & design decisions were finalized, story points were re-adjusted
Back end stories were completed nicely and in time!!
UI Story Analysis UI team sent us a document, highlighting the UI features
UI Story Analysis UI team sent us a document, highlighting the UI features Created an acceptance criteria, had one official meeting with prod manager to analyze requirements
UI Story Analysis UI team sent us a document, highlighting the UI features Created an acceptance criteria, had one official meeting with prod manager to analyze requirements Minimal changes were requested
Previous UI
Revised UI
UI Story Progress A lot of desk checks
UI Story Progress A lot of desk checks A lot of changes popped in
UI Story Progress A lot of desk checks A lot of changes popped in Back and forth between analysis + development + desk check -- back to requirements gathering and so on TIME TAKEN = 3 * (ESTIMATED TIME )
Why did that happen?
Why did that happen? Incorrect time estimation
Why did that happen? Incorrect time estimation OR New to UI development + New unit test Framework
My manager asked me to analyse, observe, talk and make suggestions
My manager asked me to analyse, observe, talk and make suggestions Why ME??
My manager asked me to analyse, observe, talk and make suggestions Why ME?? New to the way team works - can better point out defects I faced it first hand
Story Review We never had an official meeting with UI team
Story Review We never had an official meeting with UI team We were sent the UI design far too early in the process
Story Review We never had an official meeting with UI team We were sent the UI design far too early in the process Failed to identify the impossible requirements early on in the process
Story Review Underestimated the changes required in old implementation
Story Review Underestimated the changes required in old implementation A lot of back and forth between stages of SDLC
Story Review Underestimated the changes required in old implementation A lot of back and forth between different phases of SDLC Inaccurate time estimate
Story Review Underestimated the changes required in old implementation A lot of back and forth between stages of SDLC Inaccurate time estimate Did not divide UI work into smaller stories
Story Review A lot of test-rewrite
Story Review A lot of test-rewrite UI teams never sent us any docs with revised UI designs
Story Review A lot of test-rewrite UI teams never sent us any docs with revised UI designs Missed scenarios
Talked to other Team Members
Researched and Observed
What was missing?
What was missing? Improper Requirements Gathering
Observation 1 Not enough requirement analysis with the product manager and ui team as it was done amongst the technical team
Observation 1 Not enough requirement analysis with the product manager and ui team as it was done amongst the technical team - Gap between their understanding
Observation 1 Not enough requirement analysis with the product manager and ui team as it was done amongst the technical team - Gap between their understanding - Emotional issues: hesitant to ask too many questions too many times (6/10)
Solution 1 Thorough analysis should be done with product managers and UI team in early stages
Solution 1 Thorough analysis should be done with product managers and UI team in early stages UI designs should be sent in after a proper story analysis
Solution 2 Behaviour Driven Development
Solution 2 Behaviour Driven Development - A recent addition to the family of Agile software engineering methods [4] - Combines ideas from TDD and DDD - Use of domain language throughout the process
Solution 2 BDD relies on use of very specific (and small) vocabulary to minimise miscommunication Four primary clauses are used to define scenarios: Given, When, Then, And
Observation 2 Improper time estimates - This created a lot of stress both on devs and QAs - Reduced productivity
Solution 3 In agile, always try to split story into parts. Bigger stories can usually come up with unexpected complications Smaller stories can be thoroughly analysed
Observation 3 Not enough QA involvement in Analysis phase
Observation 3 Not enough QA involvement in Analysis phase - A good QA is the best thing to happen to a team. - During manual testing, QAs came up with many missed scenarios
Solution 4 QAs should be kept in loop from requirement gathering till testing They act as a bridge between technical and non technical teams Identify maximum test cases before starting development - this aids understanding of requirements
Observation 4 Jumping between stories - Too many stories in progress at once - Devs tend to jump to new stories while QA on current story is not yet completed - Too much back and forth between stories leads to improper understanding of requirements of new story
Scope for Improvement Behaviour Driven Development: Requires training
Scope for Improvement Behaviour Driven Development: Requires training How much is enough?
Scope for Improvement Behaviour Driven Development: Requires training How much is enough? Expected to come with experience
Scope for Improvement Behaviour Driven Development: Requires training How much is enough? Expected to come with experience Should BDD be adopted completely?
Scope for Improvement Behaviour Driven Development: Requires training How much is enough? Expected to come with experience Should BDD be adopted completely? Exhaustive test case identification
Discussion Adaptability More analysis and communication required between participating technical and non technical teams. BDD helps improve understanding and avoids miscommunication
References 1. An integrated approach to requirements and quality management, (White Paper), June 2009 ftp://public.dhe.ibm.com/software/emea/de/rational/neu/an_integrat ed_approach_to_requirements_and_quality_management_en_200 9.pdf 2. Hull, Elizabeth, Ken Jackson, and Jeremy Dick. Requirements engineering. Springer Science & Business Media, 2010. 3. Link to DevOps white papers 4. John H Lopes, Evaluation of Behavior Driven Development, TUDelft, August 2014
Thank You! Any Questions?