Agile Software and Systems Engineering Workshop

Transcription

1 Agile Software and Systems Engineering Workshop Systems and Software Technology Conference Salt Lake City, UT April 2012 Dr. Suzette S. Johnson Agile Engineering Northrop Grumman

2 Getting To Know You! Dr. Suzette Johnson Northrop Grumman Agile Community of Practice Chair Champion of Agile practices across NGC ADAPT Executive Committee (Industry working group) As a systems engineer, project manager, and certified Scrum Professional, with an interest and passion for promoting and implementing Agile practices in largescale systems environments. Provides coaching, consulting, mentoring, training, etc. for Northrop Grumman programs Lean and Agile experience started in 1999 Dissertation focused on investigating the impact of leadership styles on software project outcomes in traditional and agile engineering environments 2

3 Agenda Defining an Agile Environment Creating the Team High-Level Agile Steps Capabilities Description and Release Planning Iteration Planning Iteration Execution Iteration Demonstration and Retrospective Delivery Some Final Notes The Great Challenges Agile Reading List 3

4 Today s Outcomes Develop an understanding of the major agile engineering practices Participate in a planning and estimating scenario Gain insight into agile testing practices from a couple practitioners Participate in a team retrospective Have fun working together! 4

5 Today s Plan Collaborate Share Brainstorm Have Fun! Ask

6 Expectations What are your expectations? What questions do you have? Time: 10 minutes 6

7 Agenda Defining an Agile Environment Creating the Team High-Level Agile Steps Capabilities Description and Release Planning Iteration Planning Iteration Execution Iteration Demonstration and Retrospective Delivery Retrospective Closing Agile Reading List 7

8 Agile Engineering What is Agile Engineering? Why Agile Practices? Includes the entire product life cycle Impacts the entire organization Inspects and adapts Focuses on the value stream Faster time-to-market Adapt to change Shortened product life cycle New technological advancements Improved transparency of progress and end-to-end accountability and ownership 8

9 What is your level of experience? Scale of 1 to 5 9

10 Agile Principles Early and Continuous Delivery of Value A Working System is the Primary Measure of Progress Welcome Changing Requirements Deliver a Working System Frequently Business People and Developers Must Work Together Daily Motivated and Empowered Individuals Face-to-face Conversation Promote Sustainable Development Continuous Attention to Technical Excellence 10 Simplicity The Best Architectures, Requirements and Designs Emerge From Self-Organizing Teams Regular Team Reflection on How to Become More Effective 10

11 Agile Manifesto Individuals and interactions Is valued more than Processes and tools A working systems Is valued more than Comprehensive documentation Customer collaboration Is valued more than Controlled negotiation Responding to change Is valued more than Following a plan That is, while there is value in the items on the right, we value the items on the left more 11

12 Agile Methods Agile is about how we believe people are best motivated to do work and about demonstrating high value on a regular basis particularly in environments that face high requirement volatility and unpredictability. Extreme Programming Dynamic Systems Development Methods Crystal Methods Agile Methods Agile Unified Process Scrum Adaptive Software Design Feature Driven Development 12

13 Four Popular Agile Practices Agile Unified Process (AUP) Scott Ambler, Craig Larman Feature Driven Development (FDD) Jeff De Luca Extreme Programming (XP) Kent Beck, Ward Cunningham, Ron Jeffries Scrum Ken Schwaber, Jeff Sutherland Research Conducted by: VersionOne Fall ,770 completed responses representing 91 countries 13

14 The Agile Framework 14 Inspect and Adapt Visibility and Transparency

15 Agile System Development Lifecycle (Stories) Iteration review & demonstration to stakeholders; Retrospectives Release system Product Backlog Reference: Dr. Dobb s Journal/Scott Ambler 15

16 Iteration 0: Project Start up Modeling an initial architecture and design for the system Identify an architectural strategy Create an initial system design. Work through the design details during future iterations in model sessions Every iteration must deliver at least some piece of business functionality The roadmap is mapped out by release Setting up the environment You need workstations, development tools, and a work areas. Start with just enough to get the team going and continue to build on this in future releases Determine release dates and iteration length The release increments have been identified with a fixed interval typically of days making it easier for customers who may not be co-located to participate and for final QA reviews Start building the team Begin with at least one or two senior developers, the Scrum Master and Product Owner and one or more stakeholder representatives Training 16

17 Tek Talk: Ball Point Game The objective of the Ball Point game is to get as many balls through the team as possible within two minutes. Each ball must be touched at least once by every team member and must end with the same person with whom it began. After two minutes the team is allowed an additional minute to discuss the process and how it could be improved. The game is played a total of five times.

18 Tek Talk: Ball Point Game

19 Tek Talk: Ball Point Game

20 Agenda Defining an Agile Environment Creating the Team High-Level Agile Steps Capabilities Description and Release Planning Iteration Planning Iteration Execution Iteration Demonstration and Retrospective Delivery Retrospective Closing Agile Reading List 20

21 Roles and Responsibilities Team 21 Chief Engineer/ Architect Product Owner ScrumMaster Ensures the integrity of the architecture Communicates the systems design Has no predefined team Adds to the Product Backlog Defines the features of the product Manages project features and release to optimize return on investment (ROI) Prioritizes features according to mission/user needs Inspects increment and makes adaptations to project Can change features and priority every iteration Communicates project progress and status Accepts or rejects work Cross-functional, seven plus/minus two members Selects the iteration goal and specifies work results Commits to what it feels it can accomplish Has authority to do everything within existing standards and guidelines to reach the iteration goal Manages itself and its work Collaborates with Product Owner to optimize value Demos work results to the Product Owner Ensures that the team is fully functional, productive and improves quality Enables close cooperation across all roles and functions and removes barriers Shields the team from external interferences Ensures that the process is followed Teaches Product Owner and Team how to fulfill their roles Does not make decisions for the team

22 Project Team Structure Transition from functional grouping to cross functional teams PM and Technical Lead Chief Engineer Chief Architect Quality Infrastructure Team Hardware Team Software Team Systems Engineers Integration and Test Configuration Management Lead TM Lead TM Lead TM Lead TM Lead TM Lead TM TM TM TM TM TM TM TM TM TM TM TM TM Isolated progress with too many hand-offs and barriers 22

23 Project Team Structure PM and Technical Lead Capabilities or Threads Product Owner Chief Engineer Chief Architect Quality Cross Functional Team 1 Cross Functional Team 2 Cross Functional Team 3 Cross Functional Team n Scrum Master Progress against end-to-end capabilities Developer Developer Integrator Configuration Management Tester Systems Engineer Developer Services Supports Cross Functional Teams Network/ Systems Administration Push accountability and ownership to the team level 23 An Example Configuration Mgt. Everyone trained

24 Self-organizing, Self-managing Teams Team organizes around the work that needs to be done Self-organization does not mean people get to do whatever they want Management guides the evolution of behaviors that emerge from the interactions The team works to solve their own problems. Management does not solve problems for the teams 24

25 Today s Scenario: RestEZ Online hotel reservation system for RestEZ Based on customer needs, your team has defined a logical architecture for the online hotel reservation system. The system is a traditional 3 tier architecture: a database layer (to persist reservations), a business logic layer (to manage reservations), and a browser-based user interface (to receive customer input). 25

26 TekTalk: Roles and Responsibilities Reflecting on the section Creating the Team and the given scenario address the following: With your team, identify the team members and roles. You will need a product owner, scrum master, and team members. Keep in mind the need for cross-functional teams Identify your responsibilities Team discussion: Whose responsibility? The Product Owner is micromanaging the team making self-managing impossible. The team is struggling to understand the priorities of the work. A team member is constantly late for the daily standup. The team is not able to deliver on their commitments. Create a team name 26 Time: 10 minutes

27 CheckPoint What we ve covered so far Questions How are we doing? 27 Time: 5 minutes

28 Agenda Defining an Agile Environment Creating the Team High-Level Agile Steps Capabilities Description and Release Planning Iteration Planning Iteration Execution Iteration Demonstration and Retrospective Delivery Some Final Notes The Great Challenges Agile Reading List 28

29 5 Levels of Planning (1) Vision, (2) Product Roadmap, (3) Release, (4) Sprint, (5) Daily (2) Product Roadmap ~2-6 months 4 weeks (fixed) Sprint1 Planning Product Backlog Release 1 Planning Goals &User Stories Story Story Story Task Task Task Task (4) (5) Daily Stand Up (3) Sprint 2 (1) Vision Customer Needs Capabilities identified Some initial analysis Sprint 3 Sprint n 29 Copyright 2010 Northrop Grumman Corporation

30 High Level Agile Stages Vision Customer Needs Product Roadmap Release Planning Iteration Planning Iteration Execution Iteration Demo and Retrospective Delivery 30 Copyright 2010 Northrop Grumman Corporation

31 Product Roadmap Release 1 Release 2 Release 3 Release 4 March 31, 2012 June 24, 2012 Sept. 30, 2012 Dec. 18, 2012 Room reservations and payment User profiles for future visits Hotel amenities Conference offerings Online chat support Local information Special discounts for room reservations Improve usability Google maps Air and hotel package deal Meeting and Business plans and reservations 31 Copyright 2010 Northrop Grumman Corporation

32 High Level Agile Stages Vision Customer Needs Product Roadmap Release Planning Iteration Planning Iteration Execution Iteration Demo and Retrospective Delivery 32 Copyright 2010 Northrop Grumman Corporation

33 Implementing and Sustaining Change: Creating the Product Backlog Creating the Vision Product Vision Captured Capabilities System Architecture Requirements mapped to stories Revisit architecture and design each release and iteration Developing Understanding Use Cases/ Requirements Architecture Sequence Diagrams Activity Flow Diagrams System Level Validations User Stories and Acceptance Tests Product Backlog Functional Non-Functional Planning and Estimating the Work 6-9 months 1 6 months 1-4 weeks Daily Product Roadmap Release Planning Sprint Planning Daily Plans and Commitment Copyright 2010 Northrop Grumman Corporation

34 Create the Product Backlog A list of all desired work on the project Ideally expressed such that each item has value to the users or customers of the product Prioritized by the product owner Product Vision Customer Needs Business value 80 As a vacationer, I want to search room availability 70 As a vacationer, I want to change my reservation 60 As a vacationer, I want to cancel my reservation 50 As a vacationer, I want to pay with a credit card 34 Copyright 2010 Northrop Grumman Corporation

35 Define the Release and Iteration Cycle 3 Month Release Cycle Planning started in Dec. 1/04/10 3/31/10 Day 1 A.M. = Release Planning Mtg P.M. = Iteration1 Planning Mtg Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5 Iteration 6 Formal release/delivery End of Each Iteration Retrospective Demonstration and Review Potentially releasable Day 1 of Each Iteration Iteration Planning Meeting 2-4 hours 35

36 Release Planning Overview Customer Product Owner Vision Customer Needs Product Roadmap Input Systems architecture Systems design Systems Engineers Capabilities and Requirements High level requirements mapped to stories Revisit architecture and design each release and iteration High Level Design, Use Cases Define end-to-end capability tests Whole Team User Stories Product Backlog Whole Team Release Planning Meeting Goals &User Stories A Release is typically ~2-6 months 36 Product Roadmap User Story updates Acceptance Criteria Estimates Commit Copyright 2010 Northrop Grumman Corporation The Release Plan Iteration Planning

37 Planning the Release Product: Hotel Website for RestEZ Capability: Make Room Reservations User Stories As a vacationer, I want to search room availability As a vacationer, I want to save my request Story Points Demonstrate. 12 Demonstrate. 8 The Release Plan What is our velocity? How many story points can we commit to? As a vacationer, I want to pay with a credit card 37 Demo with Visa Demo with AmEx Demo with MC Owned by the Project Owner with the opportunity to reprioritize each iteration 16 Business Value User stories are ranked according to business value

38 Velocity (Based on history) Velocity is the amount of work a development team completes in an iteration (story points completed) Velocity is a range; Look for the high, the low, and the mean Velocity for Team A Project Velocity Story Points Iteration Project Velocity per Iteration Team A Velocity High: 45 story points Low: 30 story points Mean: 37 story points Project Velocity per Iteration High: 155 story points Low: 120 story points Mean: 137 story points 38

39 Release Planning Meeting Meeting is time-boxed Usually ½ - 1 day depending Occurs with the entire project team INPUT Product Roadmap Product Backlog Current Product Capabilities System level Architecture/Design Drafted User Stories Velocity Release Planning Meeting OUTPUT Release Plan Stories with Relative Effort Stories with Acceptance Criteria Dependencies Commitment to the Release 39

40 Commit to the Release Plan Capabilities/Goals identified High level requirements and initial user stories mapped Velocity for Team A User stories (functional and non-functional requirements) Dependencies identified Known or assumed velocity by development team and project team Total number of user stories planned Iteration Project Velocity Planned hours (WBS element) Story Points Project Velocity per Iteration 40

41 User Stories What is a User Story? Functional stories often based off a scenario of a use case On large projects a user can be another system Non-functional stories Definition of Done Design, Write tests, code, unit tests, documentation, etc. No credit for partial work either done or not done Estimation Story Points Bigness of the task Considers: complexity, uncertainty, effort Estimated by the team Relative values As vacationer, I want to search for available rooms. 41

42 User Stories to Convey Meaning Requirements might say The product shall have a gas engine The product shall have four wheels The product shall have a rubber tire mounted to each wheel The product shall have a steering wheel The product shall have a steel body 42 Source Mike Cohn:

43 User Stories Convey Meaning As a <lawn service provider> I want to mow lawns quickly and easily. As a <lawn service provider> I want to sit comfortably while mowing lawns. 43 Reference: Mike Cohn, mountaingoatsoftware.com

44 Non-Functional Requirements As a vacationer and user of the hotel website, I want the system to be available 99.99% of the time As vacationer, I want web pages to download in <4 seconds Stories for nonfunctional requirements As the hotel website owner, I want 10,000 concurrent users to be able to access the site at the same time with no impact to performance Describes system behavior or characteristics 44 Reference: Mike Cohn, mountaingoatsoftware.com

45 A User Story A written description A Conversation about the story Includes tests that convey and document details icle_view/27 45

46 Creating User Stories Hardware Systems Engineering Software Integration and Testing Task 1 Task 1 Task 1 Task 1 Task 2 Task 2 Task 2 Task 2 Task 3 Task 3 Task 3 Task 3 Task 4 Task 4 Task 4 Task 4 Story A story or system capability often spans several functional areas.

47 TekTalk Together: Writing User Stories Scenario: Based on customer needs, your team has defined a logical architecture for an online hotel reservation system. The system is a traditional 3 tier architecture: a database layer (to persist reservations), a business logic layer (to manage reservations), and a browser-based user interface (to receive customer input). Your product owner started to create the product backlog and has provided the following Epic: As a vacationer I want to search the hotel s facilities and related amenities. As a group we will write 2 additional stories for the epic story above including acceptance (testing) criteria. 47 Time: 10 minutes

48 TekTalk: Writing User Stories Scenario: Based on customer needs, your team has defined a logical architecture for an online hotel reservation system The system is a traditional three tier architecture: a database layer (to persist reservations), a business logic layer (to manage reservations), and a browser-based user interface (to receive customer input). Your product owner started to create the product backlog and has provided two epic stories for the first release With your team complete the following: Read through the stories Write 1 additional story Include acceptance criteria for this story Time: 10 minutes 48

49 Estimating User Stories Traditional estimates focus on absolutes Agile Estimates focus on relative size estimates Why are relative size estimates preferred? People are better at comparing versus measuring in absolutes Estimates are faster Basic math still applies Easier to reach consensus House made of straw House made of sticks House made of bricks Reference: Rally Software

50 Recommendations for Estimating User Stories Keep your estimates high-level intuitive guesses Don't over-analyze the details Estimating using a number sequence (Fibonacci numbers) It gets less precise as the estimates get bigger, which builds a natural distribution curve into your estimates. Estimate as a team Track your Velocity over time You can also track your reliability i.e. the number of points delivered as a percentage of the number of points committed to at the start of the Sprint. At the start of each Sprint, look back at your Velocity Don't try to reconcile points with hours Commit as a team 50

51 Estimating Technique: Planning Poker Estimating the user stories for a release. A release is one or more iterations Going into the estimation phase, stories for the release have been identified and each has verification objectives; Stories have been discussed with the team Steps Each estimator is given a deck of cards, each card has a valid number such as (1, 2, 3, 5, 8, 13, 21,?) The teams read the stories An average story is selected The story is read to the team and discussed briefly Each estimator selects a card to reveal his estimate Cards are turned over so everyone can see them Differences in estimates are discussed; especially outliers Re-estimate until estimates converge ? 51 Reference:

52 TekTalk Together: Estimate This! Backlog Item Putting away a load of laundry Vacuuming the floors Packing lunch Cleaning handprints off the refrigerator Relative Estimate 52 Time: 10 minutes

53 High Level Agile Stages Vision Customer Needs Product Roadmap Release Planning Iteration Planning Iteration Execution Iteration Demo and Retrospective Delivery 53 Copyright 2010 Northrop Grumman Corporation

54 Release Plan to Iteration Plan Example: Hotel Website Release Plan As a vacationer, I want to search room availability As a vacationer, I want to change my reservation As a vacationer, I want to cancel my reservation Story Points Iteration Plan (Stories with Tasks) Hours Design Review 4 Write Tests 8 Code 24 Automate Test 8 Definition of Done As a vacationer, I want to pay with a credit card 3 Designed Design Review Refactored Coded Code Review Unit tested, functional tested, integration tested, User/Stakeholder Tested Documented 54

55 Sprint Planning Meeting Meeting is time-boxed Usually ½ day depending on length of the Sprint INPUT User stories with business value User stories with estimates Team capacity Team velocity Sprint Planning Meeting OUTPUT Sprint Goals Stories for the Sprint Tasks with hours for each user story 55

56 Team Capacity Capacity is the team members available hours to work in an iteration Revisited each iteration Compare planned task hours to capacity hours Example for a two-week iteration Team Member Hours per day Total hours in the iteration Bill 5 50 Scott 5 50 Chris 8 80 Andy 7 70 Cindy 7 70 Mike 8 80 TEAM TOTAL Copyright 2010 Northrop Grumman Corporation

57 TekTalk: Planning and Estimation Scenario: Based on customer needs, your team has defined a logical architecture for an online hotel reservation system The system is a traditional three tier architecture: a database layer (to persist reservations), a business logic layer (to manage reservations), and a browser-based user interface (to receive customer input). Your product owner started to create the product backlog and has provided two epic stories for the first release With your team complete the following: Read through the stories Using Planning Poker estimate story points for each story Question: How will the team decide how many stories it should assign to an iteration? How do you know which stories to select first? Time: 20 minutes 57

58 CheckPoint What we ve covered so far Questions How are we doing? 58 Time: 5 minutes

59 High Level Agile Stages Vision Customer Needs Product Roadmap Release Planning Iteration Planning Iteration Execution Iteration Demo and Retrospective Delivery 59 Copyright 2010 Northrop Grumman Corporation

60 Iteration Execution: Design, Build, Test Release Plan Story A Story B Story C Story n Iteration Plan Iteration Design Design Story A Build Test Story C Build Design Story D Build Design Test Test Iteration Demonstration and Retrospective Fixed time frame Unit Testing/Component Testing Continuous Integration Test Automation Peer/Code Reviews 60 Reference: Hallett, D. (2006). An introduction to agile and iterative project management.

61 Managing the Iteration Backlog Any team member can add, delete or change the iteration backlog Individuals sign up for work of their own choosing Work is not assigned Estimated work remaining is updated daily Work for the iteration emerges If work is unclear, define a iteration backlog item with a larger amount of time and break it down later Update work remaining as more becomes known 61

62 A Team s Iteration Burndown User Story: As a vacationer, I want to search room availability Week1 Iteration 0 Tasks Owner Status Mon Tues Wed Thur Fri Mon Tues Wed Thur Design Review Scott Completed Install baseline Bill Completed ICD updates Scott Completed Acquire test data Bill Completed Code Scott Completed Develop tests Scott Completed Run Tests Scott Completed The Team Manages its work and progress Hours Meets daily to discuss progress and commit to plan 62 Managed by the Team

63 The Daily Standup SCRUM Daily Standup 1. What did you do since the last stand up? 2. What will you do today? (Commitment) 3. Is anything in your way? SCRUM of SCRUMS Standup (usually two or three times per week) 1. What has your team done since we last met? 2. What will your team do before we meet next? 3. What s in your team s way? 4. What are you about to put in another team s way? Parameters Daily Attendance required and critical 15-minutes Stand-up Not for problem solving

64 Agile Testing Test Early Test Often Automate 64

65 Theory and Practice Agile testing is about people and communication Test documents are often incomplete, out-ofdate, ambiguous Test results should be big, public, easy-to-read charts Testers are embedded with coders The reasons? Speed, accountability, transparency, transfer of knowledge Automated testing is a must 65 Early, Often, Automate

66 High Level Agile Stages Vision Customer Needs Product Roadmap Release Planning Iteration Planning Iteration Execution Iteration Demo and Retrospective Delivery 66 Copyright 2010 Northrop Grumman Corporation

67 The Sprint Review and Retrospective Sprint Review Demonstrates new functionality Transparency and information sharing Informal Time-boxed Everyone invited What has been tested and what stories are accepted Revisit the backlog Update Metrics Retrospective Take a look at what is and is not working well Time-boxed Done at the end of each Sprint Facilitated by the Scrum Master Focus is on process improvement Whole team participates Worked well Could be improved Actions and Priorities

68 Monitoring Progress Product Burndown for the Release Focuses on the Points (Work) Remaining Story Points Planned Completed Product Burndown (team of teams) Based on story points planned Iteration1 Iteration2 Iteration 3 Iteration 4 Iteration 5 Iteration 6 Iteration 7 A project team s burndown Updated and reviewed each iteration As stories are accepted and tests passed, requirement progress is updated Story Points iteration 0 iteration 1 iteration 2 iteration 3 iteration 4 iteration 5 Story points baseline Burndown (Pts Remain) A team s product burndown 68

69 End of the Release Similar to end of iteration every iteration is potentially releasable Hardening iteration Stories demonstrated and accepted End-to-end capability testing Version description document Documentation and metrics updated Many disciplined agile teams have a parallel testing effort during the iteration where defects are found and feed back into the process. In addition, the working software becomes a working system QA testing, DT&E, etc. 69

70 Agenda Defining an Agile Environment Creating the Team High-Level Agile Steps Capabilities Description and Release Planning Iteration Planning Iteration Execution Iteration Demonstration and Retrospective Delivery Retrospective Closing Agile Reading List 70

71 TekTalk: Retrospective Reflect on today s session. What worked well Suggestions for improvement What do you think are the greatest challenges when transitioning to Agile practices? What are three important take-aways from today? Time: 10 minutes 71

72 Your Greatest Challenges Transitioning leadership and management roles Culture change Distributed teams Scaling Systems-of-systems level Hardware Earned value Customer learning and frequent customer engagement CMMI/ISO

73 Final CheckPoint Agile is about creating an adaptive organization that is able to respond to the changing needs of customers and industries Agile is not just about software development There are several Agile methods under the umbrella of Agile Practices Agile development emphasizes the need for ongoing iterative development with completed, demonstrable functionality at the end of every iteration Agile methods emphasize the need for team and customer collaboration 73

74 Kaizen: Change for the Better 74

75 75 4/21/2012

76 Special Acknowledgments Many of the ideas in this presentation originated from: My initial training by Ken Schwaber and Mike Cohn The Agile Journal Other contributions/research are noted throughout the presentation My experiences with nearly 10 programs and projects across Northrop Grumman 76

77 An Agile Reading List Adaptive Enterprise by Steven Haeckel Agile and Iterative Development: A Manager s Guide by Craig Larman Agile Estimating and Planning by Mike Cohn Succeeding with Agile by Mike Cohn Agile Project Management with Scrum by Ken Schwaber Agile Testing by Lisa Crispin and Janet Gregory Scrum and The Enterprise by Ken Schwaber Weekly articles at by Mike Cohn 77

78 78 Scaling Scrum Recommendations

79 Agile Terminology Term Iteration Product Backlog Product Burn Down Chart Product Owner Definition Fixed time-box in which development occurs; Scrum refers to this as a Sprint Requirements/User Stories to be completed Progress for the release; Focuses on the remaining user story points Owns the product backlog, assigns priority to user stories Release Scrum Master The Team User Story 79 Usually a 2 6 month timeframe; formal committed delivery of product Helps the agile team through the process and removes impediments Cross functional team Similar to a requirement As a user I want what so that purpose 79