Playing around with Risks

Transcription

1 Playing around with Risks Jurgen Cleuren April 19th CTG, Inc. Introduction Projects are done in a probabilistic environment Incomplete information Parameters change over time What is true in the beginning of the project, can be false some time later Games in a probabilistic environment Incomplete information (cardgames, ) Random element (dice, cards, ) 1

2 Which games? Poker Monopoly Backgammon Succes: First learn the rules, then play better than everyone else Rules -> requirements Every game starts with learning and agreeing on the rules -> every project starts with defining and agreeing on the requirements 2

3 TEXAS HOLD EM POKER Rules of Texas Hold em 2 Blind cards Betting round 3 Open Community cards (Flop) Betting round 1 Open Community card (Turn) Betting Round 1 Open Community Card (River) Betting Round Best hand of 5 cards wins 3

4 Hand Ranking General test flow vs Poker round Create FTT/Risk Matrix Software Delivery Full functional test Bugfixing New Software Delivery Retesting and regression Bugfixing Final Software Version Regression Test Get Blind cards evaluate risk - Bet Flop (3 cards) Re-evalute hand and Risk Betting round Turn (1 card) Re-evaluate hand and Risk Betting round River (1 card) Final bets and showdown 4

5 Similarities / Differences 3 Recurring phases 1 big phase (Complete Functional testing vs Flop) 2 small phases (retest + regression vs Turn + River) Determine Risk before 1 st test run (Risk Matrix or FTT vs 1 st bet) Adapt Risk Matrix or FTT after each test run Should the FTT or Risk Matrix be adapted? Every Test run gives new information Likelihood of risks change Failed test cases get a higher likelihood Passed test cases in unchanged code have a lower likelihood Closer to deadline => Risks can change Reporting is more clear Each Risk Matrix/FTT tree is a snapshot of the project at a certain time 5

6 6

7 7

8 Who? Test manager should take the lead Preferably Project Board Test manager can do it by himself, but the board should at least be aware that this activity is done Justification A Poker hand needs justification at any point in the game Having the best hand in the beginning doesn t imply that you are going to win You must be prepared to fold when you are not winning anymore The money you ve already bet doesn t count It is in the pot so it is not your money anymore Don t put more money in a losing hand No justification anymore: get out of the hand Cut your losses 8

9 Project justification A project needs justification at any point in time During testing: justification after each test run New information is given Risks are updated No Justification means project should be closed Previous investments do not count Don t put more money in a failing project Cut your losses Test process justification Get Blind cards evaluate risk - Bet Flop (3 cards) Re-evalute hand and Risk Betting round Turn (1 card) Re-evaluate hand and Risk Betting round River (1 card) Final bets and showdown Create FTT/Risk Matrix Software Delivery Full functional test Justification Bugfixing New Software Delivery Retesting and regression Justification Bugfixing i Final Software Version Regression Test Justification 9

10 Result Oriented Thinking A decision can be right even if the outcome is not favourable Focus more on the decision and the why instead of the result Tester A completes a test set in 4 days by skipping tests Tester B completes the same test set in 6 days through thoroughly testing No defects were discovered in production Who would you reward the most? Thought process In poker, to anticipate and understand your opponents actions, you need to think as your opponent and not as you in his situation What motivates you, does not necessarily motivate another person Successful people ask better questions WHY? is more important than HOW? or WHO? Someone who knows HOW will always have a job Someone who knows WHY will always be his boss 10

11 Luck Luck is when preparation meets opportunity - Seneca Be prepared to get lucky In poker, sometimes you need to get lucky. When you get lucky, be sure to take a big pot out of it. Sometimes a best case scenario happens, but we need to take advantage of it. Being ahead of planning is more valuable if you can actually do something with this situation MONOPOLY 11

12 Aspects of the game Investing in houses The higher the investment, the bigger the payoff Some spaces have a higher probability Random element: dice 12

13 Analogy to Risk Based Testing Different probability of landing on spaces = Likelihood Higher Payoff = Impact What can monopoly teach us? What is more important: Impact or Likelihood? Impact and likelihood on the board 13

14 Winning Monopoly strategies Complete Orange Colour group and invest as soon as possible Why Orange? It has the biggest probability of other players landing on it Be prepared to even trade down to get this colour group Complete Red Colour group as a second priority Why Red? It has the second biggest probability of other players landing on it What lessons are to be learned Likelihood >>>>>> impact The weight of Likelihood should be > 50% The weight of Impact should be < 50% 14

15 BACKGAMMON Backgammon 15

16 Important rules of Backgammon Goal: get all your tiles from one end to the other Only tiles that are standing alone on a pillar can be captured A captured tile has to be brought back in play at the beginning of the board Random element: Dice Translation to risks Your own checkers: Risks Opponents checkers: Causes Whenever one of your checkers is captured it s in fact a cause hitting a risk A risk is mitigated when the checker cannot be captured (2 or more on one pillar) 16

17 Translation of risk priority Low risk: checker in the first quadrant Medium risk: checker in the second or third quadrant High risk: checker in the fourth quadrant What can Backgammon teach us? Which risks should be mitigated and which are low priority? There might be a possibility to remove a cause, but in the same way creating a new risk. Should we do it? Eg: Back-up testmanager vs full time testing 17

18 Backgammon Strategies Checkers in the 1 st quadrant don t have to be protected. Moving forward is the better play. Risks with low priority don t have to be mitigated. The correcting cost is usually way less than the mitigation costs Checkers in the 4 th quadrant need to be protected at all costs. Risk with high priority need to be mitigated at all costs For checkers in the 2 nd and 3 rd following rule applies: Always take a chance to capture If you can remove a cause and therefore create a medium or low risk, do so CONCLUSIONS 18

19 Conclusions What Poker taught us: Adapt FTT/Risk tree after each test run Priorities of test items can change Justification has to be done after each test run Don t focus on results, focus on decisions Be prepared to get lucky! Conclusions Create Ceae FTT/Risk Matrix Software Delivery Full functional test Bugfixing New Software Delivery Retesting and regression Bugfixing Final Software Version Regression Test Create FTT/Risk Matrix Software Delivery Full functional test Adapt FTT/Risk Matrix Justification Bugfixing New Software Delivery Retesting and regression Adapt FTT/Risk Matrix Justification Bugfixing Final Software Version Regression Test Adapt FTT/Risk Matrix Justification 19

20 Conclusions Monopoly Likelihood >>>>> Impact Backgammon Don t mitigate low priority risk Always mitigate high priority risks Removing a cause and creating a medium or low risk is the way to play Jurgen Cleuren (jurgen.cleuren@ctg.com) QUESTIONS AND ANSWERS 20