Supporting Early Decision-Making in the Presence of Uncertainty

Transcription

1 Supporting Early ecision-making in the Presence of Uncertainty JEIFER HORKOFF 1, RICK SALAY 2, MARSHA CHECHIK 2, ASSIO I SARO 2 1 epartment of Information Engineering and Computer Science, University of Trento, Italy horkoff@disi.unitn.it 2 epartment of Computer Science, University of Toronto, Canada {rsalay,chechik,adisandro}@cs.toronto.edu RE 14

2 Early RE Analysis Who is involved What do they need Large space of possible alternative requirements Example Scheduler Supporting Early ecision Making - RE'14 - Horkoff et al. 2

3 Running Example: Scheduler Initiator () Quick (Q) Other Ways to Organize (OW) Alternative Be Scheduled (MBS) Organize () Low Effort () be Scheduled (MBS) Let Scheduler Schedule (LSSM) etermine ate Scheduler () ependencies (ep) Schedule (SM) Attend () Provide etails (Pr) etails () Participate in () Use Profiles (UP) Low Effort (P) Participant () Convenient ate (CM) Agreeable ate () ecide Convenient ates (C) i* Legend Actor Goal Softgoal Task Resource Actor Boundary ependency Contribution ecomposition Means-Ends Initial Analysis Label Supporting Early ecision Making - RE'14 - Horkoff et al. 3

4 Early ecision Making (e.g., Horkoff & Yu) Initiator Organize () () Quick (Q) Be Scheduled (MBS) Other Ways to Organize (OW) etermine Low Effort () ate Let Scheduler Schedule (LSSM) be Scheduled (MBS) ependencies (ep) etails Scheduler () () Schedule (SM) Attend () Provide etails (Pr) Participate Participant in () () Use Profiles (UP) Low Effort (P) Satisfied (FS) Partially Satisfied (PS) Conflict (C) (U) Convenient ate (CM) Agreeable ate () ecide Convenient ates (C) Partially enied (P) enied (F) o Label () i* Legend Actor Goal Softgoal Task Resource Actor Boundary ependency Contribution ecomposition Means-Ends Initial Analysis Label Supporting Early ecision Making - RE'14 - Horkoff et al. 4

5 Uncertainty in Early RE uring RE elicitation, it is common to uncover uncertainties Supporting Early ecision Making - RE'14 - Horkoff et al. 5

6 Alternative esigns Hmm, I don t yet know all the possibilities. 6

7 Conflicting Stakeholder Opinions What do I do until they decide 7

8 Incomplete Information I don t know everything about this, yet. 8

9 Uncertainty in Requirements Engineering Many design alternatives Incomplete information Conflicting stakeholder opinions Uncertainty about the content of the model. 9

10 Scheduler Uncertainty Initiator () Quick (Q) Other Ways to Organize (OW) unknown types of contributions Be Scheduled (MBS) unknown which actor will perform this task Organize () Low Effort () Let Scheduler Schedule (LSSM) be Scheduled (MBS) unknown set of other ways to organize meetings etermine ate Scheduler () unknown dependencies ependencies (ep) Schedule (SM) Attend () Provide etails (Pr) etails () unknown if we need this Participate in () Use Profiles (UP) Low Effort (P) unknown what or how many details there will be Participant () Convenient ate (CM) Agreeable ate () ecide Convenient ates (C) unknown whether these are actually different goals unknown if we need this i* Legend Actor Goal Softgoal Task Resource Actor Boundary ependency Contribution ecomposition Means-Ends Initial Analysis Label Supporting Early ecision Making - RE'14 - Horkoff et al. 10

11 Early ecision Making with Uncertainty May not be able to resolve all uncertainties before decisions must be made! We need methods and tools to support early decisionmaking and trade-off analysis in the presence of uncertainty Supporting Early ecision Making - RE'14 - Horkoff et al. 11

12 Our Approach: Goal Model Analysis + MAVO To tackle this challenge we make use of existing, established RE Techniques Goal modeling and goal satisfaction analysis (e.g., Horkoff &Yu, 2010, 2102) The MAVO framework for formally capturing and reasoning over model uncertainty (Salay et al., 2012) We focus on the design-time uncertainty of the modeler, and not the intrinsic, run-time uncertainty of environment Use possibilistic rather than probabilistic uncertainty Supporting Early ecision Making - RE'14 - Horkoff et al. 12

13 Background: Capturing Uncertainty with MAVO (Salay et al., RE 12) unknown which actor will perform this task unknown dependencies ep unknown if we need this i* Legend Actor Goal Q MBS M CM Softgoal Task OW LSSM Pr UP Resource Actor Boundary unknown types of contributions MBS unknown set of other ways to organize meetings SM P unknown what or how many details there will be C unknown whether these are actually different goals unknown if we need this ependency Contribution ecomposition Means-Ends Initial Analysis Label Supporting Early ecision Making - RE'14 - Horkoff et al. 13

14 Background: Capturing Uncertainty with MAVO (Salay et al., RE 12) unknown which actor will perform this task unknown dependencies ep unknown if we need this i* Legend Actor Goal Q MBS M CM Softgoal Task OW LSSM Pr UP Resource Actor Boundary unknown types of contributions MBS unknown set of other ways to organize meetings SM P C C unknown whether these are actually different goals unknown what or how many details there will be ependency Contribution ecomposition Means-Ends Initial Analysis Label Supporting Early ecision Making - RE'14 - Horkoff et al. 14

15 Background: Capturing Uncertainty with MAVO (Salay et al., RE 12) unknown which actor will perform this task unknown dependencies ep unknown if we need this i* Legend Actor Goal Q MBS M (V) (V) (V) CM Softgoal Task OW LSSM Pr UP (V) Resource Actor Boundary unknown types of contributions MBS unknown set of other ways to organize meetings SM P unknown what or how many details there will be C C ependency Contribution ecomposition Means-Ends Initial Analysis Label Supporting Early ecision Making - RE'14 - Horkoff et al. 15

16 Background: Capturing Uncertainty with MAVO (Salay et al., RE 12) unknown which actor will perform this task unknown dependencies ep unknown if we need this i* Legend Actor Goal Q MBS M (V) (V) (V) CM Softgoal Task OW LSSM Pr Pr UP (V) Resource Actor Boundary unknown types of contributions MBS unknown set of other ways to organize meetings SM P C C ependency Contribution ecomposition Means-Ends Initial Analysis Label Supporting Early ecision Making - RE'14 - Horkoff et al. 16

17 Capturing Uncertainty with MAVO () () ep () Q MBS (V) () Pr () (V) (VM) (V) (VM) CM OW LSSM M () () UP C MBS SB P (comp) Supporting Early ecision Making - RE'14 - Horkoff et al. 17

18 Background: Concretizations Set of certain concrete models, concretizations Q OW MBS LSSM MBS () () ep (V) M SB () () () Pr () () P UP (V) (VM) (comp) (V) (VM) CM C MAVO model capturing uncertainty Q MBS OW LSSM M ep Q PrMBS UP OW P LSSM M C CM ep Pr UP CM Q MBS OW LSSM M ep Pr P UP CM C MBS SM MBS SM P C MBS SM Supporting Early ecision Making - RE'14 - Horkoff et al. 18

19 Methodology Q1 What are the analysis results given a particular analysis alternative in the goal model, considering model uncertainties Q2 Can viable choices be made over the set of results from Q1 Q3 Can viable choices be achieved simultaneously Q4 Given choices, what uncertainty reductions are forced How can we target elicitation Q5 Are suggested uncertainty reductions reasonable If not, iterate over the model and backtrack. [Further analysis/ elicitation] Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 19

20 Answering Q1: etermining Labels Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Further analysis/ elicitation] [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 20

21 Answering Q1: etermining Labels () () ep () [Further analysis/ elicitation] Start Q Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability OW [Choices simultaneously achievable] MBS LSSM MBS (V) M SB () (V) () Pr () () UP P (V) (VM) (VM) CM C (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 21

22 Answering Q1: etermining Labels () () ep () [Further analysis/ elicitation] Start Q Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability OW [Choices simultaneously achievable] MBS LSSM MBS (V) M SB () () Pr () () (V) (VM) (V) UP (VM) CM C P (comp) (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 22

23 Answering Q2: Making Choices () () ep () [Further analysis/ elicitation] Start Q Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability OW [Choices simultaneously achievable] MBS LSSM MBS (V) M SB () () Pr () () (V) (VM) (V) UP (VM) CM C P (comp) (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 23

24 Q3: Checking Simultaneous Achievement () () ep () [Further analysis/ elicitation] Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices Q [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions OW MBS LSSM MBS (V) M SB () In our example, choices are simultaneously achievable (achievable together in at least one concretization) () Pr () () UP P (V) (VM) (comp) (V) (VM) CM C [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 24

25 Q4: Finding forced Uncertainty Reductions () () ep () [Further analysis/ elicitation] Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices Q [Acceptable choices found] (Q3) Check Simultaneous Achievability OW [Choices simultaneously achievable] MBS LSSM MBS (V) M SB () () Pr () (comp) () P UP (V) (VM) (V) (VM) CM C Forced Uncertainty Reduction (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 25

26 Q4: Finding forced Uncertainty Reductions [Further analysis/ elicitation] Q OW Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] MBS () MBS () ep LSSM M () () (V) () Pr () SB (comp) () P (V) (VM) (V) (VM) CM UP C Forced Uncertainty Reduction (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 26

27 Q5: Evaluate Uncertainty Reductions () () ep () [Further analysis/ elicitation] Start Q Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability OW [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found MBS LSSM MBS (V) M SB () () Pr () (comp) UP P (V) (VM) Answering Q3 produces an example concretization (V) (VM) CM C Supporting Early ecision Making - RE'14 - Horkoff et al. 27

28 Q5: Evaluate Uncertainty Reductions [Further analysis/ elicitation] Start Q Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability OW [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found MBS LSSM MBS () () ep (V) M SB () () () Pr () (comp) P UP (V) (VM) Answering Q3 produces an example concretization (V) (VM) CM CM C C Supporting Early ecision Making - RE'14 - Horkoff et al. 28

29 Q5: Evaluate Uncertainty Reductions [Further analysis/ elicitation] Start Q Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability OW [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found MBS LSSM MBS () () ep (V) M SB () () () Pr () (comp) P UP (V) (VM) Answering Q3 produces an example concretization (V) (VM) CM CM C C Supporting Early ecision Making - RE'14 - Horkoff et al. 29

30 Q5: Evaluate Uncertainty Reductions Q MBS OW LSSM ep M Will not provide details, instead will use profiles UP CM C [Further analysis/ elicitation] Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] MBS SB P Removed in concretization Uncertainty Removed (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 30

31 Q5: Evaluate Uncertainty Reductions ep Q MBS OW LSSM M UP CM C [Further analysis/ elicitation] Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] MBS SB P Removed in concretization Uncertainty Removed (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 31

32 Q5: Evaluate Uncertainty Reductions ep Q MBS OW LSSM M UP CM Acceptable task C [Further analysis/ elicitation] Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] MBS SB P Removed in concretization Uncertainty Removed (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 32

33 Apply Changes to Uncertain Model () () ep () Q MBS (V) (V) (V) (V) (V) CM OW LSSM M UP C Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices MBS SB P [Further analysis/ elicitation] [Acceptable choices found] (Q3) Check Simultaneous Achievability (comp) [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 33

34 Re-evaluate () () ep () Q MBS (V) (V) (V) (V) (V) CM OW LSSM M UP C Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices MBS SB P [Further analysis/ elicitation] [Acceptable choices found] (Q3) Check Simultaneous Achievability (comp) [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 34

35 Re-evaluate () () ep () Q MBS (V) (V) (V) (V) (V) CM OW LSSM M UP C [Further analysis/ elicitation] [Further analysis/ elicitation] Start Pose Analysis Question (Alternative) (Q1) Apply Analysis (Q2) Make choices [Acceptable choices found] (Q3) Check Simultaneous Achievability [Choices simultaneously achievable] (Q4) Find Forced Uncertainty Reductions MBS SB (comp) P [Forced reductions acceptable] (Q5) Evaluate Uncertainty Reductions [Choices achieved with certainty] Viable Alternative Found Supporting Early ecision Making - RE'14 - Horkoff et al. 35

36 Final Result Without explicit uncertainty () () ep () With explicit uncertainty Q OW MBS LSSM (V) M (V) (V) (V) CM (V) UP C MBS SB P (comp) Supporting Early ecision Making - RE'14 - Horkoff et al. 36

37 How Implementation details Supporting Early ecision Making - RE'14 - Horkoff et al. 37

38 Formal Background: Goal Model Analysis Goal model analysis has been implemented using propositional logic (Giorgini et al. 12, Horkoff & Yu 10, 12) Supporting Early ecision Making - RE'14 - Horkoff et al. 38

39 Formal Background: Uncertainty with MAVO i Metamodel FOL Theory: Σ, Φ Σ Signature - Sorts representing entity types (e.g., Actor, Intention, Task) Predicates representing relations (e.g., task decomposes goal) Φ Sentences - i* well-formedness constraints Scheduler i* Model FO G = Σ Σ G, Φ Φ G Σ G and Φ G are model G-specific predicates and constraints Supporting Early ecision Making - RE'14 - Horkoff et al. 39

40 Formal Background: Uncertainty with MAVO i Metamodel FOL Theory: Σ, Φ Σ Signature - Sorts representing entity types (e.g., Actor, Intention, Task) Predicates representing relations (e.g., task decomposes goal) Φ Sentences - i* well-formedness constraints Scheduler i* Model FO G = Σ Σ G, Φ Φ G Σ G and Φ G are model G-specific predicates and constraints Supporting Early ecision Making - RE'14 - Horkoff et al. 40

41 Formal Background: Uncertainty with MAVO i Metamodel FOL Theory: Σ, Φ Σ Signature - Sorts representing entity types (e.g., Actor, Intention, Task) Predicates representing relations (e.g., task decomposes goal) Φ Sentences - i* well-formedness constraints Scheduler i* Model FO G = Σ Σ G, Φ Φ G Σ G and Φ G are model G-specific predicates and constraints () Supporting Early ecision Making - RE'14 - Horkoff et al. 41

42 Formal Background: Uncertainty with MAVO i Metamodel FOL Theory: Σ, Φ Σ Signature - Sorts representing entity types (e.g., Actor, Intention, Task) Predicates representing relations (e.g., task decomposes goal) Φ Sentences - i* well-formedness constraints Scheduler i* Model FO G = Σ Σ G, Φ Φ G Σ G and Φ G are model G-specific predicates and constraints (V) () Supporting Early ecision Making - RE'14 - Horkoff et al. 42

43 GM Analysis with MAVO Uncertainty Formalization Add i* analysis to FO MAVO encoding Extended encoding: FO e G = Σ Σ G Σ label, Φ Φ G Φ l Φ p Languagespecific (i*) Instancemodel specific Example Σ label : Analysis Labels Initial Analysis Labels Propagation Constraints Example constraint: Example constraint: Supporting Early ecision Making - RE'14 - Horkoff et al. 43

44 Answering Q1: etermining Labels Goal model analysis with MAVO, basic idea: assign an analysis label to an intention if there exists a concretization such that the intention has that label E.g., there is at least one concretization where has a label, at least one where it has a label, and at least one where it has a label Φ Φ G Φ l Φ p ( i Intention i FS i ) Supporting Early ecision Making - RE'14 - Horkoff et al. 44

45 Q3: Checking Simultaneous Achievement Add choices to the formalism as constraints and check for satisfiability: Φ Φ G Φ l Φ p Φ c Where is the encoding of the user s choices, for example: Q MBS () () ep (V) () () Pr Supporting Early ecision Making - RE'14 - Horkoff et al. 45 M ) () UP (V) (VM) (V) (VM) CM Answering Q4: Use method from Salay et al., FASE 13

46 Experience Implemented the automated parts of our method (Q1, Q3, Q4) in the Model Management Tool Framework (MMTF) Encoded FO representation and passed it to an SMT solver (z3) Running times for Q1, 3 and 4 on Scheduler model ranged from 0.18 to seconds Applied to method to two larger cases Inflo study (described briefly in study, available online) Smart Grid Study (current work) Supporting Early ecision Making - RE'14 - Horkoff et al. 46

47 Conclusions and Future Work Provided a method to support decision making in early RE with the presence of uncertainty Used existing RE approaches: goal model analysis + MAVO Provided methodology, showing how to answer 5 analysis questions (Q1-5) Provided examples + tooling Future work: Improve usability of MAVO labels + analysis labels Extend method and implementation to support backward analysis More examples and validation Supporting Early ecision Making - RE'14 - Horkoff et al. 47

48 Thank you! Questions Contact: ( Supporting Early ecision Making - RE'14 - Horkoff et al. 48