Jeux finiment répétés avec signaux semi-standards

Transcription

1 Jeux finiment répétés avec signaux semi-standards P. Contou-Carrère 1, T. Tomala 2 CEPN-LAGA, Université Paris 13 7 décembre Université Paris 1, Panthéon Sorbonne 2 HEC, Paris

2 Introduction Repeated games are dynamic interactions, played by stages This talk: the players repeat over and over the same stage game, which is perfectly known In the standard model (with perfect monitoring), the actions played at a given stage are publicly observed before the next stage is reached. We have the Folk Theorem: the equilibrium payoffs of the repeated game are the feasible and individually rational payoffs. We study the model with imperfect monitoring: at the end of each stage, the players receive some signal depending on the action profile. e.g.: Principal-Agent problems Computing the equilibrium payoffs is not known.

3 Introduction Repeated games are dynamic interactions, played by stages This talk: the players repeat over and over the same stage game, which is perfectly known In the standard model (with perfect monitoring), the actions played at a given stage are publicly observed before the next stage is reached. We have the Folk Theorem: the equilibrium payoffs of the repeated game are the feasible and individually rational payoffs. We study the model with imperfect monitoring: at the end of each stage, the players receive some signal depending on the action profile. e.g.: Principal-Agent problems Computing the equilibrium payoffs is not known.

4 Introduction Repeated games are dynamic interactions, played by stages This talk: the players repeat over and over the same stage game, which is perfectly known In the standard model (with perfect monitoring), the actions played at a given stage are publicly observed before the next stage is reached. We have the Folk Theorem: the equilibrium payoffs of the repeated game are the feasible and individually rational payoffs. We study the model with imperfect monitoring: at the end of each stage, the players receive some signal depending on the action profile. e.g.: Principal-Agent problems Computing the equilibrium payoffs is not known.

5 Lots of equilibrium notions: NE, uniform equilibrium Refinements: SPE, sequential... Also correlation and communication This talk: Equilibrium payoffs in finitely repeated games with semi-standard signals Actions sets partitioned into equivalence classes When a player chooses an action, the cell of the partition that contains this action is publicly announced to the rest of the players NE and SPE payoffs of the finitely repeated game with pure strategies. Characterize the limit set of equilibrium payoffs as T + (in the Hausdorff distance)

6 Lots of equilibrium notions: NE, uniform equilibrium Refinements: SPE, sequential... Also correlation and communication This talk: Equilibrium payoffs in finitely repeated games with semi-standard signals Actions sets partitioned into equivalence classes When a player chooses an action, the cell of the partition that contains this action is publicly announced to the rest of the players NE and SPE payoffs of the finitely repeated game with pure strategies. Characterize the limit set of equilibrium payoffs as T + (in the Hausdorff distance)

7 Lots of equilibrium notions: NE, uniform equilibrium Refinements: SPE, sequential... Also correlation and communication This talk: Equilibrium payoffs in finitely repeated games with semi-standard signals Actions sets partitioned into equivalence classes When a player chooses an action, the cell of the partition that contains this action is publicly announced to the rest of the players NE and SPE payoffs of the finitely repeated game with pure strategies. Characterize the limit set of equilibrium payoffs as T + (in the Hausdorff distance)

8 Outline 1 Repeated Games with Imperfect Monitoring The model Aspects of imperfect monitoring 2 The model Equilibrium Notions Feasible Payoffs Robust to Undetectable Deviations Punishment levels Necessary Conditions on Equilibrium payoffs Characterization Elements of the proof

9 Outline 1 Repeated Games with Imperfect Monitoring The model Aspects of imperfect monitoring 2 The model Equilibrium Notions Feasible Payoffs Robust to Undetectable Deviations Punishment levels Necessary Conditions on Equilibrium payoffs Characterization Elements of the proof

10 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Data: Repeated games with imperfect monitoring (also called repeated games with signals, or supergames) A finite stage game G given by a set of players N = {1,..., n}, and for each player i a set of actions A i and a payoff functions g i : A R where A = i A i denotes the set of actions profiles. a monitoring structure: for each player i, a finite set of signals Y i, and a signalling function l : A (Y), where Y = i Y i stands for the set of signal profiles. Play: at every stage t = 1, 2,..., the players independently choose an action in their own set of actions. If a t A is the joint played action, a profile of signals y t = (y i,t ) i is selected according to l(a t ). The stage payoff for player i is then g i (a t ), but what player i learns before starting stage t + 1 is y i,t.

11 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Data: Repeated games with imperfect monitoring (also called repeated games with signals, or supergames) A finite stage game G given by a set of players N = {1,..., n}, and for each player i a set of actions A i and a payoff functions g i : A R where A = i A i denotes the set of actions profiles. a monitoring structure: for each player i, a finite set of signals Y i, and a signalling function l : A (Y), where Y = i Y i stands for the set of signal profiles. Play: at every stage t = 1, 2,..., the players independently choose an action in their own set of actions. If a t A is the joint played action, a profile of signals y t = (y i,t ) i is selected according to l(a t ). The stage payoff for player i is then g i (a t ), but what player i learns before starting stage t + 1 is y i,t.

12 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Data: Repeated games with imperfect monitoring (also called repeated games with signals, or supergames) A finite stage game G given by a set of players N = {1,..., n}, and for each player i a set of actions A i and a payoff functions g i : A R where A = i A i denotes the set of actions profiles. a monitoring structure: for each player i, a finite set of signals Y i, and a signalling function l : A (Y), where Y = i Y i stands for the set of signal profiles. Play: at every stage t = 1, 2,..., the players independently choose an action in their own set of actions. If a t A is the joint played action, a profile of signals y t = (y i,t ) i is selected according to l(a t ). The stage payoff for player i is then g i (a t ), but what player i learns before starting stage t + 1 is y i,t.

13 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Illustration: The prisoner s dilemma C 2 D 2 C 1 ( 1, 1) ( 10, 0) D 1 (0, 10) ( 8, 8) (unique equilibrium payoff for the one-shot game: ( 8, 8)). Standard case of perfect monitoring: Y i = A and y i,t = a t for each player i. Trivial observation for player i: Y i is a singleton (play in the dark) Public signals: all players observe the same signal (Fudenberg, Levine, Maskin 94) Observable payoffs: each player can deduce his payoff from his own action and signal (Tomala, 99)

14 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Illustration: The prisoner s dilemma C 2 D 2 C 1 ( 1, 1) ( 10, 0) D 1 (0, 10) ( 8, 8) (unique equilibrium payoff for the one-shot game: ( 8, 8)). Standard case of perfect monitoring: Y i = A and y i,t = a t for each player i. Trivial observation for player i: Y i is a singleton (play in the dark) Public signals: all players observe the same signal (Fudenberg, Levine, Maskin 94) Observable payoffs: each player can deduce his payoff from his own action and signal (Tomala, 99)

15 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Illustration: The prisoner s dilemma C 2 D 2 C 1 ( 1, 1) ( 10, 0) D 1 (0, 10) ( 8, 8) (unique equilibrium payoff for the one-shot game: ( 8, 8)). Standard case of perfect monitoring: Y i = A and y i,t = a t for each player i. Trivial observation for player i: Y i is a singleton (play in the dark) Public signals: all players observe the same signal (Fudenberg, Levine, Maskin 94) Observable payoffs: each player can deduce his payoff from his own action and signal (Tomala, 99)

16 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Illustration: The prisoner s dilemma C 2 D 2 C 1 ( 1, 1) ( 10, 0) D 1 (0, 10) ( 8, 8) (unique equilibrium payoff for the one-shot game: ( 8, 8)). Standard case of perfect monitoring: Y i = A and y i,t = a t for each player i. Trivial observation for player i: Y i is a singleton (play in the dark) Public signals: all players observe the same signal (Fudenberg, Levine, Maskin 94) Observable payoffs: each player can deduce his payoff from his own action and signal (Tomala, 99)

17 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Strategies and payoffs A strategy for player i: σ i = (σ i,t ) t 1 where σ i,t : (A i Y i ) t 1 (A i ) gives the lottery played at stage t depending on his current information. A strategy profile σ induces a probability over plays. Average T-payoff for player i: γ i,t (σ) = E σ [ 1 T T g i (a t )] t=1 Denote γ T = (γ i,t ) i the vector payoff function.

18 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Equilibrium concepts A Nash equilibrium for G T is sa strategy profile σ such that for every player i and strategy σ i for player i, γ i,t(σ i, σ i) γ i,t (σ). A Uniform Equilibrium for G is a strategy profile σ such that 1 For each player i, γ i,t(σ) converges to some limit γ i(σ) as T +. 2 For all ε > 0, σ is an ε-ne for any sufficiently long finitely repeated game i.e. there exists T ε N such that for all T T ε, for each player i and strategy σ i for player i, γ i,t(σ i, σ i) γ i,t(σ) + ε. Denote E T (resp. E ) the set of NE payoffs for G T (resp. UE payoffs). Remarks: - E T is a non empty set (apply Nash/Glicksberg: compact convex strategy sets, multi-linear continuous payoffs) - E 1 E T E

19 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Equilibrium concepts A Nash equilibrium for G T is sa strategy profile σ such that for every player i and strategy σ i for player i, γ i,t(σ i, σ i) γ i,t (σ). A Uniform Equilibrium for G is a strategy profile σ such that 1 For each player i, γ i,t(σ) converges to some limit γ i(σ) as T +. 2 For all ε > 0, σ is an ε-ne for any sufficiently long finitely repeated game i.e. there exists T ε N such that for all T T ε, for each player i and strategy σ i for player i, γ i,t(σ i, σ i) γ i,t(σ) + ε. Denote E T (resp. E ) the set of NE payoffs for G T (resp. UE payoffs). Remarks: - E T is a non empty set (apply Nash/Glicksberg: compact convex strategy sets, multi-linear continuous payoffs) - E 1 E T E

20 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Equilibrium concepts A Nash equilibrium for G T is sa strategy profile σ such that for every player i and strategy σ i for player i, γ i,t(σ i, σ i) γ i,t (σ). A Uniform Equilibrium for G is a strategy profile σ such that 1 For each player i, γ i,t(σ) converges to some limit γ i(σ) as T +. 2 For all ε > 0, σ is an ε-ne for any sufficiently long finitely repeated game i.e. there exists T ε N such that for all T T ε, for each player i and strategy σ i for player i, γ i,t(σ i, σ i) γ i,t(σ) + ε. Denote E T (resp. E ) the set of NE payoffs for G T (resp. UE payoffs). Remarks: - E T is a non empty set (apply Nash/Glicksberg: compact convex strategy sets, multi-linear continuous payoffs) - E 1 E T E

21 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Example: the prisoner s dilemma with perfect monitoring C 2 D 2 C 1 ( 1, 1) ( 10, 0) D 1 (0, 10) ( 8, 8) ( 1, 1) is an equilibrium payoff of the infinitely repeated game: Play C i as long as your opponent does, otherwise play D i forever. The unique equilibrium payoff of the finitely repeated game is ( 8, 8): Play the unique Nash equilibrium of the stage game (D 1, D 2 ) at every stage.

22 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Example: the prisoner s dilemma with perfect monitoring C 2 D 2 C 1 ( 1, 1) ( 10, 0) D 1 (0, 10) ( 8, 8) ( 1, 1) is an equilibrium payoff of the infinitely repeated game: Play C i as long as your opponent does, otherwise play D i forever. The unique equilibrium payoff of the finitely repeated game is ( 8, 8): Play the unique Nash equilibrium of the stage game (D 1, D 2 ) at every stage.

23 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Define two sets. Feasible payoffs: conv g(a) Min max level for player i: v i = The standard Folk theorem min max p i j i (A j) p i (A i) g i (p i, p i ) Individually rational Payoffs: IR = {(u i ) i R n : i, u i v i } Standard Folk theorem: The equilibrium payoffs for the infinitely repeated game are the payoffs which are both feasible (that can be achieved) and individually rational (every player gets at least his min max level). E = conv g(a) IR

24 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Define two sets. Feasible payoffs: conv g(a) Min max level for player i: v i = The standard Folk theorem min max p i j i (A j) p i (A i) g i (p i, p i ) Individually rational Payoffs: IR = {(u i ) i R n : i, u i v i } Standard Folk theorem: The equilibrium payoffs for the infinitely repeated game are the payoffs which are both feasible (that can be achieved) and individually rational (every player gets at least his min max level). E = conv g(a) IR

25 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Define two sets. Feasible payoffs: conv g(a) Min max level for player i: v i = The standard Folk theorem min max p i j i (A j) p i (A i) g i (p i, p i ) Individually rational Payoffs: IR = {(u i ) i R n : i, u i v i } Standard Folk theorem: The equilibrium payoffs for the infinitely repeated game are the payoffs which are both feasible (that can be achieved) and individually rational (every player gets at least his min max level). E = conv g(a) IR

26 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring The Folk theorem for SPE A subgame perfect equilibrium (SPE) is strategy profile σ such that σ induces an equilibrium after every finite sequence of actions. Denote E T (resp. E ) the set of SPE payoffs for G T (resp. G ). Aumann, Shapley (1976), Rubinstein (1977) Proof: E = conv g(a) IR Objective = Define strategy profile such that deviations from main path are deterred and punishers have incentive to punish any deviator from main path. Solution = Punish deviator for a long but finite number of stages then return to main path.

27 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring The Folk theorem for SPE A subgame perfect equilibrium (SPE) is strategy profile σ such that σ induces an equilibrium after every finite sequence of actions. Denote E T (resp. E ) the set of SPE payoffs for G T (resp. G ). Aumann, Shapley (1976), Rubinstein (1977) Proof: E = conv g(a) IR Objective = Define strategy profile such that deviations from main path are deterred and punishers have incentive to punish any deviator from main path. Solution = Punish deviator for a long but finite number of stages then return to main path.

28 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring The Finitely Repeated Game (1) Benoit and Krishna (1987) If for each player i, there exist e i E 1 such that e i (i) > v i in the Hausdorff distance. lim E T = conv g(a) IR T + Proof: Equilibrium Strategy = Main Path + Punishment phase The main path is followed by a sequence of Nash equilibria of the stage game: players get e 1 for R 1 stages then e 2 for R 2 stages etc. If player i deviates from the main path he receives v i from the day after the deviation on.

29 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring The Finitely Repeated Game (1) Benoit and Krishna (1987) If for each player i, there exist e i E 1 such that e i (i) > v i in the Hausdorff distance. lim E T = conv g(a) IR T + Proof: Equilibrium Strategy = Main Path + Punishment phase The main path is followed by a sequence of Nash equilibria of the stage game: players get e 1 for R 1 stages then e 2 for R 2 stages etc. If player i deviates from the main path he receives v i from the day after the deviation on.

30 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring The Finitely Repeated Game (2) Conditions C1 and C2: 1 For every player i, there exist e i, f i E 1 such that e i (i) > f i (i) 2 dim conv g(a) = n Benoit and Krishna (1985) Under conditions C1 and C2, in the Hausdorff distance. lim E T = conv g(a) IR T + Proof: Equilibrium Strategy = Main Path + Punishment + Reward Remark: Benoit and Krishna (1985) consider pure strategies. Gossner (1995) extends this result to mixed strategies.

31 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring The Finitely Repeated Game (2) Conditions C1 and C2: 1 For every player i, there exist e i, f i E 1 such that e i (i) > f i (i) 2 dim conv g(a) = n Benoit and Krishna (1985) Under conditions C1 and C2, in the Hausdorff distance. lim E T = conv g(a) IR T + Proof: Equilibrium Strategy = Main Path + Punishment + Reward Remark: Benoit and Krishna (1985) consider pure strategies. Gossner (1995) extends this result to mixed strategies.

32 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring The Finitely Repeated Game (2) Conditions C1 and C2: 1 For every player i, there exist e i, f i E 1 such that e i (i) > f i (i) 2 dim conv g(a) = n Benoit and Krishna (1985) Under conditions C1 and C2, in the Hausdorff distance. lim E T = conv g(a) IR T + Proof: Equilibrium Strategy = Main Path + Punishment + Reward Remark: Benoit and Krishna (1985) consider pure strategies. Gossner (1995) extends this result to mixed strategies.

33 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Signals do matter. Example: The prisoner s dilemma in the dark C 2 D 2 C 1 ( 1, 1) ( 10, 0) E = {( 8, 8)} D 1 (0, 10) ( 8, 8) With imperfect monitoring, there might be in general: 1 Undetectable deviations (Lehrer 1990, 1992) 2 A deviation which is detected by some players, but not the others (Renault, Tomala 1998) 3 A deviation which is detected but the identity of the deviator is unknown (Tomala 1998, Renault et al 05 and 08) 4 Computing the punishment levels may be difficult (Gossner, Tomala 07)

34 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Signals do matter. Example: The prisoner s dilemma in the dark C 2 D 2 C 1 ( 1, 1) ( 10, 0) E = {( 8, 8)} D 1 (0, 10) ( 8, 8) With imperfect monitoring, there might be in general: 1 Undetectable deviations (Lehrer 1990, 1992) 2 A deviation which is detected by some players, but not the others (Renault, Tomala 1998) 3 A deviation which is detected but the identity of the deviator is unknown (Tomala 1998, Renault et al 05 and 08) 4 Computing the punishment levels may be difficult (Gossner, Tomala 07)

35 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Signals do matter. Example: The prisoner s dilemma in the dark C 2 D 2 C 1 ( 1, 1) ( 10, 0) E = {( 8, 8)} D 1 (0, 10) ( 8, 8) With imperfect monitoring, there might be in general: 1 Undetectable deviations (Lehrer 1990, 1992) 2 A deviation which is detected by some players, but not the others (Renault, Tomala 1998) 3 A deviation which is detected but the identity of the deviator is unknown (Tomala 1998, Renault et al 05 and 08) 4 Computing the punishment levels may be difficult (Gossner, Tomala 07)

36 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Signals do matter. Example: The prisoner s dilemma in the dark C 2 D 2 C 1 ( 1, 1) ( 10, 0) E = {( 8, 8)} D 1 (0, 10) ( 8, 8) With imperfect monitoring, there might be in general: 1 Undetectable deviations (Lehrer 1990, 1992) 2 A deviation which is detected by some players, but not the others (Renault, Tomala 1998) 3 A deviation which is detected but the identity of the deviator is unknown (Tomala 1998, Renault et al 05 and 08) 4 Computing the punishment levels may be difficult (Gossner, Tomala 07)

37 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Signals do matter. Example: The prisoner s dilemma in the dark C 2 D 2 C 1 ( 1, 1) ( 10, 0) E = {( 8, 8)} D 1 (0, 10) ( 8, 8) With imperfect monitoring, there might be in general: 1 Undetectable deviations (Lehrer 1990, 1992) 2 A deviation which is detected by some players, but not the others (Renault, Tomala 1998) 3 A deviation which is detected but the identity of the deviator is unknown (Tomala 1998, Renault et al 05 and 08) 4 Computing the punishment levels may be difficult (Gossner, Tomala 07)

38 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Signals do matter. Example: The prisoner s dilemma in the dark C 2 D 2 C 1 ( 1, 1) ( 10, 0) E = {( 8, 8)} D 1 (0, 10) ( 8, 8) With imperfect monitoring, there might be in general: 1 Undetectable deviations (Lehrer 1990, 1992) 2 A deviation which is detected by some players, but not the others (Renault, Tomala 1998) 3 A deviation which is detected but the identity of the deviator is unknown (Tomala 1998, Renault et al 05 and 08) 4 Computing the punishment levels may be difficult (Gossner, Tomala 07)

39 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Signals do matter. Example: The prisoner s dilemma in the dark C 2 D 2 C 1 ( 1, 1) ( 10, 0) E = {( 8, 8)} D 1 (0, 10) ( 8, 8) With imperfect monitoring, there might be in general: 1 Undetectable deviations (Lehrer 1990, 1992) 2 A deviation which is detected by some players, but not the others (Renault, Tomala 1998) 3 A deviation which is detected but the identity of the deviator is unknown (Tomala 1998, Renault et al 05 and 08) 4 Computing the punishment levels may be difficult (Gossner, Tomala 07)

40 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Undetectable deviations Example: Two-player game with public signals Payoffs y 1 y 2 y 3 x 1 (2, 2) (5, 1) (7, 0) x 2 (1, 5) (4, 4) (6, 3) x 3 (0, 7) (3, 6) (5, 5) Signals y 1 y 2 y 3 x 1 a c c x 2 b d d x 3 b d d Player 1 chooses the row and player 2, the column. E.g.: if player 1 chooses the row x 1 and player the column y 2 then player 1 receives a payoff equal to 5 and player 2, a payoff equal to 1. The signal c is publicly announced. Remark: The action x 3 for player 1 is strictly dominated. Player 2 cannot distinguish between the actions x 2 and x 3. Player 2 cannot force player 1 to play the action x 3. Therefore (5, 5) is not an equilibrium payoff but (5, 5) is both feasible and individually rational.

41 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Undetectable deviations Example: Two-player game with public signals Payoffs y 1 y 2 y 3 x 1 (2, 2) (5, 1) (7, 0) x 2 (1, 5) (4, 4) (6, 3) x 3 (0, 7) (3, 6) (5, 5) Signals y 1 y 2 y 3 x 1 a c c x 2 b d d x 3 b d d Player 1 chooses the row and player 2, the column. E.g.: if player 1 chooses the row x 1 and player the column y 2 then player 1 receives a payoff equal to 5 and player 2, a payoff equal to 1. The signal c is publicly announced. Remark: The action x 3 for player 1 is strictly dominated. Player 2 cannot distinguish between the actions x 2 and x 3. Player 2 cannot force player 1 to play the action x 3. Therefore (5, 5) is not an equilibrium payoff but (5, 5) is both feasible and individually rational.

42 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring A deviation which is detected by some players, but not the others E.g.: Proximity games (Renault, Tomala 1998) Players are the vertices of a network and can observe the actions chosen by their closest neighbors but receive no signal about the actions played by the other players. Each player can only observe deviations from part of a strict subset of players. A deviation which is detected, but the identity of the deviator is unknown E.g.: Minority games (Renault et al 05 and 08) An odd number of players have to vote for one of two alternatives A or B. Each player who votes for the least chosen alternative receives a reward of one euro. Between the stages, only the current majority alternative is publicly announced. Games with public signals and observable payoffs.

43 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring A deviation which is detected by some players, but not the others E.g.: Proximity games (Renault, Tomala 1998) Players are the vertices of a network and can observe the actions chosen by their closest neighbors but receive no signal about the actions played by the other players. Each player can only observe deviations from part of a strict subset of players. A deviation which is detected, but the identity of the deviator is unknown E.g.: Minority games (Renault et al 05 and 08) An odd number of players have to vote for one of two alternatives A or B. Each player who votes for the least chosen alternative receives a reward of one euro. Between the stages, only the current majority alternative is publicly announced. Games with public signals and observable payoffs.

44 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Computing the punishment levels may be difficult Example: Three-player game L R Payoffs for P3: T 1 0 B 0 0 W L R T 0 0 B 0 1 E Perfect monitoring: -1/4 (coincides with min max level) P1 and P2 see each other but P3 can only see P2 (Gossner, Tomala 07) P1 and P2 can force P3 s payoff below -3/8: At odd stages, P1 and P2 both play (1/2, 1/2). At even stages, play (T, L) if P1 s previous action was T, otherwise play (B, R). Therefore, P3 s payoff is lower than 1/2 at even stages and 1/4 at odd stages. Punishment level for P3: 0.401

45 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Computing the punishment levels may be difficult Example: Three-player game L R Payoffs for P3: T 1 0 B 0 0 W L R T 0 0 B 0 1 E Perfect monitoring: -1/4 (coincides with min max level) P1 and P2 see each other but P3 can only see P2 (Gossner, Tomala 07) P1 and P2 can force P3 s payoff below -3/8: At odd stages, P1 and P2 both play (1/2, 1/2). At even stages, play (T, L) if P1 s previous action was T, otherwise play (B, R). Therefore, P3 s payoff is lower than 1/2 at even stages and 1/4 at odd stages. Punishment level for P3: 0.401

46 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Computing the punishment levels may be difficult Example: Three-player game L R Payoffs for P3: T 1 0 B 0 0 W L R T 0 0 B 0 1 E Perfect monitoring: -1/4 (coincides with min max level) P1 and P2 see each other but P3 can only see P2 (Gossner, Tomala 07) P1 and P2 can force P3 s payoff below -3/8: At odd stages, P1 and P2 both play (1/2, 1/2). At even stages, play (T, L) if P1 s previous action was T, otherwise play (B, R). Therefore, P3 s payoff is lower than 1/2 at even stages and 1/4 at odd stages. Punishment level for P3: 0.401

47 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Semi-standard signals: If the players choose the action profile (a j ) j then the signal (a j ) j is publicly announced (a j stands for the cell element that contains a j ). Remark: ā j depends on a j, only. If a deviation is detected by a player then it is detected by all the players. If a deviation is detected, the identity of the deviator is known. If player i plays a i such that a i a i, everyone knows that player i is the deviator. Punishment cannot be more severe than the min max level. Lehrer (1990) characterizes the set of equilibrium payoffs of the infinitely repeated game: It is the set of feasible payoff vectors that are robust to undetectable deviations and individually rational.

48 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring Semi-standard signals: If the players choose the action profile (a j ) j then the signal (a j ) j is publicly announced (a j stands for the cell element that contains a j ). Remark: ā j depends on a j, only. If a deviation is detected by a player then it is detected by all the players. If a deviation is detected, the identity of the deviator is known. If player i plays a i such that a i a i, everyone knows that player i is the deviator. Punishment cannot be more severe than the min max level. Lehrer (1990) characterizes the set of equilibrium payoffs of the infinitely repeated game: It is the set of feasible payoff vectors that are robust to undetectable deviations and individually rational.

49 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring This talk: We characterize the set of equilibrium payoffs of the finitely repeated game as T + (in the Hausdorff distance). Difficulties: Undetectable deviations The finite duration of the game: the backward induction effects imply tight constraints on the equilibrium strategy. Refinement of equilibrium (pure strategies): Subgame perfect equilibrium We define the semi-standard min max level for each player, which represents the punishment level which is compatible with the constraint of robustness to undetectable deviations outside the equilibrium path.

50 The model of repeated game with imperfect monitoring Aspects of Imperfect Monitoring This talk: We characterize the set of equilibrium payoffs of the finitely repeated game as T + (in the Hausdorff distance). Difficulties: Undetectable deviations The finite duration of the game: the backward induction effects imply tight constraints on the equilibrium strategy. Refinement of equilibrium (pure strategies): Subgame perfect equilibrium We define the semi-standard min max level for each player, which represents the punishment level which is compatible with the constraint of robustness to undetectable deviations outside the equilibrium path.

51 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Economic applications of imperfect public monitoring Industrial competition à la Cournot Two firms produce the same good. The price of a unit of good depends on the quantities produced by the firms. The signal here is the price, which is publicly observed but the firms cannot observe the quantity produced by its competitor. Principal/Agent relationships (e.g.: Insurer (principal) / Insured (agent)) The agent s action determines an outcome which can be observed by the principal. But the principal cannot observe the action taken by the agent. Strategic market games Players trade assets and goods over time and propose at each period, bids and offers, which determines the vector price. The signal here, is the vector price which is publicly observed by the players. But the players cannot observe the other players holdings.

52 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems The model Players: N = {1,..., n} Actions: X 1,..., X n Stage payoff function: g i : n i=1 X i R Partitions: X 1,..., X n ( x, y X i, x i y x = y) Monitoring structure: X = n i=1 X i is the set of signals and l : n i=1 X i X the signalling function such that for all (x i ) i n i=1 X i, l((x i ) i ) = ( x i ) i where x i denotes the element of X i that contains x i. G T = T-Fold Repeated Game ( ) 1 T Payoffs: T t=1 g i(x t ) ((x t ) 1 t T = action profile) i N

53 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems We consider pure strategies. Strategies Private Histories: H i,t = (X i X) t 1 (H i,1 singleton) Private Strategies: σ i = (σ i,t ) 1 t T où σ i,t : T t=1 H i,t X i Public Histories: H t = X t 1 (H 1 singleton) Public Strategies: σ i = (σ i,t ) 1 t T où σ i,t : T t=1 H t X i Tomala (1998) For each private strategy σ i for player i, there exists a public strategy σ i equivalent to σ i i.e. such that for each strategy profile σ i of the other players and for each stage t, x t (σ i, σ i ) = x t ( σ i, σ i ). Note: We identify each strategy with the public equivalent strategy.

54 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems We consider pure strategies. Strategies Private Histories: H i,t = (X i X) t 1 (H i,1 singleton) Private Strategies: σ i = (σ i,t ) 1 t T où σ i,t : T t=1 H i,t X i Public Histories: H t = X t 1 (H 1 singleton) Public Strategies: σ i = (σ i,t ) 1 t T où σ i,t : T t=1 H t X i Tomala (1998) For each private strategy σ i for player i, there exists a public strategy σ i equivalent to σ i i.e. such that for each strategy profile σ i of the other players and for each stage t, x t (σ i, σ i ) = x t ( σ i, σ i ). Note: We identify each strategy with the public equivalent strategy.

55 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Equilibrium notions (σ i ) i is a Nash Equilibrium (NE) for G T if for each player i, and each strategy σ i for player i, γ i,t(σ i, σ i) γ i,t (σ). Denote E T the set of NE payoffs for G T. A Subgame Perfect Equilibrium (SPE) is a NE which induces a NE after each public history. Denote E T the set of SPE payoffs for G T.

56 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Feasible payoffs robust to undetectable deviations Lehrer (1989) defines for each player i, the set D i of action profiles (x i, x i ) such that given x i, player i cannot profitably deviate from x i without being detected. Denote D = i D i. Definition (Lehrer (1990)) (x i, x i ) D i x i x i, g i (x i, x i ) g i (x i, x i ) The set of feasible payoffs which are robust to undetectable deviations is the convex hull of g(d), where g(d) = {g(x) : x D}. Remarks: E 1 g(d) We assume that G 1 admits a Nash equilibrium in pure strategies. In particular, g(d).

57 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Compute D for particular semi-standard monitoring structures Perfect monitoring: x i i x i x i = x i For each player i, D i = X and D = X. Trivial monitoring: (x i, x i) X i X i, x i = x i D i is the set of joint action (x i, x i ) such that x i is a best response to x i within X i. Therefore D = i D i is the set of Nash equilibria of G 1. y 1 y 2 y 3 y 4 x 1 (3, 4) (1, 3) (3, 2) (1, 1) Example: x 2 ( 1, 1) (0, 0) (4, 2) (3, 0) x 3 (2, 1) ( 1, 2) (5, 0) (1, 1) x 4 ( 1, 1) (4, 0) ( 1, 1) (0, 0) where X 1 = {{x 1, x 2 }, {x 3, x 4 }} and X 2 = {{y 1, y 2 }, {y 3, y 4 }} D = {(x 1, y 1 ), (x 2, y 3 ), (x 3, y 3 )}

58 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Compute D for particular semi-standard monitoring structures Perfect monitoring: x i i x i x i = x i For each player i, D i = X and D = X. Trivial monitoring: (x i, x i) X i X i, x i = x i D i is the set of joint action (x i, x i ) such that x i is a best response to x i within X i. Therefore D = i D i is the set of Nash equilibria of G 1. y 1 y 2 y 3 y 4 x 1 (3, 4) (1, 3) (3, 2) (1, 1) Example: x 2 ( 1, 1) (0, 0) (4, 2) (3, 0) x 3 (2, 1) ( 1, 2) (5, 0) (1, 1) x 4 ( 1, 1) (4, 0) ( 1, 1) (0, 0) where X 1 = {{x 1, x 2 }, {x 3, x 4 }} and X 2 = {{y 1, y 2 }, {y 3, y 4 }} D = {(x 1, y 1 ), (x 2, y 3 ), (x 3, y 3 )}

59 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Compute D for particular semi-standard monitoring structures Perfect monitoring: x i i x i x i = x i For each player i, D i = X and D = X. Trivial monitoring: (x i, x i) X i X i, x i = x i D i is the set of joint action (x i, x i ) such that x i is a best response to x i within X i. Therefore D = i D i is the set of Nash equilibria of G 1. y 1 y 2 y 3 y 4 x 1 (3, 4) (1, 3) (3, 2) (1, 1) Example: x 2 ( 1, 1) (0, 0) (4, 2) (3, 0) x 3 (2, 1) ( 1, 2) (5, 0) (1, 1) x 4 ( 1, 1) (4, 0) ( 1, 1) (0, 0) where X 1 = {{x 1, x 2 }, {x 3, x 4 }} and X 2 = {{y 1, y 2 }, {y 3, y 4 }} D = {(x 1, y 1 ), (x 2, y 3 ), (x 3, y 3 )}

60 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Punishment levels Standard min max level for player i: v i = Semi-Standard min max level for player i: v i = min x i D i min max g i (x i, x i ) x i X i x i X i max g i (x i, x i ) x i X i where D i denotes the projection of D on X i. Define IR = {u R n u i v i, i N} the set of individually rational payoffs with respect to the standard min max level. Define IR = {u R n u i v i, i N} the set of individually rational payoffs with respect to the semi-standard min max level.

61 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Punishment levels Standard min max level for player i: v i = Semi-Standard min max level for player i: v i = min x i D i min max g i (x i, x i ) x i X i x i X i max g i (x i, x i ) x i X i where D i denotes the projection of D on X i. Define IR = {u R n u i v i, i N} the set of individually rational payoffs with respect to the standard min max level. Define IR = {u R n u i v i, i N} the set of individually rational payoffs with respect to the semi-standard min max level.

62 Example: 2-Player Game Actions and payoffs Partitions: X 1 = The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems a 2 b 2 c 2 a 1 3, 4 6, 0 2, 1 b 1 1, 2 5, 1 0, 1 c 1 2, 4 } 3, 0 3, 3 and X 2 = { {a 1, b 1 }, {c 1 } { {a 2, b 2 }, {c 2 } Action profiles Robust to unilateral Undetectable Profitable Deviations } } D = {a 1, c 1 {a 2, c 2 Min Max of Player 2 v 2 = min x 1 b 1 max x 2 X 2 g 2 (x 1, x 2 ) = 4 > v 2 = 2 }

63 Example: 2-Player Game Actions and payoffs Partitions: X 1 = The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems a 2 b 2 c 2 a 1 3, 4 6, 0 2, 1 b 1 1, 2 5, 1 0, 1 c 1 2, 4 } 3, 0 3, 3 and X 2 = { {a 1, b 1 }, {c 1 } { {a 2, b 2 }, {c 2 } Action profiles Robust to unilateral Undetectable Profitable Deviations } } D = {a 1, c 1 {a 2, c 2 Min Max of Player 2 v 2 = min x 1 b 1 max x 2 X 2 g 2 (x 1, x 2 ) = 4 > v 2 = 2 }

64 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Necessary Condition on Equilibrium Payoffs: Feasibility Lemma 1 1 If σ is a NE of G T then at each stage t, x t (σ) D. 2 If σ is a SPE of G T then after each public history h, σ(h) D. Corollary E T E T conv(g(d))

65 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Necessary Condition on Equilibrium Payoffs: Individual Rationality Lemma 2 If σ is a NE of G T then γ T (σ) IR. If σ is a SPE of G T then γ T (σ) IR. From Lemmas 1 and 2, Corollary E T conv(g(d)) IR E T conv(g(d)) IR

66 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Limit set of NE payoffs Folk theorem 1 If for every player i, there exists e i E 1 such that e i (i) > v i then lim E T = E T + in the Hausdorff distance, where E = cl (conv(g(d)) IR). Proof : Adapt Benoit and Krishna (1987) proof to semi-standard signals.

67 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Limit set of SPE payoffs Conditions C1 and C2 (Benoit and Krishna (1985)): 1 For every player i, there exist e i, f i E 1 such that e i (i) > f i (i) 2 dim conv(g(d)) = n Folk Theorem 2 Under conditions C1 and C2, lim E T = E T + in the Hausdorff distance, where E = cl (conv(g(d)) IR ).

68 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Standard vs. Semi-Standard Signals Under standard monitoring: D = X which implies that D i = X i and v i = v i and we retrieve the content of the standard folk theorem (Benoit, Krishna (1987)) for finitely repeated games. Remark: Under conditions C1 et C2, when signals are standard, lim T E T = lim T E T. when signals are semi-standard, lim T E T lim T E T.

69 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Standard vs. Semi-Standard Signals Under standard monitoring: D = X which implies that D i = X i and v i = v i and we retrieve the content of the standard folk theorem (Benoit, Krishna (1987)) for finitely repeated games. Remark: Under conditions C1 et C2, when signals are standard, lim T E T = lim T E T. when signals are semi-standard, lim T E T lim T E T.

70 Elements of the proof The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems We adapt Gossner (1995) method to semi-standard signals. We first study finitely repeated games G T (W) with terminal payoffs where payoffs are given by, for all strategy profile σ, θ i,t (σ) = 1 T T t=1 g i (x t (σ)) + 1 T Term i((x t (σ) 1 t T ) } {{ } =Reward {0,W} Denote E T (W) = Set of SPE payoffs of G T(W) Lemma ε > 0, u int(e ), W 0 > 0, T 0 > 0, W W 0, T T 0, w E T (W), w u ε

71 SPE for G T (W) The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Target Payoff: u int(e ). W.l.o.g. suppose u = n+1 α i i=1 α g(q i) where (q 1,...q n+1 ) D n+1, (α i ) n+1 i=1 are positive integers and α = n+1 i=1 α i. Cyclic Play: Repeat (q 1,..., q } {{ } 1,..., q n+1,..., q n+1 ) cyclically } {{ } α 1 times α n+1 times Define σ = Main Path + Punishing Phase + Reward Phase Main path: Play q m at date t if t = m mod(α) Punishment phase: If player i deviates from Main path then play x (i) D such that v i = g i (x (i)) for P stages then return to Main path Reward phase: If player j punished player i then he receives W as terminal payoff otherwise 0.

72 SPE for G T (W) The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Target Payoff: u int(e ). W.l.o.g. suppose u = n+1 α i i=1 α g(q i) where (q 1,...q n+1 ) D n+1, (α i ) n+1 i=1 are positive integers and α = n+1 i=1 α i. Cyclic Play: Repeat (q 1,..., q } {{ } 1,..., q n+1,..., q n+1 ) cyclically } {{ } α 1 times α n+1 times Define σ = Main Path + Punishing Phase + Reward Phase Main path: Play q m at date t if t = m mod(α) Punishment phase: If player i deviates from Main path then play x (i) D such that v i = g i (x (i)) for P stages then return to Main path Reward phase: If player j punished player i then he receives W as terminal payoff otherwise 0.

73 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Link between E T and E T (W): Lemma Under conditions C1 and C2, there exits T 1 > 0 s.t. dim (conv(e T 1 )) = n. Since for all integer k, k.t 1 E T 1 kt 1 E kt 1 and from the previous lemma E T 1 contains an open ball, there exists for all terminal payoff function term taking its values in {0, W} n, a duration T 2 = kt 1 such that for all public history h, term(h) can be sustained by a SPE of G T2. Use last stages of G T to simulate terminal payoff vectors.

74 The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Link between E T and E T (W): Lemma Under conditions C1 and C2, there exits T 1 > 0 s.t. dim (conv(e T 1 )) = n. Since for all integer k, k.t 1 E T 1 kt 1 E kt 1 and from the previous lemma E T 1 contains an open ball, there exists for all terminal payoff function term taking its values in {0, W} n, a duration T 2 = kt 1 such that for all public history h, term(h) can be sustained by a SPE of G T2. Use last stages of G T to simulate terminal payoff vectors.

75 Extensions Repeated Games with Imperfect Monitoring The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Discounted repeated games Folk Theorem 3 If dim conv(g(d)) = n then the set of SPE payoffs of the disccounted repeated game with discount factor λ (0, 1] converges to E (in the Hausdorff distance) as λ 0. Proof: Adapt Fudenberg Maskin (1986) to semi-standard signals Mixed actions and observable distribution of signals Our results adapt to this setup.

76 Extensions Repeated Games with Imperfect Monitoring The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Discounted repeated games Folk Theorem 3 If dim conv(g(d)) = n then the set of SPE payoffs of the disccounted repeated game with discount factor λ (0, 1] converges to E (in the Hausdorff distance) as λ 0. Proof: Adapt Fudenberg Maskin (1986) to semi-standard signals Mixed actions and observable distribution of signals Our results adapt to this setup.

77 Mixed strategies: The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Observable distribution of signals Finite sets of pure actions A i, partitions Ā i. Mixed actions X i = (A i ), partition X i such that for all x, x X i, where x i (ā i ) = a i ā i x i (a i ). x = x ā i, x i (ā i ) = x i(ā i ) Denote D m the set of mixed action profiles which are robust to undetectable deviations. Denote IR m the set of payoff vectors which are IR w.r.t. the v i (computed in mixed strategies). Denote E m the closure of conv(g(d m )) IR m. Hausdorff-convergence of E T to the set E m is obtained under conditions C1 and C2.

78 Mixed strategies: The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Observable distribution of signals Finite sets of pure actions A i, partitions Ā i. Mixed actions X i = (A i ), partition X i such that for all x, x X i, where x i (ā i ) = a i ā i x i (a i ). x = x ā i, x i (ā i ) = x i(ā i ) Denote D m the set of mixed action profiles which are robust to undetectable deviations. Denote IR m the set of payoff vectors which are IR w.r.t. the v i (computed in mixed strategies). Denote E m the closure of conv(g(d m )) IR m. Hausdorff-convergence of E T to the set E m is obtained under conditions C1 and C2.

79 Open problems The model Equilibrium notions Feasible payoffs robust to undetectable deviations Punishment levels Necessary Conditions on Equilibrium Payoffs Characterization Elements of the proof Extensions Open problems Mixed strategies and non observable distribution of signals How to detect deviations from the main path or deviations from the punishing phase? Gossner (1995) (standard repeated game) defines statistical test functions: After a punishing phase h (sequence of action profiles), test j (h) = 1 if player j did punish the deviator, and test j (h) = 0 if not. Remark: The (empirical) frequency of actions can be controlled thanks to the assumption of perfect observability of past played actions. Under imperfect monitoring, only empirical distributions of signals can be controlled but not the frequencies of actions within the classes. This can lead to payoffs far away the target payoff...