Strategy Machines. Representation and Complexity of Strategies in Infinite Games

Transcription

1 Strategy Machnes Representaton and Complexty of Strateges n Infnte Games Von der Fakultät für Mathematk, Informatk und Naturwssenschaften der RWTH Aachen Unversty zur Erlangung des akademschen Grades enes Doktors der Naturwssenschaften genehmgte Dssertaton vorgelegt von Dplom-Informatker Marcus Geldere aus Stolberg Rhld. Berchter: Unverstätsprofessor Dr. Dr. h. c. Dr. h. c. Wolfgang Thomas Chargé de Recherche Dr. habl. Ncolas Markey Tag der mündlchen Prüfung: Dese Dssertaton st auf den Internetseten der Hochschulbblothek onlne verfügbar.

2

3 Abstract Ths thess studes the representaton of wnnng strateges n nfnte games as Turng machnes. We show that Turng-machne-based strategy representaton (by strategy machnes ) f compared to the standard settng of Mealy automata allows for a fner classfcaton of strateges, provdes a better understandng of the structure of strateges, and can successfully be used n areas that are dffcult to reason about wth automaton strateges, such as composton of games and strateges. We gve a formal defnton of strategy machnes and adapt the classcal Turngmachne complexty measures to our settng called latency, space requrement, and sze. The latency s the number of computaton steps that s requred to produce a next move, the space requrement states how much memory s requred for ths computaton and the sze s the number of control states of the machne. These complexty measures are not present n representatons of strateges that are based on automata and thus provde a fner vew on strateges than currently used models. We show that strategy machnes of polynomal sze wth a polynomal space requrement can be syntheszed for Muller games. Moreover, we show that for Streett games a polynomal latency can be guaranteed n addton to these bounds. The synthess procedure s an adapton of Zelonka s algorthm, now makng use of the computatonal power of a Turng machne to spread costly computatons across the nfnte play. Turnng to composton of games and strateges, we study reachablty and Büch games on arenas that are products of several component arenas. We focus on composng a wnnng strategy on the product arena from wnnng strateges n sutably chosen games on the components. We defne strategy composton based on subroutne calls and show that as long as the component arenas cannot communcate, a composton of polynomal sze and latency can be computed n polynomal tme. If components may communcate, a smlar composton result exsts ff Pspace = Exptme. Fnally, we study mean-payoff party and mean-penalty party games as examples of quanttatve games. We prove that optmal and ε-optmal strateges n mean-payoff party games have a very homogeneous structure and can can be composed of a lnear number of postonal strateges. Usng ths, we derve a strategy machne mplementaton of polynomal sze of optmal and ε-optmal strateges. For ε-optmal strateges, the latency s moreover logarthmc n ε. Ths bound s optmal. We lft ths result to mean-penalty party games. For ths we ntroduce permssve strategy machnes and translate strateges va a reducton that does not allow for the transfer of automaton strateges. Ths agan demonstrates the usefulness of Turng machne representatons of strateges.

4

5 Contents Introducton 1 1. Games, Transton Systems and Languages Basc Concepts Infnte Games and Strateges Wnnng Condtons Strategy Machnes Strategy Machnes and ther Semantcs Complexty Effcency & Encodng Muller and Streett Games Zelonka s Algorthm Muller Games The Strategy Machne Streett Games Strategy Composton Compostonal Arenas Translatons Compostonal Games and Polynomal Compostons Compostonal Reachablty and Büch Games Games on Parallel Products Games on Synchronzed Products and Channel Systems

6 Contents 5. Quanttatve Games Basc Concepts n Quanttatve Games Mean-Payoff Party Games Mean-Penalty Party Games Concluson 169 A.Turng Machne Constructons 173 A.1. Representng the Arena A.2. Attractor Computaton A.3. Representng the Wnnng Condton A.4. Computng the Wnnng Regon n a Muller Game Bblography 181 Symbols & Notaton 189 Index 193

7 Introducton Controller synthess s the act of dervng a controller from a gven specfcaton n an automatc way. The problem tself dates back to Alonzo Church, who posed the followng problem n 1957 [Chu57]: Gven a requrement whch a crcut s to satsfy, we may suppose the requrement expressed n some sutable logstc system whch s an extenson of restrcted recursve arthmetc. The synthess problem s then to fnd recurson equvalences representng a crcut that satsfes the gven requrement (or alternatvely, to determne that there s no such crcut). The requrement to whch Church was referrng s a requrement on nfnte strngs: the synthess problem s about behavor of non-termnatng systems. The crcut to be syntheszed should react to an envronment that produces an endless sequence of uncontrolled sgnals. The crcut receves these sgnals (successvely) as nput and produces a correspondng sequence of output sgnals. It s also often referred to as a controller. The problem above s now known as Church s synthess problem and has been subject to ntensve research. Several partal solutons to ths problem were gven early n the 1960 s [Chu62]. The most fundamental soluton, however, was gven by Büch and Landweber [BL69]. They gve a constructon of a fnte state automaton wth output that satsfes a regular specfcaton for nfnte behavor provded n the form of a Muller automaton. Ths result covers specfcatons gven n monadc second-order logc over the natural numbers. For surveys of the fundamental results regardng Church s synthess problem, the reader s referred to [Tho08a, Tho08b, Tho09]. An mportant observaton already made by McNaughton [McN65] s that solutons to Church s problem may be understood as wnnng strateges n a sutable class of games, namely nfnte games. Here, two players play an nfnte game on a fnte graph called an arena by movng a pebble along ts edges. The objectve of the frst player 1

8 Introducton s to fulfll the specfcaton gven as part of the synthess problem, regardless of the other player s actons. Today, the game theoretc formulaton of Church s problem s well establshed [Tho08b]. It mples that n order to solve Church s synthess problem, one has to fnd wnnng strateges n a certan class of games. In ths framework, the specfcaton s seen as a wnnng condton, the logcal formalsm specfes the class of condtons from whch ths specfc wnnng condton s drawn, and the crcut to be syntheszed becomes a wnnng strategy. We adopt ths vew throughout ths thess and use the game theoretc termnology from now on. Church s synthess problem names three key components of synthess: a specfcaton ( requrement ), the logcal formalsm n whch t s expressed ( sutable logstc system ) and fnally, the structure of the controller to be syntheszed a crcut n Church s orgnal statement. These components characterze the problem and cover all of ts aspects. These components can be thought of as parameters of the problem. We can restrct each of them to a sutable class and obtan a dfferent flavor of synthess. In fact, Church hmself already does so n the orgnal problem statement where he requres the wnnng condton to be drawn from some sutable logstc system whch s an extenson of restrcted recursve arthmetc. Thus, the frst parameter that has been studed extensvely s the wnnng condton class n whch to express the requrement. Church hmself gave an overvew of early results n ths area [Chu62]. We also mentoned that [BL69] provded a soluton for a very large class of wnnng condtons. In addton, one can study the effect of restrctng the wnnng condton class on the complexty of decdng the wnner and on the sze of the resultng strategy. Here, we menton [GH82, PR89, EJ91, MS95, Tho95, Ze98, DJW97, HD05, DHH11]. Even for a fxed class of wnnng condtons, one can consder the mpact of the representaton of ths wnnng condton. For example, consder Muller games, one promnent example of a class of wnnng condtons where the set of vertces n the arena that are vsted nfntely often determnes who wns. One can gve the wnnng condton explctly and lst all possble sets that are allowed. One can also gve the sets ndrectly, usng a propostonal formula. Both representatons gve rse to the same class of wnnng condtons, but the complexty of decdng the synthess problem s dfferent [Hor08, HD05]. Smlar questons arse n the context of other classes of wnnng condtons. Fnally, one can change the representaton of the wnnng strategy. Church hmself referred to a crcut as a means of representng the controller. Interestngly enough, no results that actually synthesze a Boolean crcut seem to exst. Instead, from 2

9 the earlest pont onward, the object that was syntheszed was a fnte automaton [Chu62, BL69]. Wth certan classes of wnnng condtons that were consdered, the requrement that the automaton should have a fnte state-space was dropped. For example, Serre consders hgher order stack-machnes to mplement wnnng strateges n games of arbtrarly hgh Borel complexty [Ser06]. Hs work extends earler results, n whch (frst-order) stack-machnes were syntheszed [Wal01, BSW03]. However, all of these results remaned frmly rooted n the world of state-space representatons of strateges. A state-space representaton s an abstracton of a computatonal process. Durng the run of a gven program, the nternal memory of the executng machne s manpulated. Ths manpulaton of memory gves rse to certan states of computaton, whch are, formally, pars of a memory confguraton and a poston n the program. At certan ponts n the program, a new nput wll be read (recall that the program reads an nfnte sequence of sgnals successvely). Once an nput has been read, the sequence of states up to the pont where the next nput s read s unquely determned, as we are dealng wth determnstc programs. One retans only those states where the program reacts to the nput, and collapses ntermedate states nto one transton. The result s a state-space representaton that abstracts away from parts of the computaton and retans only a certan relevant subset. The crucal pont of state-space abstractons s that they focus on a certan subset of states that are more nterestng than others. The exact nterpretaton of nterestng s subject to the problem at hand. All other aspects of the computaton are dscarded. The state-space model s state centered n the sense that t does not specfy how a transton from one state to the next occurs n realty that nvolves a long sequence of borng states. It merely specfes that such a transton occurs on a gven nput. One may thnk of a brd s-eye vew of a computaton, consderng all states of a program and the results of updatng them, but not the actual process of updatng tself. In contrast to the state-centered vew of state-space abstractons, there are other concepts of computaton that take the opposte approach. Turng machnes, for nstance, specfy a program n terms of the procedure that s requred to perform the stateupdate. Gven a Turng machne, t s far from straghtforward to make a statement about all ts memory confguratons (and, n fact, most problems about the state-space are undecdable). However, what s straghtforward s the ablty to take a gven memory confguraton, agan a par consstng of memory (the tape) and poston n the program (the control-state), and produce the mmedate successor confguraton. Turng machnes capture precsely the nformaton requred to perform ths update; 3

10 Introducton nothng more. The Turng machne model of computaton s thus transton centered. It does not specfy whch states a computaton may have, but how to get from one state to the next. As we mentoned above, strategy synthess n the context of Church s synthess problem has been frmly rooted n the state-centered vew on computatons. The strategy representatons obtaned by vrtually all exstng algorthms yeld state-space based representatons of strateges (for example, see [BL69, PR89, McN93, Tho95, Ze98, Tho08b]). Ths s a surprsng fact, gven that Church asked for a crcut, whch s clearly rooted n the transton-centered vew of computaton. The surprse abates somewhat f one recalls the benefts of state-space centered models of computaton: they are objects of a smple structure and therefore well-suted for study n mathematcal contexts. Whle the advantage of state-space representatons s ther structural smplcty, ther dsadvantage s ther sze. The state-space of a program can be exponentally larger than the program tself a phenomenon coned as state-space exploson [EC80, CE81, CES86]. State-space representatons also hde certan aspects of a computaton, such as ts runtme and ts space-complexty. Moreover, they mngle two very dfferent notons of space-complexty: the statc complexty of storng the program and the dynamc complexty of executng the program. All of these measures are present n a Turng-machne-based vew on computaton. In ths thess, we propose to represent wnnng strateges n controller synthess as Turng machnes nstead of state-space centered models, such as fnte automata. We develop a sutable model of Turng-machne-based strategy representaton, called a strategy machne. There are four man contrbutons of ths thess: Frst, we ntroduce strategy representaton va strategy machnes, ncludng a detaled dscusson of all relevant complexty measures. Second, we apply ths model of strategy representaton to large classes of nfnte games, dervng novel synthess procedures to obtan effcent strateges (n terms of sze, runtme and space-consumpton). Thrd, we use strategy machnes to defne a noton of strategy composton that allows us to study the composablty of strateges n games that are themselves compostonal (product-lke) n nature. Fourth and fnally, we employ strategy machnes representatons n two select classes of quanttatve games and study the optmalty of strateges n these games as a functon of the computatonal resources of strategy machnes mplementng them. We now descrbe each of these four contrbutons n greater detal. We motvate the questons that are addressed and the solutons that are provded. In addton, we provde an overvew of related work and llustrate connectons as well as dfferences. 4

11 Contrbutons of ths Thess Strategy Machnes and Complexty As mentoned above, there s a fundamental dfference n the modelng of strateges usng Turng machnes and usng fnte automata. The frst model gves a transton centered vew on computatons, the second gves a state-centered vew. Adoptng a transton centered vew n the world of strategy representaton has several benefts. Frst of all, syntheszng a Turng machne nstead of a state-space abstracton s a more natural answer to the synthess problem. It s more natural not only because Church asked for crcut, rather than an automaton, but also because any possble applcaton of the procedure would expect somethng akn to a program. Second, the Turng-machne-based model provdes a wder range of crtera by whch to compare strateges. The result s a more honest and realstc estmaton of the complexty of usng the controller than s the number of ts states. Indeed, the state-space based vew holds no ntrnsc value to the task of mplementng a controller. We ntroduce strategy machnes as a Turng-machne-based model of strategy representaton to make use of the benefts. A strategy machne s a model of reactve Turng machne that reads opponent moves n the game, encoded as bnary strngs, and produces moves n response, also n the form of bnary strngs. It retans the content of some of ts tapes between successve moves to model a persstent memory that the machne may nspect and modfy durng ts computatons. We defne three key complexty measures: sze (number of control-states) whch measures statc, non-reusable memory, latency (worst-case tme to produce the next move), and space requrement (worst-case number tape cells requred to compute the next move) whch s a measure of dynamc, reusable storage. We then show that convertng any functon nto an exponentally more succnct strategy machne that also runs effcently, s not possble. It s therefore mportant to not detour over a state-space-representaton when dervng a strategy machne, but nstead to derve the strategy machne drectly. Strategy machnes were ntroduced and appled to Muller and Streett games n [Gel12a, Gel12b] a result that we descrbe n greater detal below. Let us frst menton some related work. The model of a reactve Turng machne as an mplementaton of functons of the form f : M M was tself ntroduced and studed n [GSW01], though n a context dfferent from games. In a settng closer to games, Fearnley, Peled, and Schewe consdered Turng machne representatons of models of CTL and LTL formulas, so called succnct models [FPS12]. Specfcally n the settngs of games and synthess, we menton that Dzembowsk, Jurdznsk, and Walukewcz study 5

12 Introducton strategy representaton va so called p-automata, a noton of communcatng automaton [DJW97]. Below, we elaborate further on ther results. At ths tme, t s suffcent to note that although p-automata are not a purely state-centered model of representatons, they lack characterstc features of transton-centered models, such as an establshed concept of runtme and a dstncton between statc and dynamc space-complexty. Fnally, Madhusudhan studed the synthess of structured programs, a class of whle programs over Boolean varables [Mad11]. In ths work, structured programs are derved from a specfcaton that s gven n the form of a Büch automaton whch accepts the nondesred behavor. The synthess procedure s parametrzed by a number of Boolean varables the program may use. The runtme of structured programs obtaned n ths way s not analyzed. Lkewse, there s no analyss of the number of Boolean varables requred. Ths approach was further studed by Brütsch [Brü13]. Strategy Machnes for Muller and Streett Games Muller and Streett games are two large and promnent classes of nfnte games. The frst class, Muller games, s defned by condtons that specfy a collecton of sets of vertces. A play s won by the controller f the set of vertces t vsts nfntely often s contaned n ths collecton. Streett games are a subclass of Muller games n that any Streett condton can be transformed nto an equvalent Muller condton on the same arena (though wth a possbly exponental ncrease n sze). Here the condton specfes a collecton of pars of sets of vertces, so-called Streett pars. If the play vsts the frst set n a par nfntely often, t must also vst the second set nfntely often. In contrast to Muller games, Streett games are not symmetrc: the opposng player has a dfferent knd of wnnng condton (called a Rabn condton) and needs no memory n general. The Muller condton was ntroduced n [Mul63] (see also [McN66]). The Streett condton was ntroduced n [Str82] and used n [MS95] n a game theoretc context. There are well-known synthess procedures for Muller and for Streett games. Muller games can be solved usng latest appearance records (LARs) [GH82, Tho95] or usng a tree-lke memory structure [Ze98] (whch s already mplct n [BL69]). Both constructons derve fnte-automaton solutons and have been shown to be optmal n terms of number of states [DJW97]. Streett games can be solved usng ndex appearance records [MS95]. Agan ths constructon s optmal [Hor05, Hor07] (see also [DJW97]). Usng strategy machnes to represent strateges n Muller and Streett games, we gve constructons that derve strategy machnes drectly from the problem nstance, as opposed to frst constructng a state-space representaton that s then converted to a 6

13 strategy machne [Gel12a, Gel12b]. The problem nstance conssts of an arena and a wnnng condton, whch s a Boolean formula for Muller games, and a set of Streett pars for Streett games. Our approach yelds exponental benefts over state-space based approaches n terms of sze, whle retanng a polynomal space requrement and, n the case of Streett games, even a polynomal latency. The strategy machne constructon reles on Zelonka s algorthm [Ze98]. It also uses elements from the presentaton n [DJW97]. There, the authors already observed that succnct representatons of wnnng strateges can be obtaned usng Zelonka s constructon (though they dd not provde a full formal proof of ths clam). As mentoned before, the authors use p-automata, communcatng fnte automata, to represent strateges n a succnct way. Our constructon, relyng on strategy machnes, extends ths observaton n two ways. Frst of all, usng strategy machnes as strategy mplementors, we can dstngush and optmze a greater varety of complexty measures. Our constructon ensures that the dfference between statc and dynamc memory, not present n p-automata, s exploted to reuse tape cells wherever possble. In optmzng the latency the second contrbuton of our constructon comes nto play. To acheve a polynomal latency for Streett games, we use a new technque for strategy mplementaton whereby costly computatons are spread across the duraton of the nfnte play. In ths way, the strategy adapts to the play as t unfolds. Asde from syntheszng a succnct representaton of a wnnng strategy drectly, there have been other research efforts drected at fndng small controllers. The frst approach targets automaton representatons of strateges and seeks to mnmze ther state-space. One possble method for ths s to employ smulaton relatons n such a way that the property of mplementng a wnnng strategy s preserved. We menton here [HL07, GH11]. The second approach seeks to convert automaton-based representatons nto transton-centered representatons va heurstc approaches such as BDDs [BGJ + 07b, BGJ + 07a]. Observe that ths approach s dfferent from our own: Instead of syntheszng a transton-centered representaton drectly, frst an automaton s constructed and then converted. Results n [BGJ + 07b] show that the approach cannot fully mtgate the effects of state-space exploson. Strategy Composton Every approach to strategy synthess that we consdered prevously works under the assumpton that the arena s gven explctly, as an adjacency matrx or as an adjacency lst. However, one should recall that the arena s a state-space representaton of the system we want to derve a controller for. In partcular, the arena suffers from the same 7

14 Introducton state-space exploson problem that we seek to avod on the controller sde. Perhaps most problematc s the fact that we base our complexty estmaton on the sze of the nput. If the controller s polynomal n the arena, t may stll be exponentally larger than the underlyng system. As an example, the reader may thnk of an assembly plant wth k assembly lnes wth n statons each. The state-space s n k, exponental n the number of assembly lnes. If the sze of a controller s polynomal n n k, t s therefore exponental n the sze of the actual system, whch can be descrbed n space n k. A possble remedy for ths s to study synthess procedures that do not take the explct state-space representaton of a system as nput but nstead work on a succnct representaton. In the example above, the nput would be a descrpton of the k assembly lnes. Ths leads to the noton of a compostonal arena whch s a product structure gven by several consttuent arenas that are multpled to obtan the overall system. It seems natural to ask f a controller for the overall system can then be composed of controllers for the consttuent arenas. Put dfferently: Gven consttuent arenas and a wnnng condton over the compostonal arena, can we effectvely derve wnnng condtons on the consttuent factors such that controllers whch satsfy these consttuent wnnng condtons may be composed to form a controller for the compostonal arena that satsfes the global wnnng condton? Ths s the queston of strategy composton. An obvous beneft of such a composton would be that the overall strategy has a small representaton. Moreover, from a theoretcal standpont, the queston of strategy composton gves deep nsghts nto the structure of wnnng strateges n nfnte games. To answer the queston of strategy composton, there are two unknowns that have to be specfed. Frst, the noton of arena composton must be fxed. Second, a concept of strategy composton needs to be specfed. To address the frst aspect, the arena composton, we ntroduce several classes of products: parallel products (no communcaton between arenas), synchronzed products (communcaton through synchronzaton), and channel systems (communcaton through asynchronous channels). To specfy the second unknown, we use strategy machnes to defne strategy composton. Informally, strategy composton s defned usng the concept of subroutne calls. In order for k strateges to be composed, there must be a composng program that uses these k strateges as subroutnes at certan ponts n ts computaton. Some techncal restrctons are requred to avod trval pathologcal cases, but we do not elaborate on ths now. Buldng on these defntons, we show that reachablty and Büch games on parallel products (no communcaton) admt compostons of strateges. Moreover, these compostons can be computed n polynomal tme. An mmedate corollary of ths 8

15 s that solvng reachablty and Büch games on parallel products s Ptme-complete. However, we show that f one ntroduces communcaton nto the system, by consderng ether synchronzed products or channel systems, then provng these classes admt compostons s equvalent to provng Pspace = Exptme. Ths result s largely a consequence of the fact that solvng reachablty and Büch games on these arenas s Exptme-hard. We also show that varous restrctons of communcaton do not prevent the complexty jump to Exptme-hardness and are therefore not suffcently restrctve to enable a strategy composton theorem. As a way of avodng the state-space exploson, composton s a fundamentally mportant subject n model checkng and synthess. Lustg and Vard consdered synthess from so called component-lbrares [LV09, LNV11, LV11, LV13]. In ther work, the authors consder several forms of composng the behavor of a controller from a lbrary of exstng controller-behavor. One of these forms of composton, called control-flow composton, s also essentally based on subroutne calls. However, ther formalzaton of composton s stll rooted n the state-space vew of controllers. Moreover, nput s of a dfferent format: n ncludes a lbrary that s to be used to construct controllers. Amnof, Kupferman, and Murano consder herarchcal systems [AKM10, AMM11, AKM12]. A herarchcal system s composed of components arranged n a herarchcal fashon. On may thnk of functons callng other functons n an mperatve programmng language. The authors consder model checkng of CTL formulas on such systems, as well as synthess of herarchcal systems. The latter part s also based on a lbrary approach; that s, a set of components s gven as part of the nput and a system s ultmately derved from these components. The controllers syntheszed are agan automata and thus represented n a state-space model. Quanttatve Games Church s synthess problem s stated n terms of a requrement. Tradtonally, ths requrement has been seen as beng bnary; t s ether fulflled, or t s not. The noton of a wnnng condton reflects ths: ether one player wns, or the other one does. However, from a very early pont onwards, there has been research on quanttatve games, where the outcome of a play s no longer bnary (e.g. [EM79]). The chef motvaton for these knds of games and the correspondng varant of the synthess problem s that t s often not natural to express a requrement n bnary form. For nstance, consder a requrement that adds a noton of cost to an exstng bnary wnnng condton: wn the followng game, but do so ncurrng as lttle cost as possble. Snce plays are no longer clearly labeled as won or lost by ether player, the concept 9

16 Introducton of a wnnng strategy s no longer approprate. Instead, one s nterested n optmal strateges that acheve the hghest possble gan for a gven player. As a weaker noton, one also consders ε-optmal strateges, where the gan only needs to be wthn dstance ε of the optmal value. We study strategy machne mplementatons n two classes of quanttatve games: mean-payoff party games and mean-penalty party games. Both are extensons of classcal party games by a quanttatve noton of utlty. In mean-payoff party games, the objectve s to maxmze an average weght that s accumulated along the play accordng to a set of edge weghts on top of satsfyng a party condton. Mean-penalty party games on the other hand are a tool desgned to study permssve strateges n the context of party games. A strategy s permssve f t offers a set of possble next moves to a player nstead of ndcatng to precsely one next move. The objectve n a mean-penalty party game s now to wn the party game wth a most permssve strategy. The quanttatve nature enters the stage as the average number of next moves that are blocked by the strategy. In ths way, a strategy s more permssve than another f t blocks fewer edges on average. We nvestgate the computatonal structure of strateges n both classes of games. We prove a structure theorem about optmal and ε-optmal strateges n mean-payoff party and mean-penalty party games. Ths theorem states that there exsts a lnear (n the number of colors) number of postonal strateges, such that an ε-optmal strategy for any choce of ε, and even an optmal strategy can be constructed from ths collecton of strateges. Moreover, the collecton can be effectvely computed. Buldng on ths theorem we show that strategy machnes mplementng ε-optmal strateges (for any ε) and even optmal strateges can be constructed such that they have polynomal sze n the arena. Moreover, the sze s ndependent of the optmalty of the strategy; that s, the sze of the strategy machne s the same, regardless of whether an optmal or ε-optmal strategy s mplemented. The latency and space requrement, on the other hand, depend on the optmalty of the strategy. Both grow only logarthmcally n ε. If the strategy s to be optmal, both latency and space requrement are unbounded. Ths result shows that the optmalty of a strategy s not a property of the underlyng algorthm but s a functon of the computatonal resources allotted to that algorthm. Turnng to mean-penalty party games, we ntroduce permssve strategy machnes, a machne model based on non-determnsm, as Turng-machne-based mplementatons of permssve strateges. Usng ths model we lft the result about mean-payoff party games to the doman of mean-penalty party games. In partcular, we obtan permssve strategy machnes of polynomal sze that compute optmal strateges n mean-penalty 10

17 party games. Agan, ε-optmal strateges requre the same sze as optmal strateges. Optmalty of a strategy s agan completely determned by the computatonal resources of the permssve strategy machne mplementng t. The latency and space requrement of ε-optmal strateges are both logarthmc n ε. The process of lftng the result from mean-payoff party games to mean-penalty party games uses an mportant observaton that s worth a dscusson of ts own. Namely, we use a reducton from mean-penalty party games to mean-payoff party games ntroduced n [BMOU11a]. The crucal pont s that ths reducton only transfers the values of vertces (where the value of a vertex s the hghest utlty a player can enforce n plays startng there), but does not normally allow the transfer of (automaton) strateges. However, usng (permssve) strategy machnes as a strategy representaton enables us to transfer strateges from one model to the other. Ths s agan mplemented va subroutne calls, smlar to the approach underlyng strategy composton. Ths result further underlnes the power of the transton-centered vew on strategy representaton. Mean-payoff party games were ntroduced n [CHJ05]. In that paper, the authors show the exstence of optmal and ε-optmal strateges. The strategy constructon they provde s mplct and the structure of the resultng strateges s not clearly descrbed. In our structure theorem, we revst the decomposton of mean-payoff party games from [CHJ05]. We make the strategy constructon explct and gve a formal descrpton of the structure of the resultng strateges. Buldng on ths structure, we can show that optmal (and ε-optmal) strateges can be mplemented by strategy machnes of polynomal sze and wth logarthmc latency n ε. Mean-payoff party games are extensons of both classcal party games [EJ91] and mean-payoff games [EM79]. Both classes of games are well studed [GTW02, Jur00, VJ00, ZP96]. Nevertheless, the complexty of decdng the wnner s an open problem for both classes. There were several subsequent papers relatng the model to other classes of quanttatve games, such as energy games. We menton here [CD10, CD12]. Permssve strateges have been studed n prevous work. For ths we menton [BJW02]. Mean-penalty party games were ntroduced n [BMOU11b, BMOU11a]. In ther work, the authors also revst mean-payoff party games. They provde two reductons from mean-penalty party games to mean-payoff party games, one of yeldng games of exponental sze and one yeldng games of polynomal sze. The frst allows for a transfer of strateges whereas the latter does not. Our results add the observaton that the Turng-machne-based strategy representaton can crcumvent the problem of non-transferable strateges. 11

18

19 1 Games, Transton Systems and Languages In ths chapter we revst some basc concepts from the area of languages, automata, logc, complexty theory, and game theory. Whle ths chapter s farly self-contaned, a thorough ntroducton nto all of these felds s beyond the scope of ths thess. In partcular, we omt the proof of several theorems. We cte reference to a publcaton provdng the formal proof n such cases. For ntroductons to the areas dscussed n ths chapter, we recommend [GTW02, HU00, Hro10, PP04, EFT94, Tho08a, Pap94, AG11]. In the frst secton, we revst elementary concepts, such as automata, languages, logc, graphs, Turng machnes and complexty. Specfcally, we ntroduce the notatonal conventons we use throughout ths thess and recall a few mportant results, partcularly from the area of complexty theory. The followng secton we ntroduce nfnte games. We recall the most fundamental defntons, such as arenas, strateges, wnnng condtons and determnacy. Fnally, we end ths chapter wth a bref tour of the varous classes of games that we study n ths thess. Each class s gven by a partcular type of wnnng condton. For every class of games, we recall the most fundamental results about algorthms solvng games n ths class and complextes as well as methods of computng wnnng strateges. Varous ways of representng wnnng condtons of a gven class n a fnte way are dscussed. For each condton, we gve ponters to lterature coverng the games wth the correspondng wnnng condton n greater detal. 13

20 1. Games, Transton Systems and Languages 1.1. Basc Concepts Functons and Sets Gven two sets A and B we wrte f : A B for a partal functon (or partal mappng) f from A to B. The doman of f s the set dom( f ) = {a A f (a) s defned} A. The mage of f s the set m( f ) = f (A) = { f (a) a dom( f )}. If dom( f ) = A then f s a functon (or mappng) and we wrte f : A B. Gven a dom( f ) we let f [a b] denote the functon f (x) x = a, f [a b](x) = b x = a. Let f : A B. Gven S A we wrte f S for the restrcton of f to S. If f : A B and g : B C, we wrte g f : A C for the mappng a g( f (a)) (f a dom( f ) and f (a) dom(g)). For any two sets A and B, the set of all functons f : A B s denoted by B A. Gven a set X Y where Y s clear from context, we wrte X C = Y \ X for the complement of X n Y. The powerset of X s denoted by P(X) = {Y Y X}. The most common sets we use n ths thess are the set B = {0, 1} of bnary dgts, the set N = {1, 2, 3,..., } of natural numbers, the set Z = {..., 2, 1, 0, 1, 2,...} of ntegers, the set Q of ratonal numbers and the set R of real numbers. We wrte N 0 for N {0}. An ntal segment of N s a set of the form k = {1,..., k} for some k N. If we want to nclude zero, we wrte k 0 = {0,..., k} for k N 0. Fnte and Infnte Words We brefly recall the elementary notaton concernng fnte and nfnte words. Let Σ be a set. A sequence w = w 1 w k where ether k = 0 and w s the empty sequence or where k 1 and w Σ for 1 k s called a (fnte) word. The set of all fnte words over Σ s denoted by Σ. If we have the empty sequence (.e., k = 0), we wrte w = ε and call w the empty word. The set of all non-empty fnte words s Σ + = Σ \ {ε}. The number k s the length of w and s denoted by w. We wrte Σ k for all words over Σ of length k N. Furthermore, we wrte Σ k for all words of length at most k; that s, Σ k = {ε} k =1 Σ. In the context of words, the elements of Σ are sometmes called letters. It s worth notng that letters, dependng upon context, may have a dfferent meanng as well (such as vertces of a graph, for nstance). Note also that a word w as above s, formally, a mappng w : k Σ. Consequently, we sometmes 14

21 1.1. Basc Concepts wrte w() = w for the -th letter n w. We use the two notatons w = w 1 w k and w = w(1) w(k) nterchangeably, choosng whchever s more convenent from a typographc pont of vew. An nfnte word over Σ s an nfnte sequence α = α 1 α 2 of letters α Σ. Agan α s a mappng α : N Σ and we may wrte α() = α Σ. The set of all nfnte words s denoted by Σ ω. Fnally, the set of all fnte or nfnte words s denoted by Σ = Σ Σ ω. We usually use lowercase Roman letters for fnte words and lowercase Greek letters for nfnte words. However, ths s no strct rule and we wll devate from t whenever t s convenent. Let w Σ. We wrte w[, j] = w w j for all j dom(w). If j < then w[, j] = ε s the empty word. A word w[1, j] s called a prefx of w. The empty word s a prefx of every w Σ va ε = w[1, 0]. A suffx of w s a word of the form w[, w ]. If w Σ ω we addtonally wrte w[, ] for the suffx w w +1 w +2 Σ ω. We wrte x w f x s a prefx of w Σ. We do not ntroduce specal notaton for suffxes. The last letter of a (fnte) word w = ε s denoted by last(w). Gven two words w Σ and α Σ, we defne the concatenaton of w and α, wrtten w α or smply wα, as the mappng w() α( w ) w > w An mportant example of a set of fnte words s the set B of all bnary strngs. Often, B s nterpreted as the bnary representaton of N 0. To do so, we requre two functons bn( ) : N 0 B and ( ) 2 : B N 0. Both mappngs are defned by vewng the rghtmost bt be the least sgnfcant one n the standard 2-adc representaton of natural numbers. We let (ε) 2 = 0. Note that ( ) 2 s not njectve. In partcular, the two functons bn( ) and ( ) 2 are not nverse. Languages, Automata & Logc Let Σ be a fnte alphabet. A subset L Σ s called a language (of fnte words over Σ). Gven two languages L and K, ther product s defned as L K = {x y x L, y K}. In addton, we wrte L = {w Σ w = x 1 x r for x 1,..., x r L, r N 0 }. In partcular, we have ε L. If L Σ ω we call L a language (of nfnte words over Σ). If K Σ s a language of fnte words and L Σ ω s a language of nfnte words, then K L = {x α x K, α L} Σ ω s a language of nfnte words. Gven a language L Σ + of nonempty fnte words, we defne L ω = {w 1 w 2 w 3 w 4 Σ ω w L, N}. We assume a certan famlarty wth the theory of languages of fnte or nfnte words. In partcular, we wll assume the reader s famlar wth the classes of ω-regular 15

22 1. Games, Transton Systems and Languages and regular languages. Ths ncludes varous automaton theoretc characterzatons of those classes. For a comprehensve ntroducton to the feld, the reader s referred to [GTW02, Löd11, PP04, HU00]. A Mealy automaton s a tuple M = (M, Σ, Γ, m 0, upd, out) consstng of a fnte set M of memory states (or smply states), a fnte nput alphabet Σ, a fnte output alphabet Γ, an ntal state m 0 M, a memory update functon upd: M Σ M and an output functon out: M Σ Γ. Often the sets Σ and Γ concde, and we omt Γ from the tuple, wrtng M = (M, Σ, m 0, upd, out). As usual, upd extends naturally to a functon upd: M Σ M acceptng words nstead of letters. The same holds for out. The sze of a Mealy automaton M s the number M = M of ts memory states. Mealy automata have functonal semantcs. A Mealy automaton M defnes a functon f M : Σ Γ by lettng f M (ε) = ε and f M (w a) = f M (w) out(m 0, w a) for all w Σ and a Σ. We say M mplements f M. On occason we also consder Mealy automata n whch the set M of control states s nfnte. We call such a structure an nfnte state Mealy automaton to emphasze the nfnte state space (and thereby nfnte memory update functon and nfnte output functon). In partcular, a Mealy automaton (wthout the prefx nfnte state ) wll always be fnte. Note that an nfnte state Mealy automaton s a computatonally very powerful object because wthout further restrctons t may mplement non-computable functons. Languages, as we have ntroduced them above, can not just be descrbed by automata, but also usng logc. We do not present a formal ntroducton of logc. That would be beyond the scope of ths thess. However, we do requre two partcular logcs, whch we wll recall now: propostonal logc and lnear temporal logc. Propostonal logc s defned as usual to be the set of formulas constructble from of a set X of varables usng negaton, dsjuncton and conjuncton. The set of all propostonal formulas over X s denoted by PL(X). Formulas of the form X, where X X, are called atoms. A lteral s an atom or the negaton of an atom,.e. X or X for X X. A model of a propostonal formula φ s a set M X such that φ s true when evaluatng all varables X M as true and all X X \ M as false. We wrte M = φ n ths case. If S X and φ PL(X), then φ[s] denotes the formula where all varables X S that occur n φ have been replaced wth 1, and all other varables occurrng n φ have been replaced wth 0. Thus, φ[s] s a Boolean expresson and t evaluates to true ff S = φ. Turnng to lnear temporal logc (LTL), we now requre a fnte set P of propostons. The set of all LTL-formulas over P s denoted by LTL(P). It s defned nductvely: 16

23 1.1. Basc Concepts For every p P p LTL(P). For every φ LTL(P) both X φ LTL(P) and φ LTL(P). For every φ, ψ LTL(P), formulas φ U ψ LTL(P), φ ψ LTL(P) and φ ψ LTL(P). We defne a number of shorthand expressons for convenence. Frst, we may assume true and false are gven constants; for nstance, as true = p p for some p P and false = true. Furthermore, we defne for every φ LTL(P): F φ := true U φ G φ := F φ Lnear temporal logc s evaluated on nfnte strngs α P ω. The semantcs are defned nductvely. α = p ff α 1 = p α = Xφ ff α[2, ] = φ α = φ ψ ff α = φ or α = ψ α = φ ψ ff α = φ and α = ψ α = φ ff α = φ α = φ U ψ ff α = ψ or there exsts 1 < N wth α[j, ] = φ for all 1 j < and α[, ] = ψ An LTL formula φ LTL(P) defnes a language L(φ) = {α P ω α = φ}. Ths language s always ω-regular [GTW02, Löd11, EFT94]. Graphs In ths thess, we usually consder nfnte games played on fnte drected graphs wth no termnal vertces. A drected graph s a tuple G = (V, E) consstng of a set V of vertces and a set E V V of edges. The sze G of G s the number G = V N of ts vertces. The neghborhood of a vertex v V s the set ve = {v V (v, v ) E}. A subgraph of a graph G = (V, E) s a graph H = (V, E ) where both V V and E E. A subgraph H = (V, E ) s called an nduced subgraph of G = (V, E) f E = E (V V ). By defnton, an nduced subgraph s already determned by ts 17

24 1. Games, Transton Systems and Languages vertex set. Therefore, we often dentfy the nduced subgraph H wth ts vertex set V and call V an nduced subgraph as well. The transpose of a graph G = (V, E) s the graph G T = (V, E T ), where E T = {(v, v ) (v, v) E}. A vertex v s called termnal, f ve =. Two vertces v and v are adjacent f v ve or v v E. The n-degree of a vertex v s the number γ n (v) = {v V (v, v) E}. Lkewse, the out-degree of a vertex v s the number γ out (v) = {v V (v, v ) E}. A path s a sequence p = v 1 v 2 v 3 v k of vertces wth (v, v +1 ) E for all 1 k. Paths are sequences of vertces, just lke words. We therefore use the same notaton. In partcular we wrte p = k for the length of p (provded p s fnte). Infnte paths are defned n the same way as fnte paths. The set of all fnte or nfnte paths n A = (V, E) s denoted by Paths(A) V. A cycle s a fnte path p = v 1 v 2 v k wth v k = v 1. We wrte v G v f there s a path from v to v n G. If G s clear from context, we drop the subscrpt and wrte v v. If p s a path from v to v, we also wrte v p G v, or v p v. Trees are an mportant subclass of graphs. A tree s a graph wthout cycles where every vertex has n-degree at most one and precsely one vertex has n-degree zero. In ths thess we only consder rooted trees. Instead of usng the notaton for graphs as ntroduced above, we represent trees as sets of words. More formally, a tree (over Σ) s a prefx closed set T Σ. The root of T s the vertex ε. There exsts an edge between any two nodes x and x a for all x a T and a Σ. For a node x T, nodes of the form xa for a Σ are called the chldren of x. Conversely, x s called the parent of all ts chldren. A tree T Σ s a fnte tree, f t s fnte as a set. A labeled tree s a par (T, λ) of a tree T and a functon λ : T Λ for some set Λ of labels. The noton of a labeled tree extends naturally to multple labelng functons. We shall apply the term rrespectve of the number of labelng functons. If Σ s not explctly gven, we usually use the set Σ = N. In ths case, we assume that the tree T N s normalzed n the followng sense. Gven a node x and a number n N such that xn T, we assume that xm T for all 1 m n. An mportant class of trees we wll use on several occasons s that of bnary search trees. A bnary search tree s a tree T B. Turng Machnes We assume the reader s famlar wth Turng machnes, complexty classes and fundamental results of complexty theory. For a detaled treatment of the concepts requred to follow the results n ths thess, the reader s referred to [HU00, Pap94]. We now ntroduce our notaton and conventons regardng Turng machnes and complexty. Let t N. A t-tape Turng machne s a tuple M = (Q, Σ, Γ, q 0,, F) wth 18

25 1.1. Basc Concepts a fnte set Q of control states a fnte nput alphabet Σ a fnte tape alphabet Γ Σ a blank symbol # Γ \ Σ an ntal state q 0 a set F Q of fnal states a transton relaton ( (Q \ F) Γ t) ( Q Γ t { 1, 0, 1} t) M s called determnstc f s a functon, say δ : (Q \ F) Γ t Q Γ t { 1, 0, 1} t. In ths event we wrte M = (Q, Σ, Γ, q 0, δ, F). If M s not determnstc, we say t s non-determnstc. Unless explctly stated otherwse, any Turng machne n ths thess s determnstc. All Turng machnes n ths thess are assumed to have rghtnfnte tapes. The tape cells are therefore numbered wth the set N of natural numbers. We always assume that the machne never attempts to move left on the frst tape cell (whch can be acheved usng a specal marker symbol). A confguraton s, as usual, a tuple c = (q, w, h) Q (Γ ) t N t consstng of a state q, the tape content of all t tapes and the head poston on every tape. A confguraton c = (q, y, k) s a successor confguraton of c = (q, x, h) f there exsts (q, x 1 (h 1 ),..., x t (h t ), q, γ 1,..., γ t, d 1,..., d t ) such that y = x [h γ ] and k = h + d. Ths gves rse to the noton of the successor relaton M ( Q (Γ ) t N t) ( Q (Γ ) t N t) defned by c M c ff c s a successor confguraton of c. If M s clear from context, we drop the subscrpt and wrte nstead. The reflexve and transtve closure of s denoted by. A computaton of M s a sequence c 1 c 2 of successor confguratons. A computaton termnates as soon as M enters a fnal state q F (note that does not nclude any transtons leavng states n F). A confguraton s called termnal f ts state s n F. Gven a Turng machne M we denote ts runtme by T (M). Note that the runtme T (M) : Σ N { } s a functon assgnng the number of steps T (M, w) n any 19

26 1. Games, Transton Systems and Languages computaton 1 of M on any possble nput w. Smlarly, we denote the space requrement of M by S(M). Agan S(M) s a mappng assgnng the space requrement S(M, w) N { } to every admssble nput w Σ. The concept of non-determnsm can be extended to alternaton. An alternatng Turng machne s a Turng machne M = (Q, Σ, Γ, q 0,, F) wth a partton Q = Q Q. We sometmes wrte M = (Q, Q, Σ, Γ, q 0,, F). We assume the reader s famlar wth alternatng Turng machnes and ther semantcs. Detals can agan be found n [Pap94]. For the purposes of ths thess t s suffcent to menton that the semantcs of an alternatng Turng machne are gven by a reachablty game 2 on ts confguraton graph. We wrte G M (w) for the reachablty game assocated wth M and nput w Σ. Complexty classes are defned as usual and typeset n smallcaps, e.g. Ptme, NP, Pspace, and so forth. Classes defned by non-determnstc machnes are always prefxed wth an N, e.g. NP, NPspace. Classes defned by alternatng Turng machnes are prefxed wth an A, e.g. APtme, APspace. We use the followng, well-known equvalences: Theorem (see [Pap94]) 1) APspace = Exptme 2) APtme = Pspace In addton, we state Savtch s theorem for reference: Theorem (Savtch, [Sav70]) Pspace = NPspace 1 Although most Turng machnes n ths thess are determnstc, n Chapter 5 we use non-determnstc machnes. Therefore, we defne the complexty measures n terms of non-determnstc machnes nstead of determnstc ones. 2 The reader not famlar wth reachablty games may want to skp ahead and read Sectons 1.2 and 1.3 frst. 20

27 1.2. Infnte Games and Strateges 1.2. Infnte Games and Strateges All games we study n ths thess are played on fnte drected graphs, called arenas. We start ths secton by formally ntroducng the concept of an arena: Defnton (Arena) An arena s a drected graph A = (V, E, V (0), V (1) ) such that 1) The vertces are parttoned nto V = V (0) V (1). Vertces n V (0) are called 0-vertces, those n V (1) are called 1-vertces. 2) No vertex s termnal: ve = for all v V. We usually drop V (0) and V (1) from the tuple and smply wrte A = (V, E). Gven an arena A we wrte V A for ts set of vertces and E A for ts set of edges. A subarena of A s an nduced subgraph B = (V, E ) of A whch s agan an arena. As wth all nduced subgraphs, we often dentfy B wth ts vertex set V. An arena A s sad to be bpartte f E b B V (b) V (1 b). Every game (to be defned shortly) consdered n ths thess s played on an arena. To get an ntuton about games, let us fx an arena A = (V, E). We pck v 1 V, called the ntal vertex. The ntuton s the followng. There are two players, Player 0 and Player 1. A token s placed on v 1. If v 1 V (0) then Player 0 chooses a vertex v 2 v 1 E. Otherwse v V (1) and Player 1 chooses v 2 v 1 E. The process then repeats wth v 2 n place of v 1. In ths way the two players construct an nfnte sequence v 1 v 2 v 3 of adjacent vertces, called a play. The set of all possble plays s parttoned nto a set W V ω of plays won by Player 0 and plays W C of plays won by Player 1. Formally we have the followng: Defnton (Game) A game s a tuple G = (A, W) consstng of an arena A = (V, E) a wnnng condton W V ω (for Player 0). 21