Factored Models for Probabilistic Modal Logic

Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008 Factored Models for Probabilistic Modal Logic Afsaneh Shirazi and Eyal Air Coputer Science Departent, University of Illinois at U-C Urbana, IL 61801, USA {hajiain, eyal}@uiuc.edu Abstract Modal logic represents knowledge that agents have about other agents knowledge. Probabilistic odal logic further captures probabilistic beliefs about probabilistic beliefs. Models in those logics are useful for understanding and decision aking in conversations, bargaining situations, and copetitions. Unfortunately, probabilistic odal structures are ipractical for large real-world applications because they represent their state space explicitly. In this paper we scale up probabilistic odal structures by giving the a factored representation. This representation applies conditional independence for factoring the probabilistic aspect of the structure (as in Bayesian Networks (BN. We also present two exact and one approxiate algorith for reasoning about the truth value of probabilistic odal logic queries over a odel encoded in a factored for. The first exact algorith applies inference in BNs to answer a liited class of queries. Our second exact ethod applies a variable eliination schee and is applicable without restrictions. Our approxiate algorith uses sapling and can be used for applications with very large odels. Given a query, it coputes an answer and its confidence level efficiently. Introduction Reasoning about knowledge plays an iportant role in various contexts ranging fro conversations to iperfectinforation gaes. Foral odels of reasoning about knowledge use odal operators and logic to express knowledge and belief. These enable agents to take into account not only facts that are true about the world, but also the knowledge of other agents. For exaple, in a bargaining situation, the seller of a car ust consider what the potential buyer knows about the car s value. The buyer ust also consider what the seller knows about what the buyer knows about the value, and so on. In any applications, it is not enough to include certain knowledge or lack thereof. For exaple, the seller of a car ay not know the buyer s estiate of the car s value, but ay have a probability distribution over it. Current foral logical systes (especially odal logics (Fitting 1993; Fagin et al. 1995 cannot represent such scenarios. On the other hand, probabilistic graphical odels (Pearl 1988 can Copyright c 2008, Association for the Advanceent of Artificial Intelligence (www.aaai.org. All rights reserved. represent distributions over distributions (Blei, Ng, & Jordan 2003, but are not granular enough for ultiple levels of coplex beliefs. A nuber of works have presented fraeworks capable of capturing probabilistic beliefs about probabilistic beliefs (Fagin & Halpern 1988; Heifetz & Mongin 1998; Auann & Heifetz 2001; Shirazi & Air 2007. These rely on probabilistic odal structures that cobine accessibility graphs and probabilistic distributions. However, reasoning with such structures does not scale to doains of any states because it accesses every state explicitly. Consequently, reasoning is ipractical even for siple scenarios such as Poker ( ( ( 52 5. 47 5 > 10 12 states. In this paper we provide a copact odel and efficient reasoning algoriths for nested probabilistic odalities. Our odel is capable of representing applications with large state spaces. We describe syntax and seantics of our representation, as well as reasoning algoriths for evaluating a query on a odel. More specifically, we introduce a fraework for odeling probabilistic knowledge that uses a Bayesian Network (BN (Pearl 1988 to represent the probabilistic relationship between states. We provide two exact ethods for answering queries based on the class of queries of interest. Our first ethod perfors inference in a BN representation of the cobined query and odel. Our second ethod is a variable eliination schee in which we copute values for sub-forulas of the query in a dynaic-prograing fashion. We show theoretically and experientally that these ethods are faster than earlier techniques (Shirazi & Air 2007. We also introduce a sapling ethod that is tractable on larger odels for which the exact ethods are intractable. We show that for a specific group of queries the probability of error in the estiated value of the query does not decrease when the nuber of saples increases. Our ethod addresses this issue by coputing the confidence in the answer to the query. Most previous related works are liited to cobining probability with a special case of odal logic in which accessibility relations are equivalence relations, which is called probabilistic knowledge. Aong those, (Fagin & Halpern 1988; Heifetz & Mongin 1998 are ainly concerned with providing a sound and coplete axioatization for the logic of knowledge and probability. (Shirazi & Air 541

2007 provides reasoning ethods for general probabilistic odal structures but does not scale up to large doains. Another related work is (Milch & Koller 2000 in which probabilistic episteic logic is used to reason about the ental states of an agent. This logic is a special case of probabilistic knowledge with the additional assuption of agents having a coon prior probability distribution over states. Adding probabilistic notions to odal logic is also considered in (Herzig 2003; Nie 1992. The forer adds a unary odal operator expressing that a proposition is ore probable than its negation, whereas the latter defines an extension of fuzzy odal logic to perfor probabilistic seanticbased approaches for finding docuents relevant to a query. Related works in the gae theory literature ainly focus on iperfect inforation gaes. For exaple, (Koller & Pfeffer 1995 provides an algorith for finding optial randoized strategies in two-player iperfect inforation copetitive gaes. The state of these gaes can be represented with our odel. Factored Probabilistic Modal Structures In this section, we provide a copact Bayesian Network (BN representation for probabilistic Kripke structures. A BN is a directed acyclic graph in which nodes represent rando variables, and the joint distribution of the node values can be written as P r(x 1,..., X n = Π n i=1 P r(x i parents(x i. According to (Shirazi & Air 2007, a probabilistic Kripke structure consists of a set of states and a probabilistic accessibility relation (we redefine it in the next section. The probabilistic accessibility relation is defined for each pair of states. The probabilistic Kripke structure is basically a graph of states with labeled edges. Therefore, the size of this structure is quadratic in the nuber of states and it is not scalable to large doains. In the following section we review the probabilistic Kripke structures. Then, in Section we provide coplete specifications of our representation. Probabilistic Kripke Structures For siplicity we assue that the agent wishes to reason about a world that can be described in ters of a nonepty set, Z, of state variables. Probabilistic odal forulas are built up fro a countable set of state variables Z using equality (=, propositional connectives (,, and the odal function K. We use and for truth and falsehood constants. First, we need to define non-odal forulas. The foration rules are: 1. For every state variable X and value x, X = x is a nonodal forula. 2. and are non-odal forulas. 3. If X is a non-odal forula so is X. 4. If X and Y are non-odal forulas so is X Y. 5. Every non-odal forula is a probabilistic odal forula. 6. If X is a probabilistic odal forula so is (K(X r when 0 < r 1 and {<, =}. Note that we have a different odal function K i for each agent i in the doain. We take, and to be abbreviations in the usual way. We use Texas Holde poker gae as an exaple of a gae with iperfect knowledge that can be odeled using our fraework. In Holde, players receive two downcards as their personal hand, after which there is a round of betting. Three boardcards are turned siultaneously (called the flop and another round of betting occurs. The next two boardcards are turned one at a tie, with a round of betting after each card. A player ay use any five-card cobination fro the board and personal cards. There are soe rules that applied to the cards to rank hands. For ore info refer to http://www.texasholde-poker.co/handrank. In the Holde exaple, suppose that we introduce two new propositional sybols, w 1 and w 2, to show whether player 1 or player 2 wins the hand, respectively. The value of these sybols is deterined based on the gae rules applied to players hands and boardcards (players hands and boardcards are state variables. In this exaple there are two players, therefore we have two odal functions, K 1 and K 2 corresponding to player 1 and player 2. K 1 (w 1 < 1/2 is an exaple of a probabilistic odal forula whose truth value can be evaluated on the current state of the world. K 1 (w 1 < 1/2 deonstrates that the probability of player 1 winning the hand is less than 1/2 fro her perspective. Now, we describe the seantics of our language. Our approach is in ters of possible worlds which is siilar to Kripke structures in odal logic. Definition 0.1 A probabilistic Kripke structure M is a tuple (S, P, V in which 1. S is a nonepty set of states or possible worlds. 2. P is a conditional probability function. P(s s denotes the probability of accessing state s given that we are in state s. P(s s > 0 indicates that s is accessible fro s. Therefore, it is siilar to the accessibility relation of odal logic. Since P is a probability function, we should ensure that the following constraints hold: 0 P(s s 1 For each state s S: s S P(s s = 1 3. V is an interpretation that associates with each state in S a value assignent to the state variables in Z. Probabilistic knowledge is a special case of probabilistic odal logic in which the accessibility relation is an equivalence relation. This odel captures the intuition that agent i considers state t accessible fro state s if in both s and t she has the sae knowledge about the world. The above definition is for one odal function. When we have j odal functions (K 1,... K j we need a probability function P i for each odal function K i. Intuitively, the probability function P i represents the probability of accessing a state fro the perspective of agent i. In probabilistic odal logic, the true state of the world is a state in S. An agent has a probability distribution over all the states that are possible to her given the true state of the world. Exaple. Let (KJ, K3, KKQ32 be the cards of players and the boardcards. Since player 1 does not know her opponent s hand, P 1((KJ, 65, KKQ32 (KJ, K3, KKQ32 should be > 0. P 1 is unifor on all the possible states since the player does not have any inforation about her opponents hand. 542

Figure 2: Left: GKM of War. Right: Kripke structure of War. Figure 1: GKM of Holde. The truth value of any forula in a given state is evaluated with the following definitions. Definition 0.2 Let M = (S, P, V be a probabilistic Kripke structure. Let s be a state in S. For any probabilistic odal forula ϕ, we define val(s, ϕ recursively as follows. 1. val(s, X = x = (V(s, X = x for a state variable X. 2. val(s, = true. 3. val(s, = false. 4. val(s, X = val(s, X. 5. val(s, X Y = val(s, X val(s, Y. 6. val(s, K(X r = true iff P(s s r. s S,val(s,X=true We use val s definition to define logical entailent, =. Definition 0.3 M, s = X iff val(s, X is true. Graphical Kripke Models A probabilistic Kripke structure as defined above, (S, P, V, has size O( S 2. This representation is ipractical for large state spaces. In this section we provide a ore copact representation for probabilistic Kripke structures. In our new odel, a state is represented by a set of state variables, Z. P is represented by a BN with 2 Z variables: Z(i a and Zh (i for each Z (i Z. Z(i a stands for a state variable for the actual state of the world, whereas Z(i h represents a variable for a hypothetical state of the world (an agent cannot distinguish with certainty between this state and the actual state. P r(z h Z a is represented by the BN which serves as P of probabilistic Kripke structures (e.g., see Figure 1. Definition 0.4 A graphical Kripke odel (GKM M on a set of rando variables Z is a BN which is defined as follows: 1. The nodes are: Z a (1,..., Za ( Z, Zh (1,..., Zh ( Z 2. P r(z h Z a is defined by the edges of M. 3. There are no edges between the nodes in Z a. The above definition is for one odal function, K. For cases with j odal functions, K 1,... K j, we need to define Z hi and P r(z hi Z a for each odal function K i. Figure 1 shows the GKM of our Holde exaple. The nodes in the first row represent the actual state of the world, whereas the second row represents a possible state of the world. Each node takes values fro {A,2,...,10,J,Q,K} {,,, }. The first and second nodes are observed by player 1 to have values K and J, respectively. In each row, the first two nodes correspond to player 1 s hand, the second two nodes correspond to player 2 s hand, and the last five are the boardcards. Fro the perspective of player 1, player 2 can have any cards except the boardcards and the cards in her hand. In the BN, this is shown by the edges to the third and forth node in Z h. The boardcards and player 1 s hand cards are the sae in the actual state of the world and the hypothetical state of the world. Let Z 1,..., Z 9 stand for the nodes in each row (Z 1 be the leftost node, the conditional probability functions are: ( P r(zi h Zi a 1 if Zi h = Zi a ; = for i {1, 2, 5,..., 9} 0 otherwise. P r(z h 3 Z a 1, Z a 2, Z a 5,..., Z a 9 = 1 α ( 1 if Z h 3 {Z a 1, Z a 2, Z a 5,..., Z a 9 }; 0 otherwise. As shown in the above equation, Z h 3 has a unifor distribution. α is the noralization factor. The conditional probability function for Z h 4 is the sae as Z h 3 except that Z h 4 is a child of Z h 3 and should not be equal to Z h 3 as well. Theore 0.5 Let Z be the set of state variables and k be the nuber of agents. GKM has O(k Z nodes and O(k Z 2 edges and each node has at ost 2 Z 1 parents. Note that this odel is ost useful when the size of the largest Conditional Probability Table (CPT is uch saller than S or when the CPTs can be represented copactly (e.g., unifor distribution. In those cases, the size of GKM is uch saller than the size of the corresponding probabilistic Kripke structure (O(2 Z nodes when state variables are binary. Exaple. We define a sipler 2-player gae, naed War, for the purpose of exposition with a saller set of states. In War, there is a deck of three cards, A, B, and C. One card is dealt to each player and the third card is face down. The player with the highest card wins the gae (A > B > C. The Kripke odel has six states, and so can be easily analyzed. The equivalence classes for player 1 are shown in the right part of Figure 2. The first rounded rectangle corresponds to the class in which player 1 has A and player 2 either has B or C (the actual state of the world is either AB or AC. In this equivalence class, player 1 knows that she is winning with probability 1. Player 1 has a probability distribution over each of these equivalence classes. The GKM representing the equivalence classes of player 1 is shown in the left part of Figure 2. Z a represent the actual 543

FUNCTION Q2BN a (query q, set of state variables Z P a: associates with each node a set of parent nodes. This function returns the query node of the BN. 1. if q = K i(x then 2. Z h i a set of new nodes for all state variables 3. P a(z h i Z with conditional probability of P i 4. return Q2BN(x, Z h i 5. else 6. n new node 7. if q is Z (i = x then P a(n = Z (i 8. else if q = x then P a(n = Q2BN(x, Z 9. else if q = x y then 10. P a(n = {Q2BN(x, Z, Q2BN(y, Z} 11. return n a We do not delve into the details of CPTs in this function. FUNCTION QuAn(query q, state s 1. Z a a set of new nodes corresponding to all state variables 2. return P r(q2bn(q, Z a = 1 Z a = s //the probability is coputed using any BN inference ethod Figure 3: Query Answering (QuAn algorith. state of the world and Z h represent the hypothetical state of the world that player 1 considers possible. In this exaple P (Z h 2 Z a 1 is unifor when Z h 2 Z a 1 (P (Z h 2 = B Z a 1 = A = 1 2 and P (Zh 2 = C Z a 1 = A = 1 2. P (Zh 1 Z a 1 is equal to 1 when Z h 1 = Z a 1 and is 0 otherwise. Query Answering In this section we provide reasoning ethods for answering queries over GKMs. Previous sections showed that using GKMs potentially reduces the size of the odel exponentially. The reasoning ethods known for probabilistic Kripke structures cannot be used on GKMs in practice. This is because they enforce explicit access to every state in the probabilistic Kripke structure. In this section we design new ethods for reasoning with GKMs and show that they are ore efficient than their counterparts for probabilistic Kripke structures. In Section we investigate a class of queries that can be answered by inference in BNs. We also introduce a ethod that answers these queries by taking advantage of the BN structure of GKMs. In Sections and we provide an exact and a sapling algorith for answering probabilistic odal queries (defined in Section, respectively. Answering Expectation Queries Any probabilistic odal query with no nested odal function can be answered by coputing the arginal probability of a node in a BN. For exaple, for K 1 (x < r we add a node x as a child of the hypothetical state of the corresponding GKM. Since x is a non-odal forula, it can be easily added as a node to the BN. In this new odel P r(x Z a = s is equal to the value of K 1 (x on state s which can be copared to r, thus answering K 1 (x < r. A probabilistic odal query with nested odal functions cannot be odeled with a BN since the paraeter of its odal function is inequality. For exaple, BNs cannot represent queries such as K 1 (K 2 (x < 1 2 < 3 4. In this exaple K 2 (x can be represented by the arginal probability of a node given the actual state of the world in a BN. However, we do not know a way to introduce another node that copares this value with a nuber. Inference can answer queries with no inequality (e.g., K 1 (K 2 (x. The answer to such queries is a nuber between 0 and 1. In these queries we treat the odal function as an expectation function. K 1 (K 2 (x denotes the expected value fro the perspective of agent 1 of the expected value of x fro the perspective of agent 2. Based on Definition 0.2, the value of K 1 (x is equal to the expected value of x fro the perspective of agent 1. K 1(x = X s S x(s P 1(s s = E 1(x where x(s denotes the value of x on state s, and P 1 (s s is the probability of accessing state s when the true state of the world is s fro the perspective of agent 1. The expectation queries have the following forat: 1. Every non-odal forula is a query. 2. If Q is a query so is K(Q. Algorith QuAn of Figure 3 coputes the answer to such queries. First, it transfors the query into a BN (using the GKM, and then coputes the answer to the query by perforing inference in the BN (any inference ethod can be used (Zhang & Poole 1994; Jordan et al. 1999; Yedidia, Freean, & Weiss 2004. Exaple. Suppose that the query is K 1 (K 2 (x on state s. First, we need to transfor this query into a BN. The BN is shown in Figure 4. Z a, Z h1, and Z h2 each represent a set of nodes. Z a is the actual state of the world. Z h1 is the hypothetical state fro the perspective of player 1 (since the first odal function is K 1. Z h2 is the hypothetical state fro the perspective of player 2. The last node, x, is a non-odal forula on the state variables, therefore, can be Figure 4: BN for represented by a node. After creating K 1 (K 2 (x the BN, the value of P r(x Z a = s is coputed by perforing inference in the BN. This value is equal to the value of K 1 (K 2 (x on state s and so is the answer to the query. The following equations justify our ethod. It shows that the value of P r(x Z a = s is equal to the value of K 1 (K 2 (x on state s. P r(x Z a = s = X X P r(x, Z h 1, Z h 2 Z a = s Z h1 Z h 2 = X X P r(x Z h 2 P r(z h 2 Z h 1 P r(z h 1 Z a = s Z h1 Z h 2 = X P r(z h 1 Z a = s X P r(x Z h 2 P r(z h 2 Z h 1 Z h1 Z h2 = X P r(z h 1 Z a = s K 2(x on Z h 1 Z h1 = K 1(K 2(x on s 544

In the following sections we provide efficient algoriths for queries with inequalities (probabilistic odal queries. Ordered Variable Eliination In this section we provide an algorith to answer probabilistic odal queries. In the previous section we entioned that existing BN inference ethods cannot answer these queries. The algorith that we introduce is called Ordered Variable Eliination (OVE. The following exaple justifies that the original variable eliination (see (Pearl 1988 does not answer the following query because soe of the suations participate in inequalities. Therefore, the order of soe of the suations cannot be changed. Exaple. Assue the query K 1 (K 2 (x < 1 2 < 3 4 on s. This query is calculated as follows: K 1(K 2(x < 1 2 < 3 4 on s X = P r(z h 1 Z a = s K 2(x < 1 «2 on Zh 1 < 3 4 Z h 1 X = P r(z h 1 Z a = s Z h 1 `X P r(x Z h 2 P r(z h 2 Z h 1 < 1 «< 3 2 4 Z h 2 In this forula we cannot ove Z h 1 inside Z h 2, since the latter participates in an inequality. OVE perfors variable eliination on this forula in two rounds. It eliinates variables Z h2 in the first round and variables Z h1 in the second round. Assue that Z which is a subset of Z h1 is the set of parents of Z h2 in the BN calculated by Function Q2BN of the previous section. After the first round of variable eliination ( Z h 2 P r(x Z h2 P r(z h2 Z h1 < 1 2 is replaced by f(z < 1 2. The result is a suation over Z h 1 which is coputed in the second round of variable eliination. The algorith is shown in figure 5. There are a few standard ways to speed up this function. For exaple, instead of suing over all Zs we can su over those in which P r(z Z is not zero. This will provide a faster approach when P r(z Z is sparse. Theore 0.6 Let q be a query, s be a state, v be the axiu size of the doain of rando variables, and t be the size of the largest factor. Function OVE calculates the value of q on s in O(v t tie. Eliination is deriven by an ordering on variables. OVE does not allow all the orders. Therefore, for soe graphs its running tie is worse than the variable eliination s. Typically, v t << S. However, the worst-case running tie of this algorith is the sae as the running tie of GBU in (Shirazi & Air 2007 which is the fastest exact ethod in that paper. Sapling with Confidence In this section we provide a sapling ethod to answer queries on GKMs. Our ethod is based on probabilistic logic sapling of (Henrion 1986 which is the siplest and first proposed sapling algorith for BNs. This ethod is optial for our query answering because our evidence nodes FUNCTION OVE(query q, state s q: the query of the for K (1 (... (K ( (x < n (... < n (1 in which K (i {K 1,..., K j} for j agents a s: list of actual-state values Z a : list of actual-state variables Y : query node b Z (i : list of hypothetical-state variables corresponding to K (i 1. Z a set of new nodes for all state variables 2. Y Q2BN (q, Z a 3. F list of conditional probabilities in the odel 4. for i to 1 5. while Z (i is not epty 6. reove node z fro Z (i 7. su-out(f, z 8. f(z the result of previous loop /*Z Z (i 1 */ 9. add f(z < n (i to F 10. h the ultiplication of all factors in F 11. return h(y P Y h(y a For siplicity we only treat <. The inequality can be > as well. b Q2BN is siilar to the one in the previous section except that these queries have inequalities. But it does not affect the BN. Figure 5: Ordered Variable Eliination (OVE algorith. are root nodes. The details of the ethod is provided in the rest of this section. First, we show that the estiated values of soe queries ay not converge to the true values by increasing the nuber of saples. Consequently, the only way to answer these queries is to use an exact ethod. The following theore states this result. Theore 0.7 Let K(x < n be a query, s be a state, and s 1, s 2,... be a sequence of independent and identically distributed states sapled fro P r(z h Z = s. Define ˆK = x(s1+...+x(s to be the observed value of K(x using saples. P r( li ( ˆK < n does not exist = 1 when n is equal to the value of K(x on s and 0 < n < 1. Proof Sketch. We show the proof for n = 1 2. We show that P r( li ( ˆK < 1 does not exist = 1 when the value of 2 K(x on s is equal to 1 2. The proof for other n s uses the solution of the ruin proble for generalized one-diensional rando walks (Feller 1968. Since the value of ˆK < n for a specific is either 0 or 1, li ( ˆK < 1 2 (if exists should be either 0 or 1. First, we show that P r(li ( ˆK < 1 2 = 1 = 0 (the proof for value 0 is siilar to the proof for value 1. By definition of liit of a function at infinity, li ( ˆK < 1 2 = 1 if and only if for each ɛ > 0 there exists an N such that ( ˆK < 1 2 1 < ɛ whenever > N. Since ˆK < 1 2 is binary, our definition would be ˆK < 1 2 for > N. Each saple is drawn fro a Bernoulli distribution with ean 1 2. To copute the above probability we need to answer the following question. In a one-diensional rando 545

walk, what is the probability that no return to the origin occurs up to and including tie 2? A rando walk is odeled with X(t + 1 = X(t + Φ(t. In this notation, Φ(ts are independent and identically distributed Bernoulli rando variables that have value +1 and 1 with probability 1/2 at all ties. By lea 1 of chapter III.3 of (Feller 1968, the probability that no return to the origin occurs up to and including tie 2 is the sae as the probability that a return occurs at tie 2 (i.e., the nuber of +1s is equal to the nuber of -1s. This probability is: ( 2 P r(return at tie 2 = 2 2 We calculate the probability that in an infinite sequence no return to the origin occurs by coputing the liit of above probability at infinity. ( 2 P r( li ( ˆK < 1 = 1 = li 2 n 2 2 = 0 Siilarly we can show that P r(li ( ˆK < 1 2 = 0 = 0. Using these results, it holds that P r( li ( ˆK < n does not exist = 1. Based on this theore, the accuracy of a sapling ethod does not necessarily increase with the nuber of saples. To estiate the accuracy of the value of a query, our sapling ethod not only calculates the truth value of the query but also returns the confidence level of the ethod in that value. The confidence level is the probability of the query being true given the set of saples. Function CoSa shown in Figure 6 presents our sapling ethod. It returns the probability of the query being true. The function first transfors the query to its BN representation. Then, it calculates the probability of q = 1 recursively using Function RecCS (e.g., if RecCS returns 0 the truth value of the query is equal to 0. For queries with no odal function, RecCS calculates the value of q on s (the details of this calculation is not shown in the function and returns the probability of q = 1 based on this value. For queries with odal functions such as q = K i (q < n, RecCS repeats the following step ties. It saples a new state s accessible fro s and recursively coputes the probability of q on s. Then, RecCS calculates the probability of q = 1 using the probabilities of the values of these saples. Function RecCS calls Function CalculateProb to perfor this calculation. In the next few paragraphs we explain how CalulateProb coputes the probability of the query. Iagine the query q = K 1 (x < 0.4 on s. First, we saple states fro the probability distribution P r(z h Z = s and we calculate the value of x on each sapled state. Then, we copute the probability of q = 1 given these values. Let there be k saples with value 1. Each saple is a Bernoulli trial whose probability of success is p = K 1 (x. The saple proportion ˆp is the fraction of saples with value 1 so ˆp = k. When is large, ˆp has an approxiately noral distribution. The standard deviation of the saple pro- FUNCTION CoSa(query q, state s q: the query of the for K (1 (... (K ( (x < n (... < n (1 in which K (i {K 1,..., K j} for j agents s: list of actual-state values Z a : list of actual-state variables Y : query node Z (i : list of hypothetical-state variables corresponding to K (i 1. Z a set of new nodes for all state variables 2. Y Q2BN (q, Z a 3. return RecCS(q, s, Z a FUNCTION RecCS(query q, state s, set of nodes Z 1. if q = `K (i (q < n then 2. T 3. for j 1 to 4. s j saple according to P r(z (i Z = s 5. s value of non-leaf nodes in Z (i 6. if s / T then 7. T add s to T with weight 1 8. conf(s RecCS(q, s j, Z (i 9. else 10. weight(s weight(s +1 11. return CalculateProb(q, T 12. else if q = x then 13. if value x on s is true then return 1 14. else return 0 Figure 6: Confidence Sapling (CoSa algorith. p(1 p portion is σ =. Since the true population proportion (p is unknown, we use standard error instead of σ. The standard error provides an unbiased estiate of the standard deviation. It can be calculated fro the equation ˆp(1 ˆp SE =. Therefore, K 1(x ˆp SE N(0, 1and the probability of the query is calculated fro: n ˆp P r(k 1(x < n = Φ (1 q ˆp(1 ˆp where Φ is the cuulative distribution function of the standard noral distribution. For queries with nested odal functions, the exact values of the sub-query on the sapled states are unknown. We only have the probability of the sub-query on those states. To calculate the probability of K 1 (q < n where q has a odal function, we use the equation bellow: P r(k 1(q < 0.4 = X (v 1,...,v {0,1} P r(k 1(q < 0.4 q s 1 = v 1,..., q s = v P r(q s 1 = v 1... P r(q s 1 = v 1 where s 1,..., s are sapled states and q s i = v i eans the value of q on sapled state s i is equal to v i. P r(q s i = v i is calculated recursively. P r(k 1 (q < 0.4 q s 1 = v 1,..., q s = v is calculated using Forula 1 with ˆp equal to saple proportion. Theore 0.8 Let q be a query, s be a state, k be the nuber of nested odal functions, and be the nuber of saples at each stage. Function CoSa calculates the truth value of q on s in O( k+2 tie. 546

Running Tie (s 60 50 40 30 20 10 0 KBU ToDo ARea 10 OVE CoSa 10 CoSa 50 1 2 3 4 5 6 Nuber of Nested Modal Functions Figure 7: Running tie coparison of exact ethods (KBU, ToDo, OVE with approxiate ethods (ARea 10, CoSa 10, CoSa 50. Experiental Results In this section we copare the running tie of OVE and CoSa with their counterparts in (Shirazi & Air 2007(KBU (Knowledge Botto-Up, ToDo (Top-Down, and ARea 10 (Approxiate Reasoning. This section confirs the theoretical results of section about the running tie of our algoriths. Figure 7 shows the running tie of these ethods on Holde. ToDo and KBU are exact ethods defined for probabilistic Kripke structures and ARea 10 is a sapling ethod in which the nuber of saples used to evaluate each odal function is equal to 10. As shown in the figure, OVE and KBU grow linearly with the degree of nesting in queries. Moreover, OVE takes advantage of the unifor conditional probability distribution over states in our tests and therefore is even faster. Both approxiate ethods (ARea and CoSa grow exponentially with the degree of nesting, however for the sae nuber of saples Cosa is uch faster (copare ARea 10 with CoSa 10. The figure shows that even when we increase the nuber of saples to 50, CoSa 50 returns the answer faster than ARea 10. Note that in typical real-world situations the degree of nesting in queries is sall (e.g., less than 4; in poker a player at ost cares about what the opponent knows about what the player knows. In Holde the nuber of states is sall, so CoSa does not have any advantage over OVE. OVE is slow when the size of the state space is large, since its running tie is linear in the size of the state space. In those cases CoSa returns the approxiate answer uch faster. Consequently, CoSa should be used only for doains with large state spaces (e.g., five card poker with ( ( 52 5. 47 5 states. In sall doains, however, our exact ethod returns the answer reasonably fast. Conclusions & Future Work We provided a factored representation for probabilistic odal structures. We also introduced exact and approxiate reasoning ethods for answering queries on such odels. Our odel is ore copact than previous representations enabling larger-scale applications. Further, we show theoretically and experientally that our ethods are faster than their counterparts in previous representations. Investigating the belief update in our language is one of our iediate future-work directions. There, it is open how to update a odel with observation of opponent actions. Also, so far we have assued that the GKM is available to us. In future works we also ai to find ethods that learn a GKM. Acknowledgents We wish to thank Chandra Chekuri and Reza Zaani Nasab for valuable coents and Oid Fateieh for editorial assistance. We also wish to acknowledge support fro Ary W9132T-06-P-0068 (CERL Chapaign, Ary DACA42-01-D-0004-0014 (CERL Chapaign, and NSF IIS 05-46663 (US National Science Foundation. References Auann, R. J., and Heifetz, A. 2001. Incoplete inforation. In Handbook of Gae Theory with Econoic Applications. Blei, D.; Ng, A.; and Jordan, M. 2003. Latent dirichlet allocation. Fagin, R., and Halpern, J. Y. 1988. Reasoning about knowledge and probability. In TARK. Fagin, R.; Halpern, J.; Moses, Y.; and Vardi, M. 1995. Reasoning about knowledge. MIT Press. Feller, W. 1968. An Introduction to Probability Theory and Its Applications, Volue 1. Wiley, 3rd edition. Fitting, M. 1993. Basic odal logic. In Handbook of Logic in Artificial Intelligence and Logic Prograing, Volue 1: Logical Foundations. Heifetz, A., and Mongin, P. 1998. The odal logic of probability. In TARK. Henrion, M. 1986. Propagating uncertainty in bayesian networks by probabilistic logic sapling. In UAI, 149 164. Herzig, A. 2003. Modal probability, belief, and actions. Funda. Inf. 57(2-4:323 344. Jordan, M. I.; Ghahraani, Z.; Jaakkola, T.; and Saul, L. K. 1999. An introduction to variational ethods for graphical odels. Machine Learning 37(2:183 233. Koller, D., and Pfeffer, A. 1995. Generating and solving iperfect inforation gaes. In IJCAI. Milch, B., and Koller, D. 2000. Probabilistic odels for agents beliefs and decisions. Nie, J.-Y. 1992. Towards a probabilistic odal logic for seanticbased inforation retrieval. In SIGIR. ACM Press. Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systes : Networks of Plausible Inference. Morgan Kaufann. Shirazi, A., and Air, E. 2007. Probabilistic odal logic. In Proc. AAAI 07. AAAI Press. Yedidia, J.; Freean, W.; and Weiss, Y. 2004. Constructing free energy approxiations and generalized belief propagation algoriths. Zhang, N. L., and Poole, D. 1994. A siple approach to bayesian network coputations. In Proc. of the Tenth Biennial Canadian AI Conference. 547