A Comparison of Two Interval-valued Probabilistic Reasoning Methods

Transcription

1 A Comparison of Two Interval-valued Probabilistic Reasoning Methods Y.Y. Yao Department of Mathematical Sciences, Lakehead University Thunder Bay, Ontario, Canada P7B 5E1 Abstract Two complementary interval-valued probabilistic reasoning approaches, the incidence calculus proposed by Bundy and the cautious probabilistic reasoning method suggested by Quinlan, are analyzed and compared in this study. The correspondences are drawn between the sets of inference axioms employed by each model. In incidence calculus, the set of inference axioms produces the tightest probability bounds. In contrast, Quinlan s axioms do not necessarily generate the tightest probability bounds. Based on the results of such an comparison, Quinlan s axioms may be refined to produce tighter probability bounds. 1 Introduction The well established probability theory has been a dominant tool for uncertainty management. There are two fundamental issues involved, the representation of uncertainty using probability functions and inference with such information. Traditionally, a single probability function is used in which the uncertainty of a proposition is expressed by a single number. There are a number of practical problems associated with such an approach [12]. For example, it may be unrealistic to expect an expert

2 to provide precise and reliable probability values. The maintenance of consistency using a single probability function may also be difficult. To resolve these problems, various proposals have been suggested based on the notion of interval-valued probability [1, 5, 7, 11]. These proposals have resulted in many non-standard tools for uncertainty management, such as the incidence calculus [2, 3], the certainty factor model of MYCIN [14], Quinlan s method of cautious probabilistic reasoning [12], the probabilistic logic proposed by Nilsson [10], and the theory of belief and plausibility functions [8, 13, 15]. In interval-valued approaches, the probability of a proposition is not represented by a single number but by an interval. Instead of providing the exact probability value, one provides a range within which lies the true probability value of the proposition. Uncertainty of a proposition is characterized by a family of probability functions bounded by such probability intervals. Inference using interval-valued probability may be carried out in different ways. For example, one may convert probability intervals into a single probability function by using the maximum entropy principle [6]. Alternatively, one may consider probability intervals as constraints on the set of probability functions representing the uncertainty of the propositions. A probabilistic inference process is formulated as a constraint propagation [11]. An advantage of the latter approach is that no ad hoc assumption is introduced. In this paper, we present a critical and comparative study of two complementary interval-valued probabilistic reasoning approaches. One is a non-numeric approach called incidence calculus proposed by Bundy [2, 3], and the other is a numeric approach introduced by Quinlan [12]. Our analysis focuses on the inference axioms employed by these two models. We will examine the similarities between these models and the limitations of each model. The results of such a comparative study clearly indicate that on the one hand, these models are closely related to each other, although each uses entirely different notations; on the other hand, each model captures

3 different aspects of interval-valued probabilistic reasoning. Probabilistic reasoning using incidence calculus corresponds to uncertain reasoning with belief and plausibility functions which may be interpreted as a special type of interval-valued probabilities [4, 16]. In this case, one can derive the tightest probability bounds using the inference axioms of incidence calculus. In contrast, Quinlan s methods can be applied to perform more general type of interval-valued probabilistic reasoning. However, the inference axioms used by Quinlan do not necessarily produce the tightest probability bounds. By refining these axioms, it is possible to infer tighter probability bounds. 2 Non-numeric probabilistic reasoning Let Φ be a finite and non-empty set of propositions of interest. A propositional language formed from Φ is denoted by L(Φ). It is the smallest set containing the truth values (true and false), and being closed under negation ( ), conjunction ( ), disjunction ( ), and implication ( ). Let W be a non-empty set of possible worlds. It represents the states or situations of the system being modeled. Each possible world can be considered as a partial interpretation of some logical formulas in the propositional language L(Φ). With respect to a possible world w W, a proposition φ L(Φ) is either true or false. In incidence calculus, instead of using a numeric value, a subset i(φ) W is assigned to a given proposition φ L(Φ) to indicate that φ is true for all w i(φ), and φ is false for all w i(φ). The set i(φ) is referred to as the incidence set of φ. Bundy [2] suggested that an incidence mapping i : L(Φ) 2 W should satisfy the following axioms: (IC1) (IC2) i( φ) = W i(φ), i(φ ψ) = i(φ) i(ψ).

4 These axioms directly follow from the semantics of propositional logic. Axiom (IC1) says that for any situation w W, if φ is true, then φ is false. Axiom (IC2) states that for any situation w W, if both φ and ψ are true, then φ ψ is true. A mapping i : L(Φ) 2 W satisfying axioms (IC1) and (IC2) is also called an incidence structure [16]. An incidence structure has the following additional properties: (IC3) i(true) = W, (IC4) i(f alse) =, (IC5) i(φ ψ) = i(φ) i(ψ). In fact, an incidence structure can be equivalently defined by another set of axioms consisting of (IC1) and (IC5). It should be noted that axioms (IC1), (IC2) and (IC5) suggest that, with an incidence structure, logic connectives, and are interpreted in terms of set-theoretic operations on the possible worlds. Moreover, an incidence structure is truth-functional. In practice, it may be difficult to specify precisely the incidence set of a proposition. One may be able to provide only lower and upper bounds of the incidence sets of certain propositions. In the absence of any information about a proposition, one may use the trivial lower bound and the trivial upper bound W. The assignment of incidence bounds can be formally described by two mappings i : L(Φ) 2 W and i : L(Φ) 2 W. They specify an interval set [i (φ), i (φ)] within which lies the true incidence set of the proposition [17]. A set of lower and upper bounds is said to be consistent if there exists an incidence structure i such that for all φ L(Φ), i (φ) i(φ) i (φ). (1) On the other hand, if an incidence structure i satisfies equation (1), we say that i is bounded by the pair (i, i ) [16]. A consistent set of lower and upper bounds (i, i )

5 can be interpreted as constraints on incidence structures. In fact, they define the following maximal family of incidence structures: I = {i i (φ) i(φ) i (φ) for every φ L(Φ)}. (2) For the set I, a pair of bounds i 0 : L(Φ) 2 W and i 0 : L(Φ) 2 W is called the tightest bounds if every incidence structure i I is bounded by (i 0, i 0 ) and there does not exist another pair of bounds inside (i 0, i 0 ) having this property. If a pair of lower and upper bounds is consistent, the tightest bounds are unique. In many situations, incidence bounds of some propositions are not known and are represented by the trivial bounds [, W]. The initial pair (i, i ) may not necessarily be the tightest bounds. Even worse, the initial pair may be inconsistent. The main tasks of reasoning with incidence calculus are to infer information about propositions whose incidence bounds are not given, to sharpen the initial bounds, and to resolve inconsistency. Given the set of incidence structures I derived from the initial bounds i and i, the tightest bounds are given by: i 0 (φ) = i(φ), i I i 0 (φ) = i(φ). (3) i I However, they do not directly offer an efficient algorithm for the construction of the tightest bounds. Bundy [2, 3] introduced a set of inference axioms for resolving this problem. The following list is a modified version of these axioms: (I1) (I2) i (φ) i (φ) (W i ( φ)); i (φ) i (φ) (W i ( φ));

6 (I3) (I4) (I5) (I6) i (φ ψ) i (φ ψ) (i (φ) i (ψ)); i (φ ψ) i (φ ψ) i (φ) i (ψ); i (φ) i (φ) i (φ ψ); i (φ) i (φ) (i (φ ψ) (W i (ψ))). For simplicity, we only consider the primitive connectives (negation) and (conjunction). A proposition expressed by the non-primitive connectives such as (disjunction), (implication), (equivalence), and the logical constants, true and false, can be reexpressed by a normal form containing only and. The symbol is an assignment operator which assigns a new value to a lower or an upper bound based on its old value. It is interesting to note that inference rules (I1)-(I6) may also be obtained from the set-theoretic operations of the interval-set model [17]. Obviously, rules (I1)-(I6) will increase lower bounds and decrease upper bounds. These Inference rules are applied repeatedly until the values of i and i are unchanged. Wong, Wang and Yao [16] have shown that these axioms indeed produce the tightest bounds. Moreover, the tightest bounds satisfy the following important properties: i (φ ψ) = i (φ) i (ψ), i (φ ψ) = i (φ) i (ψ). (4) That is, the tightest bounds form an interval structure [16]. Probabilistic reasoning with incidence calculus is carried out using a probability function on W. Let P W denote a probability function defined on W. The probability of a proposition φ is defined using its incidence set by: P(φ) = P W (i(φ)). (5) If only lower and upper bounds of the incidence sets are given and refined used using inference rules (I1)-(I6), the corresponding lower and upper probabilities of a

7 proposition are defined by: P (φ) = P W (i (φ)), P (φ) = P W (i (φ)). (6) Using properties given in equation (4), Wong, Wang and Yao [16] have shown that the lower and upper probabilities define a pair of belief and plausibility functions [13]. Thus, the interval-valued probabilistic reasoning using incidence calculus is similar to evidential reasoning using belief functions [4, 16]. However, it is also important to point out that the interval-valued probabilistic interpretation of belief and plausibility functions is only one of the several views, and such an interpretation may not be accepted by some authors [13, 15]. 3 Numeric probabilistic reasoning Interval-based probabilistic reasoning methods have also been proposed based on a direct assignment of probability bounds and the manipulation of such bounds [1, 11]. In the light of the incidence calculus, this section examines a modified and simpler version of the approach proposed by Quinlan [12]. Our analysis suggests that the inference axioms can be refined to produce tighter probability bounds. Suppose a pair of lower and upper probabilities P (φ) and P (φ) is associated with a proposition φ to indicate the bounds of its true probability, i.e., P(φ) [P (φ), P (φ)]. We refer to [P (φ), P (φ)] as interval-valued probability of φ and P(φ) as point-valued probability of φ. If one is totally ignorant of the probability of a proposition, the trivial bounds [0, 1] can be used. The lower and upper bounds can be described by two mappings P : L(Φ) [0, 1] and P : L(Φ) [0, 1]. A pair of probability bounds (P, P ) is said to be consistent if there exists a probability

8 function P such that for all φ L(Φ), P (φ) P(φ) P (φ). (7) A consistent pair of lower and upper bounds (P, P ) can be interpreted as constraints on probability functions. They characterize the maximal family of probability functions: P = {P P (φ) P(φ) P (φ) for every φ L(Φ)}. (8) This set can be equivalently defined by the pair of tightest bounds: P 0 (φ) = inf P(φ), P P P 0 (φ) = sup P(φ). (9) P P The main task of probabilistic inference is to propagate, under the constraint set P, the probability bounds of certain propositions to other propositions whose probabilities are not available. Quinlan analyzed several types of relationships between propositions and proposed the corresponding inference axioms [12]. A subset of inference axioms related to the primitive connectives and is summarized below: (P1) (P2) P (φ) max{p (φ), 1 P ( φ)}; P (φ) min{p (φ), 1 P ( φ)}; (P3) P (φ ψ) max{p (φ ψ), P (φ) + P (ψ) 1}; (P4) (P5) (P6) P (φ ψ) min{p (φ ψ), P (φ), P (ψ)}; P (φ) max{p (φ), P (φ ψ)}; P (φ) min{p (φ), P (φ ψ) + (1 P (ψ))}.

9 They can be easily extended to more than two propositions as in Quinlan s original rules. Inference rules similar to (P1)-(P6) have been used in other interval-valued probabilistic reasoning approaches [1, 8]. The application of inference rules (P1)-(P6) will increase lower bounds and decrease upper bounds. However, they do not necessarily produce the tightest bounds. This may stem from the fact that they are derived from the following inequality: P(φ) + P(ψ) 1 P(φ ψ) min{p(φ), P(ψ)}, (10) which is an expression of bounds for P(φ ψ) based on point-valued probabilities of φ and ψ. In the case where interval-valued probabilities of φ and ψ are given, these rules can be refined to produce tighter bounds. For any two propositions φ and ψ, the following equality holds: P(φ ψ) = P(φ) + P(ψ) P(φ ψ). (11) If the interval-valued probabilities are given instead of point-valued probabilities, one can express the right hand side of equation (11) in an interval form: [P (φ), P (φ)] + [P (ψ), P (ψ)] [P (φ ψ), P (φ ψ)]. (12) The symbols + and in the above equation are interpreted as interval arithmetic operations introduced by Moore [9]: [a 1, a 2 ] + [b 1, b 2 ] = [a 1 + b 1, a 2 + b 2 ], [a 1, a 2 ] [b 1, b 2 ] = [a 1 b 2, a 2 b 1 ], (13) where [a 1, a 2 ] and [b 1, b 2 ] are two interval numbers. Therefore, the value of equation (12) can be simplified into: [P (φ) + P (ψ) P (φ ψ), P (φ) + P (ψ) P (φ ψ)], (14)

10 which gives the bounds of the probability P(φ ψ). Incorporating this information into the inference rules (P3) and (P4), we obtain the following refined rules: (P3 ) P (φ ψ) max{p (φ ψ), P (φ) + P (ψ) P (φ ψ)}; (P4 ) P (φ ψ) min{p (φ ψ), P (φ), P (ψ), P (φ) + P (ψ) P (φ ψ)}. In general, P (φ ψ) 1, hence: P (φ) + P (ψ) 1 P (φ) + P (ψ) P (φ ψ). (15) This implies that inference rule (P3 ) may produce a larger lower bound compared with the original rule (P3). By definition, rule (P4 ) may produce a smaller upper bound. Using the same technique, we obtain the refined rules corresponding to (P5) and (P6): (P5 ) P (φ) max{p (φ), P (φ ψ), P (φ ψ) + P (φ ψ) P (ψ)}; (P6 ) P (φ) min{p (φ), P (φ ψ) + P (φ ψ) P (ψ)}. Clearly, (P5 ) may produce a larger lower bound compared with rule (P5). From P (φ ψ) 1 for any φ, ψ L(Φ), one can conclude that: P (φ ψ) + 1 P (ψ) P (φ ψ) + P (φ ψ) P (ψ). (16) This implies that (P6 ) may produce a smaller upper bound than rule (P6). In summary, the refined rules (P3 )-(P6 ) may produce tighter probability bounds. These rules can also be extended to more than two propositions. However, it will be much difficult to compute. In other words, the gain in accuracy is obtained from a higher computational cost.

11 4 Comparison of the two reasoning methods The incidence calculus of Bundy and cautious probabilistic reasoning method of Quinlan represent two complementary approaches of interval-valued probabilistic reasoning. One is a non-numeric method using the notion of incidence sets to assign probability bounds indirectly, while the other is a numeric approach dealing with the direct assignment of probability bounds. Each model captures certain important aspects of probabilistic reasoning. The discussion of the last two sections clearly shows that the inference paradigms by Bundy and Quinlan are significantly similar. Both approaches are based on the same assumption that one may not use a single value to represent the uncertainty about a proposition, but can use an interval to specify certain range. In incidence calculus, incidence bounds are interpreted as constraints defining a family of incidence structures. In a similar way, probability bounds are regarded to as constraints characterizing a family of probability functions in Quinlan s model. It is assumed that incidence or probability bounds are provided only for a subset of propositions in L(Φ). It is also assumed that experts underestimate lower bounds and overestimate upper bounds. Consequently, two sets of inference rules (I1)-(I6) and (P1)-(P6) are used to refine the initial expert assignments, and to infer information about other propositions. More specifically, the inference rules increase lower bounds and decrease upper bounds. The inferred new bounds are consistent with the initial bounds. The process of such inference can be interpreted as the propagation of uncertainty under the constraints given by the initial bounds. That is, the inference patterns in both models conform to a more general paradigm known as constraint propagation [11]. Wong, Wang and Yao [16] have shown the correspondences between incidence structures and probability functions, and between interval structures (i.e., the tightest incidence bounds) and belief and plausibility functions. By applying a similar

12 argument, one can show that there is a close relationship between inference rules (I1)-(I6) and (P1)-(P6). Let P W be a probability function on W. It may be applied to inference rule (I1)-(I6) in the same way as defined by equation (6). For example, if P W is appled to the right hand side of rule (I3), from the properties of probability function, we have: P W (i (φ ψ) (i (φ) i (ψ))) max{p W (i (φ ψ)), P W (i (φ) i (ψ))} = max{p W (i (φ ψ)), P W (i (φ)) + P W (i (ψ)) P W (i (φ) i (ψ))} max{p W (i (φ ψ)), P W (i (φ)) + P W (i (ψ)) 1}; = max{p (φ ψ), P (φ) + P (ψ) 1}. (17) This is exactly the right hand side of inference rule (P3). Thus, rule (I3) may be considered as the non-numeric counterpart of rule (P3). However, it should be emphasized that such a correspondence is established based on a very restrictive assumption P W (i (φ) i (ψ)) = 1. This is not surprising as similar kind of assumption is indeed used in the derivation of inference rules (P1)-(P6). The same argument can be used to show the correspondences between other inference rules. One may therefore say that the inference rules adopted by both approaches are consistent and compatible with each other. There are differences between two approaches. An incidence structure is truthfunctional with respect to logical connectives, and, i.e., the incidence set of a composite expression can be evaluated solely from the incidence sets of its components. The use of truth-functional inference rules (I1)-(I6) seems to be appropriate. In fact, because of the truth-functionality of incidence structure, inference rules (I1)- (I6) are not only correct and but also deduce the tightest bounds. On the other hand, a probability function is not truth-functional. The adoption of truth-functional inference rules (P1)-(P6) for probability bounds may not be suitable. It is not surprising

13 that they do not infer the tightest bounds. Nevertheless, inference rules (P1)-(P6) are useful by virtue of its low computational cost. The refined inference rules (P3 )-(P6 ), which may produce tighter bounds, are not truth-functional. They seem more appropriate for interval-valued probabilistic reasoning. The main problem with inference rules (P3 )-(P6 ) is their computational complexity. The inference rules can easily become complicated with more than two propositions. It is also not clear if they will produce the tightest bounds. Given a set of possible worlds W and a probability function P W on W, an incidence structure i determines a unique probability function P on L(Φ). Conversely, given a probability function, it may be converted into different incidence structures [2]. That is, incidence sets of propositions incorporate more dependence information about the individual propositions than probabilities. Probabilistic reasoning with incidence calculus is therefore not equivalent to direct probabilistic reasoning. In fact, as mentioned in Section 2, interval-valued probabilistic reasoning using incidence calculus leads to evidential reasoning with belief and plausibility functions. In contrast, direct assignment of probability is less restrictive, which only leads to lower and upper probabilities. This makes the task of finding the tightest bounds much more difficult. 5 Conclusion In this paper, we have analyzed and compared two complementary interval-valued probabilistic reasoning approaches, the incidence calculus proposed by Bundy and the cautious probabilistic reasoning method introduced by Quinlan. Our analysis has shown that each of these approaches captures different aspects of interval-valued probabilistic reasoning. The formulation of incidence calculus is limited to a special class of interval-valued probabilities known as belief and plausibility functions. Quinlan s method, although computational efficient, fails to obtain the tightest prob-

14 ability bounds based on the available information. Using interval arithmetic, we have refined these inference rules. The new set of rules is theoretical more appropriate for reasoning about interval-valued probabilities, and can in fact produce tighter bounds. However, it may be difficult to use in practice because of its computational complexity. In the studies of Bundy and Quinlan, the notions of conditional information and inconsistency have been considered. It may be useful to extend the present study along this line and to establish a unified framework for the study of interval-valued probabilistic reasoning. Acknowledgement The author is grateful for financial support from the NSERC Canada. References [1] Bonissone, P. Plausible reasoning. Encyclopedia of Artificial Intelligence, , [2] Bundy, A. Incidence calculus: a mechanism for probabilistic reasoning. Journal of Automated Reasoning, 1, , [3] Bundy, A. Correctness criteria of some algorithms for uncertain reasoning using incidence calculus. Journal of Automated Reasoning, 2, , [4] Correa da Silva, F. and Bundy A. On some equivalence relations between incidence calculus and Dempster-Shafer theory of evidence. Proceedings of 6th International Workshop on Uncertainty in Artificial Intelligence, , 1990.

15 [5] Dempster, A.P. Upper and lower probabilities induced by a multivalued mapping. Annals of Mathematical Statistics, 38, [6] Jaynes, E.T. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus (Eds.), The Maximum Entropy Formalism, The MIT Press, Cambridge, Massachusetts, [7] Kyburg, Jr. H.E. The reference class. Philosophy of Science, 50, , [8] Lowrance, J.D. and Garvey, T.D. Evidential reasoning: a developing concept. Proceedings of the International Conference on Cybernetics and Society, 6-9, [9] Moore, R.E. Interval Analysis, Prentice-Hall, Englewood Cliffs, New Jersey, [10] Nilsson, N.J. Probabilistic logic. Artificial Intelligence, 28, 71-87, [11] Pearl, J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Mateo, California, [12] Quinlan, J.R. Inferno: a cautious approach to uncertain inference. The Computer Journal, 26, , [13] Shafer, G. Perspectives on the theory and practice of belief functions. International Journal of Approximate Reasoning, 4, , [14] Shortliffe, E.H. Computer-Based Medical Consultations: MYCIN, North- Holland, New York, 1976.

16 [15] Smets, P. Belief functions (with discussions). In P. Smets, A. Mamdani, D. Dubois, H. Prade (Eds.), Non-standard Logics for Automated Reasoning, Academic Press, New York, , [16] Wong, S.K.M., Wang, L.S., and Yao, Y.Y. Interval structures: a framework for representing uncertain information. Proceedings of Eighth Conference on Uncertainty in Artificial Intelligence, , [17] Yao, Y.Y. Interval-set algebra for qualitative knowledge representation. Proceedings of the Fifth International Conference on Computing and Information, , 1993.