Ranking Community Answers by Modeling Question-Answer Relationships via Analogical Reasoning

Size: px
Start display at page:

Download "Ranking Community Answers by Modeling Question-Answer Relationships via Analogical Reasoning"

Transcription

1 Ranking Community Answers by Modeling Question-Answer Relationships via Analogial Reasoning Xin-Jing Wang Mirosoft Researh Asia 4F Sigma, 49 Zhihun Road Beijing, P.R.China Xudong Tu,Dan Feng Huazhong Si.&Teh. Univ Luoyu Road, Wu Han Hu Bei, P.R.China Lei Zhang Mirosoft Researh Asia 4F Sigma, 49 Zhihun Road Beijing, P.R.China ABSTRACT The method of finding high-quality answers has signifiant impat on user satisfation in ommunity question answering systems. However, due to the lexial gap between questions and answers as well as spam typially existing in usergenerated ontent, filtering and ranking answers is very hallenging. Previous solutions mainly fous on generating redundant features, or finding textual lues using mahine learning tehniques; none of them ever onsider questions and their answers as relational data but instead model them as independent information. Moreover, they only onsider the answers of the urrent question, and ignore any previous knowledge that would be helpful to bridge the lexial and semanti gap. We assume that answers are onneted to their questions with various types of latent links, i.e. positive links indiating high-quality answers, negative links indiating inorret answers or user-generated spam, and propose an analogial reasoning-based approah whih measures the analogy between the new question-answer linkages and those of previous relevant knowledge whih ontains only positive links; the andidate answer whih has the most analogous link is assumed to be the best answer. We onduted experiments based on 29.8 million Yahoo!Answer question-answer threads and showed the effetiveness of our approah. Categories and Subjet Desriptors H.3.3 [Information Storage and Retrieval]: Information Searh and Retrieval information filtering, searh proess; G.3 [Probability and Statistis]: orrelation and regression analysis; H.3.5 [Information Storage and Retrieval]: Online Information Servies web-based servies General Terms Algorithms, Experimentation This work was performed in MSRA. Permission to make digital or hard opies of all or part of this work for personal or lassroom use is granted without fee provided that opies are not made or distributed for profit or ommerial advantage and that opies bear this notie and the full itation on the first page. To opy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speifi permission and/or a fee. SIGIR 09, July 19 23, 2009, Boston, Massahusetts, USA. Copyright 2009 ACM /09/07...$5.00. Keywords Community Question Answering, Analogial Reasoning, Probabilisti Relational Modeling, Ranking 1. ITRODUCTIO User-generated ontent (UGC) is one of the fastest-growing areas of the use of the Internet. Suh ontent inludes soial question answering, soial book marking, soial networking, soial video sharing, soial photo sharing, et. UGC web sites or portals providing these servies not only onnet users diretly to their information needs, but also hange everyday users from ontent onsumers to ontent reators. One type of UGC portals that has beome very popular in reent years is the ommunity question-answering (CQA) sites, whih have attrated a large number of users both seeking and providing answers to a variety of questions on diverse subjets. For example, by Deember, 2007, one popular CQA site, Yahoo!Answers, had attrated 120 million users worldwide, and had 400 million answers to questions available [21]. A typial harateristi of suh sites is that they allow anyone to post or answer any questions on any subjet, whih intuitively results in high variane in the quality of answers, e.g. UGC data typially has many ontent spam produed for fun or for profit. Thus, the ability, or inability, to obtain a high-quality answer has signifiant impat on user satisfation [28]. Distinguishing high-quality answers from others in CQA sites is not a trivial task, e.g. a lexial gap typially exists between a question and its high-quality answers. The lexial gap in ommunity questions and answers is aused by at least two fators: (1) textual mismath between questions and answers; and (2) user-generated spam or flippant answers. In the first ase, questions and answers are generally short, and the words that appear in a question are not neessarily repeated in its high-quality answers. Moreover, a word itself an be ambiguous or have multiple meanings, e.g., apple an either refer to apple omputer or apple the fruit. Meanwhile, the same onept an be desribed with different words, e.g. ar and automobile. In the seond ase, user-generated spam and flippant answers usually have a negative effet and greatly inrease the number of answers, thereby make it diffiult to identify the highquality ones. Figure 1 gives several examples of the lexial gap between questions and answers. To bridge the lexial gap for better answer ranking, various tehniques have been proposed. Conventional teh- 179

2 niques for filtering answers primarily fous on generating omplementary features provided by highly strutured CQA sites [1, 4, 5, 14, 22], or finding textual lues using mahinelearning tehniques [2, 3, 19, 20], or identifying user authority via graph-based link analysis whih assumes that authoritative users tend to generate high-quality answers [17]. There are two disadvantages of these work. Firstly, only the answers of the new question are taken into onsideration. Intuitively it suffers greatly from the word ambiguity sine questioners and answerers may use different words to desribe the same objets (e.g. ar and automobile ). Seondly, questions and answers are assumed as independent information resoures and their impliit orrelations are ignored. We argue that questions and answers are relational. Reall the traditional QA approahes based on natural language proessing (LP) tehniques [25] whih attempted to disover natural language properties suh as targeted answer format, targeted answer part-of-speeh, et., and used them as lues to figure out right answers. These are examples of impliit semanti lues, whih is more valuable for right answer detetion than the lexial lues suggested by terms. We address these two problems in this study. In the offline stage, a large arhive of questions and their answers are olleted as a knowledge base. We then assume that there are various types of latent linkages between questions and answers, e.g. the high-quality answers are onneted to their questions via semanti links, the spam or flippant answers are via noisy links, while low-quality or inorret answers are through inferior links, et. We denote the links assoiated with high-quality answers as positive links and the rest as negative ones and train a Bayesian logisti regression model for link predition. By this means we are able to expliitly model the impliit q-a relationships. In the online stage, given a new question, we retrieve a set of relevant q-a pairs from the knowledge base with the hope that they would over the voabulary of the new question and its answers. These q-a pairs onstrut the prior knowledge on a similar topi to help bridge the lexial gap. Moreover, to disover the semanti lues, instead of prediting diretly based on the new question and its answers, we measure the analogy of a andidate linkage to the links embedded in the prior knowledge, and the most analogous link indiates the best answer. This is what we all analogial reasoning(ar). Intuitively, taking an ideal ase as an example, if the rawled knowledge base ontains all questions in the world, to identify the best answer for a new question, we an just find the dupliate question in the knowledge base and use its best answer to evaluate a andidate answer s quality. However, sine it is impratial to obtain all available questions as new questions are being submitted every day, we an instead searh for a set of previously answered questions that best math the new question aording to some speified metris and disover analogous relationships between them. Intuitively, the retrieved similar questions and their bestanswers are more likely to have words in ommon with the orret answers to the new question. Moreover, the retrieved set provides the knowledge of how the questions similar to the new question were answered, while the way of answering an be regarded as a kind of positive linkage between a question and its best answer. We rawled 29.8 million Yahoo!Answers questions to evaluate our proposed approah. Eah question has about 15 answers on average. We used 100,000 q-a pairs to train Q: How do you pronoune the Congolese ity Kinshasa? A: Kin as in your family sha as in sharp sa as in sargent. Q: What is the minimum positive real number in Matlab? A: Your IQ. Q: How will the sun enigma affet the earth? A: Only God truly knows the mysteries of the sun, the universe and the heavens. They are His serets! Figure 1: Real examples from Yahoo!Answers, whih suggest the lexial gap between questions and answers. the link predition model and tested the entire proess with about 200,000 q-a pairs. We ompared our method with two widely adopted information retrieval metris and one stateof-the-art method, and ahieved signifiant performane improvement in terms of average preision and mean reiproal rank. These experimental results suggest that taking the struture of relational data into aount is very helpful in the noisy environment of a CQA site. Moreover, the idea of leveraging previous knowledge to bridge the lexial gap and ranking a doument with ommunity intelligene is more effetive than traditional approahes whih only rely on individual intelligene. The paper is organized as follows. In Setion 2, the most related work is disussed, whih overs the state-of-the-art approahes on ommunity-driven answer ranking. In Setion 3, we detail the proposed approah. We evaluate its performane in Setion 4, and disuss several fators whih affet the ranking performane. We onlude our paper in Setion 5 with disussions on future work. 2. RELATED WORK 2.1 Community Answer Quality Ranking Different from traditional QA systems whose goal is to automatially generate an answer for the question of interest [25, 16], the goal of ommunity answer ranking, however, is to identify from a losed list of andidate answers one or more that semantially answers the orresponding question. Sine CQA sites have rih strutures, e.g. question/answer assoiations, user-user interations, user votes, et., they offer more publily available information than traditional QA domains. Jeon et al. [14] extrated a number of non-textual features whih over the ontextual information of questions and their answers, and proposed a language modeling-based retrieval model for proessing these features in order to predit the quality of answers olleted from a speifi CQA servie. Agihtein and his olleagues [1, 23] made great efforts on finding powerful features inluding strutural, textual, and ommunity features, and proposed a general lassifiation framework for ombining these heterogeneous features. Meanwhile, in addition to identifying answer quality, they also evaluated question quality and users satisfation. Blooma et al. [5] proposed more features, textual and nontextual, and used regression analyzers to generate preditive features for the best answer identifiation. Some researhers resorted to mahine learning tehniques. Ko et al. [19] proposed a unified probabilisti answer rank- 180

3 Figure 2: Sketh of the AR-based approah: 1)given a question Q (green blok), some previous positive q-a pairs are retrieved (red dots in dotted blue irle); 2) eah andidate q-a pair is sored aording to how well it fits into the previous knowledge w.r.t. their linkages properties. The magenta blok highlights the highest-sored q-a pair andidate, whih is assumed to ontain the best answer. ing model to simultaneously address the answer relevane and answer similarity problems. The model used logisti regression to estimate the probability that a andidate answer is orret given its relevane to the supporting evidene. A disadvantage of this model is that it onsidered eah andidate answer separately. To solve this problem, the authors improved their solution and proposed a probabilisti graphial model to take into aount the orrelation of andidate answers [20]. It estimated the joint probability of the orretness of all answers, from whih the probability of orretness of an individual answer an be inferred, and the performane was improved. Bian et al. [3] presented the GBRank algorithm whih utilizes users interations to retrieve relevant high-quality ontent in soial media. It explored the mehanism to integrate relevane, user interation, and ommunity feedbak information to find the right fatual, well-formed ontent to answer a user s question. Then they improved the approah by expliitly onsidering the effet of maliious users interations, so that the ranking algorithm is more resilient to vote manipulation or shilling [2]. Instead of diretly solving the answer ranking problem, some researhers proposed to find experts in ommunities with the assumption that authoritative users tend to produe high quality ontent. For example, Jurzyk et al. [17] adapted the HITS [18] algorithm to a CQA portal. They ran this algorithm on the user-answer graph extrated from online forums and showed promising results. Zhang et al. [30] further proposed ExpertiseRank to identify users with high expertise. They found that the expertise network is highly orrelated to answer quality. 2.2 Probabilisti Relational Modeling Probabilisti Relational Models [11] are Bayesian etworks whih simultaneously onsider the onepts of objets, their properties, and relations. Getoor et al. [10] inorporated models of link predition in relational databases, and we adopt the same idea to learn the logisti regression model for link predition. 3. THE APPROACH 3.1 Proess Overview Figure 2 shows the entire proess of our approah. The red dots in the CQA Arhives stand for positive q-a pairs whose answers are good, while the blak rosses represent negative pairs whose answers are not good (not neessarily noisy). Eah q-a pair is represented by a vetor of textual and non-textual features as listed in Table 2. In the offline stage, we learn a Bayesian logisti regression model based on the rawled QA arhive, taking into aount both positive and negative q-a pairs. The task of the model is to estimate how likely a q-a pair ontains a good answer. In the online stage, a supporting set (enlosed by the dotted blue irle in Figure 2) of positive q-a pairs is first retrieved from the CQA arhives using only the new question Q (the green blok) as a query. The supporting set, along with the learnt link predition model, is used for soring and ranking eah new q-a pair, and the top-ranked q-a pair (the magenta blok) ontains the best answer. Some real q-a examples are given to better illustrate the idea. 3.2 Learning the Link Predition Model Modeling latent linkage via logisti regression We train a Bayesian logisti regression (BLR) model with finite-dimensional parameters for latent linkage predition and set multivariate Gaussian priors for the parameters. The advantage of introduing a prior is that it helps to integrate over funtion spaes. Formally, let X ij = [Φ 1(Q i, A j ), Φ 2(Q i, A j ),..., Φ K(Q i, A j )] be a K-dimensional feature vetor of the pair of question Q i and answer A j, where Φ defines the mapping Φ : Q A R K. Let C ij {0, 1} be an indiator of linkage types, where C ij = 1 indiates a positive linkage, i.e. A j is a high-quality answer of Q i, and C ij = 0 otherwise. Let Θ = [Θ 1, Θ 2,..., Θ K] be the parameter vetor of the logisti regression model to be learnt, i.e. P(C ij = 1 X ij, Θ) = exp(θ T X ij ) (1) 181

4 Figure 3: (a) The plate model of the Bayesian logisti regression model. (b) When C = 1, Q and A is positively linked whih shares information embedded in Θ, represented by the undireted edges. where X ij X train = {X ij, 1 i D q, 1 j D a} is a training q-a pair and D q, D a are the number of training questions and answers respetively. Generally we have D a D q. Figure 3(a) shows the plate model. The C node is introdued to expliitly model the fators that link a question Q and an answer A. Sine C represents a link while Q and A represent ontent, this model seamlessly reinfores the ontent and linkage in relational data. If C = 1 is observed, as shown in Figure 3(b), it means that there exists a positive link between the orresponding Q and A, and this link is analogous to links whih onstrut Θ. We use undireted edges to represent these impliations. To ahieve a more disriminative formulation, we use both positive q-a pairs and negative q-a pairs to train this model. Meanwhile, sine noisy answers generally oupy a larger population, to balane the number of positive and negative training data, we randomly sample a similar number of negative and positive points Learning the Gaussian prior P(Θ) The reason that we use a Gaussian prior is threefold: Firstly, instead of diretly using the BLR model s outputs, we evaluate how probably the latent linkage embedded in a new q-a pair belongs to the subpopulation defined by the previous knowledge, with respet to the BLR predition (we detail this approah in Setion 3.4.2). Therefore Gaussian is a good prior andidate. Seondly, Gaussian distribution has a omparatively small parameter spae (mean and variane), whih makes the model easier for ontrol. Thirdly, Gaussian distribution is onjugate to itself, i.e. the posterior is still a Gaussian, whih results in a simple solution. To ensure the preiseness of the prior, we adopt the approah suggested by Silva et al.[26]. Firstly the BLR is fit to the training data using Maximum Likelihood Estimation (MLE), whih gives an initial ˆΘ. Then the ovariane matrix ˆΣ is defined as a smoothed version of the MLE estimated ovariane: (ˆΣ) 1 = (XT ŴX) (2) where is a salar; is the size of the training data set; X = {X ij } is the K feature matrix of the training q-a pairs, either positive or negative. Ŵ is a diagonal matrix with Ŵ ii = ˆp(i) (1 ˆp(i)), where ˆp(i) is the predited probability of a positive link for the ith row of X. The prior for Θ is then the Gaussian (ˆΘ, ˆΣ). 3.3 Previous Knowledge Mining To our knowledge, none of previous CQA answer ranking approahes ever leveraged a supporting set. In fat, suh an idea is advantageous in at least two aspets: 1) bridging the lexial gap: it enlarges the voabulary, whih is more likely to over the words not appearing in a question but in its orret answers or vie versa. In fat, some traditional QA approahes have disovered that the amount of impliit knowledge whih assoiates an answer to a question an be quantitatively estimated by exploiting the redundany in a large data set [6, 24]. 2) bridging the semanti gap: its positive linkages enable the analogy reasoning approah for new link predition. Intuitively, it is more advantageous to rank a doument based on ommunity intelligene than simply on individual intelligene [7, 12]. We use information retrieval tehniques to identify suh a supporting set. However, ommunity q-a pairs retrieval again is not a trivial task. Jeon and his olleagues [15] proposed a translation-based retrieval model using the textual similarity between answers to estimate the relevane of two questions. Xue et al [29], furthermore, ombined the wordto-word translation probabilities of question-to-question retrieval and answer-to-answer retrieval. Lin et al. [22] and Bilotti et al. [4], on the other hand, adopted the traditional information retrieval method but utilized strutural features to ensure retrieval preision. We adopt Lin and Bilotti s way for q-a pair retrieval for simpliity. In partiular, let Q q be a new question and its answer list be {A j q, 1 j M}, the supporting q-a pairs S = {(Q 1 : A 1 ), (Q 2 : A 2 ),..., (Q L : A L )} ontain those whose questions osine similarities to the new question are above a threshold: S = {(Q i : A i ) os(q q, Q i ) > λ, i 1,..., D} (3) where D is the size of our rawled Yahoo!Answer arhive and λ is the threshold. Eah question is represented in the bag-of-word model. The effet of λ is shown in Figure 5. As analyzed before, whether the retrieved questions are semantially similar to the new question is not ritial. This is beause the inferene in our ase is based on the strutures of q-a pairs rather than on the ontents. Moreover, aording to the exhaustive experiments onduted by Dumais et al. [8], when the knowledge base is large enough, the auray of question answering an be promised with simple doument ranking and n-gram extration tehniques. ote that S ontains only positive q-a pairs. Therefore if the linkage of a andidate q-a pair is predited as analogous to the linkages in the subpopulation S, we an say that it is a positive linkage whih indiates a high-quality answer. 3.4 Link Predition The idea of analogial reasoning There is a large literature on analogial reasoning in artifiial intelligene and psyhology, whih ahieved great suess in many domains inluding lustering, predition, and dimensionality redution. Interested readers an refer to Frenh s survey [9]. However, few previous work ever applied analogial reasoning onto IR domain. Silva et al. [27] used it to model latent linkages among the relational objets ontained in university webpages, suh as student, ourse, department, staff, et., and obtained promising result. An analogy is defined as a measure of similarity between strutures of related objets (q-a pairs in our ase). The key aspet is that, typially, it is not so important how eah individual objet of a andidate pair is similar to individual 182

5 objets of the supporting set (i.e. the relevant previous q-a pairs in our ase). Instead, implementations that rely on the similarity between the pairs of objets will be used to predit the existene of the relationships. In other words, similarities between two questions or two answers are not as important as the similarity between two q-a pairs. Silva et al. [27] proposed a Bayesian analogial reasoning (BAR) framework, whih uses a Bayesian model to evaluate if an objet belongs to a onept or a luster, given a few items from that luster. We use an example to better illustrate the idea. Consider a soial network of users, there are several diverse reasons that a user u links to a user v: they are friends, they joined the same ommunities, or they ommented the same artiles, et. If we know the reasons, we an group the users into subpopulations. Unfortunately, these reasons are impliit; yet we are not ompletely in the dark: we are already given some subgroups of users whih are representative of a few most important subpopulation, although it is unlear what reasons are underlying. The task now beomes to identify whih other users belong to these subgroups. Instead of writing some simple query rules to explain the ommon properties of suh subgroups, BAR solves a Bayesian inferene problem to determine the probability that a partiular user pair should be a member of a given subgroup, or say they are linked in an analogous way Answer ranking via analogial reasoning The previous knowledge retrieved is just suh a subpopulation to predit the membership of a new q-a pair. Sine we keep only positive q-a pairs in this supporting set and high-quality answers generally answer a question semantially rather than lexially, it is more likely that a andidate q-a pair ontains a high-quality answer when it is analogous to this supporting q-a set. To measure suh an analogy, we adopt the soring funtion proposed by Silva et al. [27] whih measures the marginal probability that a andidate q-a pair (Q q, A j q) belongs to the subpopulation of previous knowledge S: sore(q q, A j q) = log P(C j q = 1 X j q,s,c S = 1) log P(C j = 1 X j q)) where A j q is the j-th andidate answer of the new question. X j q represents the features of (Q q, A j q). C S is the vetor of link indiators for S, and C S = 1 indiates that all pairs in S is positively linked, i.e. C 1 = 1, C 2 = 1,..., C L = 1. The idea underlying is to measure to what extent (Q q, A j q) would fit into S, or to what extent S explains (Q q, A j q). The more analogous it is to the supporting linkages, the more probability the andidate linkage is positive. Aording to the Bayes Rule, the two probabilities in Eq.(4) an be solved by Eq.(5) and Eq.(6) respetively: P(C j q = 1 X j q,s,c S = 1) = P(C j = 1 X j q, Θ)P(Θ S,C S = 1)dΘ (4) (5) P(Cq j = 1 Xq) j = P(C j = 1 Xq, j Θ)P(Θ)dΘ (6) where Θ is the parameter set. P(C j = 1 X j q, Θ) is given by the BLR model defined in Eq.(1). The solution of these two equations is given in the appendix. The entire proess is shown in Table 1. Table 1: Summary of our AR-based method I. Training Stage: Input: feature matrix X train of all the training q-a pairs Output: BLR model parameters Θ and its prior P(Θ). Algorithm: 1. Train BLR model on X train using Eq.(1). 2. Learn Prior P(Θ) using Eq.(2). II. Testing Stage: Input: a query QA thread: {Q q : A i q} M i=1 Output: sore(q q, A i q), i = 1,..., M Algorithm: 1. Generate the feature matrix X test = {X j q } with X j q the features of the j-th q-a pair (Q q : A j q); 2. Retrieve a set of positive q-a pairs S from the CQA arhive using Eq.(3); 3. Do Bayesian inferene to obtain P(Θ S, C S = 1); 4. For eah (Q q, A j q), estimate the probability of a positive linkage using Eq.(4); 5. Ranking the answers {A j q, j = 1,..., M} in desending order of the sores. The top-ranked answer is assumed as the best answer of Q q. 4. EVALUATIO We rawled 29.8 million questions from the Yahoo!Answers web site; eah question has answers on average. These questions over 1,550 leaf ategories defined by expert and all of them have user-labeled best answers, whih is a good test bed to evaluate our approah. 100,000 randomly seleted q-a pairs were used to train the link predition model, and 16,000 q-a threads were used for testing eah ontains answers on average, whih resulted in about 200,000 testing q-a pairs. A typial harateristi of CQA sites is its rih struture whih offers abundant meta-data. Previous work have shown the effetiveness of ombining strutural features with textual features [1, 14, 23]. We adopted a similar approah and defined about 20 features to represent a q-a pair, as listed in Table 2. Two metris were used for the evaluation. One is Average For a given query, it is the mean fration of relevant answers ranked in the top K results; the higher the preision, the better the performane is. We use the best answer tagged by the Yahoo!Answers web site as the ground truth. Sine average preision ignores the exat rank of a orret answer, we use the Mean Reiproal Rank (MRR) metri for ompensation. The MRR of an individual query is the reiproal of the rank at whih the first relevant answer is returned, or 0 if none of the top K results ontain a relevant answer. The sore for a set of queries is the mean of eah individual query s reiproal ranks: MRR = 1 1 (7) Q r r q q Q r 183

6 Table 2: The BAR Algorithm in CQA Formulation Textual Features of Questions-Answers Pairs Q.(A.) TF Question (Answer) term frequeny, stopwords removed, stemmed ovel word TF Term frequeny of non-ditionary word, e.g. ms #Common-words umber of ommon words in Q and A Statistial Features of Questions-Answers Pairs Q.(A.) raw length umber of words, stopwords not removed Q/A raw length ratio Question raw length / answer raw length Q/A length ratio Q/A length ratio, stopword removed Q/A anti-stop ratio #stopword-inquestion/#stopword-in-answer Common n-gram len. Length of ommon n-grams in Q and A #Answers Answer position umber of answers to a question The position of an answer in the q-a thread User interation / Soial Elements Features #Interesting mark umber of votes mark a question as interesting Good mark by users umber of votes mark an answer as good Bad mark by users umber of votes mark an answer as bad Rating by asker The asker-assigned sore to an answer Thread life yle The time span between Q and its latest A where Q r is the set of test queries; r q is the rank of the first relevant answer for the question q. Three baseline methods were used: the earest eighbor Measure (), the Cosine Distane Metri (COS), and the Bayesian Set Metri (BSets)[12]. The first two diretly measure the similarity between a question and an answer, without using a supporting set; neither do they treat questions and answers as relational data but instead as independent information resoures. The BSets model uses the soring funtion Eq.(8): sore(x) = log p(x S) log p(x) (8) where x represents a new q-a pair and S is the supporting set. It measures how probably x belongs to S. The differene of BSets to our AR-based method is that the former ignores the q-a orrelations, but instead join the features of a question and an answer into a single row vetor. 4.1 Performane Average Figure 4 illustrates the performane of our method and the baselines with the Average metri when K = 1, 5, 10. Sine eah question has less than 15 answers on average, the average preision at K > 10 is not evaluated Our BSets COS K=1 K=5 K=10 Figure 4: Average Our method signifiantly outperformed the baselines. Table 3: MRR for Our Method and Baselines Method MRR Method MRR 0.56 Cosine 0.59 BSets [12] 0.67 Our Method 0.78 The red bar shows the performane of our approah. The blue, yellow, and purple bars are of the BSets, COS and methods respetively. In all ases, our method signifiantly out-performed the baselines. The gap between our method and BSets shows the positive effet of modeling the relationships between questions and answers, while the gap between BSets and, COS shows the power of ommunity intelligene. The superiority of our model to BSets shows that modeling ontent as well as data strutures improves the performane than modeling only ontent; this is beause in CQA sites, questions are very diverse and the retrieved previous knowledge are not neessarily semantially similar to the query question (i.e. they are still noisy). Moreover, from the experimental result that method performed the worst, we an tell that the lexial gap between questions and answers, questions and questions, and q-a pairs annot be ignored in the Yahoo!Answers arhive Mean Reiproal Rank (MRR) Table 3 gives the MRR performane of our method and the baselines. The trend oinides with the one that is suggested by the Average measure. Again our method signifiantly out-performed the baseline methods, whih means that generally the best answers rank higher than other answers in our method. 4.2 Effet of Parameters Figure 5 evaluates how our method performs with the two parameters:, the salar in Eq.(2), and λ, the threshold in Eq.(3). MRR metri is used here. Figure 5(a) shows the joint effet of these two parameters on the MRR performane. The best performane was ahieved when λ = 0.8, = 0.6. To better evaluate the individual effet of the parameters, we illustrate the urve of MRR vs. λ and MRR vs. in Figure 5(b) and Figure 5() respetively by fixing the other to its optimal value. As desribed in Setion 3.3, the threshold λ ontrols the relevane of the supporting q-a pair set. Intuitively, if λ is set too high, few q-a pairs will be retrieved. This auses too small a subpopulation of the supporting q-a pairs suh that the linkage information beomes too sparse and thus is inadequate to predit the analogy of a andidate. On the 184

7 MRR 0.4 MRR delta λ / (a) The effet of and λ on the performane with MRR metri (b) The trend of performane on MRR vs. λ. = 0.6. () The trend of performane on MRR vs.. λ = 0.8. Figure 5: The effet of the prior salar in Eq.(2) and the similarity threshold λ in Eq.(3). The Mean Reiproal Rank metri was used. The best performane was ahieved at λ = 0.8, = 0.6. other hand, if λ is set too low, likely too many q-a pairs will be retrieved whih will introdue too muh noise into the subpopulation. Therefore the semanti meaning of the supporting links is obsured. Figure 5(b) exatly reflets suh a ommon sense. The best performane was ahieved when λ was about 0.8. After that the performane dropped quikly. However, before λ was inreased to 0.8, the performane improved slowly and smoothly. This indiates that the noisy q-a pairs orresponding to diverse link semantis were removed gradually and the subpopulation was getting more and more foused, thus strengthened the power of analogial reasoning. is used to sale the ovariane matrix of the prior P(Θ). As shown in Figure 5(), it is a sensitive parameter and is stable in a omparatively narrow range (i.e. about ). This, intuitively, oinides with the property of analogial reasoning approahes [9, 27, 26], where a datadependent prior is quite important for the performane. In our approah, we used the same prior for the entire testing dataset whih ontains q-a pairs from diverse ategories. A better performane an be foreseen if we learn different priors for different ategories. 5. COCLUSIO A typial harateristi of ommunity question answering sites is the high variane of the quality of answers, while a mehanism to automatially detet a high-quality answer has a signifiant impat on users satisfation with suh sites. The typial hallenge, however, lies in the lexial gap aused by textual mismath between questions and answers as well as user-generated spam. Previous work mainly fouses on deteting powerful features, or finding textual lues using mahine learning tehniques, but none of them ever took into aount previous knowledge and the relationships between questions and their answers. Contrarily, we treated questions and their answers as relational data and proposed an analogial reasoning-based method to identify orret answers. We assume that there are various types of linkages whih attah answers to their questions, and used a Bayesian logisti regression model for link predition. Moreover, in order to bridge the lexial gap, we leverage a supporting q-a set whose questions are relevant to the new question and whih ontain only high-quality answers. This supporting set, together with the logisti regression model, is used to evaluate 1) how probably a new q-a pair has the same type of linkages as those in the supporting set, and 2) how strong it is. The andidate answer that has the strongest link to the new question is assumed as the best answer that semantially answers the question. The evaluation on 29.8 million Yahoo!Answers q-a threads showed that our method signifiantly out-performed the baselines both in average preision and in mean reiproal rank, whih suggests that in the noisy environment of a CQA site, leveraging ommunity intelligene as well as taking the struture of relational data into aount are benefiial. The urrent model only uses ontent to reinfore strutures of relational data. In the future work, we would like to investigate how latent linkages reinfore ontent, and vie versa, with the hope of improved performane. 6. ACKOWLEDGEMET We thank Chin-Yew Lin and Xinying Song s help on the q-a data. 7. REFERECES [1] E. Agihtein, C. Castillo, and et. Finding high-quality ontent in soial media. In Pro. of WSDM, [2] J. Bian, Y. Liu, and et. A few bad votes too many? towards robust ranking in soial media. In Pro. of AIRWeb, [3] J. Bian, Y. Liu, and et. Finding the right fats in the rowd: Fatoid question answering over soial media. In Pro. of WWW, [4] M. Bilotti, P. Ogilvie, and et. Strutured retrieval for question answering. In Pro. of SIGIR, [5] M. Blooma, A. Chua, and D. Goh. A preditive framework for retrieving the best answer. In Pro. of SAC, [6] E. Brill, J. Lin, M. Banko, and et. Data-intensive question answeringa. In TREC, [7] J. Chu-Carroll, K. Czuba, and et. In question answering, two heads are better than one. In Pro. of HLT/AACL, [8] S. Dumais, M. Banko, and et. Web question answering: Is more always better? In Pro. of SIGIR,

8 [9] R. Frenh. The omputational modeling of analogy-marking. Trends in ognitive Sienes, 6, [10] L. Getoor,. Friedman, and et. Learning probabilisti models of link struture. JMLR, 3: , [11] L. Getoor,. Friedman, and et. Probabilisti relational models. Introdution to Statistial Relational Learning, [12] Z. Ghahramani and K. Heller. Bayesian sets. In Pro. of IPS, [13] T. Jaakkola and M. Jordan. Bayesian parameter estimation via variational methods. Statistis and Computing, 10:25 37, [14] J. Jeon, W. Croft, and et al.. a framework to predit the quality of answers with non-textual features. In Pro. of SIGIR, [15] J. Jeon, W. Croft, and et. Finding similar questions in large question and answer arhives. In Pro. of CIKM, [16] V. Jijkoun and M. Rijke. Retrieving answers from frequently asked questions pages on the web. In Pro. of CIKM, pages 76 83, [17] P. Jurzyk and E. Agihtein. Disovering authorities in question answer ommunities by using link analysis. In Pro. of CIKM, [18] J. Kleinberg. Authoritative soures in a hyperlinked environment. Journal of the ACM, 46(5): , [19] J. Ko, L. Si, and E. yberg. A probabilisti framework for answer seletion in question answering. In Pro. of AACL/HLT, [20] J. Ko, L. Si, and E. yberg. A probabilisti graphial model for joint answer ranking in question answering. In Pro. of SIGIR, [21] J. Leibenluft. Librarianaŕs worst nightmare: Yahoo!answers, where 120 million users an be wrong. Slate Magazine, [22] J. Lin and B. Katz. Question answering from the web using knowledge annotation and knowledge mining tehniques. In Pro. of CIKM, [23] Y. Liu, J. Bian, and E. Agihtein. Prediting information seeker satisfation in ommunity question answering. In Pro. of SIGIR, pages , [24] B. Magnini and M. e. a. egri. Is it the right answer? exploiting web redundany for answer validation. In Pro. of ACL, pages , [25] D. Molla and J. Viedo. Question answering in restrited domains: An overview. In Pro. of ACL, [26] R. Silva, E. Airoldi, and K. Heller. Small sets of interating proteins suggest latent linkage mehanisms through analogial reasoning. In Gatsby Tehnial Report, GCU TR , [27] R. Silva, K. Heller, and Z. Ghahramani. Analogial reasoning with relational bayesian sets. In Pro. of AISTATS, [28] Q. Su, D. Pavlov, and et. Internet-sale olletion of human-reviewed data. In Pro. of WWW, pages , [29] X. Xue, J. Jeon, and W. Croft. Retrieval models for question and answer arhives. In Pro. of SIGIR, pages , [30] J. Zhang, M. Akerman, and L. Adami. Expertise networks in online ommunities: Struture and algorithms. In Pro. of WWW, pages , APPEDIX Jaakkola et al. [13] suggested a variational approximation solution to the BLR model. Let g(ξ) be the logisti funtion, g(ξ) = (1 + e ξ ) 1, and onsider the ase for the single data point evaluation, Eq.(6), the method lower-bounds the integrand as follows: P(C X, Θ) = g(θ T X) g(ξ) exp{ HC ξ 2 λ(ξ)(h 2 C ξ 2 )} where H C = (2C 1)Θ T X and λ(ξ) = tanh( ξ 2 ). tanh( ) is 4ξ the hyperboli tangent funtion. Thus P(Θ X, C) an be approximated by normalizing P(C X, Θ)P(Θ) Q(C X, Θ)P(Θ) where Q(C X, Θ) is the right-hand side of Eq.(9). Sine this bound assumes a quadrati form as a funtion of Θ and our priors are Gaussian, the approximate posterior will be Gaussian, whih we denote by (µ pos, Σ pos). However, this bound an be loose unless a suitable value for the free parameter ξ is hosen. The key step in the approximation is then to optimize the bound with respet to ξ. Let the Gaussian prior P(Θ) be denoted as (µ, Σ). The proedure redues to an iterative optimization algorithm where for eah step the following updates are made: Σ 1 pos = Σ 1 + 2λ(ξ)XX T µ pos = Σ 1 pos[σ 1 µ + (C 1 2 )X] ξ = (X T Σ posx + (X T µ pos) 2 ) 1 2 To approximate Eq.(5), a sequential update is performed: starting from the prior P(Θ) for the first data point (X, C) in (S,C S = 1), the resulting posterior (µ pos, Σ pos) is treated as the new prior for the next point. The ordering is hosen from an uniform distribution in our implementation. Finally, given the optimized approximate posterior, the preditive integral Eq.(5) an be approximated as: log(q(c ij X ij,s,c S )) = log g(ξ ij) ξij 2 + λ(ξij)ξ2 ij 1 2 µt SΣ 1 S µ T S µt ijσ 1 ij µ T ij log Σ 1 ij Σ 1 S where parameters ( S, Σ S ) are the ones in the approximate posterior Θ (S,C S ) (µ S, Σ S ), and (µ ij, Σ ij) ome from the approximate posterior Θ (S,C S, X ij, C ij ). Parameters (ξ ij, ξ S) ome from the respetive approximate posteriors. And Eq.(6) an be approximated as: log Q(C ij X ij ) = log g(ξ ij) xiij 2 + λ(ξij)ξ2 ij (9) 186

Ranking Community Answers by Modeling Question-Answer Relationships via Analogical Reasoning

Ranking Community Answers by Modeling Question-Answer Relationships via Analogical Reasoning Ranking Community Answers by Modeling Question-Answer Relationships via Analogical Reasoning Xin-Jing Wang Microsoft Research Asia 4F Sigma, 49 Zhichun Road Beijing, P.R.China xjwang@microsoft.com Xudong

More information

A Holistic Method for Selecting Web Services in Design of Composite Applications

A Holistic Method for Selecting Web Services in Design of Composite Applications A Holisti Method for Seleting Web Servies in Design of Composite Appliations Mārtiņš Bonders, Jānis Grabis Institute of Information Tehnology, Riga Tehnial University, 1 Kalu Street, Riga, LV 1658, Latvia,

More information

Hierarchical Clustering and Sampling Techniques for Network Monitoring

Hierarchical Clustering and Sampling Techniques for Network Monitoring S. Sindhuja Hierarhial Clustering and Sampling Tehniques for etwork Monitoring S. Sindhuja ME ABSTRACT: etwork monitoring appliations are used to monitor network traffi flows. Clustering tehniques are

More information

Subordinating to the Majority: Factoid Question Answering over CQA Sites

Subordinating to the Majority: Factoid Question Answering over CQA Sites Journal of Computational Information Systems 9: 16 (2013) 6409 6416 Available at http://www.jofcis.com Subordinating to the Majority: Factoid Question Answering over CQA Sites Xin LIAN, Xiaojie YUAN, Haiwei

More information

Recommending Questions Using the MDL-based Tree Cut Model

Recommending Questions Using the MDL-based Tree Cut Model WWW 2008 / Refereed Trak: Data Mining - Learning April 2-25, 2008 Beijing, China Reommending Questions Using the MDL-based Tree Cut Model Yunbo Cao,2, Huizhong Duan, Chin-Yew Lin 2, Yong Yu, and Hsiao-Wuen

More information

' R ATIONAL. :::~i:. :'.:::::: RETENTION ':: Compliance with the way you work PRODUCT BRIEF

' R ATIONAL. :::~i:. :'.:::::: RETENTION ':: Compliance with the way you work PRODUCT BRIEF ' R :::i:. ATIONAL :'.:::::: RETENTION ':: Compliane with the way you work, PRODUCT BRIEF In-plae Management of Unstrutured Data The explosion of unstrutured data ombined with new laws and regulations

More information

Chapter 1 Microeconomics of Consumer Theory

Chapter 1 Microeconomics of Consumer Theory Chapter 1 Miroeonomis of Consumer Theory The two broad ategories of deision-makers in an eonomy are onsumers and firms. Eah individual in eah of these groups makes its deisions in order to ahieve some

More information

Weighting Methods in Survey Sampling

Weighting Methods in Survey Sampling Setion on Survey Researh Methods JSM 01 Weighting Methods in Survey Sampling Chiao-hih Chang Ferry Butar Butar Abstrat It is said that a well-designed survey an best prevent nonresponse. However, no matter

More information

A Survey of Usability Evaluation in Virtual Environments: Classi cation and Comparison of Methods

A Survey of Usability Evaluation in Virtual Environments: Classi cation and Comparison of Methods Doug A. Bowman bowman@vt.edu Department of Computer Siene Virginia Teh Joseph L. Gabbard Deborah Hix [ jgabbard, hix]@vt.edu Systems Researh Center Virginia Teh A Survey of Usability Evaluation in Virtual

More information

BUILDING A SPAM FILTER USING NAÏVE BAYES. CIS 391- Intro to AI 1

BUILDING A SPAM FILTER USING NAÏVE BAYES. CIS 391- Intro to AI 1 BUILDING A SPAM FILTER USING NAÏVE BAYES 1 Spam or not Spam: that is the question. From: "" Subjet: real estate is the only way... gem oalvgkay Anyone an buy real estate with no

More information

Improved Vehicle Classification in Long Traffic Video by Cooperating Tracker and Classifier Modules

Improved Vehicle Classification in Long Traffic Video by Cooperating Tracker and Classifier Modules Improved Vehile Classifiation in Long Traffi Video by Cooperating Traker and Classifier Modules Brendan Morris and Mohan Trivedi University of California, San Diego San Diego, CA 92093 {b1morris, trivedi}@usd.edu

More information

Discovering Trends in Large Datasets Using Neural Networks

Discovering Trends in Large Datasets Using Neural Networks Disovering Trends in Large Datasets Using Neural Networks Khosrow Kaikhah, Ph.D. and Sandesh Doddameti Department of Computer Siene Texas State University San Maros, Texas 78666 Abstrat. A novel knowledge

More information

A Keyword Filters Method for Spam via Maximum Independent Sets

A Keyword Filters Method for Spam via Maximum Independent Sets Vol. 7, No. 3, May, 213 A Keyword Filters Method for Spam via Maximum Independent Sets HaiLong Wang 1, FanJun Meng 1, HaiPeng Jia 2, JinHong Cheng 3 and Jiong Xie 3 1 Inner Mongolia Normal University 2

More information

An Efficient Network Traffic Classification Based on Unknown and Anomaly Flow Detection Mechanism

An Efficient Network Traffic Classification Based on Unknown and Anomaly Flow Detection Mechanism An Effiient Network Traffi Classifiation Based on Unknown and Anomaly Flow Detetion Mehanism G.Suganya.M.s.,B.Ed 1 1 Mphil.Sholar, Department of Computer Siene, KG College of Arts and Siene,Coimbatore.

More information

A Context-Aware Preference Database System

A Context-Aware Preference Database System J. PERVASIVE COMPUT. & COMM. (), MARCH 005. TROUBADOR PUBLISHING LTD) A Context-Aware Preferene Database System Kostas Stefanidis Department of Computer Siene, University of Ioannina,, kstef@s.uoi.gr Evaggelia

More information

Chapter 5 Single Phase Systems

Chapter 5 Single Phase Systems Chapter 5 Single Phase Systems Chemial engineering alulations rely heavily on the availability of physial properties of materials. There are three ommon methods used to find these properties. These inlude

More information

) ( )( ) ( ) ( )( ) ( ) ( ) (1)

) ( )( ) ( ) ( )( ) ( ) ( ) (1) OPEN CHANNEL FLOW Open hannel flow is haraterized by a surfae in ontat with a gas phase, allowing the fluid to take on shapes and undergo behavior that is impossible in a pipe or other filled onduit. Examples

More information

RATING SCALES FOR NEUROLOGISTS

RATING SCALES FOR NEUROLOGISTS RATING SCALES FOR NEUROLOGISTS J Hobart iv22 WHY Correspondene to: Dr Jeremy Hobart, Department of Clinial Neurosienes, Peninsula Medial Shool, Derriford Hospital, Plymouth PL6 8DH, UK; Jeremy.Hobart@

More information

Behavior Analysis-Based Learning Framework for Host Level Intrusion Detection

Behavior Analysis-Based Learning Framework for Host Level Intrusion Detection Behavior Analysis-Based Learning Framework for Host Level Intrusion Detetion Haiyan Qiao, Jianfeng Peng, Chuan Feng, Jerzy W. Rozenblit Eletrial and Computer Engineering Department University of Arizona

More information

Capacity at Unsignalized Two-Stage Priority Intersections

Capacity at Unsignalized Two-Stage Priority Intersections Capaity at Unsignalized Two-Stage Priority Intersetions by Werner Brilon and Ning Wu Abstrat The subjet of this paper is the apaity of minor-street traffi movements aross major divided four-lane roadways

More information

Classical Electromagnetic Doppler Effect Redefined. Copyright 2014 Joseph A. Rybczyk

Classical Electromagnetic Doppler Effect Redefined. Copyright 2014 Joseph A. Rybczyk Classial Eletromagneti Doppler Effet Redefined Copyright 04 Joseph A. Rybzyk Abstrat The lassial Doppler Effet formula for eletromagneti waves is redefined to agree with the fundamental sientifi priniples

More information

Big Data Analysis and Reporting with Decision Tree Induction

Big Data Analysis and Reporting with Decision Tree Induction Big Data Analysis and Reporting with Deision Tree Indution PETRA PERNER Institute of Computer Vision and Applied Computer Sienes, IBaI Postbox 30 11 14, 04251 Leipzig GERMANY pperner@ibai-institut.de,

More information

An integrated optimization model of a Closed- Loop Supply Chain under uncertainty

An integrated optimization model of a Closed- Loop Supply Chain under uncertainty ISSN 1816-6075 (Print), 1818-0523 (Online) Journal of System and Management Sienes Vol. 2 (2012) No. 3, pp. 9-17 An integrated optimization model of a Closed- Loop Supply Chain under unertainty Xiaoxia

More information

FIRE DETECTION USING AUTONOMOUS AERIAL VEHICLES WITH INFRARED AND VISUAL CAMERAS. J. Ramiro Martínez-de Dios, Luis Merino and Aníbal Ollero

FIRE DETECTION USING AUTONOMOUS AERIAL VEHICLES WITH INFRARED AND VISUAL CAMERAS. J. Ramiro Martínez-de Dios, Luis Merino and Aníbal Ollero FE DETECTION USING AUTONOMOUS AERIAL VEHICLES WITH INFRARED AND VISUAL CAMERAS. J. Ramiro Martínez-de Dios, Luis Merino and Aníbal Ollero Robotis, Computer Vision and Intelligent Control Group. University

More information

Open and Extensible Business Process Simulator

Open and Extensible Business Process Simulator UNIVERSITY OF TARTU FACULTY OF MATHEMATICS AND COMPUTER SCIENCE Institute of Computer Siene Karl Blum Open and Extensible Business Proess Simulator Master Thesis (30 EAP) Supervisors: Luiano Garía-Bañuelos,

More information

Pattern Recognition Techniques in Microarray Data Analysis

Pattern Recognition Techniques in Microarray Data Analysis Pattern Reognition Tehniques in Miroarray Data Analysis Miao Li, Biao Wang, Zohreh Momeni, and Faramarz Valafar Department of Computer Siene San Diego State University San Diego, California, USA faramarz@sienes.sdsu.edu

More information

GABOR AND WEBER LOCAL DESCRIPTORS PERFORMANCE IN MULTISPECTRAL EARTH OBSERVATION IMAGE DATA ANALYSIS

GABOR AND WEBER LOCAL DESCRIPTORS PERFORMANCE IN MULTISPECTRAL EARTH OBSERVATION IMAGE DATA ANALYSIS HENRI COANDA AIR FORCE ACADEMY ROMANIA INTERNATIONAL CONFERENCE of SCIENTIFIC PAPER AFASES 015 Brasov, 8-30 May 015 GENERAL M.R. STEFANIK ARMED FORCES ACADEMY SLOVAK REPUBLIC GABOR AND WEBER LOCAL DESCRIPTORS

More information

Paid Placement Strategies for Internet Search Engines

Paid Placement Strategies for Internet Search Engines Paid Plaement Strategies for Internet Searh Engines Hemant K. Bhargava Smeal College of Business Penn State University 342 Beam Building University Park, PA 16802 bhargava@omputer.org Juan Feng Smeal College

More information

Improved SOM-Based High-Dimensional Data Visualization Algorithm

Improved SOM-Based High-Dimensional Data Visualization Algorithm Computer and Information Siene; Vol. 5, No. 4; 2012 ISSN 1913-8989 E-ISSN 1913-8997 Published by Canadian Center of Siene and Eduation Improved SOM-Based High-Dimensional Data Visualization Algorithm Wang

More information

CIS570 Lecture 4 Introduction to Data-flow Analysis 3

CIS570 Lecture 4 Introduction to Data-flow Analysis 3 Introdution to Data-flow Analysis Last Time Control flow analysis BT disussion Today Introdue iterative data-flow analysis Liveness analysis Introdue other useful onepts CIS570 Leture 4 Introdution to

More information

Interpretable Fuzzy Modeling using Multi-Objective Immune- Inspired Optimization Algorithms

Interpretable Fuzzy Modeling using Multi-Objective Immune- Inspired Optimization Algorithms Interpretable Fuzzy Modeling using Multi-Objetive Immune- Inspired Optimization Algorithms Jun Chen, Mahdi Mahfouf Abstrat In this paper, an immune inspired multi-objetive fuzzy modeling (IMOFM) mehanism

More information

Scalable Hierarchical Multitask Learning Algorithms for Conversion Optimization in Display Advertising

Scalable Hierarchical Multitask Learning Algorithms for Conversion Optimization in Display Advertising Salable Hierarhial Multitask Learning Algorithms for Conversion Optimization in Display Advertising Amr Ahmed Google amra@google.om Abhimanyu Das Mirosoft Researh abhidas@mirosoft.om Alexander J. Smola

More information

Static Fairness Criteria in Telecommunications

Static Fairness Criteria in Telecommunications Teknillinen Korkeakoulu ERIKOISTYÖ Teknillisen fysiikan koulutusohjelma 92002 Mat-208 Sovelletun matematiikan erikoistyöt Stati Fairness Criteria in Teleommuniations Vesa Timonen, e-mail: vesatimonen@hutfi

More information

arxiv:astro-ph/0304006v2 10 Jun 2003 Theory Group, MS 50A-5101 Lawrence Berkeley National Laboratory One Cyclotron Road Berkeley, CA 94720 USA

arxiv:astro-ph/0304006v2 10 Jun 2003 Theory Group, MS 50A-5101 Lawrence Berkeley National Laboratory One Cyclotron Road Berkeley, CA 94720 USA LBNL-52402 Marh 2003 On the Speed of Gravity and the v/ Corretions to the Shapiro Time Delay Stuart Samuel 1 arxiv:astro-ph/0304006v2 10 Jun 2003 Theory Group, MS 50A-5101 Lawrene Berkeley National Laboratory

More information

AUDITING COST OVERRUN CLAIMS *

AUDITING COST OVERRUN CLAIMS * AUDITING COST OVERRUN CLAIMS * David Pérez-Castrillo # University of Copenhagen & Universitat Autònoma de Barelona Niolas Riedinger ENSAE, Paris Abstrat: We onsider a ost-reimbursement or a ost-sharing

More information

Deadline-based Escalation in Process-Aware Information Systems

Deadline-based Escalation in Process-Aware Information Systems Deadline-based Esalation in Proess-Aware Information Systems Wil M.P. van der Aalst 1,2, Mihael Rosemann 2, Marlon Dumas 2 1 Department of Tehnology Management Eindhoven University of Tehnology, The Netherlands

More information

Magnetic Materials and Magnetic Circuit Analysis

Magnetic Materials and Magnetic Circuit Analysis Chapter 7. Magneti Materials and Magneti Ciruit Analysis Topis to over: 1) Core Losses 2) Ciruit Model of Magneti Cores 3) A Simple Magneti Ciruit 4) Magneti Ciruital Laws 5) Ciruit Model of Permanent

More information

Robust Classification and Tracking of Vehicles in Traffic Video Streams

Robust Classification and Tracking of Vehicles in Traffic Video Streams Proeedings of the IEEE ITSC 2006 2006 IEEE Intelligent Transportation Systems Conferene Toronto, Canada, September 17-20, 2006 TC1.4 Robust Classifiation and Traking of Vehiles in Traffi Video Streams

More information

Customer Efficiency, Channel Usage and Firm Performance in Retail Banking

Customer Efficiency, Channel Usage and Firm Performance in Retail Banking Customer Effiieny, Channel Usage and Firm Performane in Retail Banking Mei Xue Operations and Strategi Management Department The Wallae E. Carroll Shool of Management Boston College 350 Fulton Hall, 140

More information

Granular Problem Solving and Software Engineering

Granular Problem Solving and Software Engineering Granular Problem Solving and Software Engineering Haibin Zhu, Senior Member, IEEE Department of Computer Siene and Mathematis, Nipissing University, 100 College Drive, North Bay, Ontario, P1B 8L7, Canada

More information

WORKFLOW CONTROL-FLOW PATTERNS A Revised View

WORKFLOW CONTROL-FLOW PATTERNS A Revised View WORKFLOW CONTROL-FLOW PATTERNS A Revised View Nik Russell 1, Arthur H.M. ter Hofstede 1, 1 BPM Group, Queensland University of Tehnology GPO Box 2434, Brisbane QLD 4001, Australia {n.russell,a.terhofstede}@qut.edu.au

More information

Channel Assignment Strategies for Cellular Phone Systems

Channel Assignment Strategies for Cellular Phone Systems Channel Assignment Strategies for Cellular Phone Systems Wei Liu Yiping Han Hang Yu Zhejiang University Hangzhou, P. R. China Contat: wliu5@ie.uhk.edu.hk 000 Mathematial Contest in Modeling (MCM) Meritorious

More information

An Enhanced Critical Path Method for Multiple Resource Constraints

An Enhanced Critical Path Method for Multiple Resource Constraints An Enhaned Critial Path Method for Multiple Resoure Constraints Chang-Pin Lin, Hung-Lin Tai, and Shih-Yan Hu Abstrat Traditional Critial Path Method onsiders only logial dependenies between related ativities

More information

chapter > Make the Connection Factoring CHAPTER 4 OUTLINE Chapter 4 :: Pretest 374

chapter > Make the Connection Factoring CHAPTER 4 OUTLINE Chapter 4 :: Pretest 374 CHAPTER hapter 4 > Make the Connetion 4 INTRODUCTION Developing seret odes is big business beause of the widespread use of omputers and the Internet. Corporations all over the world sell enryption systems

More information

Henley Business School at Univ of Reading. Pre-Experience Postgraduate Programmes Chartered Institute of Personnel and Development (CIPD)

Henley Business School at Univ of Reading. Pre-Experience Postgraduate Programmes Chartered Institute of Personnel and Development (CIPD) MS in International Human Resoure Management For students entering in 2012/3 Awarding Institution: Teahing Institution: Relevant QAA subjet Benhmarking group(s): Faulty: Programme length: Date of speifiation:

More information

Context-Sensitive Adjustments of Cognitive Control: Conflict-Adaptation Effects Are Modulated by Processing Demands of the Ongoing Task

Context-Sensitive Adjustments of Cognitive Control: Conflict-Adaptation Effects Are Modulated by Processing Demands of the Ongoing Task Journal of Experimental Psyhology: Learning, Memory, and Cognition 2008, Vol. 34, No. 3, 712 718 Copyright 2008 by the Amerian Psyhologial Assoiation 0278-7393/08/$12.00 DOI: 10.1037/0278-7393.34.3.712

More information

Intelligent Measurement Processes in 3D Optical Metrology: Producing More Accurate Point Clouds

Intelligent Measurement Processes in 3D Optical Metrology: Producing More Accurate Point Clouds Intelligent Measurement Proesses in 3D Optial Metrology: Produing More Aurate Point Clouds Charles Mony, Ph.D. 1 President Creaform in. mony@reaform3d.om Daniel Brown, Eng. 1 Produt Manager Creaform in.

More information

FOOD FOR THOUGHT Topical Insights from our Subject Matter Experts

FOOD FOR THOUGHT Topical Insights from our Subject Matter Experts FOOD FOR THOUGHT Topial Insights from our Sujet Matter Experts DEGREE OF DIFFERENCE TESTING: AN ALTERNATIVE TO TRADITIONAL APPROACHES The NFL White Paper Series Volume 14, June 2014 Overview Differene

More information

INCOME TAX WITHHOLDING GUIDE FOR EMPLOYERS

INCOME TAX WITHHOLDING GUIDE FOR EMPLOYERS Virginia Department of Taxation INCOME TAX WITHHOLDING GUIDE FOR EMPLOYERS www.tax.virginia.gov 2614086 Rev. 01/16 Table of Contents Introdution... 1 Important... 1 Where to Get Assistane... 1 Online File

More information

SOFTWARE ENGINEERING I

SOFTWARE ENGINEERING I SOFTWARE ENGINEERING I CS 10 Catalog Desription PREREQUISITE: CS 21. Introdution to the systems development life yle, software development models, analysis and design tehniques and tools, and validation

More information

INCOME TAX WITHHOLDING GUIDE FOR EMPLOYERS

INCOME TAX WITHHOLDING GUIDE FOR EMPLOYERS Virginia Department of Taxation INCOME TAX WITHHOLDING GUIDE FOR EMPLOYERS www.tax.virginia.gov 2614086 Rev. 07/14 * Table of Contents Introdution... 1 Important... 1 Where to Get Assistane... 1 Online

More information

Design Implications for Enterprise Storage Systems via Multi-Dimensional Trace Analysis

Design Implications for Enterprise Storage Systems via Multi-Dimensional Trace Analysis Design Impliations for Enterprise Storage Systems via Multi-Dimensional Trae Analysis Yanpei Chen, Kiran Srinivasan, Garth Goodson, Randy Katz University of California, Berkeley, NetApp In. {yhen2, randy}@ees.berkeley.edu,

More information

User s Guide VISFIT: a computer tool for the measurement of intrinsic viscosities

User s Guide VISFIT: a computer tool for the measurement of intrinsic viscosities File:UserVisfit_2.do User s Guide VISFIT: a omputer tool for the measurement of intrinsi visosities Version 2.a, September 2003 From: Multiple Linear Least-Squares Fits with a Common Interept: Determination

More information

The Optimal Deterrence of Tax Evasion: The Trade-off Between Information Reporting and Audits

The Optimal Deterrence of Tax Evasion: The Trade-off Between Information Reporting and Audits The Optimal Deterrene of Tax Evasion: The Trade-off Between Information Reporting and Audits Yulia Paramonova Department of Eonomis, University of Mihigan Otober 30, 2014 Abstrat Despite the widespread

More information

Trade Information, Not Spectrum: A Novel TV White Space Information Market Model

Trade Information, Not Spectrum: A Novel TV White Space Information Market Model Trade Information, Not Spetrum: A Novel TV White Spae Information Market Model Yuan Luo, Lin Gao, and Jianwei Huang 1 Abstrat In this paper, we propose a novel information market for TV white spae networks,

More information

Performance Analysis of IEEE 802.11 in Multi-hop Wireless Networks

Performance Analysis of IEEE 802.11 in Multi-hop Wireless Networks Performane Analysis of IEEE 80.11 in Multi-hop Wireless Networks Lan Tien Nguyen 1, Razvan Beuran,1, Yoihi Shinoda 1, 1 Japan Advaned Institute of Siene and Tehnology, 1-1 Asahidai, Nomi, Ishikawa, 93-19

More information

RISK-BASED IN SITU BIOREMEDIATION DESIGN JENNINGS BRYAN SMALLEY. A.B., Washington University, 1992 THESIS. Urbana, Illinois

RISK-BASED IN SITU BIOREMEDIATION DESIGN JENNINGS BRYAN SMALLEY. A.B., Washington University, 1992 THESIS. Urbana, Illinois RISK-BASED IN SITU BIOREMEDIATION DESIGN BY JENNINGS BRYAN SMALLEY A.B., Washington University, 1992 THESIS Submitted in partial fulfillment of the requirements for the degree of Master of Siene in Environmental

More information

Sebastián Bravo López

Sebastián Bravo López Transfinite Turing mahines Sebastián Bravo López 1 Introdution With the rise of omputers with high omputational power the idea of developing more powerful models of omputation has appeared. Suppose that

More information

A Comparison of Service Quality between Private and Public Hospitals in Thailand

A Comparison of Service Quality between Private and Public Hospitals in Thailand International Journal of Business and Soial Siene Vol. 4 No. 11; September 2013 A Comparison of Servie Quality between Private and Hospitals in Thailand Khanhitpol Yousapronpaiboon, D.B.A. Assistant Professor

More information

State of Maryland Participation Agreement for Pre-Tax and Roth Retirement Savings Accounts

State of Maryland Participation Agreement for Pre-Tax and Roth Retirement Savings Accounts State of Maryland Partiipation Agreement for Pre-Tax and Roth Retirement Savings Aounts DC-4531 (08/2015) For help, please all 1-800-966-6355 www.marylandd.om 1 Things to Remember Complete all of the setions

More information

Recovering Articulated Motion with a Hierarchical Factorization Method

Recovering Articulated Motion with a Hierarchical Factorization Method Reovering Artiulated Motion with a Hierarhial Fatorization Method Hanning Zhou and Thomas S Huang University of Illinois at Urbana-Champaign, 405 North Mathews Avenue, Urbana, IL 680, USA {hzhou, huang}@ifpuiuedu

More information

RELEASING MICRODATA: DISCLOSURE RISK ESTIMATION, DATA MASKING AND ASSESSING UTILITY

RELEASING MICRODATA: DISCLOSURE RISK ESTIMATION, DATA MASKING AND ASSESSING UTILITY Setion on Survey Researh Methods JSM 008 RELEASING MICRODATA: DISCLOSURE RISK ESTIMATION, DATA MASKING AND ASSESSING UTILITY Natalie Shlomo 1 1 Southampton Statistial Sienes Researh Institute, University

More information

Impedance Method for Leak Detection in Zigzag Pipelines

Impedance Method for Leak Detection in Zigzag Pipelines 10.478/v10048-010-0036-0 MEASUREMENT SCIENCE REVIEW, Volume 10, No. 6, 010 Impedane Method for Leak Detetion in igzag Pipelines A. Lay-Ekuakille 1, P. Vergallo 1, A. Trotta 1 Dipartimento d Ingegneria

More information

DSP-I DSP-I DSP-I DSP-I

DSP-I DSP-I DSP-I DSP-I DSP-I DSP-I DSP-I DSP-I Digital Signal Proessing I (8-79) Fall Semester, 005 IIR FILER DESIG EXAMPLE hese notes summarize the design proedure for IIR filters as disussed in lass on ovember. Introdution:

More information

Neural network-based Load Balancing and Reactive Power Control by Static VAR Compensator

Neural network-based Load Balancing and Reactive Power Control by Static VAR Compensator nternational Journal of Computer and Eletrial Engineering, Vol. 1, No. 1, April 2009 Neural network-based Load Balaning and Reative Power Control by Stati VAR Compensator smail K. Said and Marouf Pirouti

More information

protection p1ann1ng report

protection p1ann1ng report f1re~~ protetion p1ann1ng report BUILDING CONSTRUCTION INFORMATION FROM THE CONCRETE AND MASONRY INDUSTRIES Signifiane of Fire Ratings for Building Constrution NO. 3 OF A SERIES The use of fire-resistive

More information

SLA-based Resource Allocation for Software as a Service Provider (SaaS) in Cloud Computing Environments

SLA-based Resource Allocation for Software as a Service Provider (SaaS) in Cloud Computing Environments 2 th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing SLA-based Resoure Alloation for Software as a Servie Provider (SaaS) in Cloud Computing Environments Linlin Wu, Saurabh Kumar

More information

Electrician'sMathand BasicElectricalFormulas

Electrician'sMathand BasicElectricalFormulas Eletriian'sMathand BasiEletrialFormulas MikeHoltEnterprises,In. 1.888.NEC.CODE www.mikeholt.om Introdution Introdution This PDF is a free resoure from Mike Holt Enterprises, In. It s Unit 1 from the Eletrial

More information

Software Ecosystems: From Software Product Management to Software Platform Management

Software Ecosystems: From Software Product Management to Software Platform Management Software Eosystems: From Software Produt Management to Software Platform Management Slinger Jansen, Stef Peeters, and Sjaak Brinkkemper Department of Information and Computing Sienes Utreht University,

More information

i_~f e 1 then e 2 else e 3

i_~f e 1 then e 2 else e 3 A PROCEDURE MECHANISM FOR BACKTRACK PROGRAMMING* David R. HANSON + Department o Computer Siene, The University of Arizona Tuson, Arizona 85721 One of the diffiulties in using nondeterministi algorithms

More information

A Reputation Management Approach for Resource Constrained Trustee Agents

A Reputation Management Approach for Resource Constrained Trustee Agents A Reputation Management Approah for Resoure Constrained rustee Agents Han Yu, Chunyan Miao, Bo An 2, Cyril Leung 3, and Vitor R. Lesser 4 Nanyang ehnologial University, Singapore 2 he Key Lab of Intelligent

More information

Henley Business School at Univ of Reading. Chartered Institute of Personnel and Development (CIPD)

Henley Business School at Univ of Reading. Chartered Institute of Personnel and Development (CIPD) MS in International Human Resoure Management (full-time) For students entering in 2015/6 Awarding Institution: Teahing Institution: Relevant QAA subjet Benhmarking group(s): Faulty: Programme length: Date

More information

The Basics of International Trade: A Classroom Experiment

The Basics of International Trade: A Classroom Experiment The Basis of International Trade: A Classroom Experiment Alberto Isgut, Ganesan Ravishanker, and Tanya Rosenblat * Wesleyan University Abstrat We introdue a simple web-based lassroom experiment in whih

More information

Deduplication with Block-Level Content-Aware Chunking for Solid State Drives (SSDs)

Deduplication with Block-Level Content-Aware Chunking for Solid State Drives (SSDs) 23 IEEE International Conferene on High Performane Computing and Communiations & 23 IEEE International Conferene on Embedded and Ubiquitous Computing Dedupliation with Blok-Level Content-Aware Chunking

More information

Lemon Signaling in Cross-Listings Michal Barzuza*

Lemon Signaling in Cross-Listings Michal Barzuza* Lemon Signaling in Cross-Listings Mihal Barzuza* This paper analyzes the deision to ross-list by managers and ontrolling shareholders assuming that they have private information with respet to the amount

More information

A Three-Hybrid Treatment Method of the Compressor's Characteristic Line in Performance Prediction of Power Systems

A Three-Hybrid Treatment Method of the Compressor's Characteristic Line in Performance Prediction of Power Systems A Three-Hybrid Treatment Method of the Compressor's Charateristi Line in Performane Predition of Power Systems A Three-Hybrid Treatment Method of the Compressor's Charateristi Line in Performane Predition

More information

Unit 12: Installing, Configuring and Administering Microsoft Server

Unit 12: Installing, Configuring and Administering Microsoft Server Unit 12: Installing, Configuring and Administering Mirosoft Server Learning Outomes A andidate following a programme of learning leading to this unit will be able to: Selet a suitable NOS to install for

More information

In order to be able to design beams, we need both moments and shears. 1. Moment a) From direct design method or equivalent frame method

In order to be able to design beams, we need both moments and shears. 1. Moment a) From direct design method or equivalent frame method BEAM DESIGN In order to be able to design beams, we need both moments and shears. 1. Moment a) From diret design method or equivalent frame method b) From loads applied diretly to beams inluding beam weight

More information

Procurement auctions are sometimes plagued with a chosen supplier s failing to accomplish a project successfully.

Procurement auctions are sometimes plagued with a chosen supplier s failing to accomplish a project successfully. Deision Analysis Vol. 7, No. 1, Marh 2010, pp. 23 39 issn 1545-8490 eissn 1545-8504 10 0701 0023 informs doi 10.1287/dea.1090.0155 2010 INFORMS Managing Projet Failure Risk Through Contingent Contrats

More information

Social Network Analysis Based on BSP Clustering Algorithm

Social Network Analysis Based on BSP Clustering Algorithm Soial Network Analysis Based on BSP Clustering Algorithm ong Shool of Business Administration China University of Petroleum ABSRAC Soial network analysis is a new researh field in data mining. he lustering

More information

Chemical Equilibrium. Chemical Equilibrium. Chemical Equilibrium. Chemical Equilibriu m. Chapter 14

Chemical Equilibrium. Chemical Equilibrium. Chemical Equilibrium. Chemical Equilibriu m. Chapter 14 Chapter 14 Chemial Equilibrium Chemial Equilibriu m Muh like water in a U-shaped tube, there is onstant mixing bak and forth through the lower portion of the tube. reatants produts It s as if the forward

More information

Impact Simulation of Extreme Wind Generated Missiles on Radioactive Waste Storage Facilities

Impact Simulation of Extreme Wind Generated Missiles on Radioactive Waste Storage Facilities Impat Simulation of Extreme Wind Generated issiles on Radioative Waste Storage Failities G. Barbella Sogin S.p.A. Via Torino 6 00184 Rome (Italy), barbella@sogin.it Abstrat: The strutural design of temporary

More information

Srinivas Bollapragada GE Global Research Center. Abstract

Srinivas Bollapragada GE Global Research Center. Abstract Sheduling Commerial Videotapes in Broadast Television Srinivas Bollapragada GE Global Researh Center Mihael Bussiek GAMS Development Corporation Suman Mallik University of Illinois at Urbana Champaign

More information

Board Building Recruiting and Developing Effective Board Members for Not-for-Profit Organizations

Board Building Recruiting and Developing Effective Board Members for Not-for-Profit Organizations Board Development Board Building Reruiting and Developing Effetive Board Members for Not-for-Profit Organizations Board Development Board Building Reruiting and Developing Effetive Board Members for Not-for-Profit

More information

TRENDS IN EXECUTIVE EDUCATION: TOWARDS A SYSTEMS APPROACH TO EXECUTIVE DEVELOPMENT PLANNING

TRENDS IN EXECUTIVE EDUCATION: TOWARDS A SYSTEMS APPROACH TO EXECUTIVE DEVELOPMENT PLANNING INTERMAN 7 TRENDS IN EXECUTIVE EDUCATION: TOWARDS A SYSTEMS APPROACH TO EXECUTIVE DEVELOPMENT PLANNING by Douglas A. Ready, Albert A. Viere and Alan F. White RECEIVED 2 7 MAY 1393 International Labour

More information

university of illinois library AT URBANA-CHAMPAIGN BOOKSTACKS

university of illinois library AT URBANA-CHAMPAIGN BOOKSTACKS university of illinois library AT URBANA-CHAMPAIGN BOOKSTACKS CENTRAL CIRCULATION BOOKSTACKS The person harging this material is responsible for its renewal or its return to the library from whih it was

More information

Research Data Management ANONYMISATION

Research Data Management ANONYMISATION ANONYMISATION Sensitive Data Sensitive Data is information overing: The raial or ethni origin of the Data Subjet Politial opinions Religious or other beliefs of a similar nature Membership of trade unions

More information

The Application of Mamdani Fuzzy Model for Auto Zoom Function of a Digital Camera

The Application of Mamdani Fuzzy Model for Auto Zoom Function of a Digital Camera (IJCSIS) International Journal of Computer Siene and Information Seurity, Vol. 6, No. 3, 2009 The Appliation of Mamdani Fuzzy Model for Auto Funtion of a Digital Camera * I. Elamvazuthi, P. Vasant Universiti

More information

Chapter 1: Introduction

Chapter 1: Introduction Chapter 1: Introdution 1.1 Pratial olumn base details in steel strutures 1.1.1 Pratial olumn base details Every struture must transfer vertial and lateral loads to the supports. In some ases, beams or

More information

MEMBER. Application for election MEMBER, NEW GRADUATE. psychology.org.au. April 2015

MEMBER. Application for election MEMBER, NEW GRADUATE. psychology.org.au. April 2015 MEMBER Appliation for eletion MEMBER, NEW GRADUATE April 2015 psyhology.org.au MEMBER Belonging to the Australian Psyhologial Soiety (APS) means you are part of an ative, progressive organisation whih

More information

Professional Certificate Training in Business Writing

Professional Certificate Training in Business Writing Professional Certifiate Training in Business Writing About Training in Business Writing ZeebraCross Centre for Management Exellene (ZCME) is an initiative of ZeebraCross (Unit of InfousRx Marketing and

More information

Supply chain coordination; A Game Theory approach

Supply chain coordination; A Game Theory approach aepted for publiation in the journal "Engineering Appliations of Artifiial Intelligene" 2008 upply hain oordination; A Game Theory approah Jean-Claude Hennet x and Yasemin Arda xx x LI CNR-UMR 668 Université

More information

Agile ALM White Paper: Redefining ALM with Five Key Practices

Agile ALM White Paper: Redefining ALM with Five Key Practices Agile ALM White Paper: Redefining ALM with Five Key Praties by Ethan Teng, Cyndi Mithell and Chad Wathington 2011 ThoughtWorks ln. All rights reserved www.studios.thoughtworks.om Introdution The pervasiveness

More information

Findings and Recommendations

Findings and Recommendations Contrating Methods and Administration Findings and Reommendations Finding 9-1 ESD did not utilize a formal written pre-qualifiations proess for seleting experiened design onsultants. ESD hose onsultants

More information

ROSE SCHOOL A SIMPLIFIED MECHANICS BASED PROCEDURE FOR THE SEISMIC RISK ASSESSMENT OF UNREINFORCED MASONRY BUILDINGS

ROSE SCHOOL A SIMPLIFIED MECHANICS BASED PROCEDURE FOR THE SEISMIC RISK ASSESSMENT OF UNREINFORCED MASONRY BUILDINGS I.U.S.S. Istituto Universitario di Studi Superiori di Pavia Università degli Studi di Pavia EUROPEAN SCHOOL OF ADVANCED STUDIES IN REDUCTION OF SEISMIC RISK ROSE SCHOOL A SIMPLIFIED MECHANICS BASED PROCEDURE

More information

THE PERFORMANCE OF TRANSIT TIME FLOWMETERS IN HEATED GAS MIXTURES

THE PERFORMANCE OF TRANSIT TIME FLOWMETERS IN HEATED GAS MIXTURES Proeedings of FEDSM 98 998 ASME Fluids Engineering Division Summer Meeting June 2-25, 998 Washington DC FEDSM98-529 THE PERFORMANCE OF TRANSIT TIME FLOWMETERS IN HEATED GAS MIXTURES John D. Wright Proess

More information

OpenScape 4000 CSTA V7 Connectivity Adapter - CSTA III, Part 2, Version 4.1. Developer s Guide A31003-G9310-I200-1-76D1

OpenScape 4000 CSTA V7 Connectivity Adapter - CSTA III, Part 2, Version 4.1. Developer s Guide A31003-G9310-I200-1-76D1 OpenSape 4000 CSTA V7 Connetivity Adapter - CSTA III, Part 2, Version 4.1 Developer s Guide A31003-G9310-I200-1-76 Our Quality and Environmental Management Systems are implemented aording to the requirements

More information

Computational Analysis of Two Arrangements of a Central Ground-Source Heat Pump System for Residential Buildings

Computational Analysis of Two Arrangements of a Central Ground-Source Heat Pump System for Residential Buildings Computational Analysis of Two Arrangements of a Central Ground-Soure Heat Pump System for Residential Buildings Abstrat Ehab Foda, Ala Hasan, Kai Sirén Helsinki University of Tehnology, HVAC Tehnology,

More information

Computer Networks Framing

Computer Networks Framing Computer Networks Framing Saad Mneimneh Computer Siene Hunter College of CUNY New York Introdution Who framed Roger rabbit? A detetive, a woman, and a rabbit in a network of trouble We will skip the physial

More information