Non-Markovian Logic-Probabilistic Modeling and Inference
|
|
- Morgan Allison
- 7 years ago
- Views:
Transcription
1 Non-Markovian Logic-Probabilistic Modeling and Inference Eduardo Menezes de Morais Glauber De Bona Marcelo Finger Department of Computer Science University of São Paulo São Paulo, Brazil WL4AI 2015
2 What we are trying to solve Part-of-speech Tagging PoS Tagging associates each word in a sentence to a part-of-speech tag Example The/D horse/n raced/vb-p past/p the/d barn/n Determiner (D), Nouns (N), Past tense verb (VB-P), Past participle verb (VB-PP), Preposition (P), Adjective (ADJ) To resolve ambiguity, current algorithms inspect adjacent tags Only a local context is used (usually 2 or 3 words) to avoid an exponential blowup
3 What we are trying to solve Part-of-speech Tagging Some ambiguities can only be solved with unbounded distance dependency Example The horse raced/vb-p past the barn The horse raced/vb-pp past the barn fell/vb-p Determiner (D), Nouns (N), Past tense verb (VB-P), Past participle verb (VB-PP), Preposition (P), Adjective (ADJ) We hope to find a way to treat these dependencies without exponential blowup with Probabilistic Logic
4 What we are trying to solve Part-of-speech Tagging Some ambiguities can only be solved with unbounded distance dependency Example The horse raced/vb-p past the barn The horse raced/vb-pp past the barn fell/vb-p The horse raced/vb-pp past the old red barn fell/vb-p Determiner (D), Nouns (N), Past tense verb (VB-P), Past participle verb (VB-PP), Preposition (P), Adjective (ADJ) We hope to find a way to treat these dependencies without exponential blowup with Probabilistic Logic
5 Talk outline Outline Motivation Probabilistic Logic Hidden Markov Models Encoding Hidden Markov Models with PSAT Non-markovian extension Conclusion
6 Next Topic Motivation What we are trying to solve Talk outline Probabilistic Logic Definition Properties Hidden Markov Models A algorithm for POS-Tagging Encoding Hidden Markov Models with PSAT The constraints Non-markovian extension Abandoning the Markov property Extra Constraints Conclusion
7 Definition The language Logical variables or atoms: P = {x 1,..., x n } Connectives:,,. Formulas (L P ) are inductively composed form atoms using connectives A probabilistic knowledge base is a finite set of probabilistic assignments Γ = {P(ϕ i ψ i ) = p i 1 i m} where ϕ i, ψ i L P and p i [0, 1]
8 Definition Semantics V is the set of all 2 n propositional valuations (possible worlds) v : P {0, 1} generalized for any propositional formula v : L P {0, 1} A probability distribution over propositional valuations π : V [0, 1] π(v i ) = 1 2 n i=1 Probability of a formula α according to π P π (α) = {π(v i ) v i (α) = 1}
9 Definition The PSAT Problem Probabilistic Satisfiability: are these restrictions consistent? Is there a probability distribution over the possible worlds that makes all the probabilistic assignments true?
10 Definition The PSAT Problem Probabilistic Satisfiability: are these restrictions consistent? Is there a probability distribution over the possible worlds that makes all the probabilistic assignments true? Given Γ = {P(ϕ i ψ i ) = p i 1 i m}. Is there a π such that P π (ϕ i ψ i ) = p i P π (ψ i ), for all 1 i m?
11 Definition The OPSAT Problem Similar problem: OPSAT A formula / Γ has a range of probabilities, one for each π Given Γ = {P(ϕ i ψ i ) = p i 1 i m}. Which π satisfies P π (ϕ i ψ i ) = p i P π (ψ i ), for all 1 i m and maximizes/minimizes the probability of some formula µ / Γ? Note the lack of assumptions of a priori statistical independence
12 Properties The PSAT Problem: An Algebraic Formalization Vector of probabilities p of dimension k 1 (given) Consider a large matrix A k 2 n = [a ij ] (computed) a ij = v j (α i ) {0, 1} π: probability distribution of exponential size PSAT: decide if there is vector π of dimension 2 n 1 such that Aπ = p πi = 1 π 0
13 Properties The PSAT Problem: An Algebraic Formalization Vector of probabilities p of dimension k 1 (given) Consider a large matrix A k 2 n = [a ij ] (computed) a ij = v j (α i ) {0, 1} π: probability distribution of exponential size OPSAT: decide if there is vector π of dimension 2 n 1 such that max/min Aπ = p πi = 1 π 0 P(µ)
14 Properties PSAT is NP-complete PSAT is NP-complete: [Georgakopoulos & Kavvadias & Papadimitriou 1988] A PSAT problem has a solution, then there is a solution π with at most k + 1 elements π i > 0 Carathéodory s Lemma So PSAT has a polynomial size witness 1 1 π 1 0/1 0/ π 2. 0/1 0/1 π k+1 = 1 p 1. p k
15 Next Topic Motivation What we are trying to solve Talk outline Probabilistic Logic Definition Properties Hidden Markov Models A algorithm for POS-Tagging Encoding Hidden Markov Models with PSAT The constraints Non-markovian extension Abandoning the Markov property Extra Constraints Conclusion
16 A algorithm for POS-Tagging Hidden Markov Models HMMs have been successfully used on language processing tasks as early as the 1970s Stochastic process whose states cannot be observed directly A sequence of symbols produced by another stochastic process is observed The sequence of symbols produced dependent only on the current hidden state
17 A algorithm for POS-Tagging Markov Chains Sunny Rainy Cloudy 0.5
18 A algorithm for POS-Tagging Hidden Markov Model Sunny Rainy Cloudy Walk Shop Play
19 A algorithm for POS-Tagging Inference on HMMs On a POS-tagging context, the words are the observations and the tags are the states We want to discover the sequence of states that most likely have produced a sequence of observations (decoding). max T P(W H, T ) Efficiently solved using the Viterbi algorithm Good performance, but uses a very limited context
20 Next Topic Motivation What we are trying to solve Talk outline Probabilistic Logic Definition Properties Hidden Markov Models A algorithm for POS-Tagging Encoding Hidden Markov Models with PSAT The constraints Non-markovian extension Abandoning the Markov property Extra Constraints Conclusion
21 Encoding Hidden Markov Models with PSAT To remove some of the restrictions of HMMs, first lets model it as a PSAT problem Given a sentence o 1,..., o k, define proposition variables tij true iff word i has tag j wi true iff word i is o i
22 The constraints Attributes of an HMM Given a HMM H = (A, B, q) 1. Probability of transition between states A = [a ij ], a ij = P(t k = s j t k 1 = s i ) 2. Probability of the first state q = [q i ], q i = P(t 1 = s i ) 3. Probability of producing observations B = [b ih ], b ih = P(w k = o h t k = s i )
23 The constraints Probabilistic constraints 0. Only one state per word, m states total m P(t i,j t i,k ) = 0 P( t i,j ) = 1 1. Probability of transition between states P(t i,j t i 1,k ) = a k,j 2. Probability of the first state P(t 1,j ) = q j 3. Probability of producing observations P(w i t i,j ) = b j,w(i) j=1
24 The constraints Properties of an HMM A HMM H = (A, B, q) has the following properties: 4. Markov property ( lack of memory ) P(t k t k 1 ) = P(t k t k 1,..., t 1 ) 5. Independence property (the observations depend only on the current state) P(w k t k ) = P(w k t M,..., t 1, w k 1,..., w 1 )
25 The constraints Probabilistic constraints 4. Markov property ( lack of memory ) P(t i,hi t i 1,hi 1 t 1,h1 ) = P(t i,hi t i 1,hi 1 ), for any T. 5. Independence property (the observations depend only on the current state) P(W t 1,h1 t M,hM ) = M b hi,w(i), for any T. Expressing those two properties requires an exponential number of formulas i=1
26 The constraints Using constraints 0-5 we create a PSAT instance Γ H such that if π is satisfies Γ H : P π (T W ) has a unique probability measure for any π This probability measure is the same probability as the HMM entails
27 Next Topic Motivation What we are trying to solve Talk outline Probabilistic Logic Definition Properties Hidden Markov Models A algorithm for POS-Tagging Encoding Hidden Markov Models with PSAT The constraints Non-markovian extension Abandoning the Markov property Extra Constraints Conclusion
28 Abandoning the Markov property Extending Markov Models The objective of representing HMMs as a PSAT instance is to modify HMMs removing some of its limitations Since our objective is to treat long distance dependencies, we can remove the Markov property Removing the Independence Property leads to bad performance (other probabilities become irrelevant) To avoid an exponential number of probabilistic assignments, we also restrict the Independence Property
29 Abandoning the Markov property Restricting the Independence Property We limit the Independence to a l-sized window 6. P(w i w i+l 1 t i,hi t i+l 1,hi+l 1 ) = = = i+l 1 j=i i+l 1 j=i P(w j t j,hj ), for any T k and 1 i n l + 1 b hj,w(j), for any T k and 1 i n l + 1.
30 Abandoning the Markov property Choosing the right model P π (T W ) now is not a single value, but an interval To choose the best π, we maximize P π (W ) OPSAT problem OPSAT solution is a distribution over possible worlds, each world encodes a tagging Tagging: choose world with the maximum probability
31 Abandoning the Markov property Algorithm 5.1 Performing inference in a relaxed HMM Input: A HMM H and a list of observed words (w 1,..., w m ) Output: A tag sequence T with maximum a posteriori probability. 1: Γ H { probability assignments in (0)-(3) } 2: Γ l Γ H { probability assignments in (6)} 3: (π, P W ) OPSAT (Γ l, P π (W )) 4: T argmax T {P π (T W )} 5: return T
32 Extra Constraints Adding more information We can quantify the influence of a tag on all subsequent tags, independent of distance 7. P(t i,hi t j,hj ) = r hi,h j,i j With these new constraints the problem may become unsatisfiable
33 Extra Constraints Adding more information We can quantify the influence of a tag on all subsequent tags, independent of distance 7. P(t i,hi j<i t j,k ) = r h i,k With these new constraints the problem may become unsatisfiable
34 Extra Constraints Measuring Inconsistency Since the knowledge base may be inconsistent, we can try to minimize the inconsistency Error variables ε i = P π (ϕ i ψ i ) p i P π (ψ i ) Minimize p-norm of errors ε p = m p ε i p i=1 Tratable (linear program) if p = 0 or p =
35 Extra Constraints Algorithm 5.2 Inconsistency-tolerant inference Input: A HMM H, list of observed words (w 1,..., w m ) and a set of extra constraints Ψ. Output: A tag sequence T with maximum a posteriori probability. { probability assignments in (0)-(3) } 1: Γ H 2: Γ l Γ H { probability assignments in (6) } 3: Γ + l Γ l Ψ //Ψ has the form of (7) 4: (π ε, E) OPSAT ε (Γ + l, ε p) 5: Γ ++ l Γ + l { ε p = E} 6: (π, P W ) OPSAT ε (Γ ++ l, P π (W )) 7: T argmax T {P π (T W )} 8: return T
36 Next Topic Motivation What we are trying to solve Talk outline Probabilistic Logic Definition Properties Hidden Markov Models A algorithm for POS-Tagging Encoding Hidden Markov Models with PSAT The constraints Non-markovian extension Abandoning the Markov property Extra Constraints Conclusion
37 Non-local phenomena can be modeled by Probabilistic Propositional Logic theories avoiding exponential explosion With the extra flexibility comes inconsistent theories and worse complexity Implementation is under way Future work will deal with combining local and non-local dependencies
Tagging with Hidden Markov Models
Tagging with Hidden Markov Models Michael Collins 1 Tagging Problems In many NLP problems, we would like to model pairs of sequences. Part-of-speech (POS) tagging is perhaps the earliest, and most famous,
More informationCS510 Software Engineering
CS510 Software Engineering Propositional Logic Asst. Prof. Mathias Payer Department of Computer Science Purdue University TA: Scott A. Carr Slides inspired by Xiangyu Zhang http://nebelwelt.net/teaching/15-cs510-se
More informationLogic in general. Inference rules and theorem proving
Logical Agents Knowledge-based agents Logic in general Propositional logic Inference rules and theorem proving First order logic Knowledge-based agents Inference engine Knowledge base Domain-independent
More informationGrammars and introduction to machine learning. Computers Playing Jeopardy! Course Stony Brook University
Grammars and introduction to machine learning Computers Playing Jeopardy! Course Stony Brook University Last class: grammars and parsing in Prolog Noun -> roller Verb thrills VP Verb NP S NP VP NP S VP
More informationTesting LTL Formula Translation into Büchi Automata
Testing LTL Formula Translation into Büchi Automata Heikki Tauriainen and Keijo Heljanko Helsinki University of Technology, Laboratory for Theoretical Computer Science, P. O. Box 5400, FIN-02015 HUT, Finland
More informationPOS Tagsets and POS Tagging. Definition. Tokenization. Tagset Design. Automatic POS Tagging Bigram tagging. Maximum Likelihood Estimation 1 / 23
POS Def. Part of Speech POS POS L645 POS = Assigning word class information to words Dept. of Linguistics, Indiana University Fall 2009 ex: the man bought a book determiner noun verb determiner noun 1
More informationA linear algebraic method for pricing temporary life annuities
A linear algebraic method for pricing temporary life annuities P. Date (joint work with R. Mamon, L. Jalen and I.C. Wang) Department of Mathematical Sciences, Brunel University, London Outline Introduction
More informationWe shall turn our attention to solving linear systems of equations. Ax = b
59 Linear Algebra We shall turn our attention to solving linear systems of equations Ax = b where A R m n, x R n, and b R m. We already saw examples of methods that required the solution of a linear system
More informationLECTURE 4. Last time: Lecture outline
LECTURE 4 Last time: Types of convergence Weak Law of Large Numbers Strong Law of Large Numbers Asymptotic Equipartition Property Lecture outline Stochastic processes Markov chains Entropy rate Random
More informationNear Optimal Solutions
Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.
More informationDevelopment of dynamically evolving and self-adaptive software. 1. Background
Development of dynamically evolving and self-adaptive software 1. Background LASER 2013 Isola d Elba, September 2013 Carlo Ghezzi Politecnico di Milano Deep-SE Group @ DEIB 1 Requirements Functional requirements
More informationIntroduction to Logic in Computer Science: Autumn 2006
Introduction to Logic in Computer Science: Autumn 2006 Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam Ulle Endriss 1 Plan for Today Now that we have a basic understanding
More informationHow To Understand The Relation Between Simplicity And Probability In Computer Science
Chapter 6 Computation 6.1 Introduction In the last two chapters we saw that both the logical and the cognitive models of scientific discovery include a condition to prefer simple or minimal explanations.
More informationFUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT MINING SYSTEM
International Journal of Innovative Computing, Information and Control ICIC International c 0 ISSN 34-48 Volume 8, Number 8, August 0 pp. 4 FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT
More informationImproving Data Driven Part-of-Speech Tagging by Morphologic Knowledge Induction
Improving Data Driven Part-of-Speech Tagging by Morphologic Knowledge Induction Uwe D. Reichel Department of Phonetics and Speech Communication University of Munich reichelu@phonetik.uni-muenchen.de Abstract
More informationOptimization Modeling for Mining Engineers
Optimization Modeling for Mining Engineers Alexandra M. Newman Division of Economics and Business Slide 1 Colorado School of Mines Seminar Outline Linear Programming Integer Linear Programming Slide 2
More informationRemoving Partial Inconsistency in Valuation- Based Systems*
Removing Partial Inconsistency in Valuation- Based Systems* Luis M. de Campos and Serafín Moral Departamento de Ciencias de la Computación e I.A., Universidad de Granada, 18071 Granada, Spain This paper
More informationDiscrete Optimization
Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using
More informationHidden Markov Models in Bioinformatics. By Máthé Zoltán Kőrösi Zoltán 2006
Hidden Markov Models in Bioinformatics By Máthé Zoltán Kőrösi Zoltán 2006 Outline Markov Chain HMM (Hidden Markov Model) Hidden Markov Models in Bioinformatics Gene Finding Gene Finding Model Viterbi algorithm
More informationIntroduction to Markov Chain Monte Carlo
Introduction to Markov Chain Monte Carlo Monte Carlo: sample from a distribution to estimate the distribution to compute max, mean Markov Chain Monte Carlo: sampling using local information Generic problem
More informationStatistical Machine Translation
Statistical Machine Translation Some of the content of this lecture is taken from previous lectures and presentations given by Philipp Koehn and Andy Way. Dr. Jennifer Foster National Centre for Language
More information5.1 Bipartite Matching
CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson
More informationLearning is a very general term denoting the way in which agents:
What is learning? Learning is a very general term denoting the way in which agents: Acquire and organize knowledge (by building, modifying and organizing internal representations of some external reality);
More informationWhat is Linear Programming?
Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to
More informationHardware Implementation of Probabilistic State Machine for Word Recognition
IJECT Vo l. 4, Is s u e Sp l - 5, Ju l y - Se p t 2013 ISSN : 2230-7109 (Online) ISSN : 2230-9543 (Print) Hardware Implementation of Probabilistic State Machine for Word Recognition 1 Soorya Asokan, 2
More informationTraining and evaluation of POS taggers on the French MULTITAG corpus
Training and evaluation of POS taggers on the French MULTITAG corpus A. Allauzen, H. Bonneau-Maynard LIMSI/CNRS; Univ Paris-Sud, Orsay, F-91405 {allauzen,maynard}@limsi.fr Abstract The explicit introduction
More informationNP-Completeness and Cook s Theorem
NP-Completeness and Cook s Theorem Lecture notes for COM3412 Logic and Computation 15th January 2002 1 NP decision problems The decision problem D L for a formal language L Σ is the computational task:
More informationBayesian Statistics: Indian Buffet Process
Bayesian Statistics: Indian Buffet Process Ilker Yildirim Department of Brain and Cognitive Sciences University of Rochester Rochester, NY 14627 August 2012 Reference: Most of the material in this note
More informationNatural Language Database Interface for the Community Based Monitoring System *
Natural Language Database Interface for the Community Based Monitoring System * Krissanne Kaye Garcia, Ma. Angelica Lumain, Jose Antonio Wong, Jhovee Gerard Yap, Charibeth Cheng De La Salle University
More informationSymbiosis of Evolutionary Techniques and Statistical Natural Language Processing
1 Symbiosis of Evolutionary Techniques and Statistical Natural Language Processing Lourdes Araujo Dpto. Sistemas Informáticos y Programación, Univ. Complutense, Madrid 28040, SPAIN (email: lurdes@sip.ucm.es)
More informationConditional Random Fields: An Introduction
Conditional Random Fields: An Introduction Hanna M. Wallach February 24, 2004 1 Labeling Sequential Data The task of assigning label sequences to a set of observation sequences arises in many fields, including
More informationChapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling
Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one
More informationNetwork (Tree) Topology Inference Based on Prüfer Sequence
Network (Tree) Topology Inference Based on Prüfer Sequence C. Vanniarajan and Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology Madras Chennai 600036 vanniarajanc@hcl.in,
More informationCell Phone based Activity Detection using Markov Logic Network
Cell Phone based Activity Detection using Markov Logic Network Somdeb Sarkhel sxs104721@utdallas.edu 1 Introduction Mobile devices are becoming increasingly sophisticated and the latest generation of smart
More informationSoftware Modeling and Verification
Software Modeling and Verification Alessandro Aldini DiSBeF - Sezione STI University of Urbino Carlo Bo Italy 3-4 February 2015 Algorithmic verification Correctness problem Is the software/hardware system
More informationIntroduction to Algorithmic Trading Strategies Lecture 2
Introduction to Algorithmic Trading Strategies Lecture 2 Hidden Markov Trading Model Haksun Li haksun.li@numericalmethod.com www.numericalmethod.com Outline Carry trade Momentum Valuation CAPM Markov chain
More information5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition
More informationTransportation Polytopes: a Twenty year Update
Transportation Polytopes: a Twenty year Update Jesús Antonio De Loera University of California, Davis Based on various papers joint with R. Hemmecke, E.Kim, F. Liu, U. Rothblum, F. Santos, S. Onn, R. Yoshida,
More informationOn the Unique Games Conjecture
On the Unique Games Conjecture Antonios Angelakis National Technical University of Athens June 16, 2015 Antonios Angelakis (NTUA) Theory of Computation June 16, 2015 1 / 20 Overview 1 Introduction 2 Preliminary
More informationLing 201 Syntax 1. Jirka Hana April 10, 2006
Overview of topics What is Syntax? Word Classes What to remember and understand: Ling 201 Syntax 1 Jirka Hana April 10, 2006 Syntax, difference between syntax and semantics, open/closed class words, all
More informationMonitoring Metric First-order Temporal Properties
Monitoring Metric First-order Temporal Properties DAVID BASIN, FELIX KLAEDTKE, SAMUEL MÜLLER, and EUGEN ZĂLINESCU, ETH Zurich Runtime monitoring is a general approach to verifying system properties at
More informationMatrix-Chain Multiplication
Matrix-Chain Multiplication Let A be an n by m matrix, let B be an m by p matrix, then C = AB is an n by p matrix. C = AB can be computed in O(nmp) time, using traditional matrix multiplication. Suppose
More informationIBM Research Report. A Pricing Problem under Monge Property
RC23823 (W0512-010) December 2, 2005 Mathematics IBM Research Report A Pricing Problem under Monge Property Oktay Günlük IBM Research Division Thomas J. Watson Research Center P.O. Box 218 Yorktown Heights,
More informationChapter 5. Phrase-based models. Statistical Machine Translation
Chapter 5 Phrase-based models Statistical Machine Translation Motivation Word-Based Models translate words as atomic units Phrase-Based Models translate phrases as atomic units Advantages: many-to-many
More informationReliable and Cost-Effective PoS-Tagging
Reliable and Cost-Effective PoS-Tagging Yu-Fang Tsai Keh-Jiann Chen Institute of Information Science, Academia Sinica Nanang, Taipei, Taiwan 5 eddie,chen@iis.sinica.edu.tw Abstract In order to achieve
More informationAttribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)
Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision
More informationChapter 6. Decoding. Statistical Machine Translation
Chapter 6 Decoding Statistical Machine Translation Decoding We have a mathematical model for translation p(e f) Task of decoding: find the translation e best with highest probability Two types of error
More informationIntroduction to Online Learning Theory
Introduction to Online Learning Theory Wojciech Kot lowski Institute of Computing Science, Poznań University of Technology IDSS, 04.06.2013 1 / 53 Outline 1 Example: Online (Stochastic) Gradient Descent
More informationDiscuss the size of the instance for the minimum spanning tree problem.
3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can
More informationTHREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC- TERISTICS
THREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC- TERISTICS O.U. Sezerman 1, R. Islamaj 2, E. Alpaydin 2 1 Laborotory of Computational Biology, Sabancı University, Istanbul, Turkey. 2 Computer Engineering
More information! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of
More informationDiablo Valley College Catalog 2014-2015
Mathematics MATH Michael Norris, Interim Dean Math and Computer Science Division Math Building, Room 267 Possible career opportunities Mathematicians work in a variety of fields, among them statistics,
More information24. The Branch and Bound Method
24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no
More informationVersion Spaces. riedmiller@informatik.uni-freiburg.de
. Machine Learning Version Spaces Prof. Dr. Martin Riedmiller AG Maschinelles Lernen und Natürlichsprachliche Systeme Institut für Informatik Technische Fakultät Albert-Ludwigs-Universität Freiburg riedmiller@informatik.uni-freiburg.de
More informationError Log Processing for Accurate Failure Prediction. Humboldt-Universität zu Berlin
Error Log Processing for Accurate Failure Prediction Felix Salfner ICSI Berkeley Steffen Tschirpke Humboldt-Universität zu Berlin Introduction Context of work: Error-based online failure prediction: error
More informationLinear Programming I
Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins
More informationMarkov Decision Processes for Ad Network Optimization
Markov Decision Processes for Ad Network Optimization Flávio Sales Truzzi 1, Valdinei Freire da Silva 2, Anna Helena Reali Costa 1, Fabio Gagliardi Cozman 3 1 Laboratório de Técnicas Inteligentes (LTI)
More informationFixed-Point Logics and Computation
1 Fixed-Point Logics and Computation Symposium on the Unusual Effectiveness of Logic in Computer Science University of Cambridge 2 Mathematical Logic Mathematical logic seeks to formalise the process of
More informationA Learning Based Method for Super-Resolution of Low Resolution Images
A Learning Based Method for Super-Resolution of Low Resolution Images Emre Ugur June 1, 2004 emre.ugur@ceng.metu.edu.tr Abstract The main objective of this project is the study of a learning based method
More informationCoding and decoding with convolutional codes. The Viterbi Algor
Coding and decoding with convolutional codes. The Viterbi Algorithm. 8 Block codes: main ideas Principles st point of view: infinite length block code nd point of view: convolutions Some examples Repetition
More informationLecture 2 Matrix Operations
Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrix-vector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or
More informationEnglish. Universidad Virtual. Curso de sensibilización a la PAEP (Prueba de Admisión a Estudios de Posgrado) Parts of Speech. Nouns.
English Parts of speech Parts of Speech There are eight parts of speech. Here are some of their highlights. Nouns Pronouns Adjectives Articles Verbs Adverbs Prepositions Conjunctions Click on any of the
More informationBig Ideas in Mathematics
Big Ideas in Mathematics which are important to all mathematics learning. (Adapted from the NCTM Curriculum Focal Points, 2006) The Mathematics Big Ideas are organized using the PA Mathematics Standards
More informationHidden Markov Models
8.47 Introduction to omputational Molecular Biology Lecture 7: November 4, 2004 Scribe: Han-Pang hiu Lecturer: Ross Lippert Editor: Russ ox Hidden Markov Models The G island phenomenon The nucleotide frequencies
More informationActivity recognition in ADL settings. Ben Kröse b.j.a.krose@uva.nl
Activity recognition in ADL settings Ben Kröse b.j.a.krose@uva.nl Content Why sensor monitoring for health and wellbeing? Activity monitoring from simple sensors Cameras Co-design and privacy issues Necessity
More informationCourse: Model, Learning, and Inference: Lecture 5
Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.
More information! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.
Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three
More informationInferring Probabilistic Models of cis-regulatory Modules. BMI/CS 776 www.biostat.wisc.edu/bmi776/ Spring 2015 Colin Dewey cdewey@biostat.wisc.
Inferring Probabilistic Models of cis-regulatory Modules MI/S 776 www.biostat.wisc.edu/bmi776/ Spring 2015 olin Dewey cdewey@biostat.wisc.edu Goals for Lecture the key concepts to understand are the following
More informationSearch and Data Mining: Techniques. Text Mining Anya Yarygina Boris Novikov
Search and Data Mining: Techniques Text Mining Anya Yarygina Boris Novikov Introduction Generally used to denote any system that analyzes large quantities of natural language text and detects lexical or
More informationSimilarity Search and Mining in Uncertain Spatial and Spatio Temporal Databases. Andreas Züfle
Similarity Search and Mining in Uncertain Spatial and Spatio Temporal Databases Andreas Züfle Geo Spatial Data Huge flood of geo spatial data Modern technology New user mentality Great research potential
More informationON THE DEGREE OF MAXIMALITY OF DEFINITIONALLY COMPLETE LOGICS
Bulletin of the Section of Logic Volume 15/2 (1986), pp. 72 79 reedition 2005 [original edition, pp. 72 84] Ryszard Ladniak ON THE DEGREE OF MAXIMALITY OF DEFINITIONALLY COMPLETE LOGICS The following definition
More informationPerron vector Optimization applied to search engines
Perron vector Optimization applied to search engines Olivier Fercoq INRIA Saclay and CMAP Ecole Polytechnique May 18, 2011 Web page ranking The core of search engines Semantic rankings (keywords) Hyperlink
More information6.080/6.089 GITCS Feb 12, 2008. Lecture 3
6.8/6.89 GITCS Feb 2, 28 Lecturer: Scott Aaronson Lecture 3 Scribe: Adam Rogal Administrivia. Scribe notes The purpose of scribe notes is to transcribe our lectures. Although I have formal notes of my
More information3.1 Solving Systems Using Tables and Graphs
Algebra 2 Chapter 3 3.1 Solve Systems Using Tables & Graphs 3.1 Solving Systems Using Tables and Graphs A solution to a system of linear equations is an that makes all of the equations. To solve a system
More informationGambling and Data Compression
Gambling and Data Compression Gambling. Horse Race Definition The wealth relative S(X) = b(x)o(x) is the factor by which the gambler s wealth grows if horse X wins the race, where b(x) is the fraction
More informationMaximum Likelihood Graph Structure Estimation with Degree Distributions
Maximum Likelihood Graph Structure Estimation with Distributions Bert Huang Computer Science Department Columbia University New York, NY 17 bert@cs.columbia.edu Tony Jebara Computer Science Department
More informationCHAPTER 2 Estimating Probabilities
CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a
More informationAlgorithmic Software Verification
Algorithmic Software Verification (LTL Model Checking) Azadeh Farzan What is Verification Anyway? Proving (in a formal way) that program satisfies a specification written in a logical language. Formal
More informationComputational Methods for Database Repair by Signed Formulae
Computational Methods for Database Repair by Signed Formulae Ofer Arieli (oarieli@mta.ac.il) Department of Computer Science, The Academic College of Tel-Aviv, 4 Antokolski street, Tel-Aviv 61161, Israel.
More informationApplied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
More informationOn the Modeling and Verification of Security-Aware and Process-Aware Information Systems
On the Modeling and Verification of Security-Aware and Process-Aware Information Systems 29 August 2011 What are workflows to us? Plans or schedules that map users or resources to tasks Such mappings may
More informationModel Checking: An Introduction
Announcements Model Checking: An Introduction Meeting 2 Office hours M 1:30pm-2:30pm W 5:30pm-6:30pm (after class) and by appointment ECOT 621 Moodle problems? Fundamentals of Programming Languages CSCI
More informationApproximation Algorithms
Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms
More informationDATA MINING IN FINANCE
DATA MINING IN FINANCE Advances in Relational and Hybrid Methods by BORIS KOVALERCHUK Central Washington University, USA and EVGENII VITYAEV Institute of Mathematics Russian Academy of Sciences, Russia
More informationSystem BV is NP-complete
System BV is NP-complete Ozan Kahramanoğulları 1,2 Computer Science Institute, University of Leipzig International Center for Computational Logic, TU Dresden Abstract System BV is an extension of multiplicative
More informationHow To Find An Optimal Search Protocol For An Oblivious Cell
The Conference Call Search Problem in Wireless Networks Leah Epstein 1, and Asaf Levin 2 1 Department of Mathematics, University of Haifa, 31905 Haifa, Israel. lea@math.haifa.ac.il 2 Department of Statistics,
More informationLeast-Squares Intersection of Lines
Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a
More informationDiscrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2 Proofs Intuitively, the concept of proof should already be familiar We all like to assert things, and few of us
More informationMath 1050 Khan Academy Extra Credit Algebra Assignment
Math 1050 Khan Academy Extra Credit Algebra Assignment KhanAcademy.org offers over 2,700 instructional videos, including hundreds of videos teaching algebra concepts, and corresponding problem sets. In
More informationSTATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. Clarificationof zonationprocedure described onpp. 238-239
STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. by John C. Davis Clarificationof zonationprocedure described onpp. 38-39 Because the notation used in this section (Eqs. 4.8 through 4.84) is inconsistent
More informationSoftware Verification and Testing. Lecture Notes: Temporal Logics
Software Verification and Testing Lecture Notes: Temporal Logics Motivation traditional programs (whether terminating or non-terminating) can be modelled as relations are analysed wrt their input/output
More informationMathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
More informationUniversality in the theory of algorithms and computer science
Universality in the theory of algorithms and computer science Alexander Shen Computational models The notion of computable function was introduced in 1930ies. Simplifying (a rather interesting and puzzling)
More informationPrinciples of Data Mining by Hand&Mannila&Smyth
Principles of Data Mining by Hand&Mannila&Smyth Slides for Textbook Ari Visa,, Institute of Signal Processing Tampere University of Technology October 4, 2010 Data Mining: Concepts and Techniques 1 Differences
More informationCS91.543 MidTerm Exam 4/1/2004 Name: KEY. Page Max Score 1 18 2 11 3 30 4 15 5 45 6 20 Total 139
CS91.543 MidTerm Exam 4/1/2004 Name: KEY Page Max Score 1 18 2 11 3 30 4 15 5 45 6 20 Total 139 % INTRODUCTION, AI HISTORY AND AGENTS 1. [4 pts. ea.] Briefly describe the following important AI programs.
More informationEXCEL SOLVER TUTORIAL
ENGR62/MS&E111 Autumn 2003 2004 Prof. Ben Van Roy October 1, 2003 EXCEL SOLVER TUTORIAL This tutorial will introduce you to some essential features of Excel and its plug-in, Solver, that we will be using
More informationPoS-tagging Italian texts with CORISTagger
PoS-tagging Italian texts with CORISTagger Fabio Tamburini DSLO, University of Bologna, Italy fabio.tamburini@unibo.it Abstract. This paper presents an evolution of CORISTagger [1], an high-performance
More informationPupil SPAG Card 1. Terminology for pupils. I Can Date Word
Pupil SPAG Card 1 1 I know about regular plural noun endings s or es and what they mean (for example, dog, dogs; wish, wishes) 2 I know the regular endings that can be added to verbs (e.g. helping, helped,
More informationComplexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar
Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples
More informationPermutation Betting Markets: Singleton Betting with Extra Information
Permutation Betting Markets: Singleton Betting with Extra Information Mohammad Ghodsi Sharif University of Technology ghodsi@sharif.edu Hamid Mahini Sharif University of Technology mahini@ce.sharif.edu
More information