Dual Decomposition for Parsing with Non-Projective Head Automata. Terry Koo, Alexander M. Rush, Michael Collins, David Sontag, and Tommi Jaakkola
|
|
|
- Brice Bailey
- 7 years ago
- Views:
Transcription
1 Dual Decomposition for Parsing with Non-Projective Head Automata Terry Koo, Alexander M. Rush, Michael Collins, David Sontag, and Tommi Jaakkola
2 The Cost of Model Complexity We are always looking for better ways to model natural language. Tradeoff: Richer models Harder decoding Added complexity is both computational and implementational.
3 The Cost of Model Complexity We are always looking for better ways to model natural language. Tradeoff: Richer models Harder decoding Added complexity is both computational and implementational. Tasks with challenging decoding problems: Speech Recognition Sequence Modeling (e.g. extensions to HMM/CRF) Parsing Machine Translation
4 The Cost of Model Complexity We are always looking for better ways to model natural language. Tradeoff: Richer models Harder decoding Added complexity is both computational and implementational. Tasks with challenging decoding problems: Speech Recognition Sequence Modeling (e.g. extensions to HMM/CRF) Parsing Machine Translation y = arg max f (y) y Decoding
5 Non-Projective Dependency Parsing Important problem in many languages. Problem is NP-Hard for all but the simplest models.
6 Dual Decomposition A classical technique for constructing decoding algorithms. Solve complicated models y = arg max f (y) y by decomposing into smaller problems. Upshot: Can utilize a toolbox of combinatorial algorithms. Dynamic programming Minimum spanning tree Shortest path Min-Cut...
7 A Dual Decomposition Algorithm for Non-Projective Dependency Parsing Simple - Uses basic combinatorial algorithms Efficient - Faster than previously proposed algorithms Strong Guarantees - Gives a certificate of optimality when exact Solves 98% of examples exactly, even though the problem is NP-Hard Widely Applicable - Similar techniques extend to other problems
8 Roadmap Algorithm Experiments Derivation
9 Non-Projective Dependency Parsing Starts at the root symbol * Each word has a exactly one parent word Produces a tree structure (no cycles) Dependencies can cross
10 Algorithm Outline *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Arc-Factored Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Sibling Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8
11 Algorithm Outline *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Arc-Factored Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 + Dual Decomposition *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Sibling Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8
12 Arc-Factored f (y) =
13 Arc-Factored f (y) = score(head = 0, mod =saw 2 )
14 Arc-Factored f (y) = score(head = 0, mod =saw 2 ) +score(saw 2, John 1 )
15 Arc-Factored f (y) = score(head = 0, mod =saw 2 ) +score(saw 2, John 1 ) +score(saw 2, movie 4 )
16 Arc-Factored f (y) = score(head = 0, mod =saw 2 ) +score(saw 2, John 1 ) +score(saw 2, movie 4 ) +score(saw 2, today 5 )
17 Arc-Factored f (y) = score(head = 0, mod =saw 2 ) +score(saw 2, John 1 ) +score(saw 2, movie 4 ) +score(saw 2, today 5 ) +score(movie 4, a 3 ) +...
18 Arc-Factored f (y) = score(head = 0, mod =saw 2 ) +score(saw 2, John 1 ) +score(saw 2, movie 4 ) +score(saw 2, today 5 ) +score(movie 4, a 3 ) +... e.g. score( 0, saw 2 ) = log p(saw 2 0 ) (generative model)
19 Arc-Factored f (y) = score(head = 0, mod =saw 2 ) +score(saw 2, John 1 ) +score(saw 2, movie 4 ) +score(saw 2, today 5 ) +score(movie 4, a 3 ) +... e.g. score( 0, saw 2 ) = log p(saw 2 0 ) or score( 0, saw 2 ) = w φ(saw 2, 0 ) (generative model) (CRF/perceptron model)
20 Arc-Factored f (y) = score(head = 0, mod =saw 2 ) +score(saw 2, John 1 ) +score(saw 2, movie 4 ) +score(saw 2, today 5 ) +score(movie 4, a 3 ) +... e.g. score( 0, saw 2 ) = log p(saw 2 0 ) or score( 0, saw 2 ) = w φ(saw 2, 0 ) (generative model) (CRF/perceptron model) y = arg max f (y) Minimum Spanning Tree Algorithm y
21 Sibling Models f (y) =
22 Sibling Models f (y) = score(head = 0, prev = NULL, mod = saw 2 )
23 Sibling Models f (y) = score(head = 0, prev = NULL, mod = saw 2 ) +score(saw 2, NULL, John 1 )
24 Sibling Models f (y) = score(head = 0, prev = NULL, mod = saw 2 ) +score(saw 2, NULL, John 1 ) +score(saw 2, NULL, movie 4 )
25 Sibling Models f (y) = score(head = 0, prev = NULL, mod = saw 2 ) +score(saw 2, NULL, John 1 ) +score(saw 2, NULL, movie 4 ) +score(saw 2,movie 4, today 5 ) +...
26 Sibling Models f (y) = score(head = 0, prev = NULL, mod = saw 2 ) +score(saw 2, NULL, John 1 ) +score(saw 2, NULL, movie 4 ) +score(saw 2,movie 4, today 5 ) +... e.g. score(saw 2, movie 4, today 5 ) = log p(today 5 saw 2, movie 4 )
27 Sibling Models f (y) = score(head = 0, prev = NULL, mod = saw 2 ) +score(saw 2, NULL, John 1 ) +score(saw 2, NULL, movie 4 ) +score(saw 2,movie 4, today 5 ) +... e.g. score(saw 2, movie 4, today 5 ) = log p(today 5 saw 2, movie 4 ) or score(saw 2, movie 4, today 5 ) = w φ(saw 2, movie 4, today 5 )
28 Sibling Models f (y) = score(head = 0, prev = NULL, mod = saw 2 ) +score(saw 2, NULL, John 1 ) +score(saw 2, NULL, movie 4 ) +score(saw 2,movie 4, today 5 ) +... e.g. score(saw 2, movie 4, today 5 ) = log p(today 5 saw 2, movie 4 ) or score(saw 2, movie 4, today 5 ) = w φ(saw 2, movie 4, today 5 ) y = arg max f (y) NP-Hard y
29 Thought Experiment: Individual Decoding
30 Thought Experiment: Individual Decoding score(saw 2, NULL, John 1 ) + score(saw 2, NULL, movie 4 ) +score(saw 2, movie 4, today 5 )
31 Thought Experiment: Individual Decoding score(saw 2, NULL, John 1 ) + score(saw 2, NULL, movie 4 ) +score(saw 2, movie 4, today 5 ) score(saw 2, NULL, John 1 ) + score(saw 2, NULL, that 6 )
32 Thought Experiment: Individual Decoding score(saw 2, NULL, John 1 ) + score(saw 2, NULL, movie 4 ) +score(saw 2, movie 4, today 5 ) score(saw 2, NULL, John 1 ) + score(saw 2, NULL, that 6 ) score(saw 2, NULL, a 3 ) + score(saw 2, a 3, he 7 )
33 Thought Experiment: Individual Decoding score(saw 2, NULL, John 1 ) + score(saw 2, NULL, movie 4 ) +score(saw 2, movie 4, today 5 ) 2 n 1 possibilities score(saw 2, NULL, John 1 ) + score(saw 2, NULL, that 6 ) score(saw 2, NULL, a 3 ) + score(saw 2, a 3, he 7 )
34 Thought Experiment: Individual Decoding score(saw 2, NULL, John 1 ) + score(saw 2, NULL, movie 4 ) +score(saw 2, movie 4, today 5 ) 2 n 1 possibilities score(saw 2, NULL, John 1 ) + score(saw 2, NULL, that 6 ) score(saw 2, NULL, a 3 ) + score(saw 2, a 3, he 7 ) Under Sibling Model, can solve for each word with Viterbi decoding.
35 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
36 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
37 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
38 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
39 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
40 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
41 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
42 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree.
43 Thought Experiment Continued Idea: Do individual decoding for each head word using dynamic programming. If we re lucky, we ll end up with a valid final tree. But we might violate some constraints.
44 Dual Decomposition Idea No Constraints Tree Constraints Arc- Factored Minimum Spanning Tree Sibling Model Individual Decoding
45 Dual Decomposition Idea No Constraints Tree Constraints Arc- Factored Minimum Spanning Tree Sibling Model Individual Decoding Dual Decomposition
46 Dual Decomposition Structure Goal y = arg max y Y f (y)
47 Dual Decomposition Structure Goal y = arg max y Y f (y) Rewrite as argmax z Z, y Y f (z) + g(y) such that z = y
48 Dual Decomposition Structure Goal y = arg max y Y f (y) Rewrite as argmax z Z, y Y f (z) + g(y) All Possible such that z = y
49 Dual Decomposition Structure Goal y = arg max y Y f (y) Rewrite as argmax z Z, y Y f (z) + g(y) All Possible Valid Trees such that z = y
50 Dual Decomposition Structure Goal y = arg max y Y f (y) Sibling Rewrite as argmax z Z, y Y f (z) + g(y) All Possible Valid Trees such that z = y
51 Dual Decomposition Structure Goal y = arg max y Y f (y) Sibling Arc-Factored Rewrite as argmax z Z, y Y f (z) + g(y) All Possible Valid Trees such that z = y
52 Dual Decomposition Structure Goal y = arg max y Y f (y) Sibling Arc-Factored Rewrite as argmax z Z, y Y f (z) + g(y) All Possible Valid Trees such that z = y Constraint
53 Algorithm Sketch Set penalty weights equal to 0 for all edges. For k = 1 to K
54 Algorithm Sketch Set penalty weights equal to 0 for all edges. For k = 1 to K z (k) Decode (f (z) + penalty) by Individual Decoding
55 Algorithm Sketch Set penalty weights equal to 0 for all edges. For k = 1 to K z (k) Decode (f (z) + penalty) by Individual Decoding y (k) Decode (g(y) penalty) by Minimum Spanning Tree
56 Algorithm Sketch Set penalty weights equal to 0 for all edges. For k = 1 to K z (k) Decode (f (z) + penalty) by Individual Decoding y (k) Decode (g(y) penalty) by Minimum Spanning Tree If y (k) (i, j) = z (k) (i, j) for all i, j Return (y (k), z (k) )
57 Algorithm Sketch Set penalty weights equal to 0 for all edges. For k = 1 to K z (k) Decode (f (z) + penalty) by Individual Decoding y (k) Decode (g(y) penalty) by Minimum Spanning Tree If y (k) (i, j) = z (k) (i, j) for all i, j Return (y (k), z (k) ) Else Update penalty weights based on y (k) (i, j) z (k) (i, j)
58 Individual Decoding Penalties u(i, j) = 0 for all i,j z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
59 Individual Decoding Penalties u(i, j) = 0 for all i,j z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
60 Individual Decoding Penalties u(i, j) = 0 for all i,j z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
61 Individual Decoding Penalties u(i, j) = 0 for all i,j z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
62 Individual Decoding z = arg max z Z (f (z) + i,j u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Minimum Spanning Tree Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
63 Individual Decoding z = arg max z Z (f (z) + i,j u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Minimum Spanning Tree Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
64 Individual Decoding z = arg max z Z (f (z) + i,j u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Minimum Spanning Tree Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
65 Individual Decoding z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Iteration 2 u(8, 1) -1 u(4, 6) -2 u(2, 6) 2 u(8, 7) 1 Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
66 Individual Decoding z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Iteration 2 u(8, 1) -1 u(4, 6) -2 u(2, 6) 2 u(8, 7) 1 Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
67 Individual Decoding z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Iteration 2 u(8, 1) -1 u(4, 6) -2 u(2, 6) 2 u(8, 7) 1 Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
68 Individual Decoding z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Iteration 2 u(8, 1) -1 u(4, 6) -2 u(2, 6) 2 u(8, 7) 1 Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j
69 Individual Decoding z = arg max z Z (f (z) + i,j Minimum Spanning Tree u(i, j)z(i, j)) Penalties u(i, j) = 0 for all i,j Iteration 1 u(8, 1) -1 u(4, 6) -1 u(2, 6) 1 u(8, 7) 1 Iteration 2 u(8, 1) -1 u(4, 6) -2 u(2, 6) 2 u(8, 7) 1 Key y = arg max y Y (g(y) i,j u(i, j)y(i, j)) f (z) Sibling Model g(y) Arc-Factored Model Z No Constraints Y Tree Constraints y(i, j) = 1 if y contains dependency i, j Converged y = arg max f (y) + g(y) y Y
70 Guarantees Theorem If at any iteration y (k) = z (k), then (y (k), z (k) ) is the global optimum. In experiments, we find the global optimum on 98% of examples.
71 Guarantees Theorem If at any iteration y (k) = z (k), then (y (k), z (k) ) is the global optimum. In experiments, we find the global optimum on 98% of examples. If we do not converge to a match, we can still return an approximate solution (more in the paper).
72 Extensions Grandparent Models f (y) =...+ score(gp = 0, head = saw 2, prev =movie 4, mod =today 5 ) Head Automata (Eisner, 2000) Generalization of Sibling models Allow arbitrary automata as local scoring function.
73 Roadmap Algorithm Experiments Derivation
74 Properties: Exactness Experiments Parsing Speed Parsing Accuracy Comparison to Individual Decoding Comparison to LP/ILP Training: Averaged Perceptron (more details in paper) Experiments on: CoNLL Datasets English Penn Treebank Czech Dependency Treebank
75 How often do we exactly solve the problem? Cze Eng Dan Dut Por Slo Swe Tur Percentage of examples where the dual decomposition finds an exact solution.
76 Parsing Speed Cze Eng Dan Dut Por Slo Swe Tur 0 Cze Eng Dan Dut Por Slo Swe Tur Sibling model Grandparent model Number of sentences parsed per second Comparable to dynamic programming for projective parsing
77 Accuracy Arc-Factored Prev Best Grandparent Dan Dut Por Slo Swe Tur Eng Cze Prev Best - Best reported results for CoNLL-X data set, includes Approximate search (McDonald and Pereira, 2006) Loop belief propagation (Smith and Eisner, 2008) (Integer) Linear Programming (Martins et al., 2009)
78 Eng Comparison to Subproblems Individual MST Dual F 1 for dependency accuracy
79 Comparison to LP/ILP Martins et al.(2009): Proposes two representations of non-projective dependency parsing as a linear programming relaxation as well as an exact ILP. LP (1) LP (2) ILP Use an LP/ILP Solver for decoding We compare: Accuracy Exactness Speed Both LP and dual decomposition methods use the same model, features, and weights w.
80 Comparison to LP/ILP: Accuracy LP(1) LP(2) ILP Dual Dependency Accuracy All decoding methods have comparable accuracy
81 Comparison to LP/ILP: Exactness and Speed LP(1) LP(2) ILP Dual LP(1) LP(2) ILP Dual Percentage with exact solution Sentences per second
82 Roadmap Algorithm Experiments Derivation
83 Deriving the Algorithm Goal: y = arg max y Y f (y) Rewrite: arg max z Z,y Y f (z) + g(y) s.t. z(i, j) = y(i, j) for all i, j Lagrangian: L(u, y, z) = f (z) + g(y) + i,j u(i, j) (z(i, j) y(i, j))
84 Deriving the Algorithm Goal: y = arg max y Y f (y) Rewrite: arg max z Z,y Y f (z) + g(y) s.t. z(i, j) = y(i, j) for all i, j Lagrangian: L(u, y, z) = f (z) + g(y) + i,j u(i, j) (z(i, j) y(i, j)) The dual problem is to find min u L(u) = max y Y,z Z L(u) where L(u, y, z) = max z Z + max y Y f (z) + i,j g(y) i,j u(i, j)z(i, j) u(i, j)y(i, j) Dual is an upper bound: L(u) f (z ) + g(y ) for any u
85 A Subgradient Algorithm for Minimizing L(u) L(u) = max z Z f (z) + u(i, j)z(i, j) + max g(y) u(i, j)y(i, j) y Y i,j i,j L(u) is convex, but not differentiable. A subgradient of L(u) at u is a vector g u such that for all v, L(v) L(u) + g u (v u) Subgradient methods use updates u = u αg u In fact, for our L(u), g u (i, j) = z (i, j) y (i, j)
86 Related Work Methods that use general purpose linear programming or integer linear programming solvers (Martins et al. 2009; Riedel and Clarke 2006; Roth and Yih 2005) Dual decomposition/lagrangian relaxation in combinatorial optimization (Dantzig and Wolfe, 1960; Held and Karp, 1970; Fisher 1981) Dual decomposition for inference in MRFs (Komodakis et al., 2007; Wainwright et al., 2005) Methods that incorporate combinatorial solvers within loopy belief propagation (Duchi et al. 2007; Smith and Eisner 2008)
87 Summary y = arg max f (y) NP-Hard y *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Arc-Factored Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Sibling Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8
88 Summary y = arg max f (y) NP-Hard y *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Arc-Factored Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 + Dual Decomposition *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Sibling Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8
89 Other Applications Dual decomposition can be applied to other decoding problems. Rush et al. (2010) focuses on integrated dynamic programming algorithms. Integrated Parsing and Tagging Integrated Constituency and Dependency Parsing
90 Parsing and Tagging y = arg max f (y) Slow y *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 S N V D N A D N V HMM Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 NP S VP CFG Model N V NP ADVP ADVP John saw D NP ADV D VP a N today that N V movie he liked
91 Parsing and Tagging y = arg max f (y) Slow y *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 S N V D N A D N V HMM Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 + Dual Decomposition *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 NP S VP CFG Model N V NP ADVP ADVP John saw D NP ADV D VP a N today that N V movie he liked
92 Dependency and Constituency y = arg max f (y) Slow y *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Dependency Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 NP S VP Lexicalized CFG N V NP ADVP ADVP John saw D NP ADV D VP a N today that N V movie he liked
93 Dependency and Constituency y = arg max f (y) Slow y *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 Dependency Model *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 + Dual Decomposition *0 John1 saw2 a3 movie4 today5 that6 he7 liked8 NP S VP Lexicalized CFG N V NP ADVP ADVP John saw D NP ADV D VP a N today that N V movie he liked
94 Future Directions There is much more to explore around dual decomposition in NLP. Known Techniques Generalization to more than two models K-best decoding Approximate subgradient Heuristic for branch-and-bound type search Possible NLP Applications Machine Translation Speech Recognition Loopy Sequence Models Open Questions Can we speed up subalgorithms when running repeatedly? What are the trade-offs of different decompositions? Are there better methods for optimizing the dual?
95 Appendix
96 Training the Model f (y) =... + score(saw 2,movie 4, today 5 ) +... score(saw 2, movie 4, today 5 ) = w φ(saw 2, movie 4, today 5 ) Weight vector w trained using Averaged perceptron. (More details in the paper.)
97 Early Stopping Percentage % validation UAS % certificates % match K= Maximum Number of Dual Decomposition Iterations Early Stopping
98 Caching % of Head Automata Recomputed % recomputed, g+s % recomputed, sib Iterations of Dual Decomposition Caching speed
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing Alexander M. Rush,2 Michael Collins 2 SRUSH@CSAIL.MIT.EDU MCOLLINS@CS.COLUMBIA.EDU Computer Science
Statistical Machine Translation: IBM Models 1 and 2
Statistical Machine Translation: IBM Models 1 and 2 Michael Collins 1 Introduction The next few lectures of the course will be focused on machine translation, and in particular on statistical machine translation
DEPENDENCY PARSING JOAKIM NIVRE
DEPENDENCY PARSING JOAKIM NIVRE Contents 1. Dependency Trees 1 2. Arc-Factored Models 3 3. Online Learning 3 4. Eisner s Algorithm 4 5. Spanning Tree Parsing 6 References 7 A dependency parser analyzes
Tagging with Hidden Markov Models
Tagging with Hidden Markov Models Michael Collins 1 Tagging Problems In many NLP problems, we would like to model pairs of sequences. Part-of-speech (POS) tagging is perhaps the earliest, and most famous,
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition
A Constraint Programming based Column Generation Approach to Nurse Rostering Problems
Abstract A Constraint Programming based Column Generation Approach to Nurse Rostering Problems Fang He and Rong Qu The Automated Scheduling, Optimisation and Planning (ASAP) Group School of Computer Science,
Dantzig-Wolfe bound and Dantzig-Wolfe cookbook
Dantzig-Wolfe bound and Dantzig-Wolfe cookbook thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline LP strength of the Dantzig-Wolfe The exercise from last week... The Dantzig-Wolfe
Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology
Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology Yair Weiss School of Computer Science and Engineering The Hebrew University of Jerusalem www.cs.huji.ac.il/ yweiss
Arithmetic Coding: Introduction
Data Compression Arithmetic coding Arithmetic Coding: Introduction Allows using fractional parts of bits!! Used in PPM, JPEG/MPEG (as option), Bzip More time costly than Huffman, but integer implementation
Several Views of Support Vector Machines
Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min
Scheduling Home Health Care with Separating Benders Cuts in Decision Diagrams
Scheduling Home Health Care with Separating Benders Cuts in Decision Diagrams André Ciré University of Toronto John Hooker Carnegie Mellon University INFORMS 2014 Home Health Care Home health care delivery
CHAPTER 9. Integer Programming
CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral
NLP Programming Tutorial 5 - Part of Speech Tagging with Hidden Markov Models
NLP Programming Tutorial 5 - Part of Speech Tagging with Hidden Markov Models Graham Neubig Nara Institute of Science and Technology (NAIST) 1 Part of Speech (POS) Tagging Given a sentence X, predict its
Scheduling Shop Scheduling. Tim Nieberg
Scheduling Shop Scheduling Tim Nieberg Shop models: General Introduction Remark: Consider non preemptive problems with regular objectives Notation Shop Problems: m machines, n jobs 1,..., n operations
Approximation Algorithms
Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms
Learning and Inference over Constrained Output
IJCAI 05 Learning and Inference over Constrained Output Vasin Punyakanok Dan Roth Wen-tau Yih Dav Zimak Department of Computer Science University of Illinois at Urbana-Champaign {punyakan, danr, yih, davzimak}@uiuc.edu
Discuss the size of the instance for the minimum spanning tree problem.
3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can
Integrating Benders decomposition within Constraint Programming
Integrating Benders decomposition within Constraint Programming Hadrien Cambazard, Narendra Jussien email: {hcambaza,jussien}@emn.fr École des Mines de Nantes, LINA CNRS FRE 2729 4 rue Alfred Kastler BP
Recovery of primal solutions from dual subgradient methods for mixed binary linear programming; a branch-and-bound approach
MASTER S THESIS Recovery of primal solutions from dual subgradient methods for mixed binary linear programming; a branch-and-bound approach PAULINE ALDENVIK MIRJAM SCHIERSCHER Department of Mathematical
Support Vector Machines
Support Vector Machines Charlie Frogner 1 MIT 2011 1 Slides mostly stolen from Ryan Rifkin (Google). Plan Regularization derivation of SVMs. Analyzing the SVM problem: optimization, duality. Geometric
MAP-Inference for Highly-Connected Graphs with DC-Programming
MAP-Inference for Highly-Connected Graphs with DC-Programming Jörg Kappes and Christoph Schnörr Image and Pattern Analysis Group, Heidelberg Collaboratory for Image Processing, University of Heidelberg,
! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.
Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three
Optimization Theory for Large Systems
Optimization Theory for Large Systems LEON S. LASDON CASE WESTERN RESERVE UNIVERSITY THE MACMILLAN COMPANY COLLIER-MACMILLAN LIMITED, LONDON Contents 1. Linear and Nonlinear Programming 1 1.1 Unconstrained
Tutorial: Operations Research in Constraint Programming
Tutorial: Operations Research in Constraint Programming John Hooker Carnegie Mellon University May 2009 Revised June 2009 May 2009 Slide 1 Motivation Benders decomposition allows us to apply CP and OR
Dynamic programming. Doctoral course Optimization on graphs - Lecture 4.1. Giovanni Righini. January 17 th, 2013
Dynamic programming Doctoral course Optimization on graphs - Lecture.1 Giovanni Righini January 1 th, 201 Implicit enumeration Combinatorial optimization problems are in general NP-hard and we usually
Algorithm Design and Analysis
Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;
5.1 Bipartite Matching
CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson
Distributed Machine Learning and Big Data
Distributed Machine Learning and Big Data Sourangshu Bhattacharya Dept. of Computer Science and Engineering, IIT Kharagpur. http://cse.iitkgp.ac.in/~sourangshu/ August 21, 2015 Sourangshu Bhattacharya
Course: Model, Learning, and Inference: Lecture 5
Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.
! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of
Multi-layer MPLS Network Design: the Impact of Statistical Multiplexing
Multi-layer MPLS Network Design: the Impact of Statistical Multiplexing Pietro Belotti, Antonio Capone, Giuliana Carello, Federico Malucelli Tepper School of Business, Carnegie Mellon University, Pittsburgh
Big Data Optimization at SAS
Big Data Optimization at SAS Imre Pólik et al. SAS Institute Cary, NC, USA Edinburgh, 2013 Outline 1 Optimization at SAS 2 Big Data Optimization at SAS The SAS HPA architecture Support vector machines
Applied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.
Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure
Log-Linear Models. Michael Collins
Log-Linear Models Michael Collins 1 Introduction This note describes log-linear models, which are very widely used in natural language processing. A key advantage of log-linear models is their flexibility:
Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method
Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming
Language Modeling. Chapter 1. 1.1 Introduction
Chapter 1 Language Modeling (Course notes for NLP by Michael Collins, Columbia University) 1.1 Introduction In this chapter we will consider the the problem of constructing a language model from a set
Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling
Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one
Nonlinear Optimization: Algorithms 3: Interior-point methods
Nonlinear Optimization: Algorithms 3: Interior-point methods INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2006 Jean-Philippe Vert,
Research Paper Business Analytics. Applications for the Vehicle Routing Problem. Jelmer Blok
Research Paper Business Analytics Applications for the Vehicle Routing Problem Jelmer Blok Applications for the Vehicle Routing Problem Jelmer Blok Research Paper Vrije Universiteit Amsterdam Faculteit
The Goldberg Rao Algorithm for the Maximum Flow Problem
The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }
Transition-Based Dependency Parsing with Long Distance Collocations
Transition-Based Dependency Parsing with Long Distance Collocations Chenxi Zhu, Xipeng Qiu (B), and Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science,
Discrete Optimization
Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using
Branch-and-Price Approach to the Vehicle Routing Problem with Time Windows
TECHNISCHE UNIVERSITEIT EINDHOVEN Branch-and-Price Approach to the Vehicle Routing Problem with Time Windows Lloyd A. Fasting May 2014 Supervisors: dr. M. Firat dr.ir. M.A.A. Boon J. van Twist MSc. Contents
Simplified Benders cuts for Facility Location
Simplified Benders cuts for Facility Location Matteo Fischetti, University of Padova based on joint work with Ivana Ljubic (ESSEC, Paris) and Markus Sinnl (ISOR, Vienna) Barcelona, November 2015 1 Apology
Linear Programming Notes V Problem Transformations
Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material
A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem
A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem John Karlof and Peter Hocking Mathematics and Statistics Department University of North Carolina Wilmington Wilmington,
Tekniker för storskalig parsning
Tekniker för storskalig parsning Diskriminativa modeller Joakim Nivre Uppsala Universitet Institutionen för lingvistik och filologi joakim.nivre@lingfil.uu.se Tekniker för storskalig parsning 1(19) Generative
Why? A central concept in Computer Science. Algorithms are ubiquitous.
Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online
The Binary Blocking Flow Algorithm. Andrew V. Goldberg Microsoft Research Silicon Valley www.research.microsoft.com/ goldberg/
The Binary Blocking Flow Algorithm Andrew V. Goldberg Microsoft Research Silicon Valley www.research.microsoft.com/ goldberg/ Theory vs. Practice In theory, there is no difference between theory and practice.
The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,
The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints
Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725
Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T
Equilibrium computation: Part 1
Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium
Statistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
Shallow Parsing with Apache UIMA
Shallow Parsing with Apache UIMA Graham Wilcock University of Helsinki Finland graham.wilcock@helsinki.fi Abstract Apache UIMA (Unstructured Information Management Architecture) is a framework for linguistic
Definition 11.1. Given a graph G on n vertices, we define the following quantities:
Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define
A Column Generation Model for Truck Routing in the Chilean Forest Industry
A Column Generation Model for Truck Routing in the Chilean Forest Industry Pablo A. Rey Escuela de Ingeniería Industrial, Facultad de Ingeniería, Universidad Diego Portales, Santiago, Chile, e-mail: pablo.rey@udp.cl
Semantic parsing with Structured SVM Ensemble Classification Models
Semantic parsing with Structured SVM Ensemble Classification Models Le-Minh Nguyen, Akira Shimazu, and Xuan-Hieu Phan Japan Advanced Institute of Science and Technology (JAIST) Asahidai 1-1, Nomi, Ishikawa,
Lecture 2: The SVM classifier
Lecture 2: The SVM classifier C19 Machine Learning Hilary 2015 A. Zisserman Review of linear classifiers Linear separability Perceptron Support Vector Machine (SVM) classifier Wide margin Cost function
A numerically adaptive implementation of the simplex method
A numerically adaptive implementation of the simplex method József Smidla, Péter Tar, István Maros Department of Computer Science and Systems Technology University of Pannonia 17th of December 2014. 1
Supporting Differentiated QoS in MPLS Networks
Supporting Differentiated QoS in MPLS Networks Roberto A. Dias 1, Eduardo Camponogara 2, and Jean-Marie Farines 2 1 Federal Technology Center of Santa Catarina, Florianópolis, 88020-300, Brazil 2 Federal
How To Solve A Minimum Set Covering Problem (Mcp)
Measuring Rationality with the Minimum Cost of Revealed Preference Violations Mark Dean and Daniel Martin Online Appendices - Not for Publication 1 1 Algorithm for Solving the MASP In this online appendix
Multiple Spanning Tree Protocol (MSTP), Multi Spreading And Network Optimization Model
Load Balancing of Telecommunication Networks based on Multiple Spanning Trees Dorabella Santos Amaro de Sousa Filipe Alvelos Instituto de Telecomunicações 3810-193 Aveiro, Portugal dorabella@av.it.pt Instituto
Lecture 10 Scheduling 1
Lecture 10 Scheduling 1 Transportation Models -1- large variety of models due to the many modes of transportation roads railroad shipping airlines as a consequence different type of equipment and resources
4.6 Linear Programming duality
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal
Scheduling Jobs and Preventive Maintenance Activities on Parallel Machines
Scheduling Jobs and Preventive Maintenance Activities on Parallel Machines Maher Rebai University of Technology of Troyes Department of Industrial Systems 12 rue Marie Curie, 10000 Troyes France maher.rebai@utt.fr
Chapter 13: Binary and Mixed-Integer Programming
Chapter 3: Binary and Mixed-Integer Programming The general branch and bound approach described in the previous chapter can be customized for special situations. This chapter addresses two special situations:
A Service Design Problem for a Railway Network
A Service Design Problem for a Railway Network Alberto Caprara Enrico Malaguti Paolo Toth Dipartimento di Elettronica, Informatica e Sistemistica, University of Bologna Viale Risorgimento, 2-40136 - Bologna
A Simple Introduction to Support Vector Machines
A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Large-margin linear
Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits
Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique
Natural Language Processing. Today. Logistic Regression Models. Lecture 13 10/6/2015. Jim Martin. Multinomial Logistic Regression
Natural Language Processing Lecture 13 10/6/2015 Jim Martin Today Multinomial Logistic Regression Aka log-linear models or maximum entropy (maxent) Components of the model Learning the parameters 10/1/15
An Overview Of Software For Convex Optimization. Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.
An Overview Of Software For Convex Optimization Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu In fact, the great watershed in optimization isn t between linearity
Numerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen
(für Informatiker) M. Grepl J. Berger & J.T. Frings Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2010/11 Problem Statement Unconstrained Optimality Conditions Constrained
ONLINE DEGREE-BOUNDED STEINER NETWORK DESIGN. Sina Dehghani Saeed Seddighin Ali Shafahi Fall 2015
ONLINE DEGREE-BOUNDED STEINER NETWORK DESIGN Sina Dehghani Saeed Seddighin Ali Shafahi Fall 2015 ONLINE STEINER FOREST PROBLEM An initially given graph G. s 1 s 2 A sequence of demands (s i, t i ) arriving
Data Structures Fibonacci Heaps, Amortized Analysis
Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:
Finding the M Most Probable Configurations Using Loopy Belief Propagation
Finding the M Most Probable Configurations Using Loopy Belief Propagation Chen Yanover and Yair Weiss School of Computer Science and Engineering The Hebrew University of Jerusalem 91904 Jerusalem, Israel
17.3.1 Follow the Perturbed Leader
CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.
Open Domain Information Extraction. Günter Neumann, DFKI, 2012
Open Domain Information Extraction Günter Neumann, DFKI, 2012 Improving TextRunner Wu and Weld (2010) Open Information Extraction using Wikipedia, ACL 2010 Fader et al. (2011) Identifying Relations for
Big Data - Lecture 1 Optimization reminders
Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics
Basic Parsing Algorithms Chart Parsing
Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS 2011/2012 Anna Schmidt Talk Outline Chart Parsing Basics Chart Parsing Algorithms Earley Algorithm CKY Algorithm
Single-Link Failure Detection in All-Optical Networks Using Monitoring Cycles and Paths
Single-Link Failure Detection in All-Optical Networks Using Monitoring Cycles and Paths Satyajeet S. Ahuja, Srinivasan Ramasubramanian, and Marwan Krunz Department of ECE, University of Arizona, Tucson,
Models in Transportation. Tim Nieberg
Models in Transportation Tim Nieberg Transportation Models large variety of models due to the many modes of transportation roads railroad shipping airlines as a consequence different type of equipment
An Expressive Auction Design for Online Display Advertising. AUTHORS: Sébastien Lahaie, David C. Parkes, David M. Pennock
An Expressive Auction Design for Online Display Advertising AUTHORS: Sébastien Lahaie, David C. Parkes, David M. Pennock Li PU & Tong ZHANG Motivation Online advertisement allow advertisers to specify
6.852: Distributed Algorithms Fall, 2009. Class 2
.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election
11. APPROXIMATION ALGORITHMS
11. APPROXIMATION ALGORITHMS load balancing center selection pricing method: vertex cover LP rounding: vertex cover generalized load balancing knapsack problem Lecture slides by Kevin Wayne Copyright 2005
Direct Loss Minimization for Structured Prediction
Direct Loss Minimization for Structured Prediction David McAllester TTI-Chicago mcallester@ttic.edu Tamir Hazan TTI-Chicago tamir@ttic.edu Joseph Keshet TTI-Chicago jkeshet@ttic.edu Abstract In discriminative
Online Learning of Approximate Dependency Parsing Algorithms
Online Learning of Approximate Dependency Parsing Algorithms Ryan McDonald Fernando Pereira Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104 {ryantm,pereira}@cis.upenn.edu
Fast Multipole Method for particle interactions: an open source parallel library component
Fast Multipole Method for particle interactions: an open source parallel library component F. A. Cruz 1,M.G.Knepley 2,andL.A.Barba 1 1 Department of Mathematics, University of Bristol, University Walk,
Transportation Polytopes: a Twenty year Update
Transportation Polytopes: a Twenty year Update Jesús Antonio De Loera University of California, Davis Based on various papers joint with R. Hemmecke, E.Kim, F. Liu, U. Rothblum, F. Santos, S. Onn, R. Yoshida,
Special Session on Integrating Constraint Programming and Operations Research ISAIM 2016
Titles Special Session on Integrating Constraint Programming and Operations Research ISAIM 2016 1. Grammar-Based Integer Programming Models and Methods for Employee Scheduling Problems 2. Detecting and
Lecture 2: August 29. Linear Programming (part I)
10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.
INTEGER PROGRAMMING. Integer Programming. Prototype example. BIP model. BIP models
Integer Programming INTEGER PROGRAMMING In many problems the decision variables must have integer values. Example: assign people, machines, and vehicles to activities in integer quantities. If this is
High Performance Computing for Operation Research
High Performance Computing for Operation Research IEF - Paris Sud University claude.tadonki@u-psud.fr INRIA-Alchemy seminar, Thursday March 17 Research topics Fundamental Aspects of Algorithms and Complexity
7 Gaussian Elimination and LU Factorization
7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method
Lecture 11: 0-1 Quadratic Program and Lower Bounds
Lecture : - Quadratic Program and Lower Bounds (3 units) Outline Problem formulations Reformulation: Linearization & continuous relaxation Branch & Bound Method framework Simple bounds, LP bound and semidefinite
Adaptive Online Gradient Descent
Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650
Joint Arc-factored Parsing of Syntactic and Semantic Dependencies
Joint Arc-factored Parsing of Syntactic and Semantic Dependencies Xavier Lluís and Xavier Carreras and Lluís Màrquez TALP Research Center Universitat Politècnica de Catalunya Jordi Girona 1 3, 08034 Barcelona
Efficient Scheduling of Internet Banner Advertisements
Efficient Scheduling of Internet Banner Advertisements ALI AMIRI Oklahoma State University and SYAM MENON University of Texas at Dallas Despite the slowdown in the economy, advertisement revenue remains
24. The Branch and Bound Method
24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no
Semi-Supervised Support Vector Machines and Application to Spam Filtering
Semi-Supervised Support Vector Machines and Application to Spam Filtering Alexander Zien Empirical Inference Department, Bernhard Schölkopf Max Planck Institute for Biological Cybernetics ECML 2006 Discovery