Dependency Parsing CMSC 723 / LING 723 / INST 725 MARINE CARPUAT. Slides credit: Joakim Nivre & Ryan McDonald

Transcription

1 Dependency Parsing CMSC 723 / LING 723 / INST 725 MARINE CARPUAT marine@cs.umd.edu Slides credit: Joakim Nivre & Ryan McDonald

2 Today s Agenda Midterm P1 Dependency grammars & parsing

3 Two Types of Parsing Dependency: focuses on relations between words I saw a girl with a telescope Constituency: focuses on identifying constituents and their recursive structure S VP PP NP NP NP PRP VBD DT NN IN DT NN I saw a girl with a telescope

4 Why is parsing difficult? Ambiguity I saw a girl with a telescope I saw a girl with a telescope

5 Constituency: Example The funicular which goes to the top of Victoria Peak is one of the longest in the world.

6 Dependency Grammars Syntactic structure = lexical items linked by binary asymmetrical relations called dependencies

7 Example Dependency Parse They hid the letter on the shelf Compare with constituent parse What s the relation?

8 Criteria for dependency D is likely to be the dependent of head H in construction C H can often replace C and determines the category of C H gives semantic specification of C, D specifies H H is obligatory, D may be optional H selects D and determines whether D is obligatory The form of D depends on H (agreement)

9 Some intuitive dependencies

10 Some trickier dependencies Complex verb groups Subordination Coordination Prepositions Punctuation

11 Dependencies Typed: Label indicating relationship between words nsubj det prep dobj pobj det I saw a girl with a telescope Untyped: Only which words depend I saw a girl with a telescope

12 Dependency Treebanks CoNLL File Format Standard format for dependencies Tab-separated columns, sentences separated by space ID Word Base POS POS2 Head Type 1 ms. ms. N NNP 2 DEP 2 haag haag N NNP 3 NP-SBJ 3 plays play V VBZ 0 ROOT 4 elianti elianti N NNP 3 NP-OBJ DEP

13 Data-driven dependency parsing Goal: learn a good predictor of dependency graphs Input: x Output: dependency graph/tree G Parsing strategies Transition-based Learn to predict transitions given input and history Predict new graphs using deterministic parsing algorithm Graph-based Learn to predict entire graphs given the input Predict new graphs using spanning tree algorithms

14 Shift-Reduce Process words one-by-one left-to-right Two data structures Queue [Buffer] of unprocessed words Stack of partially processed words At each point choose one action shift: move one word from queue to stack reduce left: top word on stack is head of second word reduce right: second word on stack is head of top word Learn how to choose each action with a classifier

15 Shift Reduce Example Stack Queue Stack Queue I shift I saw a girl saw a girl I saw a girl r left shift saw girl I saw saw I saw a r left shift a girl a girl girl shift I I a saw girl a r right I

16 Classification for Shift-Reduce Given a state: Stack saw a I Queue girl Which action do we choose? shift? r left? r right? I saw a girl I saw a girl I saw a girl Correct actions correct tree

17 Classification for Shift-Reduce We have a weight vector for shift reduce left reduce right w s w l w r Calculate feature functions from the queue and stack φ(queue, stack) Multiply the feature functions to get scores s s = w s * φ(queue,stack) Take the highest score s s > s l && s s > s r do shift

18 Features for Shift Reduce Features should generally cover at least the last stack entries and first queue entry stack[-2] stack[-1] Word: saw a POS: VBD DET NN queue[0] girl (-2 second-to-last) (-1 last) (0 first) φ W-2saw,W-1a = 1 φ W-2saw,P-1DET = 1 φ P-2VBD,W-1a = 1 φ P-2VBD,P-1DET = 1 φ W-1a,W0girl = 1 φ W-1a,P0NN = 1 φ P-1DET,W0girl = 1 φ P-1DET,P0NN = 1

19 Algorithm Definition The algorithm ShiftReduce takes as input: Weights w s w l w r A queue=[ (1, word 1, POS 1 ), (2, word 2, POS 2 ), ] starts with a stack holding the special ROOT symbol: stack = [ (0, ROOT, ROOT ) ] processes and returns: heads = [ -1, head 1, head 2, ]

20 Training Shift-Reduce Can be trained using perceptron algorithm Do parsing, if correct answer corr different from classifier answer ans, update weights e.g. if ans = SHIFT and corr = LEFT w s -= φ(queue,stack) w l += φ(queue,stack)

21 Keeping Track of the Correct Answer (Initial Attempt) Assume we know correct head of each stack entry: stack[-1].head == stack[-2].id corr = RIGHT stack[-2].head == stack[-1].id corr = LEFT else corr = SHIFT (left is head of right) (right is head of left) Problem: too greedy for right-branching dependencies stack[-2] stack[-1] queue[0] go go to school to id: RIGHT school head: 0 1 2

22 Keeping Track of the Correct Answer (Revised) Count the number of unprocessed children stack[-1].head == stack[-2].id (right is head of left) stack[-1].unproc == 0 (left no unprocessed children) corr = RIGHT stack[-2].head == stack[-1].id (left is head of right) stack[-2].unproc == 0 (right no unprocessed children) corr = LEFT else corr = SHIFT

23 Shift Reduce Training Algorithm ShiftReduceTrain(queue) initialize stack while queue > 0 or stack > 1: feats = MakeFeats(stack, queue) calculate ans calculate corr if ans!= corr: w ans -= feats w corr += feats perform action according to corr

24 Recap Dependency grammars What s a dependency Some relations are clearer than others Dependency treenbanks An example of a transition-based parser Shift-Reduce parser Combined with classifier Can be trained using perceptron algorithm