Discovering Textual Structures: Generative Grammar Induction using Template Trees
- 1. Discovering Textual Structures: Generative Grammar Induction using Template Trees Thomas Winters, Luc De Raedt KU Leuven, Belgium @thomas_wint firstname.lastname@cs.kuleuven.be
- 2. Motivation Generative model designers often write generative grammars by hand. Can we bootstrap this by providing some exemplars? S <Hero> is going to <Action> Hero <Name> the <Job> Name Alice | Bob | Cathy Job hunter | knight | cook Action fish | slay a dragon | relax Alice the hunter is going to fish Cathy the cook is going to slay a dragon Bob the cook is going to fish Alice the knight is going to relax ?
- 3. Motivation Automatically inducing generative context-free grammars from text with latent templates Problem: Most grammar induction algorithms don’t “acknowledge” templates, but prefer part-of-speech-like structures Generative grammars often use template-like structures! Solution: Introducing notion of “Template Trees”
- 4. Template Tree = connected acyclic directed graph where each node represents a template that is more general than the template of all its child nodes Created by iteratively merging closest templates hello world hi world hi universe howdy universe <D> universe <B> world <A> hi <C>
- 5. GITTA Grammar Induction using a Template Tree Approach 1. Creates an initial template tree 2. Prunes redundant children for each node (all descendent leaves covered by other children) 3. Merges slots (when values have similar value sets) 4. Simplifies tree using slot content 5. Step 3 & 4 until convergence
- 6. Example input 1. "I like my cat and my dog" 2. "I like my dog and my chicken" 3. "Alice the cat is jumping" 4. "Bob the dog is walking" 5. “Cathy the cat is walking”
- 7. Join closest strings iteratively into templates with slots to create Template Tree A: I like my <B> and my <C> | <D> the <E> is <F> B: cat | dog C: chicken | dog D: <G> | <I> E: <H> | cat F: walking | <J> G: Bob | Cathy H: cat | dog I: Alice | Cathy J: walking | jumping
- 8. Merge similar slots and rename slots in Template Tree A: I like my <B> and my <B> | <G> the <B> is <F> B: chicken | cat | dog C: <B> D: <G> E: <B> F: walking | jumping G: Alice | Bob | Cathy H: <B> I: <G> J: <F>
- 9. Iteratively merge and simplify tree & recalculate templates until convergence A: I like my <B> and my <B> | <G> the <B> is <F> B: chicken | cat | dog F: walking | jumping G: Alice | Bob | Cathy
- 10. Output S → I like my <B> and my <B> | <G> the <B> is <F> B → chicken | cat | dog F → walking | jumping G → Alice | Bob | Cathy
- 11. Evaluation: Reverse-engineering Twitterbots GITTA finds decent generalisations, but difficulties with subsequent slots, especially if slots have large variations in number of words for their slots values. Given 25/50/100 example generations of Tracery grammar, try to reconstruct grammar
- 12. Conclusion GITTA is a promising approach for learning generative grammars from texts with latent templates Code: https://github.com/twinters/gitta