20051128.doc
- 1. Time：11：00AM Location：New CSIE Building R110 Topic：Learning with Integer Linear Programming Inference for Constrained Output Speaker： Scott Wen-Tau Yih Abstract： In several structured classification problems, explicit and expressive constraints are crucial to enhancing the accuracy and quality of the predictions. However, it was not clear how this additional knowledge can be used in various learning frameworks. In this talk, I'll first demonstrate how constraints can be incorporated in Conditional Random Fields via a novel inference approach based on integer linear programming. Inference in CRFs and HMMs is usually done using the Viterbi algorithm, an efficient dynamic programming algorithm. In many cases, general (non-local and non-sequential) constraints may exist over the output sequence, but cannot be incorporated and exploited in a natural way by Viterbi. Our inference procedure extends CRF models to naturally and efficiently support general constraint structures. For sequential constraints, this procedure reduces to simple linear programming as the inference process. Experimental evidences of our approach will be provided in the context of an important NLP problem, semantic role labeling. One interesting phenomenon we observed in the experiments is that a simple learning plus inference scheme may outperform inference based training approaches when incorporating constraints. In the second part of my talk, I'll describe how we compared these two learning frameworks by observing their behaviors in different conditions. Experiments and theoretical justification lead to the conclusion that using inference based learning is superior when the local classifiers are difficult to learn but may require many examples before any discernible difference can be observed. Bio： Wen-tau Yih is a post-doc researcher in the Machine Learning and Applied Statistics group at Microsoft Research. He got his Ph.D. at the University of Illinois at Urbana-Champaign in May 2005. Although his current research focuses mainly on problems related to email applications and anti-spam, his research interests spread on various problems in Machine Learning and Natural Language Processing, such as learning and knowledge representation, information extraction, semantic parsing, and inference and learning for structured output. Wen-tau received both his M.S. and B.S. degrees in Computer Science from National Taiwan University. More information can be found on his homepage: http://scottyih.org/