表現学習時代の生成語彙論ことはじめ
34 likes9,062 views
The document discusses concepts related to the generative lexicon theory and statistical semantics. It provides definitions and examples of key terms, such as the generative lexicon, word sense disambiguation, and the principle of compositionality. Examples are given to illustrate how statistical semantics can be used to analyze word usage patterns and determine word meanings.
![The Generative Lexicon [Pustejovsky95]
! James PustejovskyĀdž
! ÄƸA4;ƟDA)Q4%
-
! TimeML:ĀdžŽ4EĐL
! „Å; ¿Ó813:ÚP
%Iƙ03-:40K
%-
ÊÌ5¥%
http://www.cs.brandeis.edu/~jamesp/](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-8-320.jpg)
![„Ň:„7J·¦é—:„ [·¦05]
! ·¦ǍƆ—ÐŇǖÊů
Ħ
! ®ƟQ4LǕƥ† ; _ ;
?](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-9-320.jpg)
![Æ8ħă8%-´ē
! ·¦ǍƆ—ÐŇǖÊůĦ[·¦05]:Û12
ƽ
! James Pustejovsky, Introduction to
Generative Lexicon. [Pustejovsky05]
! ě„ƲëGenerative Lexicon:NjÇ [ě„
06]
76](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-10-320.jpg)
![ƭǎ.Jä:†8LƜö:†.I
! Ɯö;ä8LE:5ƿǏ:ƤøPįŁ%3L
! ƶ8ärbg8703-Jƭǎ:^|
o;J8ÀŪ
1Ƨ:åŇĭ¸5:PGK-9
87
^[]U5@CLASSICAOP](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-28-320.jpg)
![+3
8?
45.
$.#/R%BFSY]W2M( $](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-42-320.jpg)
![WU|Rļŋ;‚ĨũdVfJ¾Ʈõ%-
! Web:íƾóČJ…’4ÿÐ%-m`€
Pܹ%3ǐˆ'Lœ¯ [Cimiano+07]
! ęĸǔ°5ðǒG¾ǒ:ƠÎPĝƯ8%3{
€V€X½ƎPƂ¹'Lœ¯ [ĻŖ+12]
5;ċ:EƴƄ4;7#+
96](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-43-320.jpg)
![ƄŇ:rWf}¿Ó
! Ÿ§:Ú;ƄŚ:ƄŇ:ÊůP¿Ó%-
! ®§;ŹŴƄŇ5:¨Ē4ÊůģOLÚ7:
4/(J7Lo~]:Êů67
L:](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-48-320.jpg)
![Principle of compositionality (Frege1892]
… the meaning of a complex expression is determined by
the meanings of its constituent expressions and the rules
used to combine them (wikipedia](https://image.slidesharecdn.com/20141016wordrepandgl-141016004120-conversion-gate01/85/-51-320.jpg)
表現学習時代の生成語彙論ことはじめ
- 1. 2014/10/16 PFIVZX^ ¿Ó½Ǝ»: —ÐŇǖĦ!5;D ĉPreferred Infrastructure Ù¦Ʋǁ (@unnonouno)
- 2. ƝLJŘ Ù¦Ʋǁ (@unnonouno) ! ŪåŇĭ¸GdVfuSh€XGÁnj½Ǝ ! ĘƬĭ¸E/H05GKA' ! “‚ú߸ýŵġIBMôLjŎPFI 8
- 3. mrep5c}P1KA%- ! řĹƑƚ8›%3ÖĨåŇ4íƾ4A' ! pip install mrep 4S€f} 9
- 5. ¿Ó½Ǝ:ÚPň3Ĉ0-!5 ƣƵ.5ăJM3LƄ Ň:+)PrWf} Į :†8ƅ%ñDJM L:4;7 ;
- 6. —ÐŇǖĦ Generative Lexicon (GL)
- 7. ®:£Ć ! —ÐŇǖĦ813:ƴƄ7šʼn ! ¿Ó½Ǝ5Êů:Ð5:¨Ē 18EA%3Œ:Ú OJ7, =
- 8. The Generative Lexicon [Pustejovsky95] ! James PustejovskyĀdž ! ÄƸA4;ƟDA)Q4% - ! TimeML:ĀdžŽ4EĐL ! „Å; ¿Ó813:ÚP %Iƙ03-:40K %- ÊÌ5¥% http://www.cs.brandeis.edu/~jamesp/
- 9. „Ň:„7J·¦é—:„ [·¦05] ! ·¦ǍƆ—ÐŇǖÊů Ħ ! ®ƟQ4LǕƥ† ; _ ; ?
- 10. Æ8ħă8%-´ē ! ·¦ǍƆ—ÐŇǖÊůĦ[·¦05]:Û12 ƽ ! James Pustejovsky, Introduction to Generative Lexicon. [Pustejovsky05] ! ě„ƲëGenerative Lexicon:NjÇ [ě„ 06] 76
- 11. É‹:Êů813ă3BI ƛŸ:8‹0- ŒP8ƍ- ¥ƺ48‹0- ! ‰č8É‹5å03EĬî5%3:É‹5æŲ 5%3:É‹L 77
- 12. gR:Êů813ă3BI Ps€V4Ǒ0- Jš03«# ! wj5%3:gR5‹Ò'L”Ï5%3:gR:Ê ůL ! é:É‹:ŗ5™I7!5ĸ!037
- 13. 78
- 14. ! ă3BL5ƄŇ;1:ƌ7LĎćP â03L!58Ñ2 ! ċOMœ8I03ÊůģO03L ! !:Êů:ģµ8;ęª:ĨƏÎ:I7E: ĐL ! %Eƌ7LåŇ4Eǃ-ÓłL 79
- 15. —ÐŇǖĦ:ăœ ! 1:Ňǖ8ęª:ļŋPâ03L ! đǓ8Ī3+:ļŋJƦ§ƌ7L Êů—Ð#M3L5ăL 7: É‹ Ĭî:É‹ æŲ:É‹ $_
- 16. ŅÊ™ŸƌĺŇ5êĺŇ;ĔĖ#ML ! êĺŇ;Êů¢8ŒJ:¨ĒPâ03LÔ çÇ ! -A-A™Ÿ870-Ň;™ŸƌĺŇ5% 3ĔĖ#ML 7;
- 17. Ù¦;7*!ÚÝ8ŰůPâ03L Ī¹‡ƄŇGƣŇ~r}:ÊůPA Ʊ7Ë݉ŏê5ă3L %(¿þņ6™Êů %(¿þ™.6ņÊů 7
- 18. Ňǖ:ĝŃļŋWU|Rļŋ5E:P ăL ŇPń'LƠÎGł:ğ 1. ļÐWU|RConstitutive ! ƋēG£Ć76Œ4ˆÒ3LPā' 2. řĚWU|RFormal ! +MŒ7:‡ì:NjōPā' 3. ²¢WU|RTelic ! ÁĊG²¢Pā' 4. ƱWU|RAgentive ! +MP—Bˆ'¾ÿGÄƉPā' 7=
- 19. Œ‚ģ+. 7
- 20. ŗò:WU|Rļŋ 7? 1. ļÐ`Syǀ k€g} 2. řĚŀKîÁ nj 3. ²¢ŀLĄĩ 'L 4. ƱæB©3L ÿL @1AKN`KBC0 L4GM*!JDIBQS[^T
- 21. Œ4!%-úßĠÇ7:
- 22. ŔŌ:ò;ş ŔŌ:ò;ngş ! ò;Ŋ¾'L5ƿǏ:Ƥø!!4; ²¢WU|RĐL ! +:-D…;«:đ:Êů.5øDŽ4 L 86
- 25. 60/
- 26. ä4
- 27. Ɯö4
- 28. ƭǎ.Jä:†8LƜö:†.I ! Ɯö;ä8LE:5ƿǏ:ƤøPįŁ%3L ! ƶ8ärbg8703-Jƭǎ:^| o;J8ÀŪ 1Ƨ:åŇĭ¸5:PGK-9 87 ^[]U5@CLASSICAOP
- 29. 7Q4WU|RļŋPăL:
- 30. ! ê:ƄŇ:Êů8¨%3é8•-I7ƿǏ:• øūĢ'L ! +:-D-Eáơ#M-:I7¿Ó4Êů ª0-K+0K7đĖ:Êů87L!5ĐL ! !:ƿǏ:•øJ³ï:đ†4:Êů—Ð# ML:4;7 88
- 31. begin8;į×:Êů—Ð#ML begin the meeting begin a dance begin the book begin the movie ! +E+Ebegin:58;‹ƒGł4;
- 34. the book 1. begin:žN8;łÒL;(.book;ł4 ;7 , 2. bookPł5%3øDŽ%I`SqÍÈ 3. WU|RJúßPÂ03L ‰¶:—Ðq^P 5Ž= 8:
- 35. cf. ƿǏ:ŝģŶ double x =
- 36. 10 1. double:žN8;double:ŞÒL;(.10; double4;7 , 2. 10Pdouble:Ş5%3øDŽ%I 3. Vxf:œ¯:úßPÂ03L ‰¶:—Ðq^P13*,5Ž= 8;
- 37. ħă³;—ÐŇǖĦ8;ĴK31:ļŋL ĝŃļŋ;—ÐŇǖĦ:41ļŋ:†:11 ! Ƴļŋ ! łļŋ ! ĝŃļŋ ! Ňǖżƕļŋ ®;ėŷ%A'I 8
- 38. !!A4:A5D ! ĥŇǖ8;+:ċOMœ8¨'LúßPâ0 3LWU|Rļŋ ! ŹŴ:ƄŇ5śćªK:øDŽ478 !:WU|R:úßPċ03¥%ÊůP—Ð 'LęÐ 8=
- 40. ! åŇĭ¸½ƒ:«430÷ ! ACL:«4300÷ 8 ÊÌ5Ĥ7
- 41. /H05!M;‚ģ ! 11:Ňǖ4!MJ:úß 5 . $./LHE, @A M5BAM
- 42. +3 8? 45. $.#/R%BFSY]W2M( $
- 43. WU|Rļŋ;‚ĨũdVfJ¾Ʈõ%- ! Web:íƾóČJ…’4ÿÐ%-m`€ Pܹ%3ǐˆ'Lœ¯ [Cimiano+07] ! ęĸǔ°5ðǒG¾ǒ:ƠÎPĝƯ8%3{ €V€X½ƎPƂ¹'Lœ¯ [ĻŖ+12] 5;ċ:EƴƄ4;7#+ 96
- 44. Ĥ%ƖJ•7%3BL ! ƄŇ6:I7ƃ:ƄŇ5‰Ƽ8ċOML 5!58›'LüŭP±ųµ%3L ! 6:I7ƄŇ5ċOML8I03ƄŇ: ÊůEģOL 97
- 45. !:Ú6!4 Ÿ§: Statistical Semantics:Ú8ǃ3L 98
- 47. Statistical Semantics:#J Statistical Semantics is the study of how the statistical patterns of human word usage can be used to figure out what people mean, at least to a level sufficient for information access”(ACL wiki) ! ŹŴƄŇ:ˆÓǔ°76:IJè¢7ÎŃJ+:Ň: ÊůPăL ! ‹ƚ‘øGhz{}ibf76:’¯¹JML 9:
- 48. ƄŇ:rWf}¿Ó ! Ÿ§:Ú;ƄŚ:ƄŇ:ÊůP¿Ó%- ! ®§;ŹŴƄŇ5:¨Ē4ÊůģOLÚ7: 4/(J7Lo~]:Êů67 L:
- 49. 9;
- 50. —ÐŇǖĦ5Statistical Semantics:¨Ē ! —ÐŇǖĦ ! ĝ¡:đǓ8LƄŇ:ˆÓ:ÊůPă3L ! 5+:I7ģµPĸ!'-D:Ňǖ:ļ ŋPă3L ! Statistical Semantics ! ŹŴƄŇ:‘Ɯ8I03‰Ɣ¢7+:ƄŇ:Êů Pă3L ! ƀ±¢7ƄŇ:ˆÓ8›'LÊů8åſ%37 ®J'LI 9
- 51. Principle of compositionality (Frege1892] … the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them (wikipedia
- 52. )
- 53. 9=
- 55. ¿ÓrWf}:Ð;ĵ%ĕIKƻĕķ [Mitchell+08] ! ĵ%ĕGƻĕ76rWf}:Ðœ¯Pılj ! ÇƑ5:ƻĕMultiplicative¬0/F0- 9?
- 56. Statistical Semantics5:¨Ē ! Statistical SemanticsŸ§:£Ć;6/J 5å5#-(:rWf}PƮõ'LŎŠ ! Compositionality:ŎŠ;+:rWf}: :ŎŠ ĞœP™8øœØEL%¿Ó½Ǝ:ˆÓ 4ƁK‡KP•)3L :6
- 57. Êů:ÐP‹ƚ4¿Ó'L (MV-RNN) [Socher +12] ! ĥƄŇ8;rWf}5‹ƚ›Ī'L ! ŵƹ#ML5ƪ:‹ƚP¼’:rWf}8ƻ3 #J8žĶř7ģŶ f PŦż¢8‹ ! RootA4ƗKľ'5đ:¿ÓrWf}õJML :7
- 58. GGd€_}. [Cruys+13] ! ÆŇ¾ǒ²¢Ň:31Pd€_}Pċ03æ) 3L:.5ăL :8
- 59. Dynamic Convolutional Neural Network [Kalchbrenner+14] ! CNNPċ03ƄŇ:A5AKPæB‡ 3 ! ļđÞ4;7Żǃ:ŜČõJML
- 60. :9
- 61. ƇőCNN5compositionality ! Ŀ~Sy4;ŧD3ijÏ¢ƄƷ7úß ! ˜Ă:~Sy87L8¶M3+MJæBO#K IKƣƵ7Êů:LúßPûŭ%3L :: Layer1 Layer3 Layer5 [Zelier+14]OP'
- 62. åŇĭ¸;ĸÔņ 4 nW^}Tb[¤‘ǂ źřŸƑƄŇ–Ú ¿Ó
- 63. ƄŇo~]đ ‘ø'LæB‡ L M- :; M-
- 64. !:œŐ8—ÐŇǖĦ:ü•Pď)7 ! —ÐŇǖĦ;CompositionalityP…’4űƘ% 3±ų2L-D8Ľ¸#M-¸Ħ5•M7
- 66. Ãà:Ú ƣƵ.5ăJM3LƄ Ň:+)PrWf} Į :†8ƅ%ñDJM L:4;7 :=
- 67. ęÐP¿ÓrWf}85K!C [Tsubaki+13] companyM JMrun run companyM) run LOQ ! ƪƪ8ŕŢPſ@%3¥%rWf}PÿL ! 4-rWf}JÊů:ЋOML : run companyM Mrun company LOQ runM JMcompany Mcompany '48
- 68. ƄŇ:Ð8›'L1:ŁÔ ! —ÐŇǖĦ ! åŇÓłPűƘ%3Awe}µ%I5'L īB ! Ť:׸we} ! űƘ#M-åŇe`P‡’ń'LI7we }:ŬƘ ! ¿Ó½Ǝųšƕ½Ǝų ! ļÐ5¿Ó:½ƎP™8‹5%3L !MJ:RSeR;ƞţ8¨Ē%3L :?
- 69. A5D ! —ÐŇǖĦ5;
- 70. ! Ňǖ;8›'LÀƊº7üŭJđǓ8Ī3Ê ů—Ð#ML5ăL ! Ň:Êů:Ð8¨'LŎŠ ! ƄŇ:¿ÓrWf}PÐ%3ƣ¿Ó:ÊůP ÿL ! ¿Ó½Ǝpv8ŀ03®İ‹03L åŇ½:ü•5åŇĭ¸:¥%’¯:ã E05ÕC
- 71. ;6
- 72. ƓƢK5$A%- ;7
- 73. ħăđƫ (1/3) ! [Pustejovsky95] James Pustejovsky. The Generative Lexicon. MIT Press. ! [·¦05] ·¦ǍƆ. (4+$. N%ˆư ! [Pustejovsky05] James Pustejovsky. Introduction to Generative Lexicon. Foundations of SemanticsƩĺ´ē. http://www.cs.brandeis.edu/~jamesp/classes/LING130/ ! [ě„06] ě„Ʋë. Generative Lexicon2. Ɛ:ğ†NJ̓:ŏÌƈ ;8
- 74. ħăđƫ(2/3) ! [Cimiano+07] Philipp Cimiano, Johanna Wenderoth. Automatic Acquisition of Ranked Qualia Structures from the Web. ACL2007 ! [ĻŖ+12] ĻŖ˜ƨ, ·Šť, ě„Ʋë. (4$ (!')
- 75. . 0. NLP2012. ! [Mitchell+08] Jeff Mitchell, Mirella Lapata. Vector-based Models of Semantic Composition. ACL2008. ! [Socher+12] Richard Socher, Brody Huval, Christopher D. Manning, Andrew Y. Ng. Semantic Compositionality through Recursive Matrix-Vector Spaces. EMNLP2012. ;9
- 76. ħăđƫ(3/3) ! [Cruys+13] Tim Van de Cruys, Thierry Poibeau, Anna Korhonen. A Tensor-based Factorization Model of Semantic Compositionality. NAACL2013. ! [Kalchbrenner+14] Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom. A Convolutional Neural Network for Modelling Sentences. ACL2014. ! [Zelier+14] Matthew D. Zeiler, Rob Fergus. Visualizing and Understanding Convolutional Networks. ECCV2014. ! [Tsubaki+13] Masashi Tsubaki, Kevin Duh, Masashi Shimbo, Yuji Matsumoto. Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks. EMNLP2013 ;: