3. Target-dependent sentiment classification
Intro
“I bought a new camera.
The picture quality is amazing but the battery life is too short”
Input：a sentence, target mention
Output： the sentiment polarity (e.g. positive, negative, neutral)
of the sentence towards the target
target = picture quality Output = positive
target = battery life Output = negative
target wordとそのcontext wordsとの意味的関係をうまくモデル化したい
target-dependentなLSTMモデルの提案
（ただfeature engineeringはlabor intensiveだからしたくない）
3
17. Attention-based LSTMモデル
Discussion
we also tried an attention-based LSTM model, which is inspired by
the recent success of attention-based neural network in machine
translation (Bahdanau et al., 2015) and document encoding (Li et
al., 2015b). We implement the soft-attention mechanism (Bah-
danau et al., 2015) to enhance TD-LSTM. We incorporate two
attention layers for preceding LSTM and following LSTM,
respectively. The output vector for each attention layer is the
weighted average among hidden vectors of LSTM, where the
weight of each hidden vector is calculated with a feedforward
neural network. The outputs of preceding and following attention
models are concatenated and fed to sof tmax for sentiment
classification. However, we cannot obtain better result with such an
attention model. The accuracy of this attention model is slightly
lower than the standard LSTM model (around 65%), which means
that the attention component has a negative impact on the model.
A potential reason might be that the attention based LSTM has
larger number of parameters, which cannot be easily optimized
with the small number of corpus. 17
18. References
18
Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014.
Adaptive recursive neural network for target-dependent twitter sentiment classification.
In ACL, pages 49–54.
Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011.
Target-dependent twitter sentiment classi- fication.
ACL, 1:151–160.
Duy-Tin Vo and Yue Zhang. 2015.
Target-dependent twitter sentiment classification with rich automatic features.
IJCAI.
![実験設定
Experiment
データセット[Dong et al., 2014]
・製品や有名人などを含むツイートを抽出
・人手で (negative, neutral, positive) を分類
・アノテーター２人（一致率: 82.5%）
・negative : neutral : positive = 25% : 50% : 25%
・train : 6248 tweet, test : 692 tweet
比較手法
・SVM-indep, SVM-dep[Jiang et al., 2011]
・Recursive NN, AdaRNN[Dong et al., 2014]
・Target-dep[Vo and Zhang, 2015]
LSTM
・100次元のベクトルをTwitterからGloveで学習
・一様分布 U(-0.003, 0.003) でパラメータをランダムに初期化
・softmax層のclipping閾値：200
・learning rate：0.01 7](https://image.slidesharecdn.com/imireading20170210-170211152506/85/Effective-LSTMs-for-Target-Dependent-Sentiment-Classification-Duyu-Tang-et-al-COLONG-2016-7-320.jpg)
![比較手法概要
Experiment
[Jiang et al., 2011]
・SVM-indep：unigram, bigram, 記号, 顔文字, ハッシュタグ,
ポジネガ単語数（辞書を使用）
・SVM-dep：SVM-indepの素性
+ target-dependentな素性（targetを項に持つ述語など）
8](https://image.slidesharecdn.com/imireading20170210-170211152506/85/Effective-LSTMs-for-Target-Dependent-Sentiment-Classification-Duyu-Tang-et-al-COLONG-2016-8-320.jpg)
![比較手法概要
Experiment
[Dong et al., 2014]
・AdaRNN-w/oE：合成関数の選択に係り受け関係を使わない
・AdaRNN-w/E：合成関数の選択に係り受け関係を使う
・AdaRNN-comb：AdaRNN-w/Eの合成ベクトル + uni/bigram を素性としてSVM
9](https://image.slidesharecdn.com/imireading20170210-170211152506/85/Effective-LSTMs-for-Target-Dependent-Sentiment-Classification-Duyu-Tang-et-al-COLONG-2016-9-320.jpg)
![比較手法概要
Experiment
[Vo and Zhang, 2015]
・Target-dep：T_twとF_twの素性のみでSVM
・Target-dep+：T_twとF_twに加えてS_twの素性も含めてSVM
10](https://image.slidesharecdn.com/imireading20170210-170211152506/85/Effective-LSTMs-for-Target-Dependent-Sentiment-Classification-Duyu-Tang-et-al-COLONG-2016-10-320.jpg)
