Aspect extraction (A survey)

By: Mahmoud El-Razzaz
ISSR, Cairo University
Cairo, Egypt










Mining Aspects
Collect Dataset
Apply Aspect Mining to dataset collected
Conduct aspect-level sentiment classifier
Preview results & compare it with same
classifiers conducted for other languages
Conclusion
Future work









Collect Dataset
Apply Aspect Mining to dataset collected
Conduct aspect-level sentiment classifier
Preview results & compare it with same
classifiers conducted for other languages
Conclusion
Future work



Vocabulary:
› Aspect[1] and feature[2]
› The two terms are used in the literature as

synonyms and represents the opinion target.
› Simply aspect here means a feature of a
product e.g. “cast” and “script” are a
features of a movie

[1] Na, J.-C., Khoo, C. S. G.. Aspect-based sentiment analysis of movie reviews on
discussion boards. 2010.
[2] Hu, Minqing and Bing Liu. mining and summarization customer reviews. In proceedings
of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
(KDD-2004). 2004.

Aspect mining or Aspect Extraction:
 For example “ the voice quality of this
phone is amazing”
 The aspect is “voice quality” of entity
represented by “this phone”


it is possible that in an application the opinion targets are given because the user is only
Interested in these particular targets (e.g., the BMW and Ford brands)

An opinion typically always has a target.
 The target is often the aspect to be
extracted from a sentence.
 Thus it is important to recognize each
opinion expression and its target from a
sentence.


some opinion expressions can play two rules, indicating a sentiment and implying an
(implicit) aspect (target). For example, in “this car is expensive” is a sentiment word also
indicates the aspect “price”

 There

are four main approaches for
aspect extraction:
1. Extraction based on frequent nouns and

noun phrases.
2. Extraction by exploiting opinion and

target relations.
3. Extraction using supervised learning.
4. Extraction using topic modeling.

This method finds explicit expressions that
are nouns and noun phrases from a
large number of reviews in a given
domain.
 Hu and Liu (2004) used a data mining
algorithm.
 Nouns and noun phrases were identified
by a part-of-peach (POS) tagger.
 Their occurrence frequency is counted
and only frequent ones are kept.


The reason that this approach works is that
when people comment on different
aspects of an entity, the vocabulary that
they use usually converges.
 Irrelevant content in reviews are often
diverse.
 The precision of this algorithm was
improved in (Popescu and Etzioni, 2005)[1]


[1] N Popescu, Ana-Maria and Oren Etzioni. Extracting product features and opinions
from reviews. In proceedings of Conference on Empirical Methods in Natural
Language Processing (EMNLP-2005). 2005.



More references for aspect extraction
based on frequent nouns:
› Blair-Goldensohn et al. (2008)[1]
 In this approach several filters were applied to
remove unlikely aspects, e.g., dropping aspects
which do not have sufficient mentions along-side
down sentiment words.
 Also they collapsed aspects at the word stem level.

[1] Blair-Goldensohn, Sasha, Kerry Hannan, Ryan Mcdonald, Tyler Neylon, George A. Reis,
and Jeff Reyner. Building a sentiment summarizer for local service reviews.
In proceedings of WWW-2008 workshop on NLP in the information Explosion Era. 2008.



› Ku, Liang and Chen, (2006)[1]
 The authors made use of TF-IDF scheme
considering terms at the document level and
the paragraph level.

[1] Ku, Lun-Weim Yu-Ting Liang, and Hsin-His Chen. Opinion extraction, summarization and
Tracking in news and blog corpora. In proceedings of AAAI-CAAW’06. 2006.



› Moghaddam and Ester, (2010)[1]
 The authors augmented the frequency-based
approach with an additional filter to remove
some non-aspect nouns.
 Their work also predicted aspect ratings.

[1] Moghaddam, Samaneh and Martin Ester. ILDA: interdependent LDA model for
learning latent aspects and their ratings from online product reviews. in Proceedings
of the Annual ACM SIGIR International conference on Research and Development in
Information Retrieval (SIGIR- 2011). 2011.



› Scaffidi et al., (2007)[1]
 The authors compared the frequency of
extracted frequent nouns in a review corpus
with their occurrence rates in generic English
corpus to identify true aspects.

[1] Scaffidi, Christopher, Kevin Bierhoff, Eric Chang, Mikhael Felker, Herman Ng, and Chun
Jin. Red Opal: product-feature scoring from reviews. in Proceedings of Twelfth ACM
Conference on Electronic Commerce (EC-2007). 2007.



› Zhu et al.,(2009)[1]
 Proposed a method based on the Cvalue
measure from (Frantzi, Ananiadou and Mima,
2000)[2] for exracting multi-word aspects.

[1] Zhu, Jingbo, Huizhen Wang, Benjamin K. Tsou, and Muhua Zhu. Multiaspect opinion
polling from textual reviews. in Proceedings of ACM International Conference on
Information and Knowledge Management (CIKM-2009). 2009.
[2] Frantzi, Katerina, Sophia Ananiadou, and Hideki Mima. Automatic recognition of multiword terms:. the C-value/NC-value method. International Journal on Digital Libraries,
2000. 3(2): p. 115-130.



More references for aspect extraction based
on frequent nouns:
› Long, Zhang and Zhu,(2010)[1]
 Extracted aspects based on frequency and information
distance.
 Their method first finds the core aspect words using the
frequency-based method.
 It then uses the information distance in (Cilibrasi and
Vitanyi, 2007) to find other related words to an
aspect, e.g., for aspect price, it may find “$” and
“dollars”.

[1] Long, Chong, Jie Zhang, and Xiaoyan Zhu. A review selection approach for accurate
feature rating estimation. in Proceedings of Coling 2010: Poster Volume. 2010.
[2] Cilibrasi, Rudi L. and Paul M. B. Vitanyi. The google similarity distance. IEEE Transactions
on Knowledge and Data Engineering, 2007. 19(3): p. 370-383.

Since opinions have targets, they are
obviously related. Their relationships can
be exploited to extract aspects which
are opinion targets because sentiment
words are often known.
 This method was used in (Hu and
Liu, 2004) for extracting infrequent
aspects.
 For example “The software is amazing.”
if we know that “amazing” is a sentiment
word, then “software” is extracted as an
aspect.




References for literature used this methid:
› Zhuang, Jingm and Zhu, 2006[1]
› Somasundaran and Wiebe, 2009[2]
› Kobayashi et al., 2006[3]



In previous literature a dependency parser was used to
identify such dependency relations for aspect
extraction.

[1] Zhuang, Li, Feng Jing, and Xiaoyan Zhu. Movie review mining and summarization. in
Proceedings of ACM International Conference on Information and Knowledge
Management (CIKM-2006). 2006.
[2] Somasundaran, S., J. Ruppenhofer, and J. Wiebe. Discourse level opinion relations: An
annotation study. in Proceedings of the 9th SIGdial Workshop on Discourse and
Dialogue. 2008.
[3] Kobayashi, Nozomi, Ryu Iida, Kentaro Inui, and Yuji Matsumoto. Opinion mining on the
Web by extracting subject-attribute-value relations. In Proceedings of AAAI-CAAW'06.2006.



Many algorithms based on supervised
learning have been proposed in the past
for information extraction (Hobbs and
Riloff, 2010[1]; Mooney and Bunescu,
2005[2]; Sarawagi, 2008[3])

[1] Hobbs, Jerry R. and Ellen Riloff. Information Extraction, in in Handbook of Natural
Language Processing, 2nd Edition, N. Indurkhya and F.J. Damerau, Editors. 2010,
Chapman & Hall/CRC Press.
[2] Mooney, Raymond J. and Razvan Bunescu. Mining knowledge from text using
information extraction. ACM SIGKDD Explorations Newsletter, 2005. 7(1): p. 3-10.
[3] Sarawagi, Sunita. Information extraction. Foundations and Trends in Databases, 2008.
1(3): p. 261-377..

The most dominant methods are based
on sequential learning.
 The current state of the art sequential
learning methods are Hidden Markov
Models (HMM) (Rabiner, 1989)[1] and
Conditional Random Fields (CRF)
(Lafferty, McCallum and Pereira, 2001)[2]


[1] Rabiner, Lawrence R. A tutorial on hidden Markov models and selected applications in
speech recognition. Proceedings of the IEEE, 1989. 77(2): p. 257-286.
[2] Lafferty, John, Andrew McCallum, and Fernando Pereira. Conditional random fields:
Probabilistic models for segmenting and labeling sequence data. in Proceedings of
International Conference on Machine Learning (ICML-2001). 2001.

Yu et al. (2012)[1] used a partially supervised learning
method called one class SVM (Manevitz and Yousef,
2002)[2] to extract aspects.
 In their case they only extracted aspects from Pos
and Cons of review format 2 as in (Liu, Hu and
Cheng, 2005)[3]
 They also clustered those synonym aspects and
ranked aspects based on their frequency and their
contributions to the overall review rating of reviews.


[1] Yu, Jianxing, Zheng-Jun Zha, Meng Wang, and Tat-Seng Chua. Aspect ranking:
identifying important product aspects from online consumer reviews. in Proceedings of
the 49th Annual Meeting of the Association for Computational Linguistics. 2011.
[2] Manevitz, Larry M. and Malik Yousef. One-class SVMs for document classification. The
Journal of Machine Learning Research, 2002. 2: p. 139- 154.
[3] Liu, Bing, Minqing Hu, and Junsheng Cheng. Opinion observer: Analyzing and
comparing opinions on the web. in Proceedings of International Conference on World
Wide Web (WWW-2005). 2005.

Ghani et al. (2006)[1] used both traditional
supervised learning and semi-supervised
learning for aspect extraction.
 Kovelamudi et al., (2011)[2] used a
supervised method but also exploited some
relevant information from Wikipedia.


[1] Ghani, Rayid, Katharina Probst, Yan Liu, Marko Krema, and Andrew Fano. Text mining for
product attribute extraction. ACM SIGKDD Explorations Newsletter, 2006. 8(1): p. 41-48.
[2] Kovelamudi, Sudheer, Sethu Ramalingam, Arpit Sood, and Vasudeva Varma. Domain
Independent Model for Product Attribute Extraction from User Reviews using Wikipedia.
in Proceedings of the 5th International Joint Conference on Natural Language
Processing (IJCNLP-2010). 2011.





Topic modeling is an unsupervised learning
method that assumes each document consists
of a mixture of topics and each topic is a
probability distribution.
There were two main basic models, pLSA
(Probabilistic Latent Semantic Analysis)
(Hofmann, 1999)[1] and LDA (Latent Dirichlet
allocation) (Blei, Ng and Jordan, 2003; Griffiths
and Steyvers, 2003; Steyvers and Griffiths, 2007).

[1] Hofmann, Thomas. Probabilistic latent semantic indexing. in Proceedings of Conference
on Uncertainty in Artificial Intelligence (UAI-1999). 1999.
[2] Blei, David M., Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. The
Journal of Machine Learning Research, 2003. 3: p. 993- 1022.
[3] Steyvers, Mark and Thomas L. Griffiths. Probabilistic topic models. Handbook of latent
semantic analysis, 2007. 427(7): p. 424-440.



In the sentiment analysis context, one can
design a joint model to model both
sentiment words and topics at the same
time, due to the observation that every
opinion has a target.



For readers who are not familiar with topic
models, a part from reading the topic
modeling literature, the “pattern
recognition and machine learning” book
by Christopher M. Bishop.





Mei et al. (2007)[1] proposed an aspect
sentiment mexture model, which was based on
aspect (topic) model, positive and negative
sentiment models learned with the help of
external training data. And their model was
based on pLSA.
Some researchers showed that global topic
models are not suitable for detecting aspects
as in (Titov and McDonald, 2008)[2].

[1] Mei, Qiaozhu, Xu Ling, Matthew Wondra, Hang Su, and ChengXiang Zhai. Topic
sentiment mixture: modeling facets and opinions in weblogs. In Proceedings of
International Conference on World Wide Web (WWW-2007). 2007.
[2] Titov, Ivan and Ryan McDonald. Modeling online reviews with multi-grain topic models.
in Proceedings of International Conference on World Wide Web (WWW-2008). 2008.





Later Brody and El Hadad (2010) [1] proposed to first
identify aspects using topic models and then identify
aspect-specific sentiment words by considering
adjectives only.
In (Mukherjee and Liu, 2012), a semi-supervised joint
model was proposed, which allows the user to
provide some seed aspect terms for some topics in
order to guide the inference to produce aspect
distributions that conform to the user’s need.

[1] Brody, Samuel and Noemie Elhadad. An Unsupervised Aspect-Sentiment Model for
Online Reviews. in Proceedings of The 2010 Annual Conference of the North American
Chapter of the ACL. 2010.
[2] Mukherjee, Arjun and Bing Liu. Aspect Extraction through Semi- Supervised Modeling. in
roceedings of 50th Anunal Meeting of Association for Computational Linguistics (ACL2012) (Accepted for publication). 2012.

 Some

other used techniques for
aspect extraction:
› Meng and Wang (2009)[1] extracted

aspects from product
specifications, which are structured
data.
[1] Meng, Xinfan and Houfeng Wang. Mining user reviews: from specification to
summarization. in Proceedings of the ACL-IJCNLP 2009 Conference Short Papers. 2009.

Identify which of those methods is better
and more reliable.
 Study the applicability of each of these
methods for Arabic Language based on
the language dependent factor of
each.


Aspect extraction (A survey)

More Related Content

Viewers also liked (20)

Similar to Aspect extraction (A survey) (20)

Recently uploaded (20)

Aspect extraction (A survey)