Social Sentiment Analysis and Its Use in Communication Campaigns

Social Sentiment Analysis and Its Use in Communication Campaigns
Sanja Vladović
Communication Management Forum 2015
Reconciling the traditional and contemporary: the new integrated communication
682
SOCIAL SENTIMENT ANALYSIS AND ITS USE IN
COMMUNICATION CAMPAIGNS
Professional Paper
Sanja Vladović, M.A.
Faculty of Humanities and Social Sciences, University of Zagreb (Ph.D. candidate), Croatia
svladovic@yahoo.com
Summary
With the development of the Internet and especially social media, companies are able to find an
unlimited number of discussions on almost any subject, with opinions containing subjective statements
and emotions of users. This paper is focused on automatic social sentiment analysis and how it can be
used to support marketers’ efforts to determine if communication campaigns are delivering the
planned results, while also giving indications on how to adapt the communication strategy, as well as
the challenges it poses. The paper defines the most important characteristics of the social sentiment
analysis and evaluates three free online sentiment analysis tools: Topsy, Sentiment140 and Social
Mention. In order to demonstrate the use of social sentiment analysis tools, a sentiment analysis will
be conducted on messages related to the Super Bowl 2015 commercials as well as the evaluation of
how effective the selected tools are when it comes to tracking and analysing the number of published
messages and the prevailing sentiment.
Keywords: marketing research, social media, social media analyses, data-mining, sentiment analyses,
Super Bowl commercials
1. Introduction
The Internet and social media are becoming increasingly important information resources for
marketers and advertisers to understand what consumers are thinking and saying about their
brands, products and campaigns. Before the advent of the Internet, when looking for the
opinion of the consumers, companies had to rely on surveys, polls and focus groups. With the
development of the Internet, and especially social media, companies are able to find an
unlimited number of discussions on almost any subject, with statements containing subjective
opinions and emotions of users.

Sanja Vladović
683
This paper is focused on automatic social sentiment analysis and how it can be used to
support marketers’ efforts to determine if communication campaigns are delivering the
planned results, while also providing indications on how to adapt the communication strategy,
as well as the challenges it poses. The paper defines the most important characteristics of the
social sentiment analysis and evaluates three free online sentiment analysis tools: Topsy,
Sentiment140 and Social Mention.
In order to demonstrate the use of social sentiment analysis tools, a sentiment analysis was
conducted on messages related to Super Bowl 2015 commercials as well as the evaluation of
how effective the selected tools are when it comes to tracking and analysing the number of
published messages and the prevailing sentiment.
An evaluation of these free tools revealed that the selected tools, despite all the challenges
that measuring social media poses to research, can produce consistent and compatible results
regarding the number of published messages and the prevailing sentiment, which affirm them
as a valid research method for marketers.
This paper builds on a relatively small quantity of relevant literature on the use of real-time
automatic sentiment analysis tools for marketing research. Authors Züll and Mikelić
Preradović (2013) tested sentiment analysis tools (Sentiment140, TweetFeel and Tweettone,
Topsy and Social Mention) to find out if they depict an audience response to the broadcast of
a TV program (Oscars - Academy Award Ceremony 2013, the American TV series
“American Dadˮ and the quiz show “Who Wants to Be a Millionaireˮ) and which sentiment
was shown towards it. The authors Jakopović and Mikelić Preradović (2013) focused on
evaluation in public relations and the use of sentiment analysis tools. The authors applied the
sentiment analysis programs SentiStrength and Social Mention for the measurement of
perception of the airline company Croatia Airlines by passengers.
The majority of the relevant literature, as that cited in this paper, focuses primarily on
analysing a single phenomenon with one sentiment analysis tool. Authors Shin, Byun and Lee
(2015) examined Twitter usage during the 2014 Super Bowl and authors Oh, Sasser and
Almahmoud (2015) studied social media word-of-mouth surrounding the 2014 Super Bowl
TV advertisement. The author is not aware of similar research that compares the results of
more freely available tools for real-time sentiment analysis for the research of advertisement
performance.

Sanja Vladović
684
2. Social Media
In its beginnings, the Internet was primarily a one-way medium intended for reading, in which
the majority of users could only browse the websites without the possibility of a two-way
communication. There were relatively few content creators compared to the vast majority of
users who acted merely as consumers of the content (Cormode, 2008).
The progress of technology and the development of applications that allow two-way
communication with the users, which started in early 2000, also led to the transformation of
the user’s role, who turned from a passive viewer into an active participant. This period is
known as Web 2.0. Defined by Tim O'Reilly (O'Reilly, 2005) as “the network as platform, spanning
all connected devices; Web 2.0 applications are those that make the most of the intrinsic advantages of that
platform: delivering software as a continually-updated service that gets better the more people use it, consuming
and remixing data from multiple sources, including individual users, while providing their own data and services
in a form that allows remixing by others, creating network effects through an ꞌarchitecture of participationꞌ, and
going beyond the page metaphor of Web 1.0 to deliver rich user experiences”.
Technologies brought about by Web 2.0 enabled the development of social media. Although
the two terms are often equated, they are not synonymous. Social media can be considered a
product of Web 2.0, but equating the Web 2.0 with social media is not proper (Beattie, 2011).
In his e-book, “What is Social Mediaˮ, Mayfield (2013) lists basic forms of social media:
social networks, blogs, Wikis, Podcasts, Forums, with the main characteristics including:
participation, openness, conversation, community and connectedness.
It was the social networks such as Facebook, Twitter, LinkedIn and MySpace that mainly
changed the way people communicate with one another, not only online but in real life as
well. They have facilitated sharing of the information, news, views and opinions in real-time
without the limitations of physical space. Users generate huge amounts of posts that are in
digital form, publicly and globally available.
Collection and analysis of posts on social media open up endless possibilities for many
professionals, from politicians, journalists through to business analysts. Before the emergence
of the Internet, when looking for the opinion of certain groups, journalists, scientists,
marketing experts, business or political analysts had to rely on research, surveys, reports of
competent experts and other persons who represent the opinion of the group which was in the
focus of interest. With the development of the Internet, especially social media, experts from

Sanja Vladović
685
different areas and organizations are able to find any number of discussions on almost any
topic and thus get the statements directly from the users, as well as attitudes and emotions on
a topic.
In this paper, we want to address the possible ways of using the collected and analysed user
posts from social networks in more detail in the field of marketing, more precisely in the field
of advertising.
3. Analysis of Social Media and Marketing Research
With the emergence of the Internet, especially social media, a new field has opened up in
which potential buyers and companies can find an enormous amount of opinions and user
recommendations in the posts that contain information about the product/service/campaign
but also their subjective attitudes and emotions. Modern customers post and share their
opinions, recommendations and criticisms on the commercial websites and on their personal
profiles on social media.
This leads to a transfer of control of the brand from the hands of the company to the hands of
consumers (Oh, Sasser, Almahmoud, 2015), which is why monitoring and analysing user
posts on social media is becoming one of the priorities of companies and marketing
professionals. An increasing number of companies are realizing how important it is to build
brand value in partnership with its customers via social media and the benefits of using the
mechanisms to measure investments in advertising and promotion that social media allows
(Peltier, 2013).
Ever since the first marketing research studies by Daniel Starch1
(Vasquez, 2011), marketing
experts have been using different research methods to gain a better understanding of
consumers, developed messages that communicate with target groups and evaluated the
effectiveness of these messages.
1
Vasquez (2011) reports that “during the early 1930s Daniel Starch developed the theory that effective
advertising must be seen, read, believed, remembered and then acted upon. Soon after, he developed a
research company that would interview people in the streets, asking them if they read certain publications. If
they did, his researchers would show them the magazines and ask if they recognized or remembered any of
the ads found in them. After collecting the data, he then compared the number of people he interviewed with
the circulation of the magazine to figure out how effective those ads were in reaching their readers. Thus
surveying or marketing research was born.”

Sanja Vladović
686
Marketing experts do not just follow criticism and opinions on various products and services,
but rather follow the attitudes of customers about promotional and advertising activities of the
company in order to evaluate their efficiency.
Prior to the development of communication campaigns, market research is used to optimize
communication for the target audience and the media, creative solutions are tested to
determine their effectiveness. At the end of the campaign tests are conducted that measure the
results of the campaign, whether the goal was to increase brand awareness, encourage
consumption, etc.
The biggest novelty that social media bring to this process is the possibility of obtaining
results in real-time, enabling the companies to make decisions in real-time as well.
Analysis of data from social media can provide interesting information for the understanding
of individual and human behaviour, detecting hot topics, as well as identifying influential
individuals, groups or communities. However, it is difficult to discover useful information
from social data without automatic data processing due to the three main features of data
obtained from social networks that are, as described by Oh, Sasser and Almahmoud (2015)
“large, noisy and dynamic”.
In order to overcome these social media challenges, in-depth techniques may be used to
research the data and their analysis in order to collect and process large amounts of data
generated on social networks.
Modern tools allow us to follow posts in real-time and perform analyses that indicate the
prevailing sentiments. Automatic sentiment analysis of posted opinions, criticisms,
recommendations and discussions becomes one of the basic tools for marketing experts
because it allows the retrieval and processing of large amounts of posts, which would take a
lot of time if using manual processing.
3.1. Automatic Sentiment Analysis
Sentiment analysis uses complex algorithms for natural language processing to analyse texts
that determine the views of the authors and the emotional content of the text. Since 2000,
sentiment analysis has developed into one of the most active areas of research in natural
language processing thanks to the development of the Internet and social media that allow
access to a large amount of data in digital form and contain the opinions and emotions of the

Sanja Vladović
687
authors. Given the wide range of applications, the interest for sentiment analysis spread from
computer science to other scientific areas (primarily economics and management) (Liu,
Sentiment Analysis and Opinion Mining, 2012) and among the professional public that
recognized the great value of sentiment analysis in practical use.
However, finding and monitoring opinions and views on the Internet is a major challenge
given the large number of different sources and posts that contain opinions and views. Often
the opinions and views are hidden inside long posts on forums or blogs with a form that
makes recognition and retrieval very difficult, while the amount of posts in the form suitable
for further use makes non-automatic searches, analysis, summarizing and organization of
posts extremely difficult (Liu, 2010).
The automated sentiment analysis developed from the need for a system that will automate
the detection and compression of opinions and views. Due to the complexity and ambiguity
of natural language, text analysis is a complex task that relies on methods from natural
language processing and machine learning.
Author Bing Liu in his paper “Sentiment Analysis and Opinion Miningˮ highlights several
challenges associated with the automatic sentiment analysis:
 different levels of analysis, i.e., whether the whole to be analysed is a document,
sentence, word, aspect;
 different types of opinions: conventional opinion and comparative opinion;
 different word sentiments: depending on the domain of use the same word can have two
different polarities; a sentence does not have to express feelings even though it contains
words with that sentiment; a sentence can express a view or opinion even though it does
not contain words with that sentiment; it is difficult to distinguish sarcasm with or
without an expressed sentiment; understanding slang, etc.
 problems of natural language processing where one should pay attention to the fact that
the automatic sentiment analysis uses limited functionality of natural language processing
because it is not necessary to fully understand the semantics of each sentence, just to
recognize positive and negative sentiments of related terms and sentence conditionalities;
 detection of false reviews (Liu, 2012)

Sanja Vladović
688
3.2. Tools for Automatic Sentiment Search
Tools for automatic monitoring and analysis of posts on the Internet and social networks
allow us to search and process relevant communication with the aim of obtaining data suitable
for further use.
As the interest of scientific and professional community for sentiment analysis grew, so did
the interest in developing tools that allow automatic analysis. The market has a large number
of tools developed by small start-ups, while lots of large companies are developing their own
internal solutions as well (SAP, IBM, Adobe) or have taken over the existing solution (e.g., in
December 2013 Apple bought TopsyLab that develops a social network search engine Topsy)
(Wakabayashi, MacMillan, 2013).
In this study, we used three freely available tools for analysing sentiment: Topsy.com,
Sentiment140 and Social Mentioning. After reviewing a number of freely available tools for
sentiment analysis, we decided to focus on the three mentioned above due to the specific
characteristics each of them has that make them distinctive: Topsy – a comparison of multiple
keywords; Sentiment140 – simplicity of functionality; Social mention – filters.
The following table provides an overview of the most important features of the selected tools:
Table 1: Comparison table: Topsy / Sentiment 140 / Social Mention
FEATURE TOPSY SENTIMENT140 SOCIAL MENTION
Source Twitter Twitter
over one hundred
social media sites
Historical analyses
all posts on Twitter
from 2006 to today
One hour
Anytime; Last
hour/day/week/month
Real-time + + +
Number of posts + + +
Sentiment analysis + + +
Identification of
influential authors
+ - +
Multiple keywords + - -
Specific time range + - +
Trends analyses + - +

Sanja Vladović
689
Insight into sentiment
classification
-
+
+
Languages 10 2 Any language
Sort by Relevance, time - Date, source
“Advanced search”
feature
+ - +
Filters
time, type (links,
tweets, photos, etc.)
language
-
sentiment, top
keywords, top users,
top hashtags and
sources
3.2.1. Short Overview of Topsy.com
Topsy.com2
is a search engine for social networks and socially shared content in real-time and
it stands out because of the possibility to search all posts on Twitter in the time span from
2006 to today. Topsy.com provides quantitative and qualitative analysis of posts. Analyses
can be done in real-time while Topsy.com offers the possibility of searching in a certain
period of time as well as comparing the number of posts for up to three different terms.
Topsy.com gives insight into the number of posts, sentiment analysis, identification of
influential authors and a comparison of the number of posts for longer terms.
Topsy.com analyses trends and allows identification of authors with an extensive online
influence on Twitter and other networks. The influence is determined by measuring the
number of responses and sharings of certain posts. The sentiment is determined on a scale of 1
to 100, but Topsy.com does not give insight into sentiment classification of individual posts
so we have no way of checking how accurate the classification is. Also, we could not find
information on the methodology used to determine sentiment. Topsy.com supports searches in
10 languages. The professional version Topsy Pro offers advanced analyses with additional
payment, at the time of writing this paper there was no the possibility of using the demo
version.
2
http://about.topsy.com/support/search/

Sanja Vladović
690
3.2.2. Short Overview of Sentiment140
Sentiment1403
is a free tool to analyse sentiment posts on a specific topic on Twitter in real-
time with very simple functionality. It is possible to search and analyse in English and
Spanish. It shows the general sentiment as a percentage and the number of positive and
negative posts for the searched term.
Posts are classified into three possible sentiments and are marked in appropriate colour:
positive - green, negative - red, neutral - white. Sentiment140 gives insight into the
classification of sentiment which allows you to check how accurate the sentiment analysis is.
Sentiment140 only gives the results for the latest posts in a range of one hour and there is no
possibility to view the results in another period or to identify influential authors.
3.3.3. Short Overview of Social Mention
Social Mention4
is defined as a platform for searching and analysing social media that collects
content created by users on the Internet and combines them into a single sequence of
information. It is a tool for monitoring and collecting relevant results on social networks,
blogs, microblogs, forums, news, networks for video and audio content. It allows you to
search by date and source.
After the analysis, Social Mention delivers measurable results for the following
characteristics:
1. Strength: the likelihood of mentioning a searched term. It is calculated by dividing the
number of mentions of a specific term by the number of all possible mentions;
2. Sentiment: the relationship between generally positive mentions and mostly negative
mentions;
3. Passion: the likelihood that those who mention the searched term will mention it several
times;
4. Reach: the reach of the impact is calculated by dividing the number of unique posts that
mention the searched term by the total number of posts.
3
http://help.sentiment140.com/
4
http://socialmention.com/about/

Sanja Vladović
691
In addition to the above features, Social Mention allows filtering by the following
characteristics:
1. classification of posts by sentiment in three categories: positive, negative, neutral.
Marking each category with a certain colour allows easy detection of the post with a
certain sentiment as well as the accuracy analysis and classification of sentiment for each
post. It also allows analysis by sentiment and thus only filtering the posts with
predominantly negative sentiments offers insight into potential sources of customer
dissatisfaction;
2. a list of the most frequently used keywords with the number of mentioning. This list
gives a very useful insight into the associated search which facilitates the planning and
implementation of further, more detailed analyses;
3. top users, i.e., authors that most commonly use the searched term. Identifying influential
authors is very useful for businesses, but the author makes it easy to analyse and identify
the so-called “opinion spammers" or authors who publish commercial posts shaped like
user posts (equivalent to covert advertising);
4. top hashtags i.e., most used hashtags and the number of uses thereof;
5. sources included in the search and the number of results per each source.
The last two filters provide marketing experts with insight into the sources and content which
are worth investing into so that their campaigns are more successful.
4. Sentiment Analysis for Ads Aired During Super Bowl 2015
In order to test the selected tools for analysing sentiment, we have conducted an analysis of
messages on social media related to the advertisements aired during the Super Bowl game in
2015. The effectiveness of the selected tools in finding and collecting the published posts and
the prevailing sentiment analysis was tested. The reports provide information on the number
of posts and the prevailing sentiments in the period during and after the event.
4.1. Super Bowl
The Super Bowl is the NFL playoffs final football game, an event that has historically been
among the most popular TV programmes in the U.S. and the world. Since 2010 Super Bowl
has officially become the most watched TV programme in the U.S. In 2015, Super Bowl

Sanja Vladović
692
XLIX became the most watched TV programme in the history of U.S. television with an
average number of viewers of 114.4 million (NFL Communications, 2015).
Thanks to the record viewership, advertising during the game is the most expensive time of
the year, resulting in companies investing big budgets in the production of top quality
commercials, which has led to the incredible popularity of these commercials. Watching and
commenting on the commercials aired during the Super Bowl has become an integral part of
the event.
Authors Shin, Byun and Lee (2015), while examining Twitter usage during the 2014 Super
Bowl Game, found that more than half of the 25.3 million tweets posted during the game
mentioned one specific advertisement aired at the time and concluded that “users of Twitter
post tweets about current active topics or events, as well as they tend to reflect their opinion
on the subjectˮ.
Simultaneously, in their study of social media word-of-mouth surrounding the 2014 Super
Bowl TV advertisement, authors Oh, Sasser and Almahmoud (2015) demonstrated that social
media measures can be a supplementary indicator of ad performance even if advertisers still
face “immense challenges in attempt to measure social initiativeˮ.
The complexity of the social media measurement are well demonstrated in the article “Which
Ads Won the Super Bowl?” published just few hours after the SuperBowl 2015 where 11
different ways to rank “Top Ads” are listed (McMains, 2015).
For the purpose of this paper, we decided to analyse the performance of 3 advertisements that
took top positions among several most notable rankings and are “The Top 3” according to
Talkwalker, one of the world’s leading social data intelligence companies (Sunley, 2015):

Sanja Vladović
693
Graph 1: The Top 3: Always, Nationwide and Budweiser. Retrieved from
http://blog.talkwalker.com/en/super-bowl-ads-online/. Accessed on 20 March 2015.
4.2. Commercials Selected for Analysis
4.2.1. Always: Like a Girl
Always: Like a Girl, aired after halftime (8:24 EST), first seen online 06/26/2014
This girl-empowering ad shows perceptions of the phrase “like a girl” and seeks to redefine it
as something strong and powerful instead of an insult. The ad features teenagers and adults,
both male and female, who were asked to run, hit or throw “like a girl” and they do it
intentionally mildly confirming that phrase “like a girl” is often perceived as an insult. But
when a group of pre-puberty girls is asked to do the same things they do it in a strong and
confident way.
According to Talkwalker (Sunley, 2015), this ad was the most mentioned on the
afternoon/evening of February 1st
, generating over 450000 online mentions.
Emotion words that were used to discuss the ad: “important”, “amazing” and “powerful”.
According to Adobe Digital Index5
(Adobe, 2015), P&G ranked 1st
on the list of “Top 10
Second Screen Winners” while the advert “Like a Girl” saw the most mentions, over 400 000
and 84% of those mentions were extremely positive.
5
To determine second-screen winners, Adobe Social analysed over 4 million social mentions on desktops,
tablets, apps and smartphones across Twitter, Instagram, Facebook, YouTube, Tumblr and more during the
game. Adobe then applied an algorithm consisting of five key factors: 1) total mentions, 2) big game buzz
growth over an average day, 3) sentiment, 4) spend efficiency and 5) international reach. The top 10 ads had

Sanja Vladović
694
According to the iSpot.tv (iSpot.tv, 2015) Super Bowl scorecard6
“Super Bowl Top 10 Ads”,
advert “Like a Girl” ranked 1st
on Game Day by Digital Activity, with the following result on
game day: digital SOV7
of 9.03%; Social Actions: 415 144; Online Plays: 2 242 166;
Engagement: 36% male / 64% female and Sentiment: 89% Liked it.
In the iSpot.tv Final Report (ranking Super Bowl Ads Overall based on data collected
between 01/18/2015 to 02/14/2015), the advert “Like a Girl” ranked 5th
, with the following
result overall: digital SOV of 5.40%; Social Actions: 632 421; Online Plays: 5 387 372;
Engagement: 36% male / 64% female and Sentiment: 85% liked it.
4.2.2. Nationwide Insurance: Make Safe Happen
Nationwide Insurance: Make Safe Happen aired during 2nd Quarter (7:25 PM EST), first seen
online 02/01/2015. The advertisement tells the story of a boy who is unable to follow his
dreams because he has died in a preventable accident. The ad generated lots of mention, but
the majority of them were negative.
According to Talkwalker (Sunley, 2015), this ad was among the most mentioned over the
, generating over 350 000 online mentions, but “a fair
amount of that reaction was negative”.
Emotion words that were used to discuss the ad: “ruined”, “horrible” “awful” “terrible” and
“depressed”.
According to the Adobe Digital Index (Adobe, 2015), the advert “Make Safe Happen” did not
rank among the “Top 10 Second Screen Winners”.
According to the iSpot.tv (iSpot.tv, 2015) Super Bowl scorecard “Super Bowl Top 10 Ads”,
the advert “Make Safe Happen” ranked 8th
on Game Day by Digital Activity, with the
following result on game day: digital SOV of 2.48%; Social Actions: 67 613; Online Plays: 1
832 635; Engagement: 61% male / 39% female and Sentiment: 27% liked it.
the highest combined score (Adobe, 2015)
6
The Super Bowl scorecard, calculated by iSpot, tracks and weights activity across YouTube, Facebook, Twitter
and search that is explicitly related to the commercials (iSpot.tv, 2015)
7
Digital SOV - The digital share of voice (SOV) or percentage of earned digital activity generated by the spot
compared to all others from the game.

Sanja Vladović
695
between 01/18/2015 and 02/14/2015), the advert “Make Safe Happen” ranked 13th
, with the
following result overall: digital SOV of 1.74%; Social Actions: 121 341; Online Plays: 6 990
587; Engagement: 62% male / 38% female and Sentiment: 33% liked it.
4.2.3. Budweiser: Lost Dog
Budweiser: Lost Dog, aired during the 2nd
quarter (7:03 PM EST), first seen online on
01/28/2015
The minute-long ad is a sequel to Budweiser last year’s ad and tells the story of a puppy that
gets lost but with a help of his horse friend eventually finds its way home. The ad generated a
large amount of mainly positive mention.
According to Talkwalker (Sunley, 2015), this ad was among the most mentioned over the
, generating just over 350 000 online mentions and “a very
positive reaction from the public”. Emotion words that were used to discuss the ad:
“awesome”, “amazing” and “perfect”.
According to the Adobe Digital Index (Adobe, 2015), Budweiser’s company Anheuser-Busch
ranked 10th
on the list of “Top 10 Second Screen Winners”.
According to the iSpot.tv (iSpot.tv, 2015) Super Bowl scorecard “Super Bowl Top 10 Ads”,
“Lost Dog” ranked 2nd
on Game Day by Digital Activity, with the following result on game
day: digital SOV of 8.90%; Social Actions: 360 620; Online Plays: 5 821 996; Engagement:
48% male / 52% female and Sentiment: 95% liked it.
between 01/18/2015 and 02/14/2015), the advert “Lost Dog” ranked 1st
, with the following
result overall: digital SOV of 11.97%; Social Actions: 2 592 902; Online Plays: 55 648 217;
Engagement: 48% male / 52% female and Sentiment: 95% liked it.
4.3. Results of Analyses with Selected Tools
On all the selected tools, we conducted analysis in two waves in order to gain insight into
immediate reaction, as well as the longevity of the buzz. The first wave was conducted during

Sanja Vladović
696
the airtime and captured data from the last hour. The second wave was conducted 16 hours
after airtime and captured data from last day (24h range).
In order to filter only the mentions associated with Super Bowl, we decided to search by the
brand name together with the keyword “super bowl”.
We also conducted research with hashtags #likeagirl, #makesafehappen and #bestbuds as
keywords in order to research mentioning related strictly to the particular advertisement and
to use it for control benchmarking. Advertisers use hashtags to drive the conversation online.
Hashtags enable tracking all posts that use the specific hashtag in real time and help identify
relevant posts.
Research based on hashtags was very important for Always since it is a brand name as well as
a generic word that could generate a larger amount of captured data, including data not related
to our research, and thus influence the results.
Also, it should be noted that both Budweiser and Nationwide aired two commercials during
the Super Bowl 2015, while Always aired only one commercial, which makes research based
on hashtags relating to particular advertisement and not only brand name even more
important.
4.3.1. Analysis with Topsy.com
Analysis of selected commercials conducted on Topsy.com showed that the keywords
“Nationwide Super Bowl” generated the highest number on tweets during the airtime (4 852),
but also in the following period after the airtime (18 362), while the keywords “Always Super
Bowl” during airtime generated less mentions (2 202) than “Budweiser Super Bowl” (3 263)
but the during period after the airtime “Always Super Bowl” generated almost 70% (16 044)
more mentions than “Budweiser Super Bowl”(9 359).

Sanja Vladović
697
Graph 2: Number of tweets reported by
Topsy.com based on brand name
Graph 3: Number of tweets reported by
Topsy.com based on the ad’s hashtag
Taking into account that Always aired only one commercial, while Nationwide and
Budweiser aired two each, these results can confirm that the impact of that single ad can be
considerable. The control research by hashtags confirms this conclusion by showing the much
larger number of tweets generated by the hashtag #likeagirl (during airtime: 106 013; after
airtime: 269 208) both during the air time and during period after the airtime, compared to
#bestbuds (during airtime: 31 063; after airtime: 44 128) and #makesafehappen (during
airtime: 1.001; after airtime: 7 006).
Topsy.com allowed us to perform historical trends analysis of the number of tweets and see
how three ads preformed over the period of 30 days that shows that in that period the
keywords “Nationwide Super Bowl” (orange: 30 886) and “Always Super Bowl” (green: 30
103) generated similar number of tweets while the keyword “Budweiser Super Bowl” (blue:
43 301) gained almost 50% more.

Sanja Vladović
698
Graph 4: Number of tweets from January 8th to February 8th reported by Topsy.com based on brand
name
Historical trends analysis of the hashtags shows the dominance of #likeagirl (blue: 351 908)
compared with #bestbuds (green: 130 999) and especial #makesafehappen (orange: 7 006)
Graph 5: Number of tweets reported by Topsy.com
In the sentiment score, the Topsy.com report for keywords is consistent with the hashtag
report regarding low sentiment score results for Nationwide’s advert, but for Always and
Budweiser’s adverts there is a slight difference. While the keywords results show higher
sentiment score results for the Always ad, in both during and after the airtime period, the
hashtag report shows higher sentiment score results for #bestbuds compared to #likeagirl
during and after airtime, as shown in Figures 5 and 6.
Graph 6: Sentiment score reported by Topsy.com
based on brand name
Graph 7: Sentiment score reported by Topsy.com
based on hashtags

Sanja Vladović
699
While analysing the results, it was difficult to determine the cause of the variances in the
value of the sentiment score since Topsy.com gives no insight into the classification of
sentiment releases (such as offered by Sentiment140 and Social Mention) or into the
associated most commonly used terms (as in Social Mention).
4.3.2. Analysis with Sentiment140
Analysis with Sentiment140 does clearly show the lower percentage of positive sentiment for
Nationwide’s advert and the very high positive sentiment for Always advert and Budweiser’s
advert, during and after airtime. Control research with hashtags confirms these results.
Graph 8: Positive sentiment percentage reported
by Sentiment140 based on brand name
Graph 9: Positive sentiment percentage reported
by Sentiment140 based on hashtags
Analysis of a number of tweets shows quite a low number of tweets for keywords during the
airtime, which could be a consequence of a problem with server since we did get a “Server
error” notice several times during the airtime research. The results of control research with the
hashtags differ from the results from Topsy.com and show a higher number of tweets for the
hashtag #bestbuds (during airtime: 44; after airtime: 63) compared to the hashtag #likeagirl
(during airtime: 53; after airtime: 47)

Sanja Vladović
700
Graph 10: Sentiment by count, counting positive,
negative and neutral posts, reported by
Sentiment140 based on brand name
Graph 11: Sentiment by count, counting positive,
negative and neutral posts, reported by
Sentiment140 based on hashtags
Sentiment140 gives insight into the classification of the sentiment for a particular tweet that
allows us to check how accurate the analysis of sentiment is.
In order to determinate the accuracy of the Sentiment140 results, we reviewed the results of
the automatic sentiment analysis from the second wave of the analysis that was conducted 16
hours after airtime for Budweiser’s advert. We reviewed the results with the hashtag
#BestBuds in order to focus on the most relevant mentions. The analysis gave results for the
time period of one hour and there were 63 posts mentioning #BestBuds. Of those, 43 were
marked with a positive sentiment, 10 were marked with a negative sentiment and 10 were
marked with a neutral sentiment.
After reading the posts and conducting our own analysis of the expressed sentiments, we
confirmed that 44 posts were correctly classified while 19 were not classified correctly.
Nine out of ten posts that were originally marked as Negative turned out to be Positive. Out of
ten posts that were originally marked as Neutral, four turned out to be Positive, six posts that
were originally marked as Positive turned out to be Neutral.
Our analysis of expressed sentiments reveals that the posts marked as Positive were much
more likely to be marked correctly than the posts marked as Negative. Dominant reason for
this is not recognizing a positive sentiment due to words with negative meaning.

Sanja Vladović
701
Other observed mistakes (from all Sentiment140 results, not only from the reviewed report
described above):
 not recognizing sentiment due to the negation: tweet classified as negative sentiment, but
is actually positive
 not recognizing slang: tweet classified as negative sentiment, but is actually positive
 not recognizing sarcasm: tweet classified as neutral sentiment, but is actually negative
 not recognizing sarcasm: tweet classified as positive sentiment, but is actually negative
 not recognizing positive sentiment
4.3.3. Analysis with Social Mention
Unlike Topsy.com and Sentiment140, which gather and analyse data only from Twitter,
Social Mention monitors and collects relevant results on social networks, blogs, microblogs,
forums, news, networks for video and audio content. Consequently, the results from Social
Mention cannot be completely comparable with those gained through Topsy.com and
Sentiment140.
Analysis shows that during airtime, the highest percent of strength was shown by the keyword
“Budweiser Super Bowl” (92%), followed by “Always Super Bowl” (77%) and “Nationwide
Super Bowl” (69%).

Sanja Vladović
702
As for the sentiment, analyses clearly confirm the lower level of positive sentiment for
Nationwide’s ad (28 / 24), the higher positive sentiment for the Always ad (32/42) and very
high for Budweiser’s ad (69/74), during and after airtime.
The results for passion (See definition on page 8) show small differences between the three
ads, during and after airtime, but it is interesting to notice that the results for passion are
significantly higher for the period after the airtime for all three ads.
As for the reach, the keywords “Budweiser Super Bowl” show the highest reach (60%/18%),
while “Always Super Bowl” (51%/16%) and “Nationwide Super Bowl” (46%/25%) follow
with small difference between them.
Graph 12: Strength, sentiment, passion and reach reported by Social Mention based on brand name
Control research with hashtags reveals different results with the highest strength, passion and
reach percentage for #makesafehappen (100%/15%/102%) during airtime, followed by
#likeagirl (68%/0%/63%) and the lowest for #bestbuds (18%/0%/18%).
The results for period after airtime follow the same pattern as those during the airtime, with
only one difference in research with hashtags where #likeagirl has the lowest strength (19%)
and reach (10%) percentage.

Sanja Vladović
703
Graph 13: Strength, sentiment, passion and reach reported by Social Mention based on hashtags
Even if Social Mention has filters that allow deepening analysis, which proved to be very
useful in some of our earlier research, for this research they did not demonstrate any
significant benefit since both the filters “Top Keywords” and “Top Hashtags” were largely
determinate by the campaign:
Graph 14: Filters by top keywords and hashtags

Sanja Vladović
704
Nevertheless filters by Top Users and Sources did reveal some influencers and that Twitter
and Facebook are the most active social media for Super Bowl.
Graph15: Filters by Top Users and Sources
5. Conclusion
None of the tools used managed to give a detailed analysis like that given using commercial
tools, but considering that these are freely available tools, we believe that we managed to
acquire some useful data. Even if the tools do not show completely comparable results on the
number of posts and the prevailing sentiment, they still confirm the mostly negative sentiment
towards Nationwide’s ad and very positive towards Always and Budweiser’s adverts.
Considering that results differ significantly even among different commercial tools as well,
this research confirmed how challenging ensuring social media is.
The possibility of viewing sentiment classification proved a major advantage because the
automatic classification still has many drawbacks so a check is necessary in order to gain
insight into the accuracy of the classification and therefore into the credibility of the obtained
data.
Nevertheless, sentiment analysis on social networks provides an important and good insight
into the movement of customer opinions and offers an excellent upgrade to traditional forms
of marketing research. However, what makes automatic sentiment analysis seem far superior
to traditional forms of marketing research is the ability to monitor and analyse the opinions of

Sanja Vladović
705
customers in real-time and in conversations in which greater honesty and openness can be
assumed when it comes to opinions and views than is the case with traditional forms of
marketing research.
This study did not have the ambition to provide a detailed analysis of the selected adverts or a
comprehensive insight into the area of sentiment analysis; what we have been shown by this
research is that the skilful use of free tools can provide very useful insight into the analysis of
selected terms.
Each of the selected tools has its advantages and disadvantages, so it is crucial to know the
possibilities of each tool and to select the tool depending on the research objective. Using
multiple tools at the same time can also improve the end result, but one must be aware of
differences in the methodology of each tool in order to be able to read and compare the
results.
From our research, we are inclined to conclude that we were given the most beneficial results
by the Social Mention browser, which despite some flaws (such as the lack of a historical
presentation of the results), thanks to other functionalities (such as insight into post
classification and filters for various criteria) allowed further analyses that provided us with a
detailed understanding of the basic results obtained through sentiment analysis.
6. Acknowledgments
This paper was developed as term paper for the course “Media and Intelligent Text Retrieval”
headed by Nives Mikelić Preradović, Ph.D., associate professor at the Faculty of Humanities
and Social Sciences, University of Zagreb, Croatia, and the author would like to thank her for
her valuable support and feedback on various drafts of this paper.
7. Reference List
• Consoli, D. (2012). A Model to Extract Sentimental Knowledge in a Semantic Web. The
Journal of Knowledge Economy & Knowledge Management, 7 (5), 5-19.
• Cormode, G. (2008). Key differences between Web 1.0 and Web 2.0. First Monday, 13
(6). Retrieved from http://journals.uic.edu/ojs/index.php/fm/article/view/2125/1972.

Sanja Vladović
706
Accessed on 26 March 2015
• Hyeonjeong Shin, C. B. (2015). The Influence of Social Media: Twitter Usage Pattern
during the 2014 Super Bowl Game. International Journal of Multimedia and Ubiquitous
Engineering, 10 (03), 109-118.
• Jakopović, H. & Mikelić Preradović, N. (2013). Evaluation in public relations – sentiment
and social media analysis of Croatia Airlines. Recent advances in information science. 17,
154-160.
• Liu, B. (2010). Sentiment analysis and subjectivity. Handbook of natural language
processing, 2, 627-666. Retrieved from http://gnode1.mib.man.ac.uk/tutorials/NLP-
handbook-sentiment-analysis.pdf . Accessed on 18 November 2014.
• Liu, B. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language
Technologies. Morgan & Claypool Publishers. Retrieved from
http://www.dcc.ufrj.br/~valeriab/DTM-SentimentAnalysisAndOpinionMining-
BingLiu.pdf. Accessed on 18 November 2014.
• Oh, C., Sasser, S., & Almahmoud, S. (2015). Social Media Analytics Framework: Case of
Twitter and Super Bowl Ads. Journal of Information Technology Management. 26 (1), 1-
18.
• Peltier, D. E. (2013). Social media's slippery slope: challenges, opportunities andfuture
research directions. Journal of Research in Interactive Marketing, 7 (2), 86-99.
• Zuell, B. & Mikelić Preradović, N. (2013). Methods and usage of sentiment analysis in the
context of the TV industry. Recent advances in information science. 13, 124-129.
Internet sources

Sanja Vladović
707
• Adobe. (2015). See which brands made the most of the second screen. Retrieved from
http://admeter.usatoday.com/2015/02/02/adobe-second-screen-top-
10/?adbid=562268782853230592&adbpl=tw&adbpr=15151711&scid=social39782577.
Accessed on 27 March 2015.
• Beattie, A. (2011). What is the difference between social media and Web 2.0? Retrieved
from http://www.techopedia.com/2/27884/internet/social-media/what-is-the-difference-
between-social-media-and-web-20. Accessed on 26 March 2015.
• Cormode, G. (2008). Key differences between Web 1.0 and Web 2.0. First Monday, 13
(6). Retrieved from http://journals.uic.edu/ojs/index.php/fm/article/view/2125/1972.
Accessed on 26 March 2015
• iSpot.tv. (2 February 2015). Top 2015 Super Bowl Commercials. Retrieved from
http://www.ispot.tv/events/top-2015-super-bowl-commercials. Accessed on 27 March
2015.
• Mayfield, A. (2013). What is social media? Retrieved from
http://www.icrossing.co.uk/fileadmin/uploads/eBooks/What_is_Social_Media_iCrossing_
ebook.pdf. Accessed on 26 March 2015.
• McMains, A. (2015, 02.02). Which Ads Won the Super Bowl? Retrieved from
http://www.adweek.com/news/advertising-branding/which-ads-won-super-bowl-here-are-
11-different-ways-rank-them-162718. Accessed on 3 March 2015.
• NFL Communications. (February 2015). Most-Watched Ever: Record 114.4 Million Fans
Watch Super Bowl XLIX on NBC. Retrieved from
http://nflcommunications.com/2015/02/03/most-watched-ever-record-114-4-million-fans-
watch-super-bowl-xlix-on-nbc/. Accessed on 27 March 2015.
• O'Reilly, T. (2005, 10 1). Web 2.0: Compact Definition? Retrieved from
http://radar.oreilly.com/2005/10/web-20-compact-definition.html. Accessed on 31 March
2015.
• Sunley, R. (2015, 02 02). Jumping on the Brandwagon: How the Super Bowl Ads Fared
Online. Retrieved from http://blog.talkwalker.com/en/super-bowl-ads-online/. Accessed
on 27 March 2015.
• Vasquez, J. (2011). The History of Marketing Research. Retrieved from
http://www.marketresearchworld.net/content/view/3754/49. Accessed on 20 March 2015.

Sanja Vladović
708
• Wakabayashi, MacMillan. (2013, 12). Apple Taps Into Twitter, Buying Social Analytics
Firm Topsy. Retrieved from
http://www.wsj.com/articles/SB10001424052702304854804579234450633315742.
Accessed on 12 August 2014.

Social Sentiment Analysis and Its Use in Communication Campaigns

More Related Content

What's hot (20)

Similar to Social Sentiment Analysis and Its Use in Communication Campaigns (20)

Social Sentiment Analysis and Its Use in Communication Campaigns