Filtering out improper user accounts from twitter user accounts for discovering individuals interested in certain topic

Editor's Notes

• #2: Thanks for coming at first, please let me introduce myself my name is xxxxx from department of xxxxxx Today I want to talk about our own research, the title of which is xxxxxx --------------
• #3: Lets begin with the background of our research Since we are living in the IT society. There is definitely growing need of xxxxx Where we can find latent participants, we considered the social network service such as twitter Twitter as an SNS is holding xxxx who we want to invite to those events But during the collection of user data based on certain keyword list, a lot of improper user appeared such as xxxx who we want to get rid of
• #4: As we mentioned before, there are a lot of web-based debate system, in this research, we concentrated on collagree Collagree is aiming for consensus generating and also used by Nagoya government to collect regional residences opinion For the better use of this platform, we think it would be great if we can invite more people from different locations with a diversity of backgrounds to offer their new ideas So we plan to develop a participants invitation agent for this system.
• #5: Here is the procedure of the whole agent Firstly the agent will receive a keyword list which can be prepared by human or extracted from the introduction of the debate topic And then the agent will collect the user set based on the list and filter out the improper user Before the agent actually sent the invitation, there will be a more specific classification of the user group to find out which user can really attend the debate And then the agent will sent the invitation to the users This research is focused on the second part, filter out the improper user. So which kind of account is improper
• #6: Here is how we defined the improper account There are three kinds of them Official user account is used by company or public facilities, they usually have specific terms in their user onscreen name or description The second is inactive user, who only retweet the campaign contents for a gift, and they usually don’t have a description but with random characters combination in their onscreen name The last one is robot user who are also likely to have specific terms in their description or onscreen name such as bot, and often there are Ads or promotion information in their tweet. to filter out these kinds of accounts
• #7: Here is the approach for this research To begin with, we collect the data with twitter search api and streaming api based on the keywords or hashtags to keep the balance of positive and negative samples Since they are mainly written in Japanese, we need mecab to tokenize and use the tfidf for vectorization We proposed two ways to generate the feature vector which are mix process and separated process which I like to demonstrate later So for the mixed process, we xxxxx And for separated process, we xxxx And we choose the rbf-kernel SVM as learning model since the SVM perform well in binary classification
• #8: There are some related work. One was done by Marco Pennacchiotti 2011 Xxxxxx They proposed a general model for user profiling and ran a deep analysis on tweet linguistic contents. They designed their feature vector with four parts, xxxxxx And we dealt with xxxxx in our research However they did not consider xxxxx Since there are xxxxxxx and over 50% xxxx And their research was focused on English twitter user But there are definitely a lot of differences between English and Japanese user such as the use habit, language As I mentioned we utilized user information and tweet contents for feature vector design Here I like to give you a walk through about the design
• #9: So firstly we got the initial data which consist of ----------- For the mixed process, we process these two parts as one document to generate one vector , and each dimension is filled with the value of TFIDF of one term this is the feature vector of mixed process for separated process, we process these these two parts separately to generate two vector by tfidf of course each dimension is filled with the value of tfidf of one term then we combine two vector into one And this is the feature vector of separated process And then we try this approach in practice
• #10: Here is the training data for experiment We firstly assume a topic for debate in collagree, child care Then we collect the data twice First time is by streaming api based on the keyword list as you can see Among the 269 users, 78% percent are improper Send time we used the twitter search api based on the hashtag which we happened to find in twitter trend The hashtag is about a song which is related to child care In this time of search, 91 percent of all 400 tweets are tweeted by individual users So the whole data consisted of 669 tweets contents with user information
• #11: Here’s the samples from each group in the data
• #12: To do that, we use SVM ran on 5 different hyperparameter settings Xxxxx with different combination of C and gamma C by the way, is the cost parameter which will trade off the misclassification against complexity of prediction surface with gamma. --------
• #13: Lets take a look at the result of experiments You can see setting2 gave the best performance on F measure And the separated process is one point higher than mixed process by setting2 On recall score separated is 4 point higher but on precision mixed one is 2 point higher And though all settings performed well on recall score only setting2 gave well performance on precision score
• #14: And we consider that the unbalance and lack of data was the reason why all settings gave well performance on recall score And about the setting2 giving the best balance performance, we think that is because large C and small gamma providing more support vectors to deal with the similarity of data Since we label the data ourselves, the result could affected by the mistaken label of human. and though the mixed process and separated process both performed well, but we think that separated process can give more feature of the data
• #15: Here the conclusions, We consider that the contents of user information and tweet can be important factors in filtering task But they are still not enough if the data is much more Some keyword list or hashtags in twitter trend probably can help collecting individual account We are considering that improper accounts may need time to react to those trend But the model is expected to be unreliable when dealing with a large amount of data Since the feature vector is similar with each other and we only consider one tweet for each user
• #16: For our future work, We would like to find a way to detect the hashtags or keywords which can help us find more individual account to help us collect training data and maybe it can reveal some features of improper users We are planning to include tweet behavior and relationship of user in feature vector design, and we are considering introduce the deep learning into our research if we can get enough data Finally we want to connect our filter system to the existed platform, which in this case, collagree, to evaluate our system in practice