Presentation, case study_event detection in twitter

Editor's Notes

• #2: Hello, Everyone Today, me and Yue we’d like to talk about our implementation of EDCoW (Event Detection with Clustering of Wavelet-based Signals) This case study was under the guidance of ...
• #3: Well, Twitter is one of the most popular microblogging services, and has received much attention recently. And, Microblogging is a form of blogging that allows user to send brief text updates. For Twitter it is 140 characters per tweets. users can post photographs or audio clips similar to text. This is the screenshot of Twitter Public Time. Twitter users write tweets several times in a single day. The number of tweets was claimed to be about 200million per day. So, there is the large number of tweets, which results in many reports related to events. For example, they include social events such as parties, baseball games and presidential campaigns. And they also include disastrous events, such as storms, traffic jams riots , heavy rain-fall and earthquakes. We want to know how we can detect these events.
• #6: Different event detection methods adapted to Twitter were studied in several papers. In general, as mentioned in a survey paper that Adrien shared, we have these approaches (1) The idea is to identify words that are particular to a fixed length time window. (2) The idea is to incrementally update the topic model in each time window using previously generated model to guide the learning of the new evolves. (3) The idea is to study the similarity between pairs of words, then to cluster the words -> EDCoW is in that approache
• #8: Just to give you an overview of EDCoW, here is the very simplified workflow. First, we have as input a list of keywords that were stored in different files each. Then, we filter them to get the significant keywords. After that, we get the cross correlation matrix of these keywords. Finally, we cluster the very correlated keywords to get bag of words of events.
• #9: So all in all, EDCoW is the composition of 3 components Signal Construction / Cross Correlation Computation / Modularity-based Graph Partition In Signal Construction, To identify burst, EDCoW uses the frequency of individual words to get wavelets, leverages Fourier; and Shannon theories to compute the change of wavelet entropy. In Cross Correlation Computation Trivial words are filtered away based on their corresponding signal’s autocorrelation, and the similarity between each pair of non-trivial words are measured then measured. In Modularity-based Graph Partition in the end, events are detected with modularity-based Graph partitionning. After having the clusters, events’significance : depends on 2 factors : - number of words - corss correlation among the words and we filter the detected events. ******************************************************* The wavelet analysis provides precise measurements regarding when and how the frequency of the signal changes over time - Determine the existence of any natural division of vertices of a network. - Use Newman’s modularity function (without a ground truth). - Disjoint communities in a graph. - Weighted network. Modularity reflects the concentration of nodes within modules compared with random distribution of links between all nodes regardless of modules. Modularity Q =(# of edges within groups) - (expected # of edges within groups) For a random network, Q = 0 (the number of edges within a community is no different from what you would expect) High modularity dense connections between the nodes within modules + sparse connections between nodes in different modules
• #14: maybe you will ask, there are too many words in the dataset, how can you group the sets of words?
• #18: maybe you will ask how to find the events from the network? (see next slide)