Big Data Analytics and Knowledge Discovery through Location-Based Social Networks (LBSN)
- 1. Big Data Analytics and Knowledge Discovery through Location-Based Social Networks (LBSN) Master’s Thesis Presentation Author: Ioannis Makridis Supervisor: Prof. Ioannis Kopanakis Technological Educational Institute of Crete Department of Informatics Engineering Postgraduate program «Informatics and Multimedia» Heraklion, July 2018
- 2. ➔ Introduction ➔ State of the art ➔ Motivation and research questions ➔ Implementation & case study ➔ Findings ➔ Conclusion & further research Outline 2
- 3. 3 Big Data Era “The process of collecting, managing, analyzing and visualizing large amounts of data to generate knowledge and expose hidden patterns.”(*) (*) ”Creating value from Social Big Data: Implications for Smart Tourism Destinations”, Vecchio et al., Information Processing & Management, vol.54, 2017.
- 4. Big Data - Characteristics 7. Value 4 1. Volume: the large size of generated datasets (Internet of Things, Artificial Intelligence, User-generated content) 2. Velocity: the high speed of data generation (more and more devices are interconnected - more and more users have internet access) 3. Variety: the diversity of data sources and formats (unstructured data, semi-structured data, structured data, multi-structured data) 4. Variability: the meaning of data can vary significantly in context as well as values can differ from each other 5. Veracity: the challenge of having inaccurate, incomplete or non-genuine data (truthfulness & reliability) 6. Visualization: the presentation of quantitative and qualitative information in a schematic form, in ways that trends, patterns, and anomalies can be observed and understood 7. Value: big data is recognized as a key source for creating value since it is believed to result in (1) more efficient operations, e.g. setting the most profitable price for products and services, optimizing supply chain flows and minimizing errors and quality problems, and (2) improving customer relationships
- 5. Big Data - Benefits 7. Value 5 ➔ Data-driven decision-making (smarter decisions based on actual data) ➔ Better predictions ➔ Creation of new business models, products and services ➔ Increase in productivity and profit ➔ Understanding of business environments ➔ Identifying customers’ needs ➔ Competitive advantage
- 6. 6 Social → Big Data Data generated every minute in 2018 (Data Never Sleeps Infographic v.6, Domo, 2018)
- 7. 7 Social Media Analytics “The analysis of structured and unstructured data collected from various social media channels.”(*) ➔ social networks (e.g. Facebook and LinkedIn) ➔ microblogs (e.g. Twitter and Tumblr) ➔ media sharing (e.g. Instagram, Flickr and YouTube) ➔ social news (e.g. Digg and Reddit) ➔ review sites (e.g. Foursquare and TripAdvisor) (*) “Big data analytics techniques: A survey”, Vashisht P. & Gupta V., International Conf. on Green Computing and Internet of Things (ICGCIoT), 2015.
- 8. Common techniques that are applied in social media analytics: ➔ Sentiment Analysis ➔ Natural language processing (NLP) ➔ Social networking analysis (influencer identification, profiling and scoring) ➔ Predictive modeling ➔ Entities identification These techniques are applied using social big data in operations such as: ➔ Opinion mining and analysis ➔ Socialized marketing ➔ Decision-making Social Media Analytics 8
- 9. The applications of social big data are divided into two main types: ➔ Content-based Applications where the text, its language and location are the most important factors as we can identify the users’ preference, emotion, interest, demand, etc. for specific regions of interest. ➔ Structure-based Applications where we can identify the users’ hobbies, interests, and relations into a clustered structure (community). Social Media Analytics 9
- 10. 10 Location Intelligence“The use of locationally referenced information as a key input in business decision-making.”(*) (*) ”Encyclopedia of GIS”, Shekhar S. & Xiong H., Springer Science & Business Media, 2007.
- 11. Location Intelligence (LI) 11Defining Location Intelligence (3 Ways Location Intelligence Is Already Part of Your Life, Hahn K., Ironsidegroup, 2015)
- 12. Location Intelligence (LI) Benefits of LI (*): ➔ Businesses and public authorities can better understand external characteristics and how they affect their operations ➔ Gaining a much more complete picture for a phenomenon by integrating location and time dimensions to internal data Modern companies must follow three main steps to integrate LI into their business properly, these are (**): ➔ Location discovery ➔ Location analytics ◆ Analysis ◆ Predictions ◆ Visualizations ➔ Location optimization 12 (*) “Location Intelligence - The Future Looks Bright”, Milton S., Forbes Media LLC., 2011. (**) “The 3 cores of Location Intelligence”, Franchet P., Galigeo, 2017.
- 13. 13 Big Data Analytics 7. Value The three principal data analytics types (Southeastern University - Lakeland, FL)
- 14. Data Analytics - Types → Descriptive Analytics 7. Value 14 Descriptive analytics is based on current and historical data to provide significant insights. Using techniques like online analytical processing (OLAP), probability analysis, trending and association of data that is already classified and categorized, descriptive analytics answers to what’s happening in the organizations regarding their sales, orders, marketing, supply chain, support, customers and financial performance. Interactive Dashboards Alerts & Reports Data Visualizations
- 15. Big Data Analytics - Techniques 7. Value 15 There are several techniques (more than 25) being used to analyze big datasets including data/text mining, pattern matching, forecasting, visualization, semantic analysis, sentiment analysis, cluster analysis, neural networks, etc.(*) Techniques used in this thesis: (*) IT Glossary: Advanced Analytics, Gartner Inc. ➔ Text Mining ➔ Sentiment Analysis ➔ Social Network Analysis ➔ Spatial Analysis ➔ Clustering ➔ Visualization
- 16. State of the art 16
- 17. Exploration of perceptions in Asian restaurants (Korean, Japanese, Chinese and Thai) using Twitter analysis (on 86K tweets).(*) ➔ Techniques ◆ Text mining ◆ Word frequency analysis ◆ Sentiment analysis ➔ Findings ◆ The average sentiment score of Chinese restaurants was significantly lower than the others ◆ The most positive tweets referred to food quality ◆ Many negative tweets suggested problems about the service quality or food culture Related work 1 17 (*) “Analyzing Twitter to explore perceptions of Asian restaurants”, Park S.B. et al., Journal of Hospitality and Tourism Technology, 2016.
- 18. Extraction and visualization of ratings and reviews for Hilton hotel using TripAdvisor.(*) ➔ Techniques ◆ Natural language processing ◆ Sentiment analysis ➔ Findings ◆ Types of travelers that are giving the lower and higher ratings (business travelers and couples) ◆ Months with the lowest and highest rates (July and December) ◆ The travelers’ emotions according to the most frequently used negative or positive words Related work 2 18 (*) “Social media analytics: Extracting and visualizing Hilton hotel ratings and reviews from TripAdvisor”, Chang Y.C. et al., Int. J. Inf. Manag., 2017
- 19. Integration of data from TripAdvisor and Booking.com to extract meaningful information for tourism planning and decision-making. ➔ Techniques ◆ Spatial analysis ◆ Spatial statistics ➔ Findings ◆ Which are the most popular destinations ◆ Why people chose those destinations ◆ What attracts tourists attention and what do they appreciate/disregard Related work 3 19 (*) “Social Media Data in Tourism Planning: Analysing Tourists’ Satisfaction in Space and Time”, Floris R. & Campagna M., REAL CORP 2014 - Plan it smart! Clever Solutions for Smart Cities, 2014.
- 20. Motivation The aim of this thesis is to design and develop a web application for discovering knowledge through the analysis of actual user-generated content on location-based social networks and emphasizing the significant impact of “where” in business operations. 20
- 21. Research questions Through the acquisition of geospatial data from four popular LBSNs including Twitter, Foursquare, Instagram and Flickr we wanted to answer the following research questions: ● “What are the visitors’ and local people’s behavior, impressions, and preferences for tourist destinations?” ● “What decisions local authorities and businesses can take to make a more efficient promotion of the tourist destinations, to improve the existing facilities and activities, and to create new experiences for attracting the interest of more potential visitors?” In order to answer these questions and find out what insights can be extracted from the analysis of LBSN data, we used the case study of two cities - Heraklion and Chania - from Crete (Greece). 21
- 22. KnowLI: An application for knowledge discovery through LBSNs Implementation
- 23. KnowLI 23 Key processes 1. Data acquisition, cleansing and storage ◆ Data fetch through the communication with LBSNs web APIs ◆ Transform data into a structured form ◆ Store data to SQL database and data warehouse 2. Data analysis ◆ Sentiment analysis ◆ Entity analysis 3. Data querying and visualization ◆ Maps and heatmaps ◆ Pie charts ◆ Line charts ◆ Bar charts ◆ Word clouds
- 24. KnowLI → Technologies Client-side Server-side 24 AngularJS HTML5 CSS3 Google Charts D3.js Cloud SQL BigQuery Cloud Storage Express.jsNode.jsJavaScript
- 25. KnowLI → System Architecture 25
- 26. MySQL vs. BigQuery (test on our dataset) 26 SELECT * FROM `data` ORDER BY `data`.`id` DESC LIMIT 100000; Query result from BigQuery is 66% faster than MySQL
- 27. KnowLI → Defining the Regions of Interest (ROIs) 27
- 28. 28 Posts Analytics ➔ Location ➔ Sentiment ➔ Entities ➔ Language ➔ Social engagement ➔ Top hashtags ➔ Top words ➔ Posts per social network ➔ Posts per day of week ➔ Posts per hour Photos Analytics ➔ Location ➔ Social engagement ➔ Top hashtags ➔ Top words ➔ Posts per social network ➔ Posts per day of week ➔ Posts per hour Business-level Analytics ➔ Places’ location ➔ Places’ details ◆ rating ◆ check-ins ◆ category ➔ Top businesses types KnowLI → Insights
- 29. The case of Crete - Regions’ overall numbers 29 Period: 1/4/2018 ~ 30/6/2018 Total users, posts, sentiment, photos and engagement per city
- 30. The case of Crete - Textual posts & sentiment 30 Period: 1/4/2018 ~ 30/6/2018 Total posts and sentiment per day for Heraklion (top) and Chania (bottom)
- 31. The case of Crete - Posts’ map 31 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018
- 32. The case of Crete - Posts’ clusters 32 Period: 1/4/2018 ~ 30/6/2018 Map with posts’ clusters for Heraklion (top) and Chania (bottom)
- 33. The case of Crete - Social engagement 33 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Social engagement line chart for both cities
- 34. The case of Crete - Social networks 34 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Total textual posts and photos per channel and ROI
- 35. The case of Crete - Posts per day of week 35 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Textual posts and photos per day of week for Heraklion (left) and Chania (right)
- 36. The case of Crete - Posts per hour 36 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Textual posts and photos per hour for Heraklion (left) and Chania (right)
- 37. The case of Crete - Posts’ sentiment 37 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Textual posts’ sentiment for Heraklion (left) and Chania (right)
- 38. The case of Crete - Posts’ sentiment 38 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Response with text’s sentiment from Google Natural Language Processing API
- 39. The case of Crete - Posts’ entities 39 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Textual posts’ entities for Heraklion (left) and Chania (right)
- 40. 40 The case of Crete - Posts’ entities Response with text’s entities from Google Natural Language Processing API
- 41. The case of Crete - Posts’ languages 41 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Textual posts’ languages for Heraklion (left) and Chania (right)
- 42. The case of Crete - Top hashtags 42 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Word cloud with the top hashtags for Heraklion (left) and Chania (right)
- 43. The case of Crete - Top hashtags 43 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Table with the top hashtags for Heraklion (left) and Chania (right)
- 44. The case of Crete - Top words 44 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Word cloud with the most used words for Heraklion (left) and Chania (right)
- 45. The case of Crete - Top words 45 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Table with the most used words for Heraklion (left) and Chania (right)
- 46. The case of Crete - Places’ map 46 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Places’ map
- 47. The case of Crete - Places’ clusters 47 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Map with places’ clusters for Heraklion (left) and Chania (right)
- 48. The case of Crete - Places’ table 48 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Places’ data table
- 49. The case of Crete – Top businesses types 49 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Top businesses types for Heraklion (left) and Chania (right)
- 50. Findings (1/3) 50 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Heraklion Chania Total users 14.5K 14.6K Total posts 441 textual posts & 63.8K photos 1.9K textual posts & 57.3K photos Social engagement 3.2M likes + 10 shares + 78.1K comments 2.1M likes + 477 shares + 56.9K comments Popular places City’s center, Koule Fortress, Natural History Museum of Crete, Heraklion Archaeological Museum, Port, local market of Heraklion City’s center, Old Venetian Harbour, Lighthouse, local market of Chania Top social networks Twitter for texting and Instagram for media sharing Twitter for texting and Instagram for media sharing
- 51. Findings (2/3) 51 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Heraklion Chania Textual posts' sentiment Positive 33.8%, Neutral: 58.6%, Negative: 7.6% Positive 32.7%, Neutral: 58.9%, Negative: 8.4% Most positive themes Places' beauty, quality of services, food tastefulness Places' beauty, customer service Most negative themes Customer service, outdoor activities Quality of services and products Most mentioned entities location 41.6%, person 35.2% location 40.5%, person 36.6% Top languages English, Greek, Russian, Spanish, French,Turkish English, Greek, Spanish, Russian, French, Finish
- 52. Findings (3/3) 52 Examined period: 1/4/2018 ~ 30/6/2018Period: 1/4/2018 ~ 30/6/2018 Heraklion Chania Top hashtags #crete, #greece, #heraklion, #summer, #sea #crete, #greece, #chania, #summer, #chaniacrete Most-used words Crete, Greece, summer, Heraklion, love Chania, Crete, Greece, summer, holiday Days with most posts Texting: Tuesday Media sharing: Tuesday, Wednesday, Thursday Texting: Saturday Media sharing: Sunday Hours with most posts Texting: morning hours from 08:00 to 10:00 Media sharing: evening hours from 19:00 to 22:00 Texting: evening hours from 18:00 to 22:00 Media sharing: evening hours from 18:00 to 22:00 Top businesses types Greek restaurants 20.5%, Cafe 20.2%, Salons/barber shops 14% Hotels 15.4%, Automotive 14.3%, Cafe 13.1%
- 53. 53
- 54. 54 Conclusion ● Big data analytics can transform the way organizations operate and increase the business adaptivity to the rapidly changing environment ● Using data processing tools and advanced analytics techniques, the organizations can predict and observe future trends, build econometric models and identify customer needs ● By combining techniques analyzing big data (textual content, photos and position) coming from LBSN, stakeholders of tourism destinations such as local authorities, enterprises etc. are able to plan and implement more efficient strategies for value creation ● Big data should be harnessed for data-driven strategic decisions and enhanced destination competitiveness ● Influencer marketing or e-word of mouth marketing can contribute to enhancing destination attractiveness or destination branding since influencers can spread messages affecting communities in the digital world
- 55. 55 Suggestions for further research ➔ Integrate data from more social media platforms ➔ Image analysis (e.g object detection) ➔ Apply predictive analytics ➔ Use of microservices architecture
- 56. Thanks for your attention max growth 56
Editor's Notes
- #34: Heraklion: - 17/5 London-based Makeup artist 107K likes on post, user has 1.1m followers (Bi5DU5jlJ8i) - 18/5 London-based Makeup artist 98K likes on post, user has 1.1m followers (Bi7muiIlmyU) - 20/6 Czech-based Youtuber 92K likes on post, user has 419K followers (BkQVipwnyGa) Chania: - 16/4 Indian singer 77K likes on post - 217K video views, user has 11M followers (BhoT7NsloBa)