Slides: Safeguarding Abila through Multiple Data Perspectives
0 likes336 views
The document outlines solutions developed for the VAST 2014 challenge focusing on data analysis and visualization, highlighting advantages such as user-centered interfaces and minimal backend data cleaning. It describes two main analytical tools: a news analyzer for exploring articles and an email analyzer for visualizing email networks, using various technologies for implementation. Additionally, it discusses point of interest identification from GPS data and analysis techniques applied to categorize user activities in different settings.
Slides: Safeguarding Abila through Multiple Data Perspectives
- 1. Safeguarding Abila through Multiple Data Perspectives VAST 2014 Grand Challenge Award: Effective Analysis and Presentation VAST 2014 Mini Challenge 2 Award: Honorable Mention for Effective Presentation Parang Saraf; Patrick Butler; Naren Ramakrishnan Discovery Analytics Center, Department of Computer Science, Virginia Tech 1 Presented By: Parang Saraf
- 3. VAST Challenge Solution From DAC • The DAC solution offers three key advantages: 1. Provides an efficient front-end interface for user-centered exploration of data 2. Very little analysis or cleaning of data is performed in the backend, thereby helping an analyst to understand the data better ! Example: Faulty news sources or GPS coordinates are displayed 3. Offers an intuitive interface to present data in several different ways • Each Interface was designed from scratch specifically for the VAST Challenge 3
- 4. Mini Challenge - I 4
- 5. Mini Challenge - I • Two Interfaces: – News Analyzer • Helps an Analyst explore news articles and Identify Events – Email Analyzer • Helps an Analyst visualize and examine Email network 5
- 10. News Analyzer 10
- 11. News Analyzer 11
- 12. News Analyzer 12
- 13. News Analyzer 13
- 14. News Analyzer 14
- 15. News Analyzer Example: How POK Leadership has changed over Time 15 1998 – 1999
- 16. News Analyzer Example: How POK Leadership has changed over Time 16 1998 – 1999 2000 – 2009
- 17. News Analyzer Example: How POK Leadership has changed over Time 17 1998 – 1999 2000 – 2009 2000 – 2014
- 18. News Analyzer Packages Used • Search Engine – Whoosh (Python search engine library) • Provides Logical Query of Articles • Returns Similar articles • Named Entity Recognition – Stanford NER Parser • Provides Person, Organization and Locations in the input document set • Visualization – NVD3 • Line Chart – D3 • Word Clouds 18
- 26. Email Analyzer 26 Spectral Co-Clustering • Given an n x m matrix of n documents and m words, the algorithm performs co-clustering of documents and words. • The clustering problem is posed in terms of finding minimum cut vertex partitions in a bipartite graph between document and words • We provide an m x m matrix where rows and columns denote employees and a cell denotes the number of emails exchanged • Implemented using the scikit-learn package
- 30. Email Analyzer - Example 30
- 31. Email Analyzer - Example 31
- 32. Email Analyzer - Example 32
- 33. Email Analyzer Packages Used • Co-Occurrence Matrix – Spectral Co-Clustering Algorithm – Visualization • D3 – Email Radial, Co-Occurrence matrix and Word Cloud 33
- 34. Mini Challenge - II 34
- 35. Mini Challenge – II Points of Interest (POIs) • KML file doesn’t provide any Point type data • A POI is defined as any location where a user stops for more than 5 minutes. – The location has a diameter of 50 meters – Example: A shopping complex. • Several users will park their car for more than 5 minutes and will park within vicinity (< 50 meters) of each other. – Assumption: We are interested in knowing which locations a user visit rather than which path he takes 35
- 36. Mini Challenge – II Points of Interest (POIs) • Total 129 POIs were identified: – 64 resulted because of the faulty GPS associated with Cars 9 and 28 – Remaining 65 POIs an analyst classifies into: • Home POIs: Where a user is present during night • Work POIs: Where the user is present on weekdays during office hours • Recreational POIs: Where the users are present during Breakfast, Lunch and Dinner hours • Suspicious POIs: None of the above • How to classify these POIs? 36
- 37. Mini Challenge - II • POI Analysis • Recreational POIs • Location Playback • Spending Analysis 37
- 38. POI Analysis Identifying Work And Home POIs 38
- 39. POI Analysis Identifying Work And Home POIs 39
- 40. POI Analysis Identifying Work And Home POIs 40
- 41. POI Analysis Identifying Work And Home POIs 41
- 42. POI Analysis Identifying Work And Home POIs 42 Work Home
- 43. POI Analysis Identifying Work And Home POIs 43
- 44. POI Analysis Identifying Work And Home POIs 44
- 45. POI Analysis Identifying Work And Home POIs 45
- 46. POI Analysis Identifying Work And Home POIs 46
- 47. POI Analysis Identifying Work And Home POIs 47 Weekdays For All Cars Weekends For All Cars
- 48. POI Analysis Identifying Work And Home POIs 48 Weekdays For All Cars Weekends For All Cars
- 49. POI Analysis Identifying Work And Home POIs 49 Weekdays For All Cars Weekends For All Cars
- 50. POI Analysis Identifying Work And Home POIs 50 Weekdays For All Cars Weekends For All Cars
- 51. POI Analysis Identifying Work And Home POIs 51 Weekdays For All Cars Weekends For All Cars
- 52. POI Analysis Identifying Work And Home POIs 52 Weekdays For All Cars Weekends For All Cars
- 59. POI Analysis Characterizing POIs 59 Home POI Recreational POI
- 61. POI Analysis Characterizing POIs 61 Home POI Recreational POI
- 67. Location Playback 67 POI Distibution By Cars
- 69. Spending Analysis Employee Vs. Employee Comparison 69
- 70. Spending Analysis Employee Vs. Employee Comparison 70
- 71. Spending Analysis Employee Vs. Employee Comparison 71
- 72. Spending Analysis Employee Spending Distribution 72
- 73. Spending Analysis Establishment Sales Distribution 73
- 74. Mini Challenge – II Packages Used • Google Map API – For displaying geographical map • D3 – For displaying all the charts • DataTables (Table Plug-in for jQuery) – For displaying Spending Information 74
- 75. Mini Challenge - III 75
- 76. Mini Challenge - III 76
- 77. Mini Challenge - III 77
- 78. Mini Challenge - III 78
- 79. Mini Challenge - III 79
- 80. Mini Challenge - III 80
- 81. Mini Challenge - III 81 Displays the most frequent words by default in the stream
- 82. Mini Challenge - III 82
- 83. Mini Challenge - III 83
- 84. Mini Challenge - III 84
- 85. Mini Challenge - III 85
- 86. Mini Challenge - III 86
- 87. Mini Challenge - III • Data streamed at 100x the speed – We stored the data beforehand • Searching and updating is performed in real-time on streaming data 87
- 88. Mini Challenge – III Packages Used • Google Maps API – For displaying the map • D3 and NVD3 – For displaying the line chart 88
- 89. Thank You 92