Traffic Analytics for Linked Data Publishers
2 likes685 views
The document discusses a traffic analytics platform designed for linked data publishers, addressing the lack of tools for measuring access to RDF datasets. It outlines challenges such as client session detection, traffic metrics extraction, and noise filtering while providing insights into heavy and light SPARQL queries. The project aims to automate access log mining, support linked data-specific metrics, and enhance understanding of user interactions with linked data infrastructure.
![4
Existing tools do not include Linked Data-specific metrics:
Linked data-specific metrics, but no platform [Moller et al, WebScience 2010]
Filling a Gap in Prior Art](https://image.slidesharecdn.com/analytics-eswc2017-20170601-170601143958/85/Traffic-Analytics-for-Linked-Data-Publishers-4-320.jpg)
![14
Visitor Session Detection
Session: sequence of requests issued with no significant
interruptions by a uniquely identified visitor. Expires after a
period of inactivity.
We use the HAC variant by [Murray et al. 2006, Mehrzadi et al. 2012]
Unsupervised, gap-based session boundary detection
Traditional web logs analysis
Benefit: visitor-specific temporal cut-off
Two-step procedure:
Set visitor-specific session cut-off as time interval that significantly
increases the variance.
Group HTTP/SPARQL requests into sessions according to the cut-off](https://image.slidesharecdn.com/analytics-eswc2017-20170601-170601143958/85/Traffic-Analytics-for-Linked-Data-Publishers-14-320.jpg)
Traffic Analytics for Linked Data Publishers
- 1. FUJITSU RESTRICTED - UK & IRELAND EYES ONLY © Copyright 2014 Fujitsu (Ireland) Limited Traffic Analytics for Linked Data Publishers Luca Costabello, Pierre-Yves Vandenbussche, Gofran Shukair Fujitsu Ireland Corine Deliot, Neil Wilson British Library
- 2. 2 The Problem: Measuring Traffic on RDF Datasets Linked Data publishers have limited awareness of how datasets are accessed by visitors. No tool to mine Linked Data servers access logs Why is this such a big deal? Justify investment in Linked Data IT infrastructure Cost control Identify abuses Interpret access peaks
- 3. 3 Which traffic metrics? Adapt conventional web analytics metrics Define Linked-Data specific extensions How to extract and compute such metrics? Which data sources? (client tracking? server access logs mining? both?) Need to support dual data access protocol (HTTP operations + SPARQL) How to filter noise? (i.e. robots, search engines crawlers) How to detect client sessions? (no client tracking, dual data access protocol) How to detect SPARQL activity peaks? Challenges
- 4. 4 Existing tools do not include Linked Data-specific metrics: Linked data-specific metrics, but no platform [Moller et al, WebScience 2010] Filling a Gap in Prior Art
- 5. 5 • Traffic analytics platform for LD servers • Metrics • Metrics Extraction • Visitor Sessions • Heavy/Light SPARQL Queries • Results & British Library Trial Insight Our Contribution / Agenda
- 6. 6 • Traffic analytics platform for LD servers • Metrics • Metrics Extraction • Visitor Sessions • Heavy/Light SPARQL Queries • Results & British Library Trial Insight
- 7. 7
- 8. 8
- 9. 9 • Traffic Analytics Platform for LD Servers • Metrics • Metrics Extraction • Visitor Sessions • Heavy/Light SPARQL Queries • Results & British Library Trial Insight
- 13. 13 • Traffic Analytics Platform for LD Servers • Metrics • Metrics Extraction • Visitor Sessions • Heavy/Light SPARQL Queries • Results & British Library Trial Insight
- 14. 14 Visitor Session Detection Session: sequence of requests issued with no significant interruptions by a uniquely identified visitor. Expires after a period of inactivity. We use the HAC variant by [Murray et al. 2006, Mehrzadi et al. 2012] Unsupervised, gap-based session boundary detection Traditional web logs analysis Benefit: visitor-specific temporal cut-off Two-step procedure: Set visitor-specific session cut-off as time interval that significantly increases the variance. Group HTTP/SPARQL requests into sessions according to the cut-off
- 15. 15 • Traffic Analytics Platform for LD Servers • Metrics • Metrics Extraction • Visitor Sessions • Heavy/Light SPARQL Queries • Results & British Library Trial Insight
- 16. 16 Heavy/Light SPARQL Queries Binary Classifier Rough estimate of heavy and light queries with supervised binary classification. Heavy SPARQL Query: if it requires considerable computational and memory resources. Light Heavy
- 17. 17 Heavy/Light SPARQL Queries Binary Classifier Feature vectors: SPARQL 1.1 syntactic features only:
- 18. 18 • Traffic Analytics Platform for LD Servers • Metrics • Metrics Extraction • Visitor Sessions • Heavy/Light SPARQL Queries • Results & British Library Trial Insight
- 19. 19 British National Bibliography access logs bnb.data.bl.uk (access logs are not public) 13 months ~ 10M HTTP requests/month DBpedia 3.9 access logs USEWOD 2015 Dataset Datasets
- 20. 20 Visitor Session Detection: Results How well do we detect the beginning of a new session? Dataset British National Bibliography access logs (3 consecutive days) ~16k HTTP/SPARQL requests • 32% Desktop browsers (115 visitors) • 68% Software libraries (10 visitors) Manually annotated records • 1=session_start | 0=internal Baseline: fixed-length cut-offs HAC outperforms fixed-length cut-offs
- 21. 21 Random distinct queries from DBpedia 3.9 access logs Run the queries multiple times on local clone of DBpedia Kept ~3.7k queries with low variance (3.1k light, 600 heavy) Cut-off threshold: 100ms Naïve Bayes and SVM Grid search & randomized search w/ 10-fold CV Heavy/Light SPARQL: Experiment Protocol
- 23. 23 Genuine calls account for 0.6% of total traffic! +30% of HTTP/SPARQL traffic over the observed 13 months Sharp increase in requests from Software Libraries (95x) SPARQL accounts for 29% of traffic 6% of heavy SPARQL queries 37 days have unusual traffic spikes Bounce rate: 48% Software Libraries have bigger, deeper, and longer sessions. Some Insights on BL Traffic Logs
- 24. 24 We relieve publishers from manual and time-consuming access log mining Support Linked Data-specific metrics Break down traffic by RDF content Capture SPARQL insights Properly interpret 303 patterns Reconstruction of Linked Data visitors sessions Heavy/light SPARQL classifier w/ SPARQL syntax + supervised learning Revealed hidden insights on 13 months of access logs of the British Library Summary
- 25. 25 Statistics on noise (i.e. web crawlers) Heavy/light classifier Feature set refinements Does it generalize to other datasets? Enhance session detection with content-based heuristics Relatedness of subsequent SPARQL queries Structure and type of requested RDF entities Future Work