GRAPH-TA 2013 - RDF and Graph benchmarking - Jose Lluis Larriba Pey
1 like609 views
The document discusses an agenda for a meeting on benchmarking RDF and graph databases. It provides an overview of the LDBC benchmarking project including its objectives to create standardized benchmarks, spur industry cooperation, and push technological improvements. It outlines the working groups and task forces that will focus on specific types of benchmarks. It also discusses common issues to be addressed like suitable use cases, benchmark methodologies, and benchmark workloads for querying, updating, and integrating RDF and graph data. Open questions are raised about benchmark realism, benchmark rules, and technological convergence of RDF and graph databases.
GRAPH-TA 2013 - RDF and Graph benchmarking - Jose Lluis Larriba Pey
- 1. RDF and Graph benchmarking www.ldbc.eu GRAPH-TA Barcelona Feb 19, 2013 Graph-TA, Barcelona Feb 19th, 2013
- 2. Agenda - Why benchmarking - Presentation of LDBC - Remarks Who is who Project overview: WP and Task Forces TUC, Technical Users Community - Benchmarking RDF and GDB - Common issues - Open questions
- 3. Why benchmarking • Two main objectives: – Allow final users to assess the performance of the software they want to buy – Push the technology to the limit to allow for progress • Main effort in DB benchmarking up to now – TPC: Transaction Processing Performance Council • Relational DBs: Transactional and DSS
- 4. LDBC • Objectives: – Benchmarks for the emerging field of RDF and Graph database management systems (GDBs) – Spur industry cooperation around benchmarks – Create LDBC foundation during 1Q 2013. – Become a technology push effort, making improvements measurable – Become the de-facto research benchmark, usable, interesting and open to inputs.
- 5. Preliminary remarks • Nature: – LDBC is different from other EC projects. – The objective for LDBC is to survive after the end. • Opportunity: – Have a benchmarking effort sponsored by EC. – Focal point for the vendor community. – Showcase to the user community. • Collaboration: – LDBC should lead to great achievements and world recognition, with the help of all the community: industry, technologists and users.
- 6. Who is involved • • • • • • • • FORTH, research centre, Greece TUM, research centre, Germany UIBK, technology centre, Austria Neo Technologies, Graph management, Sweeden OGL, RDF management, UK ONT, RDF management, Bulgaria VUA, research institution, Netherlands DAMA-UPC, research institution, Spain
- 7. Project Overview: the WPs
- 8. Matrix Organization • EU project reporting activities (WPs) • LDBC benchmark task force activities (TFs) WP1 WP2 WP3 WP4 TF1 TF2 TF3 TF4 TF5 (part of) deliverable/ task force artifact
- 9. Task forces • Focusses on specific benchmarking effort (i.e. transactional, analytical, integration for RDF/GDB) • Decides on the Use Case to be used • Procedure: – Designs and implements data generation (characteristics, scale, etc.) – Incorporates the generic methodology – Designs specific workload • Choke points specific for the effort • Incorporating the needs from users • Incorporating the opinions from industry
- 10. Technical User Community (TUC) • It will be the driving force for LDBC: – To help understand users needs and decide use cases – To decide the type of problem/scenarios to be tackled, i.e. task forces to be deployed – To provide typical queries placed to RDF and GDBs • First TUC meeting, Barcelona 19-20 Nov. – Start with an on-line questionaire: http://goo.gl/PwGtK – The outcomes will determine important directions
- 11. Common issues • Use case for RDF and GDBs: • Social Network Analysis • Semantic Publishing, specific for RDF (SP) • Methodology: • Audited benchmarks • Specific rules, similar to TPC • Workload for RDF – – – – – – Throughput, concurrency Traversals Reasoning Data updates Integration: LOD, geonames, etc. SP: semantic annotation support, relationship btwn ontologies and instances, links to other content, text and metadata. • Workload for GDBs – Throughput, concurrency – Traversals, shortest paths, pattern matching, clustering algorithms – Data updates, transactionality – Update semantics (serializable/acid vs delayed commit vs batch) – Raw traversal speed, use of indexes
- 12. Open questions • Use cases: • • • • How realistic would you see synthetic data generation? Use of real data like twitter, Facebook or Open Ontologies? Any suggestions for use cases? Any suggestion for scenarios: analytical, transactional, integration, others? • Would it make sense to propose open benchmark scenarios? • The tight rules of TPC: – Are those against the realism of benchmarks? – Can we solve this in any flexible way? • Will RDF and GDB move towards the same technology/solutions? • GDBs: no standard language. How to proceed?