Querying the Web of Data
3 likes959 views
The document discusses the Semantic Web and Linked Data. It provides an overview of RDF syntaxes, storage and querying technologies for the Semantic Web. It also discusses issues around scalability and reasoning over large amounts of semantic data. Examples are provided to illustrate SPARQL querying of RDF data, including graph patterns, conjunctions, optional patterns and value testing.
Querying the Web of Data
- 1. Querying the Web of DataKennissystemen, December 2010Rinke Hoekstra
- 2. OverviewLinked (Open) DataThe Web of DataScalability issuesTechnologyRDF SyntaxesRDF Storage and QueryingKennissystemen 2010
- 3. The Semantic Web IdeologyIdentity is everythingPartial solutions are great too!Layer cakeOWLKennissystemen 2010
- 4. The Web of Data… does it exist?Kennissystemen 2010
- 7. `Metadata’ for web pages
- 8. Since ~2006
- 10. Semantic web as data source in its own right
- 11. Linked Data
- 15. Storage (on the web)As documents.rdf, .n3, .turtle, .htmlRDF triple storesSesame, Joseki, 4Store, AllegroGraph, OpenLink Virtuoso, SDB/TDB, Open Calais, SWI Prolog Reasoners ‘on top’, or via DIGPellet, OWLIM, etc.SPARQL EndpointsResults as JSON, XML, CSV etc.Kennissystemen 2010
- 16. Data and the WebNeed to add this ‘meta’ to my ‘data’‘Linking’ data across sitesWeb of Documents and the Web of DataOld fashioned HTML:<link rel='meta' type='application/rdf+xml' href='http://www.leibnizcenter.org/~hoekstra/foaf.rdf' title='FOAF'> URL-basedHTTP 303 `see other’http://www.w3.org/TR/swbp-vocab-pub/RDFaKennissystemen 2010
- 21. Integration: 303 See OtherKennissystemen 2010
- 22. Integration: Inline RDFaAttributes on XHTML elementshttp://www.w3.org/TR/xhtml-rdfa-primerKennissystemen 2010
- 23. Integration: RDFa ExampleIn XHTML:<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd"><html xmlns:cal="http://www.w3.org/2002/12/cal/ical#"> <head><title>Jo's Friends and Family Blog</title></head> <body> <pinstanceof="cal:Vevent"> I'm holding <span property="cal:summary">one last summer Barbecue</span>, on <span property="cal:dtstart" content="20070916T1600-0500"> September 16th at 4pm. </span> </p> </body></html>In RDF:_:blanknode0rdf:typecal:Vevent; cal:summary ”one last summer Barbecue";cal:dtstart "20070916T1600-0500" .Kennissystemen 2010
- 26. So, where’s that data?I repeat: does it really exist?Kennissystemen 2010
- 27. Linked Open DataKennissystemen 2010
- 28. November 2009: 13.1 Billion triples, 142 Million links Kennissystemen 2010
- 29. September 2010: 25 Miljard triples, 395 Miljoenlinks Kennissystemen 2010
- 30. ScalabilityHow to deal with massive amounts of data?Consequences for reasoningBillion Triple Challenge(864.8 Million Triples)Consequences for queryingTable lookups, joins etc.… and what about …Dealing with change, provenance, trust?Kennissystemen 2010
- 31. A rough idea…I can crash a DL reasoner using an ontology of ~15 classes and 5 individuals (honestly)What if my ontology contains thousands of classes and billions of individuals?Kennissystemen 2010
- 32. ReasoningReasoning with inconsistent knowledgeincomplete knowledgeComplete vs. incomplete reasoningKennissystemen 2010
- 33. ReasoningWhen?
- 34. Realtime vs. in advance
- 35. Lightweight reasoning (RDFS, OWL 2 RL)
- 36. Implementable using forward chaining rules
- 37. Still problems with scalability
- 39. MaRVIN
- 40. ‘SpeedDate’ distrubution of triples across nodes
- 41. MapReduce
- 42. Full closure of BTC in 57 minutes
- 44. And what to do with the results?Kennissystemen 2010
- 45. 2 Degrees from Kevin BaconPREFIX p: http://dbpedia.org/property/SELECT ?film1 ?actor1 ?film2 ?actor2WHERE { ?film1 p:starring <http://dbpedia.org/resource/Kevin_Bacon> . ?film1 p:starring ?actor1 . ?film2 p:starring ?actor1 . ?film2 p:starring ?actor2 .}DBPedia: 150M triplesKennissystemen 2010
- 46. Another rough idea…1 Billion triples in MySQL
- 47. Load time
- 48. … a couple of hours
- 49. Simple table lookup (one-variable query)
- 50. … about 5 minutes
- 51. Single join (two-variable query)
- 52. … a couple of hours
- 53. Better indexes?
- 54. Harddisk access times are the bottleneck (9ms)
- 55. More targeted reasoning, querying, federation.Kennissystemen 2010
- 56. SPARQLQuerying the linked data cloudKennissystemen 2010
- 58. RDF, RDFS and OWL
- 59. Multiple SyntaxesRDF/XML, Turtle (Restricted N3), NtripleFunctional Syntax, Manchester Syntax, OWL XMLRDF
- 61. Distributed
- 62. Always about something else
- 63. ... but can be about other RDF triples as well.Kennissystemen 2010
- 64. Languages: RDF(S)/XML<rdf:RDFxmlns:rdf=“http://www.w3.org/1999/02/22-rdf-syntax-ns#”xmlns:rdfs="http://www.w3.org/TR/rdf-schema/"xmlns:owl=“http://www.w3.org/2002/07/owl#”xmlns:uva=“http://www.uva.nl/rdf#”xmlns=“http://www.uva.nl/people”> <rdf:Descriptionrdf:ID=“#radboud”> <rdf:typerdf:resource=“http://www.uva.nl/rdf#AssociateProfessor”/> <uva:name>RadboudWinkels</uva:name> <uva:teachesrdf:resource=“http://www.uva.nl/courses#ks2009”/> </rdf:Description> <uva:Courserdf:about=“http://www.uva.nl/courses#ks2009”/> <rdfs:Classrdf:about=“http://www.uva.nl/rdf#AssociateProfessor”> <rdfs:subClassOfrdf:resource=“http://www.uva.nl/rdf#StaffMember”/> </rdfs:Class><owl:ObjectPropertyrdf:about=“http://www.uva.nl/rdf#teaches”> <rdfs:domainrdf:resource=“http://www.uva.nl/rdf#Professor”/> <rdfs:rangerdf:resource=“http://www.uva.nl/rdf#Course”/> </owl:ObjectProperty></rdf:RDF>Kennissystemen 2010
- 65. Languages: FSNamespace(=<http://www.uva.nl/people#>)Namespace(owl=<http://www.w3.org/2002/07/owl#>)Namespace(uva=<http://www.uva.nl/rdf#>)Namespace(courses=<http://www.uva.nl/courses#>)Declaration(Class(uva:Course))Declaration(Class(uva:StaffMember))Declaration(Class(uva:AssociateProfessor))SubClassOf(uva:AssociateProfessoruva:StaffMember)Declaration(DataProperty(uva:name))Declaration(ObjectProperty(uva:teaches))ObjectPropertyDomain(uva:teachesuva:AssociateProfessor)ObjectPropertyRange(uva2:teaches uva:Course)Declaration(Individual(courses:ks2009))Declaration(Individual(radboud))ObjectPropertyAssertion(uva:teachesradboud courses:ks2009)DataPropertyAssertion(uva:nameradboud "RadboudWinkels")Kennissystemen 2010
- 66. Languages: Turtle@prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>.@prefix rdfs:<http://www.w3.org/TR/rdf-schema/>.@prefix owl:<http://www.w3.org/2002/07/owl#>.@prefix uva:<http://www.uva.nl/rdf#>.@prefix courses:<http://www.uva.nl/courses#>.@prefix :<http://www.uva.nl/people#>.uva:AssociateProfessor a rdfs:Class;rdfs:subClassOfuva:StaffMember.uva:teaches aowl:ObjectProperty;rdfs:domainuva:AssociateProfessor;rdfs:rangeuva:Course.:radboud a uva:AssociateProfessor;uva:name ”RadboudWinkels”^^xsd:string;uva:teachescourses:ks2009.courses:ks2009 a uva:Course.Kennissystemen 2010
- 67. Turtle SyntaxSimple abbreviations:xgeo:name “A’dam” . :xgeo:areacode “020” .:xgeo:name “A’dam” ; geo:areacode “020” .Nesting:nlgeo:capital [ geo:name “A’dam” ] .Blank nodes:nlgeo:capital _:bnode1 ._:bnode1 geo:name “A’dam” .Kennissystemen 2010
- 68. QueryingOriginally there were many languagesSPARQL, nRQL, SeRQL, etc. SPARQL:SPARQL Query Language for RDFhttp://www.w3.org/TR/rdf-sparql-query/Version 1.1 in the making...Kennissystemen 2010
- 69. Do you know SQL?Formulate a query on the relational modelstudents(name, age, address)Structured Query LanguageSELECT namedata neededFROM student data sourceWHERE age > 20 data constraintKennissystemen 2010
- 70. SPARQL Query SyntaxInspired by SQL (select-from-where)select: the entities (variables) you want to returnSELECT ?cityfrom: a datasource (RDF graph)FROM <http://example.org/geo.rdf>where: the (sub)graph you want to get information fromWHERE {?city geo:areacode “010”. }Including additional constraints on objects, using operatorsWHERE {?city geo:areacode ?c . FILTER (?c > 010). }PREFIX geo: <http://example.org/geo/>SELECT ?cityFROM <http://example.org/geoData.rdf>WHERE { ?city geo:areacode ?c . FILTER (?c > 010)}Kennissystemen 2010
- 71. SPARQL Graph PatternsWHERE clause specifies graph patternpattern should be matchedpattern can match more than onceGraph pattern:an RDF graphwith some nodes/edges as variables:EuropeanCountry“020”^^xsd:integer:hasCapital?rdf:type??Kennissystemen 2010
- 72. Basis: triple patternsTriples with one/more variablesTurtle syntax?xgeo:hasCapitalgeo:Amsterdam?xgeo:hasCapital ?y?xgeo:areacode “020”^^xsd:integer?x ?p ?yAll of them match the graph:“020”^^xsd:integer:Netherlands:Amsterdamgeo:areacodegeo:hasCapitalKennissystemen 2010
- 73. Conjunctions: several patternsA pattern with several graphs, all must matchequivalent toPREFIX geo: <http://example.org/geo/>SELECT ?xFROM <http://example.org/geoData.rdf>WHERE { {?xgeo:hasCapital ?y } {?ygeo:areacode “020”^^xsd:integer } }PREFIX geo: <http://example.org/geo/>SELECT ?xFROM <http://example.org/geoData.rdf>WHERE { ?xgeo:hasCapital ?y . ?ygeo:areacode “020”^^xsd:integer . }Kennissystemen 2010
- 74. Conjunctions: several patterns (2)A pattern with several graphs, all must matchequivalent toPREFIX geo: <http://example.org/geo/>SELECT ?xFROM <http://example.org/geoData.rdf>WHERE { {?xgeo:hasCapital ?y } {?ygeo:areacode “020”^^xsd:integer } }PREFIX geo: <http://example.org/geo/>SELECT ?xFROM <http://example.org/geoData.rdf>WHERE { ?xgeo:hasCapital [ geo:areacode “020”^^xsd:integer ]. }Kennissystemen 2010
- 75. Alternatives: UNIONA pattern with several graphsAt least one should matchPREFIX geo: <http://example.org/geo/>SELECT ?cityFROM <http://example.org/geoData.rdf>WHERE { { ?city geo:name “Parijs”@nl . }UNION { ?city geo:name “Paris”@fr . }}Kennissystemen 2010
- 76. Optional GraphsRDF allows for ‘partial’ representations“Give me all people with names, and if known their email address”Use an OPTIONAL graph expressionPREFIX geo: <http://example.org/geo/>SELECT ?person ?name ?emailWHERE { ?person :name ?name . OPTIONAL { ?person :email ?email }}Kennissystemen 2010
- 77. Testing values of nodesTests in FILTER clause have to be validated for matching subgraphsFunctionsisLiteral(?aNode)isURI(?aNode)str(?aResource)for resources with partially known namesfor literals with unknown language tagPREFIX geo: <http://example.org/geo/>SELECT ?x ?nWHERE { ?x ?p ?n. FILTER ( str(?p) = “areacode”)}Kennissystemen 2010
- 78. Testing values of nodesTests in FILTER clauseComparison (<=, <, =, etc.)Arithmetic operators (+, -, etc.)String matching using regular expressionsregex(?x, “netherlands”, “i”)... matches “The Netherlands”Boolean combination of these&& (and), || (or), ! (not)(?y >10 && ?y <30) || !regex(?z, “Rott”)Kennissystemen 2010
- 79. Boolean comparisons and datatypesRDF has basic datatypes for literalsxsd:integer, xsd:float, xsd:string, xsd:dateTime etc.Datatypes can be used in value comparisons?x < “21”^^xsd:integer... and be obtained from literalsdatatype(?aLiteral)Kennissystemen 2010
- 80. Solution modifiersSorting using ORDER BYLimiting the number of results: LIMITPREFIX geo: <http://example.org/geo/>SELECT ?dog ?ageWHERE { ?dog a :Dog; ?dog :age ?age .}ORDER BY DESC(?age)PREFIX geo: <http://example.org/geo/>SELECT ?dog ?ageWHERE { ?dog a :Dog; ?dog :age ?age .}ORDER BY ?dogLIMIT 10Kennissystemen 2010
- 81. SPARQL query typesSELECT: table with variable bindingsSELECT ... WHERE { ... }CONSTRUCT: returns a graphCONSTRUCT { ... } WHERE { ... }ASK: returns yes/noASK { ... }DESCRIBE: returns a graphDESCRIBE dbpedia:AmsterdamKennissystemen 2010
- 82. SELECT Query ResultsSolutions consist of variable bindingsFor each variable in the query, it gives a value (or list)The result is a table, where each column is a variable and each row a combination of variable bindingsPREFIX geo: <http://example.org/geo/>SELECT ?x >y ?xWHERE { ?xgeo:contains ?y . OPTIONAL { ?ygeo:areacode ?z }}Kennissystemen 2010
- 83. CONSTRUCT query resultsConstruct queries return RDF statementsThe query result is either a subgraph or a transformed graph.PREFIX geo: <http://example.org/geo/>CONSTRUCT {?xgeo:hasCapital ?y . }WHERE { ?xgeo:containsCity ?y . ?ygeo:name “Amsterdam”@nl. }Kennissystemen 2010
- 84. RecapSPARQL is the query language for the web of dataQueries are sent to ‘endpoints’ on the webQueries describe graph patterns with variablesGraph patterns match the graphs in the triple storeResults are typically returned as a tableKennissystemen 2010