How Linked Data Can Speed Information Discovery

Editor's Notes

• #4: Whether you’re from the business or from the business intelligence side of your company, you’re like familiar with both of these scenarios. On the first scenario, the business wants access to new data, or data that’s available but they don’t have rights to. Because it’s new, they may or may not know everything that’s there, or how to access the particular data they require so their BI team has to do a lot of analysis before they can even commit to providing data in a format that may or may not work. In the second scenario, the business knows exactly what they want but because of all the work on one central team it may take a while to get to it and by then the requirements have changed or the work is no longer needed. Both scenarios have the same problem – it takes time to process data through a data warehouse and into a format that may work for the businesses’ needs. The other question to have here is that is the data warehouse fit for what the business is trying to do? Why does it take so long to get data from source to warehouse frontend tools?
• #5: This is the typical process used to take data from source to mart. There may be more or less steps involved in your business, but generally speaking there’s a staging process that feeds the data in to be piped into the data warehouse, which is then cleansed/conformed into specialized data marts. All of this processing is typically done using data integration/ETL. If it’s a new data source, regardless of the type of architecture for your warehouse, you’re looking at weeks to months of turn around time. If it’s modification on an existing source, turn around time goes down to days to weeks. It’s time consuming because Business Intelligence has to try to analyze the requirements of any given project and ensure that it meets the overall business requirements. In addition, each ETL step has to process, clean and prune data accordingly.
• #6: So here’s the real question. What if I told you there’s another way to do data exploration? One that can have faster initial turn around time, provide a way for business areas to agree and disagree on data interpretation, and provide better business requirements for your business intelligence team? That’s what we’re here to talk about today – Linked Data!
• #7: The concept of linked data has been around for a while, with Tim Berners-Lee coining the term back in 2006. Unlike with traditional relational databases, Linked Data explicitly describes the relationships and metadata around the data stored in ‘vocabularies’. These vocabularies can be combined where they overlap to provide richer data that the source systems alone. This data is stored in what is known as a ‘triple’ format. We’ll get to both this as well as what vocabularies look like in a few slides.
• #8: Let’s start by diving into what makes up a vocabulary. There are three primary components to any vocabulary: the class hierarchy, how the classes relate to each other, and the properties of said classes. For example, here’s an open source vocabulary for beer. Classes make up the objects that are being described in any given data set. Properties of a parent are inherited by the children. In this case, if we look at BeerType, we can see that there are number of properties that are associated with all beers. Also note that there’s things listed that we don’t explicitly call out in this vocabulary. For instance, the rdfs:label and xsd:float. Those are examples of things defined in other vocabularies that we are applying in this beer vocabulary that helps provide validation and rules to the values. Also note that each object is given a unique URL (IRI). This ensures that every object in the vocabulary is unique. Vocabulary source: http://webprotege.stanford.edu/#Edit:projectId=2c3e9b0f-b7ce-49ca-a621-acb66883607a
• #9: Vocabularies also have properties that provide descriptions of the classes. Properties can be attached to any classes/relationships (domain and range). Just like with relational databases, properties can be given types that ensure the format is correct.
• #10: Material for this example from: http://www.thevergemagazine.org/career-qa-beer-brewer/
• #11: Here’s that same data (plus some more) in RDF/XML format.
• #12: Now back to our architecture from earlier. We aren’t replacing the data warehouse with Linked Data – far from it! There are a few options for implementation, with the first one being virtualization. In this case, there is a mapping between the source systems and the linked data layer that will translate linked data queries (typically written in SPARQL) into SQL or other formats to pull data from the sources as if they are physically stored in a triple store. The benefit here is that there is fast turn around and your reporting and analysis tools should be able to run SPARQL queries. The downside is that these source systems may get overwhelmed with the types of queries being executed due to the translation/interpretation of the virtualization platform.
• #14: Our other alternative, which requires a bit more engineering, is to use data integration (ETL, etc.) to morph the source systems into a triple store. The benefit here is that the data is being stored in literal triples, which minimizes impact on the source systems and provides an independent platform for data discovery. The con of course is that there needs to be a bit of work gets sources into the linked data store, but it’s fewer steps that would be required with the data warehouse/specialized data marts. Of the two options, this one is more robust for long term data discovery and proof of concepts.