Operationalize Your Linked Data

Editor's Notes

• #2: Hi everyone, I’m Matt Turner, CTO for Media & Manufacturing. That means I take care of all our customers that make the great things you all know and love And I’ve very glad to be here today to talk about the impact of Linked Data and in particular how investing in this data can help transform your industry.
• #3: I’m going to start with something that seems obvious … the importance of data But, before I get into this, I do want to remind everyone that going back even 5 years, this was not an obvious topic. We were talking about apps and especially the mobile experience and the new wave of BI … but not about the data itself as a topic
• #4: But there were people having the conversation in their industry We do a lot of work with publishers and one of the primary voices for change has been Dr. Sven Fund – then the CEO of DeGruyter a publisher over in Germany. He wrote what I think of as a battle plan for the modern publisher called integrating publishing. It’s a data driven approach to rethinking every part of the business around using data across every part of the business. From planning what content to invest in, to creating it and tailoring it to knowing how it impacts your customers, data can and should play a role … and Sven laid out the plan to get publishers to that point. This was quite a change for an industry still just thinking about content. Shelly Palmer is another voice that was early with a message about data. He worked mostly within Media but his message was to every organization highlighting how the game has changed. He says “Data Rich or Data Poor” that is the ONLY game. Every company is now competing on the battleground of data. Its not your revenue, your number of customers or their engagement. It’s the data you gather that actually matters. What’s more, you aren’t competing against what you think of as your competitors. Its Google, Apple, Facebook … and way above all of them Amazon. Shelly says this to bring people’s attention to the importance of data. And he’s not alone – he is joined by my colleague Michel de Ru. Michel works across a number of industries and at the MarkLogic 360 event last year he issued a call to arms: Industrialize your data! You invest in your processes, your machinery, your people and take care of your capital. And you need to do the same thing your data. Think about how you manage it and, just like your machinery and other assets, industrialize how you deal with it
• #5: And they aren’t alone. Who has heard this phrase Data is the new Oil? Its everywhere … there is even someone saying it’s the not the new oil it the new nuclear. I guess because it keeps delivering value forever? In fact there is so much about this, if you search for Data is the new oil infographic you get 13 million hits! This is my favorite – see the data in the ground – just pump it out and – presto – you get your value! Right? Its that easy, right?
• #6: And they aren’t alone. Who has heard this phrase Data is the new Oil? Its everywhere … there is even someone saying it’s the not the new oil it the new nuclear. I guess because it keeps delivering value forever? In fact there is so much about this, if you search for Data is the new oil infographic you get 13 million hits! This is my favorite – see the data in the ground – just pump it out and – presto – you get your value! Right? Its that easy, right?
• #7: And on this topic, we are just starting to hear from the experts. I hope Alan Morrison as one of these visionaries. He gave a keynote at the Semantic conference in August that was a real call to arms for everyone in THIS room to evangelize that you do need more than just data. He was specifically talking about the vast gap between the vision of a unified IT stack and being able to leverage AI and the reality of the many silos of applications. He specifically is looking at who is out there paying attention to this problem and he put up this slide – the top 10 companies in the world And of them, fully 9 are doing more than just collecting data. They are investing in Linked Data – creating knowledge graphs and connecting their data to realize its value He isn’t alone – Kurt Cagle makes a bold statement about the rise of Ontology will be a critical business advantage. And then there is this paper about the state of AI. Alan also goes into this in his talk – AI without the meaning and connections in the data is just going to fall short. Specifically they make this statement – that nearly every problem comes down to graphs of relationships among entities!
• #8: And on this topic, we are just starting to hear from the experts. I hope Alan Morrison as one of these visionaries. He gave a keynote at the Semantic conference in August that was a real call to arms for everyone in THIS room to evangelize that you do need more than just data. He was specifically talking about the vast gap between the vision of a unified IT stack and being able to leverage AI and the reality of the many silos of applications. He specifically is looking at who is out there paying attention to this problem and he put up this slide – the top 10 companies in the world And of them, fully 9 are doing more than just collecting data. They are investing in Linked Data – creating knowledge graphs and connecting their data to realize its value He isn’t alone – Kurt Cagle makes a bold statement about the rise of Ontology will be a critical business advantage. And then there is this paper about the state of AI. Alan also goes into this in his talk – AI without the meaning and connections in the data is just going to fall short. Specifically they make this statement – that nearly every problem comes down to graphs of relationships among entities!
• #9: So what’s going on here? Well to get to the heart of this, I want to ask you ONE question. Get ready because I’m going to ask you to raise your hand and take this very seriously
• #10: Who’s smarter, a dog or a chimpanzee? --- OK – everybody has to vote -- And it was a trick. Of course chimpanzees are much smarter. They can drive cars, talk in sign language … they are way way smarter .. But … there is something a dog can do that a chimp just can’t do
• #11: And that is understand context. If you hide a treat and then point to it, the chimp will totally ignore you and randomly guess. No matter what you do, the chimp just doesn’t care what you think. But if you do the same thing with the dog, the dog will look at you to understand what is going on. In fact they will look at the right side of your face which is how we understand emotions and act on what they see. This means you can point to the treat and they will go right to it. Puppies will do it. Dogs are so good at this, you can just move your eyes. So what does this all mean for data?
• #12: Well, machines don’t get context! They are the chimps of the technology world they can tell you it’s a link or a picture on that page … but they can’t tell you that they fit together. Or what its all about!
• #13: When you take this to the world of data, and in particular the data layer that can run your business this is what you get – traditional data structures that just fall short You have to define everything up front – all your data and everything your organization does … And then categorize it. In no way will this work – you will end up stripping off context sometimes in layer. You can’t share this data across your organization and so you get what Alan was talking about in terms of the multiple layers of appliations and data One of our customers talks about the result of all this changing of data as operating on opinion, not data!
• #14: We at MarkLogic started to address this with a document data model. This foundation of NoSQL means that you can much more flexibly store the data. You didn’t have to define it all up front and now you could adapt as things changed. But it was still cumbersome to keep track of meaning and context … more elements could be very ticky
• #15: Enter Linked Data – this concept of describing the linkages in data – and specifically of using triples or RDF as an additional data mode, is a way to bring, with data, what the machine is lacking You can now actually describe the concepts around the data. And what is more important – you can also describe the source, the provenance and even the usage of the data.
• #16: Combined, these two data models are key to creating that data layer and enabling you to actually make data the foundation of your business This is critical – because there is a balance here. In the world of semantics and linked data, there are huge gains to be made in creating ontologies that match the real world and then linking data to those ontologies But there is also a role for just a document – things that belong just to the document like dates, titles and of course the actual –these are perfectly OK in the world of XML. And also in this model the connecting triple is part of the document – also making it a graph that enables you to have data integrity.
• #17: At MarkLogic we’ve been using this combined model in an architectural pattern called an Operational Data Hub. This pattern details (and I’m not going to go throught it all) taking in data, and then with many different approaches, curating that data and creating the context around it. You end up with documents and triples that can provide that single view of the data that, as Alan and others talk about, can be the foundation for a data driven organization
• #18: Lets take a look at how these patterns are letting organizations leverage their data
• #19: Lets start in the complex regulatory compliance sector of financial services ABN Amro wanted to take a new approach to compliance. Instead of building data sets for each issue and process they wanted to create an platform for regulatory compliance so they could respond to future needs This required them to think about the problem differently – create data that then be used in what they called multiple compliance schemas. They also realized this data would be a powerful asset across the compay. They did this first to meet the tradestore regularion MiFID II and then created another hub for GDPR This projects all used semantics to describe the entities and their relationships in the bank But really be able to use the data, you need to understand the provenance of the data
• #20: To do this they did map all the entities But the also added Prov-O to the picture to record that data lineage. Now they know not just about customers and their interactions inside the bank and on their own They also know where that data came from
• #21: This lets them see a much richer set of data – tieing together internal and external systems With this data they can now delvier the much more complete view of the customer required by GDPR, describe the entities correctly AND know the source and all the details about that data Keep in mind this also follows the document and semantics route – so that profile is also in the system enabling them to see the details of the customer
• #22: This platform delivers that vision of universal enterprise data – or the semantic data layer as Alan talks about it. Focused on compliance, it is an engine for handling compliance data rather than just a single solution But it is now also a view of data across the organization that is a considerable asset to the bank
• #23: In addition to improving your own organization, you can also use this approach to improve the customer experience. Mitchell1 is in the car repair business. They provide the tools to garages to help the shops and the mechanics make the repair experience better.
• #24: They stared with the acutal information on how to make the repair. This information went from paper to digital and is exploding in complexity as cars get more complex. But they also provide the systems that run the shop – scheduling, diagnostics, parts ordering, the actual repairs and billing. One cool thing is that they also get tips from the mechanics about how to fix the car To put this data into action, they created a data hub that linked all this data And the link is this – every part on every car that is sold going back 30 years or more! They invested in ontology development to create this foundational data – how parts, compnents and systems fit together And then spend 2+ years creating the data set and linking the data.
• #25: Using this, they are trying to make the mechanics job better – for instance knowing what is wrong almost as soon as they see the car. This is a composite screen that shows the parts that you will probably have to order from a set of error codes. This saves everyone time getting to answers And as cars get connected, this is starting to happen outside of the shop - cars sending codes and getting ready for repair and maybe even making appointments You can only do this if all the data is connected
• #26: And they are also giving information about the long term – for instance what is likely to break on my car. This used to be in the heads of experts … but now it done with data – Don’t forget this is your car, with your exact specification
• #27: This data hub lets Mitchell1 make car repair a much better experience
• #28: And finally lets talk about using data to make a difference Sensing clues is an organization here in the Netherlands dedicated to conversation and specifically protecting endangered species They are focused on the interactions between people and these animals This is both when people go into their environment -> and this is often very bad for instance poachers But also protecting the animals when they go into human environments
• #29: To help with this mission they have collected a lot of data. Incident reports, field notes, signal data and even pictures and videos And they have integrated all this data so they can look at an area. Part of the project is to get information to the right people so this lets them send updates and warnings to rangers and other workers when they are actually out in the field – or actually going out since there isn’t a lot of connectivity This also allows them to undersnad what is happening. And to do this they created a semantic layer that created concepts and linkd the data
• #30: This helps them gain insights into what is happening – this is a zoom into the data for a specific area And it shows the different types of interactions – the professional incidents, likely poachers, and then where arrows are used which is a very different case They can then go back to the map and be prepared for what the situation is and also use this information for different programs to prevent both types of incidents
• #31: And this also helps them understand other incidents. For instance why were warthogs showing up? Does this mean there are poachers? Well actually because of the context of the data, they can tell that these are not usually poachers – they are people coming in to the area and camping out for 6-7 days and making charcoal. Not as high a priority and, some good news in the data
• #32: We volunterred our efforts and as they continue to develop ther data layer to make an impact on animals I’m sure they could use more help
• #33: One more factor in creating this type of semantic data layer is that in addition to using the data for your internal uses, you can share it. This is what Springer does with their rich database of scientific information … they have the actual products SprinterLink and then they just take the UI off and offer APIs for text mining and data access.
• #34: So lets take a look back – you can impact your organization, help your customers and even help the world if you have your data together So lets go back to that infographic. Maybe it is actually just so easy – if you add a few things. First consider the context of the data where you find it. And capture all of that Then lets think about the usage and the different contexts everyone will need when the access it! Using this as a guide, lets think about putting a data hub right here in the middle. This won’t be the only source of data … and even for a single ‘well’ you need a place to collect the data and make it universally accessible. I think if you do this – if you apply the principles of linked data to your operational data … well maybe you can bring some of those doggie smarts to your organization. Thank you!