Applied semantic technology and linked data

Editor's Notes

• #2: Hello, My name is william smith and today we will be talking about a project near and dear to my heart.I served as project manager for a prototype application, worked closely with 2 German teams, and we were the first customer for several of the tools used to assemble this application. I was also the chief integration point into Vulcan so am well aware of the technologies, code bases, and data sets that went into assembling this project…
• #3: So what are we discussing today?First and foremost this was a project for an internal organization at Vulcan involved in mapping the human brain. This, of course, generates petabytes of data and millions of triples worth of gene mappings – but we took a smaller slice of a couple hundred thousand genes for the initial prototype. There were also several parallel research programs generating data in a format we could use, and a conference was held of industry professionals to find the interlinking pieces of these datasets. Finally, I’m going to walk through the data pipeline, the application itself, and a set of our original use cases.
• #4: Why?Well a core problem that has been in neurobiology, and most sciences for that matter, is the inability to share and author sets of data across projects by industry professionals. This leaves an odd gap where people with computer science degrees are linking data they don’t fully understand, and the people that understand the data don’t have the ability to add the interlinks for greater vision into the data.With this problem known our original prototype soon expanded into how do we get these tools into the hands of the research community, and that in itself created 3 core questions. Ownership, Authorship, and publishing provenance of the newly linked data.
• #5: The organization that chartered this project, and provided the original data sets is the Allen Institute for Brain Science – or AIBS. When you hear me say AIBS on accident I’m referring to this organization. It was launched in 2003 by Paul G. Allen and has the explicit focus of mapping the human brain to accelerate our understand of the brain and neurological systems. Furthermore, the institute is a 501c(3) nonprofit medical research organization employing hundreds of neuroscientists, molecular biologists, informaticists, and engineers within the seattle area.----- Meeting Notes (1/28/14 12:15) -----So who requested this?accelerate our understanding
• #6: And this is the Institutes core product… or several screen shots of the core product. Here we have gene heat maps… some location data… where it all is location wise in the human brain. As odd as those screen caps are they are accessed by thousands of researchers daily and this is considered a major success.It’s open, the public right now can go to this site and browse the catalog. There are currently 3 human brains fully mapped with a 4th in progress. Each of these donors have generated genomic analysis of brain structure and have created a thorough catalog of genes with respect to location. While the captions are small they are part of a much larger suite of atlas navigation tools with several components – ie. Heat map – pinpointing genes expressed down to the cellular level.And most importantly, for our purposes, they generate terabytes of data with industry wide IDS we can link to other sources!
• #7: And here’s our prototype in screenshots. No page is hand type, no graph is hand entered, 4 static templates pulling data from our normalized mine creating all these pretty pictures and full pages of text. There are over 30 thousand of these pages.We will be discussing the first two points in depth – RDF and the LDIF pipeline. Charting tools use SPARQL which we will not be discussing in depth – however I have a hidden slide of the details should somebody be really malicious and want to ask about SPARQL queries. Finally, our navigation closely resembles the common MediaWiki installation which everybody who has been on the internet in the last 10 years is familiar with… editing on the other hand is very different and currently only bots create and maintain the pages.
• #8: Which brings us to these parallel tracks of research data I keep mentioning.To choose these sets we had a conference of industry researchers and data professionals go through the hundreds of biology mines looking for useful projects that closely relate to genes found in the human brain. The 4 prototype sets chosen were <read slides>
• #9: Our original cross section of data found these connections. Not the full dump, but with roughly 15 thousand gene connections plenty of pages produced relevant connections and filled pages with interesting data points. <read numbers>And to the right we have our simplified ontology. Looks incredible right… hey they can’t all be winners and don’t blame me – blame protégé.This was generated with basic 1-1 relations, domain-range logic, where applicable. <joke about line colors>The simplification was created in part because nobody that does anything in neuroscience agrees with another person that does the same thing. We could get them agree that in some gray area way these things are related on the domain-range level … so that generates that and it looks way worst if I try to spread the boxes out in any other way.
• #10: Which brings us to the pretty graph I hate… because it makes unifying things into that ugly protégé graph look easy.It’s not, but it does give a good overall view of what we able to convert directly to the wiki be 32 thousand 900 instances turned directly into pages with over 500 thousand properties across the set. Even more important after “same as” connections were made we had 20 thousand fully populates pages – and these are the pages with connections across the datasets. That brings up an important point, if I imported all of the gene data I would end up with a huge wiki by page count, but the better part of these pages would be nothing more than a page title and empty templates. Hence the importance of finding these connections and only tracking the useful data points – like pages with more than a title.On the right we have the simplified process which I will be going into more detail very soon <read right graph>----- Meeting Notes (1/28/14 12:15) -----But it does give a good overall view of what we are able to convert directly into the wiki.
• #11: And those parts that just turned red - <read red parts> - is the process we will be discussing for a section I like to call: Linked Data Integration Framework
• #12: \----- Meeting Notes (1/28/14 13:28) -----Created over the last 4 yearsCreated by Free University of BerlinSame team that helped build the prototypeFirst customerStill active, last update late 20132 main components, R2R and SILK
• #13: And this is why I don’t like the oversimplification of that process chart. Plenty of difficult computer science problems and none of them cut and dry to solve…Assuming we can find overlapping data sources you then have to unify vocabularies – the predicate of the triple. Once this is done and you can agree on what the name of the entity is, then you will have data sets with the same entity going by a range of names and ids. Finally, once you’ve located the same entities there’s no guarantee the normalized vocabularies will be referencing the same value.Without the normalization pipeline – LDIF - this creates queries that are silo’d to a specific data set basically creating an API… and that’s good for companies like facebook and Google but terrible for independent research. The last point is less of a problem for us because we decided long ago this was a philanthropic prototype with 501-c-3 data – but it is something to be considered when working with say – national security data.
• #14: Lucky for us, as customer 1 of the LDIF framework, we get to test all of the steps in normalization and hope for the best or fix it ourselves!If this works right we will…<read steps>
• #20: And here’s the LDIF architecture.All this stuff on the bottom are the 5 data sets, the arrows don’t really apply because they didn’t link up that well before LDIF, and then to the pipeline.After processing and RE-releasing the arrows apply, and then we shove that all in our own public triple store for use in the application.
• #21: And here’s your application.
• #22: ----- Meeting Notes (1/28/14 13:28) -----Pubby created 5 years agoUsed in dbpediaFree univeristy of berlinNo search, have to follow linksNot very modern viewing experienceNo expression of data via links
• #23: Less than helpful – FINE.
• #24: Well I am in this business to please the consumer, and my consumer understands common web architectures – even if they don’t know they do - so let’s try an installation of Semantic MediaWikiInvented roughly 5 years ago it’s a series of plugins, that run on mediawiki, which was created by the good folks that invented wikipedia! Millions of people see it everyday while researching homework they don’t feel like doing, when sloppily referencing college term papers, or in my opinion creating one of the most accurate and comprehensive encyclopedias humanity has to date. Even better we can display the semantic properties of our normalized data inline! <show arrows><can you expand> of course I can.
• #25: I’m going to build you 4 base templates by category – Gene, Drug, Disease, and Side Effect.These templates will have the base information displaying our semantic properties - <run through wireframe>
• #26: ----- Meeting Notes (1/28/14 12:15) -----This created a problem - namely how do I create 30,000 pages and not get fired for entering data over the course of 2 years. So, a lot of what you see on wikipedia isn' t actually input or maintained by humans. The gene pages all have very complex info boxes tracking ids, regions, and a variety of known properties mined from other sources. The pieces of code that do this mining and page creation are called wiki-bots.We wrote a wiki-bot to create our 30,000 pages, one for each page type, and this is the creation pipeline these bots utilized.
• #30: ----- Meeting Notes (1/28/14 12:15) -----I'll be running through 3 core use cases we used to test the project and explaining how the pages and graphs were generated. All of the graphs related to the genes, diseases, drugs, and side effects within the next few slides are generated from the wiki.However, it's far easier to view the wiki when you have access behind the vulcan firewall... so I had to run on screen shots for this portion.
• #31: ----- Meeting Notes (1/28/14 12:15) -----Calcium - difficult use case- within all creatures- has lots of connections to other entities- but we don't want to create all the pages
• #33: ----- Meeting Notes (1/28/14 13:28) ------ 15 minutes of fame 5 years ago- Powerful seditive used in anesthesiology-- You should not use it as a sleep aid- Listed as cause of death for popular musician
• #40: Fix this
• #44: ----- Meeting Notes (1/28/14 12:15) -----Finally, we head over to drug bank and search for an obscure drug page... Bicalutamide...It's an oral steroid used in the treatment of cancer that effects the androgen receptor. Thus validating our links across the data. An example of how a not-so-simple correlation of data can give researchers deeper vision by merging sets and presenting the interlinks.
• #46: ----- Meeting Notes (1/28/14 12:15) -----Aura wiki - it was used to test crowd sourcing of data authoring for a proto-AI.