TSO Organograms - Moving Linked Data into production and reaping the benefits - SemTechBiz 2012

Editor's Notes

• #3: So who are TSO?TSO have been around a long time in one way or another, we are the former publishing operations of Her Majesty’s Stationery Office and were privatised in 1996 By virtue of the types of information we process on behalf of government, for example the law, parliamentary information, insolvency information and so on the publishing processes of data capture, transformation and publication needs to a sustainable for the long term. This applies whether the output is print or digital or a mix of both.RDF and web publishing are natural progressions for TSO, we’ve actually being doing semantic work for the last 5 years or so.Semantic technologies are rapidly becoming a standard part of TSO solutions.
• #4: Coming back from the specific to the general to the specific, the organogram project is really all about what is semantic discoverabilityFundamentally it’s publishing that you can forget about! That is, once it is set up and configured you are hands-off – it happens day in, day out, reliably, on time in an automated fashion.But it also includes tracking all of the data transformations so that you users and publishers can return to those processes if necessary – so provenance plays an important part- in fact my colleague represents legislation.gov.uk onthe W3C provenance Interchange Working group.Why do it?Because if people are to invest time and money in building a system on top of published data they need to have some level of guarantee that the data will be reliably available, continuously updated and up-to-date and that the system will be around for long enough to ensure that their own system’s viability is maintained.The benefits include:centralized storage, better quality, standardized machine-readable, sharing and reusability, low cost and quicker.
• #6: Pulling all of these Open Up® platform elements together is the Government Organogram project went live in June 2011. What makes it exciting is that it is the first example of every government department publishing RDF.The organograms utilise the Harvester, RDF Store, SPARQL endpoint, Linked Data API and hosting facilities. The harvesting strategy was changed in Sep. 2011, we now accept data upload from departments.
• #7: capturing data in a simple Excel template converting data to CSV and RDF using PHP and XLWrappreviewing and publishing data through a linked data API and custom organogram visualisation supporting distributed publication by owner organisations “We found detailed guidance, early validation, vocabulary reuse and visualisation are key to data quality, and that Linked Data can support the demands of both re-users and end users. The resulting reference data will be linked to from other government datasets.”We replaced the XL Wrap with CSV2TTL, a python-based implementation of CSV to RDF. The new software supports the publishing RDF triples from CSV in a more efficient and reliable way.The data owners are required to fill up the spreadsheet following the given template. This is where the early validation taking place. The data owners upload the spreadsheet to the preview server, where previewing and signing-off happens.
• #9: Web Content Management system unable to handle RDF, so original plan publish on website register on data.gov.uk, and harvester would collect data – not possible.Data validation, and individuals introducing errors and using the system differently.Focus had been on getting the information out rather than figuring how to display the changes over time.
• #11: Government is a huge organisation, which means that there were inevitable inconsistencies that crept in across departments –breaking initial assumptions.
• #13: Excel spreadsheet improvementsPublic sector and government uses Government Secure Intranet (GSI) as a platform for its IT systems.Would have been ideal to use signed macros – but with 200 sets of users at launch (soon rising to 400) - testing on all systems and supporting that number of users was not practicalThe aim is to make the process intuitive for users.
• #15: Our approach to creating a semantic discoverability solution is to create a suite of infrastructural services on which we or others can develop applications. This has been branded the OpenUp® platform and it has developed over the past 18 months.The four major elements of OpenUp® services are:Harvesting: The ability to get hold of content in a reliable, regular wayEnrichment: The ability to extract additional information from your data using information extraction techniquesStorage: Holding the data Publishing: Making that data available on the web as well as supporting websites for that data if requiredThe platform is scalable at each level and runs on hardware in our data centres in Norwich and London. (Our hosting is used in particular for government websites as we appropriate security certifications)