Graph databases and the Panama Papers - Stefan Armbruster - Codemotion Milan 2016
1 like872 views
The document outlines the operations of a data and research unit comprising over 190 journalists across more than 65 countries, detailing their data processes and server infrastructure. It discusses methods for handling and importing vast amounts of data, including querying techniques and CSV file management. The focus is on leveraging graph database queries and APIs to streamline data integration and analysis.
1 of 61
Download to read offline
![MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total](https://image.slidesharecdn.com/cucszzqtoko5fpmpsxvq-signature-824e7d4f412a31d249d066a2073101362aa82ef866dac5866786669e3f9999ae-poli-161129142654/85/Graph-databases-and-the-Panama-Papers-Stefan-Armbruster-Codemotion-Milan-2016-61-320.jpg)
Graph databases and the Panama Papers - Stefan Armbruster - Codemotion Milan 2016
- 7. +190 journalists in more than 65 countries 12 staff members (USA, Costa Rica, Venezuela, Germany, France, Spain) 50% of the team = Data & Research Unit
- 8. raw files metadata author; sender... database search and discovery raw text
- 9. 3 million files x 10 seconds per file = 1 yr / 35 servers = 1.5 weeks
- 10. Lucene syntax queries with proximity matching! 400 users
- 11. ● ● ● ● ● ● ● ●
- 17. • • • • •
- 19. • • • •
- 20. • • • • •
- 27. • • LOAD CSV WITH HEADERS FROM "url" AS row MERGE (:Person {name:row.name, age:toInt(row.age)});
- 28. • • • CALL apoc.load.json("url") yield value as doc UNWIND doc.items as item MERGE (:Contract {title:item.title, amount:toFloat(item.amount)});
- 29. • • • bin/neo4j-import –-into people.db --nodes:Person people.csv --nodes:Company companies.csv --relationship:STAKEHOLDER stakeholders.csv
- 31. • • • • •
- 32. •
- 37. • • •
- 40. • •
- 41. • • • • •
- 42. • • •
- 43. • • •
- 54. • • • • •
- 55. • • • • • • • •
- 56. • • • • …
- 57. • • • • …
- 58. • • • • •
- 61. MATCH (boss)-[:MANAGES*0..3]->(sub), (sub)-[:MANAGES*1..3]->(report) WHERE boss.name = “John Doe” RETURN sub.name AS Subordinate, count(report) AS Total