Google BigQuery for analysis of scientific datasets: Interactive exploration and analysis of the data using KNIME Analytics Platform
1 like905 views
This document discusses using Google BigQuery for analyzing scientific datasets interactively with KNIME Analytics Platform. It acknowledges contributions from others and notes that BigQuery allows querying of giant tables with SQL. The author is enthusiastic about exploring data when questions are still being formulated rather than just searching. The workflow demonstrated will explore scientific data in BigQuery using KNIME, starting with initial database queries to pick a disease and examine compound classes.
Google BigQuery for analysis of scientific datasets: Interactive exploration and analysis of the data using KNIME Analytics Platform
- 1. © 2019 KNIME AG. All Rights Reserved. Google BigQuery for analysis of scientific datasets: Interactive exploration and analysis of the data using KNIME Analytics Platform Greg Landrum Martyna Pawletta Jeanette Prinz greg.landrum@knime.com @dr_greg_landrum
- 2. © 2019 KNIME AG. All Rights Reserved. 2 Acknowledgements • Steve Boyer (Collabra) • Lutz Weber (OntoChem) • Ian Wetherbee (Google)
- 3. © 2019 KNIME AG. All Rights Reserved. 3 Google BigQuery? • A giant collection of tables that I can query with SQL • If the tables share common keys, I can do interesting things Might be an oversimplification. ☺
- 4. © 2019 KNIME AG. All Rights Reserved. 4 An aside: searching vs exploring a.k.a. why I’m enthusiastic about this project
- 5. © 2019 KNIME AG. All Rights Reserved. 5 An aside: searching vs exploring
- 6. © 2019 KNIME AG. All Rights Reserved. 6 An aside: searching vs exploring
- 7. © 2019 KNIME AG. All Rights Reserved. 7 An aside: searching vs exploring • There are definitely arguments for specialized interfaces that are tailored to make answering a particular question super efficient and easy • But! There are times when I’m still trying to figure out exactly what the question is • For this it’s nice to have a giant pile of data and a general purpose tool for exploring it
- 8. © 2019 KNIME AG. All Rights Reserved. 8 What we’re going to do here • Do some exploration of the scientific data that’s now in BigQuery… • … with KNIME
- 9. © 2019 KNIME AG. All Rights Reserved. 9 Workflow part 1
- 10. © 2019 KNIME AG. All Rights Reserved. 10 Workflow part 2
- 11. © 2019 KNIME AG. All Rights Reserved. 11 The first database queries
- 12. © 2019 KNIME AG. All Rights Reserved. 12 Picking the disease/condition
- 13. © 2019 KNIME AG. All Rights Reserved. 13 Results
- 14. © 2019 KNIME AG. All Rights Reserved. 14 Compound classes
- 15. © 2019 KNIME AG. All Rights Reserved. 15
- 16. © 2019 KNIME AG. All Rights Reserved. 16