SC7 Hangout 2: Technical update on the pilot development for secure societies
1 like357 views
The document outlines the pilot development of workflows for analyzing big data in secure societies, focusing on satellite image processing and event detection. It describes various workflows, including image aggregation, change detection, and query handling through scalable systems like Strabon and Semantic Growth. Additionally, it provides an overview of user interface components for interacting with these services through applications like Sextant.
SC7 Hangout 2: Technical update on the pilot development for secure societies
- 1. TECHNICAL UPDATE ON THE PILOT DEVELOPMENT FOR SECURE SOCIETIES BDE Hang-out “Updates on Big Data in Secure societies”21/04/2016 George Papadakis, University of Athens Postdoctoral Researcher
- 3. 1st Workflow: Image Aggregator Rest service called by GUI & Event Detection Input: o Bounding box of the area of interest (WKT) o The time of interest o A past time, before an event of interest took place Output: o a set of satellite images downloaded from ESA’s Sentinels Scientific Data Hub. 23-mai-16www.big-data-europe.eu
- 4. 1st Workflow: HDFS Input: o Two satellite images in zip format, each occupying few GBs. Output: o Distribute parts of every image to the available cluster nodes to facilitate their efficient processing. 23-mai-16www.big-data-europe.eu
- 5. 1st Workflow: Change Detector Parallelizes the change detection algorithm using Spark. Input: o Two satellite images depicting the same geolocation. Output: o A set of bounding boxes for the areas with differences between the two snapshots. 23-mai-16www.big-data-europe.eu
- 6. Change Detector Algorithm Two steps: o Preprocessing to co-register the images 4 operators o Main algorithm to perform the actual comparison Two parallelization strategies: o Image-centric approach o Tile-centric approach 23-mai-16www.big-data-europe.eu
- 7. Common workflow: GeoTriples Converts geospatial data into RDF. Input Scenario I: o Areas of change from Change Detector Input Scenario II: o Event summaries from Event Detector Output: o RDF statements 23-mai-16www.big-data-europe.eu
- 8. Common workflow: Strabon Scalable & efficient spatiotemporal RDF store. Input: o Data coming from GeoTriples o Queries of the following types: Get N latest event summaries from location X. Get event summaries with keyword Y. Get areas with change in location X. Output: o Answers to the received queries. 23-mai-16www.big-data-europe.eu
- 9. Common Workflow: SemaGrow Federates Cassandra and Strabon. Input: o Queries from GUI about events or locations with changes. Output: o Answers to the received queries. 23-mai-16www.big-data-europe.eu
- 10. Common Workflow: Sextant - A Web application running on the client-side. Input for Change Detection: o Area selected by user through the interactive map or Location name typed by user o Time interval (optional) Output: o Calls Image Aggregator o Progress bar 23-mai-16www.big-data-europe.eu
- 11. Common Workflow: Sextant - B Input for Event Detection: o Keyword (optional) o Location name or coordinates o Time Output: o Latest relevant event summaries. By clicking the title of an event, the user is redirected to a page with the full event description. 23-mai-16www.big-data-europe.eu
- 12. Thank you! Questions? Links Strabon: http://strabon.di.uoa.gr GeoTriples: https://github.com/LinkedEOData/GeoTriples Sextant: http://sextant.di.uoa.gr 23-mai-16www.big-data-europe.eu