Real Time Semantic Analysis of Streaming Sensor Data
Download as PPTX, PDF2 likes566 views
The document discusses the evolution of web data through various phases, highlighting the transition from static files in the 1990s to dynamic user-generated content in the 2010s. It emphasizes the challenges posed by massive volumes and rapid information generation from real-time sensors and social media, proposing methods for summarizing this data effectively. The work presented at the 48th ACM Southeast Conference outlines the architecture and application of linked sensor data and feature streams to better analyze weather events and enhance data usability.
Real Time Semantic Analysis of Streaming Sensor Data
- 2. xWEB DATA evolved over timeReal-Time Sensor, Social, Multi-media data2010’sDynamic User Generated Content2000’sStatic Document and files 1990’s2
- 3. xProperties of Streaming DataHuge VolumeRapidContinuousInformation Overload!!Heterogeneous3
- 4. xSome Statistics“A cross-country flight from New York to Los Angeles on a Boeing 737 plane generates a massive 240 terabytes of data”- GigaOmni Media“Sensors Networks will produce 10-20 times the amount of generated by social media in the next few years” - GigaOmni Media“More data has been created in the last three years than in all the past 40,000 years”- TeradataSolution - “Meaningfully summarize this data”4
- 5. 48th ACM Southeast Conference. ACMSE 2010. Oxford, Mississippi. April 15-17, 2010.From Sensor Streams to Feature Streams in Real TimeHarshalPatniOhio Center of Excellence in Knowledge enabled Computing (Kno.e.sis) Wright State University, Dayton, OHPart of Semantic Sensor Web @ Kno.e.sis
- 6. xOutline Introduction Architecture Linked Sensor Data Feature Streams Demonstration6
- 8. xExplaining the titleBackground KnowledgeBlizzardRain StormABSTRACTIONHuge amount of Raw Sensor DataFeatures representing Real-World events8
- 9. xTypes of AbstractionsSummarization across Thematic DimensionSummarization over the Temporal Dimension9
- 10. xTypes of AbstractionsSummarization across Thematic DimensionSelectJoinBackground KnowledgeAnalyzeFeatures representing Real-World Events10
- 11. xAn example problem?11“Find the sequence of weather events observed near Dayton James Cox Airport between Jan 13th and Jan 18th?”SpatialThematicTemporalTechnologies required - Linked Sensor DataFeature Streams
- 12. xOutlineIntroductionArchitectureLinked Sensor DataFeature Streams Demonstration12
- 14. xOutlineIntroduction ArchitectureLinked Sensor DataFeature Streams Demonstration14
- 15. 48th ACM Southeast Conference. ACMSE 2010. Oxford, Mississippi. April 15-17, 2010.Technology1: Linked Sensor DataFind the sensor around Dayton James Cox Airport?Extract Data for the sensor near Dayton James Cox Airport?Harshal Patni, Cory Henson, Amit Sheth, 'Linked Sensor Data,' In: Proceedings of 2010 International Symposium on Collaborative Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010.
- 16. Sensor Discovery Application Weather Station IDCurrent Observations from MesoWestWeather Station CoordinatesWeather Station PhenomenaMesoWest – Project under Department of Meteorology, University of UTAHGeoNames – Geographic dataset16
- 17. What is Linked Sensor DataWeather SensorsSensor DatasetGPS SensorsSatellite SensorsCamera Sensors17
- 18. What is Linked Sensor DataRecommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Web using URIs and RDFGeoNames DatasetRDF – language for representing data on the WeblocatedNearSensor DatasetPublicly Accessible18
- 19. Linked Sensor Data on LOD - First Sensor Dataset on LOD - Among the largest dataset on LOD19
- 20. znSensor DatasetsLinkedSensorDatasetRDF Descriptions of ~20,000 weather stations in US
- 21. Average 5 sensors/weather station
- 22. Spatial attributes of the weather station
- 23. Links to locations in GeonamesLinkedObservationDataset RDF descriptions of Hurricanes and Blizzard observations in US
- 24. Observations generated by sensors described in LinkedSensorDataset20
- 26. Workflow – Phase 122
- 27. Workflow – Phase 2OGC (Open Geospatial Consortium) standard for encoding sensor observations23
- 28. Workflow – Phase 3W3C SSN ontologyOntology – formal representation of knowledge by a set of concepts and relationship between those concepts
- 29. Workflow – Phase 3Figure 1: System Components and Architecture
- 30. Workflow – Phase 4Open Source RDF store by OpenLink Software for storing RDF dataPUBBY Linked Data Front End
- 31. Summarizing Linked Sensor DataFind the sensor around Dayton James Cox Airport?Extract Data for the sensor?ObservationKBSensor KBLocation KB(Geonames)locationprocedurelocationlocationprocedure720FThermometerDayton Airport ~2 billion triples
- 32. MesoWest
- 33. Static + Dynamic
- 34. 20,000+ systems
- 35. MesoWest
- 36. ~Static
- 38. Geonames
- 39. ~StaticxOutline Introduction ArchitectureLinked Sensor DataFeature Streams Demonstration28
- 40. 48th ACM Southeast Conference. ACMSE 2010. Oxford, Mississippi. April 15-17, 2010.Technology 2: Feature StreamsWhat feature is currently being detected by sensor near Dayton Airport?Harshal Patni, Cory Henson, Amit Sheth, Pramod Ananthram, ‘From Real Time Sensor Streams to Real Time Feature Streams,' Kno.e.sis Technical Report, January 2011.
- 41. xSystem ArchitectureStreams Integration based on feature compositionIntegrated Stream Analysis to check if the feature is being detected30
- 45. xSystem ArchitectureIntegrated Stream Analysis to check if the feature is being detected34
- 46. xFeature DefinitionRainStorm = HighWindSpeed(above 35mph) AND Rain Precipitation AND Temperature(greater than 32F)SPARQL query for RainStormTemperatureRain PrecipitationWindSpeed35Rain Storm NOAA definition
- 47. xFeature AnalysisRDF Feature Stream36
- 48. xRevisiting AbstractionsSummarization across Thematic DimensionSelectJoinBackground KnowledgeAnalyzeFeatures representing Real-World Events37
- 49. Summarizing Feature StreamsFeature StreamsKBFind sequence of events near Dayton Airport?ObservationKBSensor KBLocation KB(Geonames)procedurelocationlocationprocedure720FThermometerDayton Airport ~2 billion triples
- 50. MesoWest
- 51. Static + Dynamic
- 52. 20,000+ systems
- 53. MesoWest
- 54. ~Static
- 56. Geonames
- 57. ~StaticxAnswering the query39“Find the sequence of weather events observed near Dayton James Cox Airport between Jan 13th and Jan 18th?”Feature StreamsLinked Sensor Data
- 58. xOutline Introduction ArchitectureLinked Sensor DataFeature Streams Demonstration40
- 60. xEvaluationData Used: Nevada Blizzard (April 1st – April 6th)70% Data clear30% Feature Observed42
- 61. WORKSHOP PAPERSHarshal Patni, Satya S. Sahoo, Cory Henson, Amit Sheth, Provenance Aware Linked Sensor Data, 2nd Workshop on Trust and Privacy on Social and Semantic Web,Co-Located with ESWC, Heraklion Greece, May 30th - June 3rd 2010Harshal Patni, Cory Henson, Amit Sheth, Linked Sensor Data, In: Proceedings of 2010 International Symposium on Collaborative Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010TECHNICAL REPORTHarshal Patni, Cory Henson, Amit Sheth, and Pramod Ananthram. From Real Time Sensor Streams to Real Time Feature Streams, Kno.e.sis Center Technical Report, December 2009Joshua Pschorr, Cory Henson, Harshal Patni, and Amit Sheth. Sensor Discovery on Linked Data, Kno.e.sis Center Technical Report, December 2009JOURNAL PAPER (In Progress)Semantic Sensor Web: Design and Application towards weaving a meaningful sensor webPublications43
- 63. Thank YouSemantic Sensor Web45
- 64. Demos, Papers and more at: http://wiki.knoesis.org/index.php/SSWSemantic Sensor Web @ Kno.e.sisQUESTIONS46
Editor's Notes
- #3: Good Morning Everyone. My name is Harshal Patni and I am here to present my thesis on Streaming Sensor Data but Before we begin lets have a look at how web data evolved over time
- #5: Social media is the dominant source of streaming data now, however in future sensors would …Data needs to be reduced
- #7: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #8: Move this slide above
- #10: Remove the precipitation (in) and also show the general streamAdd the image taken on the phoneRemove the stuff on left when you show select, join and analyze`
- #11: Remove the precipitation (in) and also show the general streamAdd the image taken on the phoneRemove the stuff on left when you show select, join and analyze`
- #12: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #13: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #14: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsAdd linked Sensor Data when highlightThe output of these phases is called LSD and its added on LOD
- #15: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #17: Get all sensors using well known location names – Problem to be solveAssociate sensor descriptions to well know locations.
- #20: Get all sensors using well known location names – Problem to be solve
- #21: Say the numbers in the table
- #22: RDF because of LOD
- #23: Highlight the important points in MesoWest DataThe sensor data file just 3 linesMapping file - shorten
- #24: Emphasize semantically annotated O&MAnd its an XMLTry to replace the cory/weather.owl
- #25: Use the ssn ontologyAdd the image of ontology for the (Sensor Ontology)http://www.w3.org/2005/Incubator/ssn/wiki/Report_Work_on_the_SSN_ontology
- #26: Add in block letters saying this is semantically annotated XML and RDF
- #27: Add Pubby to show derefenced dataPubby should be large to show what it is
- #29: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #31: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #32: Replace Air Temperature with Non Freezing Temperature
- #33: Replace Rain Precipitation with PrecipitationSame with airtempearure - temperature
- #35: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #36: Highlight the query with 3 boxes to show the temp,windspeed and precipitation streamHighlight the feature results too
- #37: Talk about the observations and features storage
- #38: Remove the precipitation (in) and also show the general streamAdd the image taken on the phoneRemove the stuff on left when you show select, join and analyze`
- #40: Linked Data explodes
- #41: To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
- #42: Linked Data explodes
- #43: % of FeaturesThrow the text on the top for the statisticsMiddle of storm and hence we have 70 % data reductionElse it would be more