The Power of Now! Azure Stream Analytics - Microsoft ITPro AirLift
Download as PPTX, PDF2 likes1,514 views
The document discusses Azure Stream Analytics, a fully managed stream processing service. It describes how Stream Analytics allows processing of streaming data using simple SQL queries. Examples are provided of using temporal windows like tumbling, hopping and sliding windows to analyze streaming data from scenarios like toll stations. Components of a Stream Analytics application are also outlined.
The Power of Now! Azure Stream Analytics - Microsoft ITPro AirLift
- 1. 22 de Janeiro Microsoft Lisbon Experience The Power of Now! Rui Quintino | DevScope 22 Jan 2015 rui.quintino@devscope.net rquintino.wordpress.com twitter.com/rquintino Azure Stream Analytics
- 2. The traditional data warehouse
- 4. What are customers wanting to do?
- 5. Canonical scenarios for real-time processing
- 6. KUKA Systems Group created an IoT-powered plant that centrally connects and monitors hundreds of robots.
- 7. Introducing Azure Stream Analytics
- 9. Procure HW Infrastructure and setup Code for ingress, processing and egress Plan for resiliency, such as HW failures Design solution Build Monitoring and Troubleshooting
- 11. • Built-in monitoring – View your system’s performance at a glance – Help you find the cost-optimal way of deployment
- 12. 3 lines of code in Stream Analytics Thousand lines of code in other solutions • Only SQL queries needed – Developers uses declarative SQL commands – Some functions take several lines of code versus thousands from other solutions
- 13. • Implement temporal functions • Manage out-of-order events • Manage actions on late events
- 14. End-to-end stream processing architecture
- 16. Sample Scenario – Toll Station TollId EntryTime License Plate State Make Model Type Weight 1 2014-10-25T19:33:30.0000000Z JNB7001 NY Honda CRV 1 3010 1 2014-10-25T19:33:31.0000000Z YXZ 1001 NY Toyota Camry 2 3020 3 2014-10-25T19:33:32.0000000Z ABC 1004 CT Ford Taurus 2 3800 2 2014-10-25T19:33:33.0000000Z XYZ 1003 CT Toyota Corolla 2 2900 1 2014-10-25T19:33:34.0000000Z BNJ 1007 NY Honda CRV 1 3400 2 2014-10-25T19:33:35.0000000Z CDE 1007 NJ Toyota 4x4 1 3800 … … … … … … … … EntryStream - Data about vehicles entering toll stations TollId ExitTime LicensePlate 1 2014-10-25T19:33:40.0000000Z JNB7001 1 2014-10-25T19:33:41.0000000Z YXZ 1001 3 2014-10-25T19:33:42.0000000Z ABC 1004 2 2014-10-25T19:33:43.0000000Z XYZ 1003 … … … ExitStream - Data about cars leaving toll stations LicensePlate RegistartionId Expired SVT 6023 285429838 1 XLZ 3463 362715656 0 QMZ 1273 876133137 1 RIV 8632 992711956 0 … … …. ReferenceData - Commercial vehicle registration data
- 17. Application Components Components of an Azure Stream Analytics Application Azure SQL DB Azure Event Hubs Azure Blob StorageAzure Blob Storage Azure Event Hubs Reference Data Query runs continuously against incoming stream of events Events Have a defined schema and are temporal (sequenced in time)
- 18. Temporal Windows • Tumbling Windows – Repeating, non-overlapping, fixed interval windows • Hopping Windows – Generic window, overlapping, fixed size • Sliding Windows – Slides by an epsilon and produces output at the occurrence of an event
- 19. Tumbling Window SELECT System.TimeStamp AS OutTime, TollId, COUNT (*) FROM Input TIMESTAMP BY EntryTime GROUP BY TollId, TumblingWindow(minute,5)
- 20. Hopping Windows SELECT System.TimeStamp AS OutTime, TollId, COUNT (*) FROM Input TIMESTAMP BY EntryTime GROUP BY TollId, HoppingWindow(minute, 10 , 5)
- 21. Sliding Windows SELECT System.TimeStamp AS OutTime, TollId, COUNT (*) FROM Input TIMESTAMP BY EntryTime GROUP BY TollId, SlidingWindow(minute, 3) HAVING Count(*) > 3 Finds all toll booths which have served more than 3 vehicle in the last 3 minutes
- 23. Explore the latest in SQL Server Visit www.microsoft.com/sql to learn more about our latest innovations Evaluate SQL Server 2014 Get hands on – visit Microsoft’s TechNet evaluation center and evaluate SQL Server 2014 today! Evaluate Windows Server 2012 R2 Get hands on – visit Microsoft’s TechNet evaluation center and evaluate Windows Server 2102 R2 today!Microsoft Azure