FFWD - Fast Forward With Degradation
1 like1,247 views
The document presents ffwd, a latency-aware event stream processing framework designed for real-time sentiment analysis, particularly using Twitter data. It features domain-specific load-shedding policies to manage unpredictable data arrival rates, ensuring response time remains bounded while minimizing dropped tweets. Experimental evaluations demonstrate the effectiveness of ffwd's heuristic controller and pluggable policies in accurately processing metrics under various load conditions.
![Real-time sentiment analysis 4
• Real-time sentiment analysis allows to:
– Track the sentiment of a topic over time
– Correlate real world events and related sentiment, e.g.
• Toyota crisis (2010) [1]
• 2012 US Presidential Election Cycle [2]
– Track online evolution of companies reputation, derive social
profiling and allow enhanced social marketing strategies
[1] Bifet Figuerol, Albert Carles, et al. "Detecting sentiment change in Twitter streaming data." Journal of Machine Learning Research:
Workshop and Conference Proceedings Series. 2011.
[2] Wang, Hao, et al. "A system for real-time twitter sentiment analysis of 2012 us presidential election cycle." Proceedings of the ACL
2012 System Demonstrations.](https://image.slidesharecdn.com/ffwd-160919112218/85/FFWD-Fast-Forward-With-Degradation-4-320.jpg)
FFWD - Fast Forward With Degradation
- 1. FFWD: latency-aware event stream processing via domain-specific load-shedding policies R. Brondolin, M. Ferroni, M. D. Santambrogio 2016 IEEE 14th International Conference on Embedded and Ubiquitous Computing (EUC) 1
- 2. Outline 2 • Stream processing engines and real-time sentiment analysis • Problem definition and proposed solution • FFWD design • Load-Shedding components • Experimental evaluation • Conclusion and future work
- 3. Introduction 3 • Stream processing engines (SPEs) are scalable tools that process continuous data streams. They are widely used for example in network monitoring and telecommunication • Sentiment analysis is the process of determining the emotional tone behind a series of words, in our case Twitter messages
- 4. Real-time sentiment analysis 4 • Real-time sentiment analysis allows to: – Track the sentiment of a topic over time – Correlate real world events and related sentiment, e.g. • Toyota crisis (2010) [1] • 2012 US Presidential Election Cycle [2] – Track online evolution of companies reputation, derive social profiling and allow enhanced social marketing strategies [1] Bifet Figuerol, Albert Carles, et al. "Detecting sentiment change in Twitter streaming data." Journal of Machine Learning Research: Workshop and Conference Proceedings Series. 2011. [2] Wang, Hao, et al. "A system for real-time twitter sentiment analysis of 2012 us presidential election cycle." Proceedings of the ACL 2012 System Demonstrations.
- 5. Case Study 5 • Simple Twitter streaming sentiment analyzer with Stanford NLP • System components: – Event producer – RabbitMQ queue – Event consumer • Consumer components: – Event Capture – Sentiment Analyzer – Sentiment Aggregator • Real-time queue consumption, aggregated metrics emission each second (keywords and hashtag sentiment)
- 6. Problem definition (1) 6 • Our sentiment analyzer is a streaming system with a finite queue • Unpredictable arrival rate λ(t) • Limited service rate μ(t) S λ(t) μ(t) • If λ(t) limited -> λ(t) ≃ μ(t) • Stable system • Limited response time
- 7. Problem definition (2) 7 • If λ(t) increases too much -> λ(t) >> μ(t) • The queue starts to fill • Response time increases… S λ(t) μ(t) • Our sentiment analyzer is a streaming system with a finite queue • Unpredictable arrival rate λ(t) • Limited service rate μ(t)
- 8. Problem definition (2) 8 • … until the system looses its real-time behavior S λ(t) μ(t) • Our sentiment analyzer is a streaming system with a finite queue • Unpredictable arrival rate λ(t) • Limited service rate μ(t)
- 9. Proposed solution 9 • Scale-out? – however limited to the available machines • What if we try to drop tweets? – Keep bounded the response time – Try to minimize the number of dropped tweets – Try to minimize the error between the exact computation and the approximated one • Use probabilistic approach to load shedding • domain-specific policies to enhance the accuracy in estimation
- 10. Fast Forward With Degradation (FFWD) • FFWD adds four components: 10 Event Capture Sentiment Analyzer Sentiment Aggregator account metrics output metrics analyze event Producer eventinput tweets real-time queue
- 11. Fast Forward With Degradation (FFWD) 11 • FFWD adds four components: – Load shedding filter at the beginning of the pipeline – Shedding plan used by the filter Producer Load Shedding Filter Event Capture Sentiment Analyzer Sentiment Aggregator Shedding Plan real-time queue ok ko ko count account metrics event output metricsinput tweets drop probability analyze event
- 12. Fast Forward With Degradation (FFWD) 12 • FFWD adds four components: – Load shedding filter at the beginning of the pipeline – Shedding plan used by the filter – Domain-specific policy wrapper Producer Load Shedding Filter Event Capture Sentiment Analyzer Sentiment Aggregator Policy Wrapper Shedding Plan real-time queue ok ko ko count account metrics stream statsupdated plan event output metricsinput tweets drop probability analyze event
- 13. Fast Forward With Degradation (FFWD) 13 • FFWD adds four components: – Load shedding filter at the beginning of the pipeline – Shedding plan used by the filter – Domain-specific policy wrapper – Application controller manager to detect load peaks Producer Load Shedding Filter Event Capture Sentiment Analyzer Sentiment Aggregator Policy Wrapper Controller Shedding Plan real-time queue ok ko ko count account metrics λ(t) R(t) stream statsupdated plan μ(t+1) event output metricsinput tweets drop probability Rt analyze event
- 14. Controller 14 S: (Little’s Law) (Jobs in the system) The system can be characterized by its response time and the jobs in the system Control error: Requested throughput: The requested throughput is used by the load shedding policies to derive the LS probabilities Controller
- 15. Controller 15 S: (Little’s Law) (Jobs in the system) The system can be characterized by its response time and the jobs in the system Control error: Requested throughput: The requested throughput is used by the load shedding policies to derive the LS probabilities Old response time Target response time Controller
- 16. Controller 16 S: (Little’s Law) (Jobs in the system) The system can be characterized by its response time and the jobs in the system Control error: Requested throughput: The requested throughput is used by the load shedding policies to derive the LS probabilities Requested throughput Arrival rate Controller Control error
- 17. Policies • Baseline: General drop probability computed from the requested throughput 17 en the event e component a drop queue n to perform pecific Policy computes the erence signal µ(t) = (t 1) µmax · e(t) (6) U(t) = ¯U (7) P(X) = 1 µc(t 1) µ(t) (8) Policy Wrapper
- 18. Policies • Baseline: General drop probability computed from the requested throughput • Fair: Assign to each input class the “same" number of events – Save metrics of small classes, still accurate results on big ones 18 en the event e component a drop queue n to perform pecific Policy computes the erence signal µ(t) = (t 1) µmax · e(t) (6) U(t) = ¯U (7) P(X) = 1 µc(t 1) µ(t) (8) Policy Wrapper
- 19. Policies • Baseline: General drop probability computed from the requested throughput • Fair: Assign to each input class the “same" number of events – Save metrics of small classes, still accurate results on big ones • Priority: Assign a priority to each input class – Divide events depending on the priorities – General case of Fair policy 19 en the event e component a drop queue n to perform pecific Policy computes the erence signal µ(t) = (t 1) µmax · e(t) (6) U(t) = ¯U (7) P(X) = 1 µc(t 1) µ(t) (8) Policy Wrapper
- 20. Filter 20 • For each event in the system: – looks for probabilities in shedding plan using its meta-data – if not found uses general drop probability Load Shedding Filter Load Shedding Filter Shedding Plan real-time queue batch queue ok ko drop probability Event Capture • If specified, the dropped events are placed in a different queue for a later analysis
- 21. Evaluation setup 21 • Separate tests to understand FFWD behavior: – Controller performance – Policy and degradation evaluation • Dataset: 900K tweets of 35th week of Premier League • Performed tests: – Controller: synthetic and real tweets at various λ(t) – Policy: real tweets at various λ(t) • Evaluation setup – Intel core i7 3770, 4 cores @ 3.4 Ghz + HT, 8MB LLC – 8 GB RAM @ 1600 Mhz
- 22. Controller Performance 22 case A: λ(t) = λ(t-1) case B: λ(t) = avg(λ(t)) λ(t) estimation:
- 23. Controller showcase (1) • Controller demo (Rt = 5s): – λ(t) increased after 60s and 240s – response time: 23 0 1 2 3 4 5 6 7 0 50 100 150 200 250 300 Responsetime(s) time (s) Controller performance QoS = 5s R
- 24. Controller showcase (2) • Controller demo (Rt = 5s): – λ(t) increased after 60s and 240s – throughput: 24 0 100 200 300 400 500 0 50 100 150 200 250 300 #Events time (s) Actuation lambda dropped computed mu
- 25. Degradation Evaluation 25 • Real tweets, μc(t) ≃ 40 evt/s • Evaluated policies: • Baseline • Fair • Priority • R = 5s, λ(t) = 100 evt/s, 200 evt/s, 400 evt/s • Error metric: Mean Absolute Percentage Error (MAPE %) (lower is better) 0 10 20 30 40 50 A B C D MAPE(%) Groups baseline_error fair_error priority_error λ(t) = 100 evt/s 0 10 20 30 40 50 A B C D MAPE(%) Groups baseline_error fair_error priority_error λ(t) = 200 evt/s 0 10 20 30 40 50 A B C D MAPE(%) Groups baseline_error fair_error priority_error λ(t) = 400 evt/s
- 26. Conclusions and future work 26 • We saw the main challenges of stream processing for real- time sentiment analysis • Fast Forward With Degradation (FFWD) – Heuristic controller for bounded response time – Pluggable policies for domain-specific load shedding – Accurate computation of metrics – Simple Load Shedding Filter for fast drop • Future work – Controller generalization, to cope with other control metrics (CPU) – Predictive modeling of the arrival rate – Explore different fields of application, use cases and policies