Nservicebus at scale
Download as PPTX, PDF0 likes491 views
This document discusses NServiceBus at scale. It covers why the speaker's company adopted NServiceBus, how they monitored and tuned their system using it, and an integration example with a payment service provider. Some lessons learned included managing code dependencies, testing asynchronous and long-running processes, and deploying distributed transaction coordinators.
![[Unique] Attribute](https://image.slidesharecdn.com/nservicebusatscale-140915040910-phpapp02/85/Nservicebus-at-scale-22-320.jpg)
Nservicebus at scale
- 2. About Me Charlie Barker Twitter: @porkstone Blog: http://dualbotic.com
- 3. What will this talk cover? Why we adopted NServiceBus Monitoring / Tuning Integration Example with a PSP Lessons Learned
- 4. How our platform used to communicate Website Back Office Decision Engine XML/HTTP XML/HTTP Third party services
- 6. Why NServiceBus? ● Udi says ● Robust ● Distributed ● Opinionated ● Training ● Community
- 7. Monitoring / Tuning ● Added more granular events ● Used WMI to send data ● Aggregated data every minute ● Exceptions by Message Type ● Allowed us to find bottlenecks quickly ● Allowed us to predict SLA failure
- 8. Predict Failure Point SLA Breach 1 7 14 21 28 % Busy Time Day of the month
- 11. Detailed View - Message Type Breakdown
- 12. Detailed View - Single Message
- 13. Metrics available for endpoint as a whole ● %Busy Time ● %Idle Time ● Critical Time(s)
- 14. Metrics for endpoints + individual messages ● Count ● Percentage of time ● Average Duration
- 15. Count Metrics ● Message Count ● Errors Count ● Retries Count ● Handler Failures Count
- 16. Percentage of time metrics ● Message deserialisation time ● Message handler time ● Message commit time ● Message processing time ● Failed message processing time
- 17. Average Duration metrics ● Avg message deserialize (ms) ● Avg message handler (ms) ● Avg message commit (ms) ● Avg message processing (ms) ● Avg failed message processing (ms)
- 18. Ivica
- 20. Endpoints Engine Bot HTTP Payment Service Provider NSB v3.3.5 RavenDB 2.0
- 23. RavenDB
- 25. Creating the missing docs
- 27. Payment Collection Saga - Start/Stop
- 28. Payment Collection Saga - Timeout
- 29. Lessons Learned ● Code dependencies ● Testing Async ● Testing Long Running Processes ● Deploying the DTC ● Rewrite from scratch?
- 30. What would I like to see in the future?
- 31. Questions?
- 32. fin
Editor's Notes
- #17: "Percentage of time" metrics These metrics report the average percentage of the entire available time an endpoint is spending performing specific message-processing tasks (see note on % Busy Time further below on how percentage timekeeping is performed) % Msg Deser Time - percentage of time spent deserialising message XML bodies. This time is usually tiny relative to other components. % Msg Handler Time - percentage of time spent executing a message handler. This is the component the endpoint developer has direct control over. % Msg Commit Time - percentage of time spent inside MSDTC commiting a successful message. This time is usually 1/10th of the overall Msg processing time - if it gets bigger than that, it signifies potential problems with the MSDTC. % Msg Proc Time - percentage of time spent processing successful messages. This is a sum of ( % Msg Deser Time + % Msg Handler Time + % Msg Commit Time ). % Failed Msg Proc Time - percentage of time spent processing messages that have failed.
- #18: "Average (ms) duration" metrics These metrics report average duration (in miliseconds) of the same message-processing tasks in the previous slide. A rarely occurring but long-running message type can be under-represented in the "percentage" metrics, but it will clearly stand out in the "average (ms) duration" ones. These counters are particularly useful for troubleshooting individual message types that are taking too long to execute, or are failing after a timeout. These can be easily pinpointed by enabling "Avg Msg Proc (ms)" and "Avg Failed Msg Proc (ms)" counters in the "message breakdown" view.
- #19: It would be unfair of me not to mention the talented engineer who built the majority of this monitoring solution - Ivica Nikolic