Asynchronous design with Spring and RTI: 1M events per second
7 likes2,097 views
This document discusses the development and performance enhancements of a real-time intelligence (RTI) stream processing application using the Spring framework. Key insights include lessons learned from initial builds, performance improvements through effective data ingestion, and architectural optimizations leveraging Spring Integration and Reactor. The findings highlight significant efficiency gains in processing speed and resource utilization for cloud applications.
Asynchronous design with Spring and RTI: 1M events per second
- 1. Asynchronous Design: 1M events per second – with Spring By Stuart Williams © 2014 SpringOne 2GX. All rights reserved. Do not distribute without permission.
- 2. Bio 2 Stuart Williams • Software Engineer at Pivotal – RTI project lead @pidster
- 3. What is this all about? • We built a product using Pivotal products • Learned some lessons • We found a few limitations & some room for improvement… • … but we addressed them & now things go faster. A lot faster. 3
- 4. Dogfood • Built with Spring IO Platform • Boot, Data, Integration, Reactor, AMQP, SpEL, Shell (and a little Groovy) • GemFire • RabbitMQ 4 Spring Framework Spring Data Spring Integration Spring Reactor Spring AMQP Spring Hateoas Groovy Spring Boot
- 5. Questions for you • Heard of Spring Integration? • Tried it? • In production? • Heard of Reactor? • Tried it? • In production? 5
- 6. RTI 6
- 7. What is RTI? • RTI == ‘Real Time Intelligence’ • Stream processing application • Location based services • Analytics (e.g. network health) • Telecom network data • ‘Control plane’ is meta data • ‘User plane’ is actual data (30x more) • Rich data model 7
- 8. Input Data Rates RTI* • 100k/s average • 120k/s daily peak • 1M/s annual peak 8 *Control-plane only, user-plane is 20x
- 10. Input Data Rates RTI* • 100k/s baseline • >120k/s daily peak • >1M/s annual peak 10 *Control-plane only, user-plane is 20x Twitter** • 6k/s average • 9k/s daily peak • 30k/s large events **Source @catehstn twitter.com/catehstn/status/494918021358813184
- 11. Load Characteristics • Low numbers of inbound connections • High rates, micro-bursts • Occasional peaks of nearly 2x, rare peaks of 10x • Variable payload size (200B – 300KB) • Internal fan-outs multiple event rates 11
- 12. More statistics… • 100k/s order of magnitude • 8,640,000,000 (per day) • An Integer based counter will ‘roll over’ in ~2 days • 400Mbps of raw data (‘control plane’) • 10Gbps NICs required to support traffic peaks • Logging! Any verbose errors can blow a disk away • Queues backing up == #fail 12
- 13. Architecture 13
- 14. Architecture 14 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
- 15. Architectural Challenges • Ingest • Responsibility • Micro-bursts • Infrastructure considerations • Compute • Memory • Disk • Network 15
- 16. Architecture 16 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
- 17. Ingester Architecture 17 Ingester • Spring Integration • TCP Server • Transformer • Filters • Reactor Stream • GemFire client • Single process • Multiple protocols – different rates & sizes
- 18. Architecture 18 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
- 19. Analytics Architecture 19 Analytics • Reactor • SpEL evaluation • Hundreds of expression calculations per event • Collate across nodes • Output to File or AMQP
- 20. Architecture 20 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
- 21. Architecture 21 Distribution • Spring Integration • Enrichers • Filters • Reactor Stream • Output to File / AMQP / JDBC • Membership checks • LBS, opt-in’s
- 22. First Ingester solution 22
- 23. Solution #1 – ‘Naïve’ proof of concept • Build codecs • More on this in John Davies’ “Big Data In Memory” talk later today… • Spring Integration (SI) pipeline • TCP Inbound Adapter • Transformer • Filters • Outbound adapter 23
- 24. Solution #1 – ‘Naïve’ proof of concept 24
- 25. Solution #1 results • Message UUID generation was slow! • SpEL-based method resolution was slow! • Standard Channels added overhead! • Single event output was slow! 25
- 26. Ingester Mark 2 26
- 27. Solution #2 – Use interfaces 27
- 28. Solution #2 – Use interfaces • Use the IdGenerator interface • Use specific endpoint interfaces • … we’ll come back to SpEL … • Use a Chain • Use an Aggregator to build a batch 28
- 29. Solution #2 results • IdGenerator helped a lot • Specific interfaces not recognised! • Using <int:chain helped • Aggregator helped, but is too slow • <int:tcp-inbound-adapter is too slow 29
- 30. Many whiteboards later… 30
- 31. Many whiteboards later… 31
- 33. Solution N 33 Message-only Filters Batcher Reference Data Filters GB IUPS IUCS A Radius/Diamete r 4G
- 34. Working version • Netty / Reactor TCP • IdGenerator • Specific endpoint interfaces • Chain • Reactor Stream based batching • + many improvements & enhancements 34
- 35. Roundup 35
- 36. Spring Improvements • Performance • Spring Integration • SpEL • Reactor • Spring XD benefits from these upgrades 36
- 38. Spring Integration • UUID generator • MessageBuilderFactory & MutableMessage • Dispatcher optimisation • SpEL parser caching • Counters are ‘long’ • Interfaces used directly – if you’re specific SI respects that 38
- 39. Spring Expression Language • Compilation of expressions • Configuration options • SI just re-evaluates expressions • Trade-offs & limitations • Much, much faster 39
- 40. SpEL demo 40
- 41. Reactor • Drop-in replacements • Thread pools, dispatchers • TCP/UDP Server & Client • Much faster – lower resource utilisation • Stream API • Batching and other functionality • More about Reactor • Thu, 8.30am “Building Reactive Applications…” 41
- 42. Batching endpoint code 42
- 43. Summary • Spring Integration is much faster • Good performance means better resource utilisation • For cloud applications cost savings can be dramatic • Batching payloads makes a big difference • Many applications wait on network IO • Trade-off risk of data loss for performance • Reactor FTW 43
- 44. Questions 44
- 45. 45 Learn More. Stay Connected Tweet #rti #s2gx if you’d like to go faster @pidster “Big Data in Memory” John Davis – Trinity 1-2 4.30pm @springcentral | spring.io/video