Engage 2020: Hello are you listening, There is stream for everything
0 likes4,121 views
The document is a presentation by Frank van der Linden discussing the principles of publish/subscribe architecture in stream processing, including the roles of producers, consumers, and message brokers in systems like RabbitMQ and Kafka. It introduces Spring Cloud Stream as a framework for connecting applications to message brokers and highlights the functionalities of a demo application built using these technologies. Additionally, it mentions upcoming features in the Domino event publisher and provides links to resources for further exploration.
Engage 2020: Hello are you listening, There is stream for everything
- 1. #engageug Hello are you listening There is stream for everything 1 Frank van der Linden - elstar IT
- 2. #engageug Introduction • Freelance Full stack developer • Owner of elstar IT • Curious of new technology • Spring Framework enthusiast • IBM Champion 2020 • Volleybal referee 2
- 4. #engageug ‘Everything in life can be broken down to producers, consumers and producers’ 4 Mark Heckler
- 5. #engageug How to proceed • Pub Sub principle • About streams • Spring Cloud Streams • Demo 5
- 8. #engageug Publish/Subscribe principle • Loose coupling • Scalability • Non-blocking • Message driven • 4 interfaces (Producer, Subscriber, Subscription and Processor) 8
- 9. #engageug 9
- 11. #engageug Parts of the stream • Producer or Source or Publisher, publish the message on the topic or queue • Consumer or Sink or Receiver, receiving the message from the topic or queue • Processor, can consume message, manipulate it and produce it again. • Broker, the middleware who is handling the queues or topics. 11
- 12. #engageug Message Broker • Route messages to one or more destinations • Transform messages to an alternative representation • Provide content and topic-based message routing using the publish–subscribe pattern • Perform message aggregation, decomposing messages into multiple messages and sending them to their destination, then recomposing the responses into one message to return to the use • Respond to events or errors 12
- 13. #engageug Message Broker • Kafka • RabbitMQ • OpenMQ • ActiveMQ (Amazone MQ) • Google Cloud PubSub • Azure Service Bus • Amazone Kinesis • IBM MQ • …….. 13
- 14. #engageug 14
- 16. #engageug 16 Domino Event Publisher
- 17. #engageug Domino Event Publisher • Add-on on Domino • Security Plugin for MQ’s needed to be installed • Will share UniqueID of Document and event type • Consumer needs to connect back to Domino to get all the information • May be later also other information to produce. • Private Beta will be available soon to HCL Masters. 17
- 18. #engageug Domino Event Publisher - my opinion • A good first step to embrace the decoupled world • Too much moving parts • Too complicated • Not following the pub/sub principle, because of the needed call back • Hope it will be possible in the future to share just information with the broker. 18
- 20. #engageug Spring Cloud Stream • Part of the Spring Cloud Family • Abstraction of the connection with a broker • Works in existing Spring Boot apps • Binders are the foundation • Multiple brokers are supported 20
- 21. #engageug 21
- 22. #engageug 22
- 23. #engageug 23
- 25. #engageug Demo • Spring Boot - foundation for Java API • Spring data - for connection to datastore • Spring Cloud Stream - abstraction to connect to broker • RabbitMQ • NodeJS + Typescript • MongoDb • Redis 25
- 26. #engageug 26 controller + producercontroller + producer http://localhost:3031/demo/api/todohttp://localhost:4000/api/todo processorprocessor consumer consumer post to database
- 27. #engageug Demo Flow 27 producer processor consumer message.header.type=open message.header.type=completed
- 28. #engageug Spring Cloud Stream - things to remember 28 application.yaml pom.xml
- 29. #engageug Spring Cloud Stream - things to remember 29 bindings processor producer consumer
- 30. #engageug NodeJS - things to remember 30 server.ts controller router
- 31. #engageug NodeJS - things to remember 31 processor producer
- 32. #engageug NodeJS - things to remember 32 consumer mongo service redis service
- 34. #engageug Resources • Spring Cloud Stream • https://spring.io/projects/spring-cloud-stream • Demo application • https://github.com/flinden68/streams-demo • RabbitMQ • https://www.rabbitmq.com/documentation.html • MongoDb • https://www.mongodb.com/ • Redis • https://redis.io/topics/quickstart • NodeJS • https://nodejs.org/en/ 34 • MongoDb Compass Community • https://www.mongodb.com/products/compass • Redily (GUI for Redis) • https://www.redily.app/
- 35. #engageug Thank you • @flinden68 • http://www.elstarit.nl • flinden68@elstarit.nl • https://nl.linkedin.com/in/flinden68 35