An Introduction to the Message Queuning Technology

Introduction
Introduction to Java Message Service(JMS)
Message Oriented Middleware
Demo
An Introduction to the Message Queuing
Technology
K. Harinath
ClariTrics Technologies
Buddihealth
BUDDI InterACT Session, 2017
K. Harinath An Introduction to the Message Queuing Technology

Introduction
Demo
Outline
1 Introduction
Introduction to Message and Queue
Message Queue & Purpose
2 Introduction to Java Message Service(JMS)
JMS Components/ Elements
Message Queue Model
List of Message Oriented Middleware(MOM) available
3 Message Oriented Middleware
4 Demo

Introduction
Demo
Message Queue & Purpose
What is Message and Queue.?
Message: Communication conveyed from one person or
group to another.
Example: email, chat, notications etc.
Queue: Queue is a form of arrangement, where items are
processed in sequential order.
Example: Queues in ATM, bill payment etch.

Introduction
Demo
Message Queue Purpose
What is Synchronous Communication
Synchronous operation (caller is blocked until callee returns).
Problems / drawbacks with this model of operation:
The client is blocked until the server (object operation) call
returns (tight coupling).
Connection overhead (each call needs marshalling, entails
protocol overhead for network access etc.).
Dicult to react to failures (server may throw an exception,
may not be active etc.).

Introduction
Demo
What is Asynchronous Communication
Asynchronous operation, caller sends a message and continue
its work.
Store and Forward communication.
Sender receiver are loosely coupled:
They do not need to be active at the same time.
The sender does not need to know the receiver location and
vice verse.
Analogy
Synchronous (RPC) - Telephone
Asynchronous (MOM) - Mail

Introduction
Demo
Message Queue (MQ)
Message Queue: It is design pattern used in system
architecture which is used to achieve fail safe and high
availability.
It follows store and forward communication.
Advantages
Allows application to connect in asynchronous way.
Allows to handle multiple request at same time.
Decoupling design in achieved in the application using MQ

Introduction
Demo
Uses of Message Queue
Email service
Web chat
Notication and Alerting service
Order Processing in e-commerce sites
Feed in Facebook

Introduction
Demo
Message Queue Model
Java Messaging Service
JMS: JMS is a set of standard laid by Java community to
ensure reliable and relax tightly coupled system.
Examples of tightly coupled system
RMI
CORBA
Socket Communication

Introduction
Demo
Message Queue Model
JMS Components
JMS Client or Consumer
JMS Producer
JMS Queue
JMS Message
JMS Provider

Introduction
Demo
Message Queue Model
Types of Message Queue Model
1 Point to Point - One to One Model
2 Publish and Subscribe - One to Many Model

Introduction
Demo
Message Queue Model
Types of Message
Persistent Message
Message are stored in disk/database.
We can store in ActiveMQ internal DB or data store such as
MYSQL.
Non Persistent Message
Message is kept in volatile memory.
Oers high performance compared to persistent message

Introduction
Demo
Message Queue Model
Types of Queue - Ordering
FIFO Queues
Priority Queues

Introduction
Demo
Message Queue Model
Queue Type - Conguration
Private Queue
Sending to a private queue requires sender authentication.
Public Queue
All senders may send messages without access control.
Journal Queue
Keeps a copy of every received message for logging or
monitoring purposes.
Dead Letter Queue
Queue that holds non deliverable messages.

Introduction
Demo
Message Queue Model
Advance Message Queuing Protocol (AMQP)
AMQP (Advanced Message Queuing Protocol) is wire
specication for asynchronous messaging.
API
Methods, data, and rules to interact with a component from
an application.
Message format not specied
Protocol
A Protocol denes the way two entities communicate, a
protocol API is an interface for an application to use a
protocol implementation
Every byte of transmitted data is specied
Standard Wire protocol

Introduction
Demo
Message Queue Model
List of Message Oriented Middleware Available
Apache ActiveMQ
RabbitMQ
MSMQ
Apache Kafka

Introduction
Demo
ActiveMQ
Apache ActiveMQ is JMS compliant message broker
Written in Java but support API for other language to extend
the functionality and Scala version is in alpha version.
Open Sourced and suitable for business model.
Provides reliability, performance, high availability and security
for messaging
It oer extra set of features compared to JMS standard
specication.

Introduction
Demo
Download Installation
Download the package from http:
//activemq.apache.org/activemq-5150-release.html
Unzip or Untar in desired location
Installation
bin/activemq console in linux
bin/activemq in windows
Web Console to manage ActiveMq started at IP:8161
Install stomp module in python by pip install stomp.py

Introduction
Demo
Architecture of ActiveMQ

Introduction
Demo
ActiveMQ Message Format

Introduction
Demo
Supported Protocols
Open-Wire
default in ActiveMQ; a binary protocol
Clients for C++, Java and .NET
STOMP
Simple Text Oriented Messaging Protocol; a text based
protocol
Clients for C, JavaScript, Perl, PHP, Python, Ruby
XMPP
Used extensively in chat applications
REST
HTTP POST and GET

Introduction
Demo
RabbitMQ
RabbitMQ is a message queue written in Erlang, which can
utilise all the cores present in the system.
AMQP + JMS = RabbitMQ
It provides client for many languages.

Introduction
Demo
Components of RabbitMQ
Virtual Host
Access Control
acts a name space and it has own set of queues, exchanges
and associated objects.
Exchange
Routing agent can be durable, temporary, auto deleted.
each exchange can implements its own set of routing
algorithm.
Each message can be published with routing key.

Introduction
Demo
Message Pattern in RabbitMQ
Direct
Routes message to the queues which matches the binding key
Fanout
Broadcast messages to all queues that are bound to it.
Topic
Broadcast message to all queues that are bound to it using
binding keys

Introduction
Demo
MSMQ
Message infrastructure and a development platform for
creating distributed, loosely coupled system
MSMQ is also similar to other Message queue such as
ActiveMQ.
It also oers guaranteed message delivery, ecient routing,
security and transaction support
It is developed for Microsoft Windows OS.

Introduction
Demo
Structure of MSMQ

Introduction
Demo
MSMQ Modes
MSMQ can be run in 2 modes
Workgroup Mode: Private Queues in workgroup mode.
Domain Mode: allows sending messages between multiple
domains, both private and public queues are possible here
Message size can not be more than 4 MB.
2GB Queue size limit

Introduction
Demo
Data Pipeline System
Data pipeline is a system which allows to move data from one
sub system to another sub system within system or sometimes
across system
Helps to transform data from one format to another
Removes un necessary columns or data points
Mask sensitive information
Insert Record
Ability to speak with dierent consumer and producer data.
Data Pipeline system can also mimic as message queue.

Introduction
Demo
Data Pipeline

Introduction
Demo
Kafka
Kafka is high throughput distributed message system .
Used to build data pipeline, real time streaming and messaging
system.
Acts as data collector: Collects data from various source and
aggregates to common format.
Build on top of Scala and Java
kafka allows to have separation of concern in application level.

Introduction
Demo
Kafka

Introduction
Demo
API of Kafka
Producer
Kafka Producer class provides send method to send messages
asynchronously to a topic
Consumer
Consumer group is a multi-threaded or multi-machine
consumption from Kafka topics
Connect
Handles export and import
Streams
Key operations are map, lter, aggregate and join

Introduction
Demo
Azure Queue Mechanism
Azure supports two types of queue mechanisms:
Storage queues
Service Bus queues
Cloud service allows us to focus on functionality instead of
operational nature of the application.
Load balancing comes as standard features, no worries about
scaling when demand rises or decrease.
Allows to lter the message queue based on the condition.

Introduction
Demo
Azure Storage Queue
Message queue have life time of 7 days
Message queue size more than 80 GB
Stateful nature, remebers where the application crashes and
resume from the same point.
Allows to access the storage queue using REST API.
Operations performed are not atomic.

Introduction
Demo
Azure Service Bus
It oers more features than storage queue, basically it is build
on top of storage queue
Sessions, duplicate detection, AMQP
Transactions, Automatic dead-lettering
Durable publish/subscribe capabilities
Message queue size must be less than 80 GB
Guaranteed rst-in-rst-out (FIFO) ordered delivery and TTl
can exceed 7 days.

Introduction
Demo
Azure Message Architecture

Introduction
Demo
Creating Queue Sending to Queue
SendingMessageToQueue.py
# Import by STOMP library
import stomp
# Establish the connection
conn = stomp.Connection10()
# Start the connection.
conn.start()
#Connecting the ActiveMQ with username and password
conn.connect()
# Creates the queue
conn.send('BuddiQueue', 'Hi')
conn.disconnect()

Introduction
Demo
Reading Message from Queue
ReceivingMessageFromQueue.py
class BuddiListener(object):
def on_message(self, headers, msg):
print(msg)
conn = stomp.Connection10()
conn.set_listener('SampleListener', BuddiListener())
#Start the connection.
conn.start()
# Connecting the ActiveMQ with username and password
conn.connect()
#Subscribe to a destination
conn.subscribe('BuddiQueue')

Appendix For Further Reading
Reference I
Bruce Snyder, Dejan Bosanac, Rob Davies.
ActiveMQ in Action.
Manning Press, 2011.
Emrah, Yusuf, Dotan Nahum.
Mastering RabbitMQ.
Packt Publishing Ltd, 2015.
Fernando Ribeiro.
Using Apache ActiveMQ with Python
https://www.slideshare.net/fribeiro1/
using-apache-activemq-with-python.
https://www.codeproject.com/Articles/276400/
Using-Windows-Azure-Service-Bus-Messaging

Appendix For Further Reading
Reference II
Nishant Garg
Apache Kafka.
Packt Publishing Ltd, 2013.
https://www.tutorialspoint.com/apache_kafka/
apache_kafka_tutorial.pdf
https://docs.microsoft.com/en-us/azure/
service-bus-messaging/
service-bus-azure-and-service-bus-queues-compared-contra
https://msdn.microsoft.com/en-us/library/
ms711472(v=vs.85).aspx

An Introduction to the Message Queuning Technology

More Related Content

What's hot (7)

Similar to An Introduction to the Message Queuning Technology (20)

Recently uploaded (20)

An Introduction to the Message Queuning Technology