ActiveMQ In Action - ApacheCon 2011

ActiveMQ In Action
Common Problems and Solutions

Bruce Snyder
VMware, Inc.
bsnyder@vmware.com

Wednesday, November 16, 11

Common Questions

• Should I create my JMS clients from scratch?
• How do I manage connections efﬁciently?
• How do I consume only certain messages?
• Why is ActiveMQ locking up or freezing?

• Bonus: Do I need a network of brokers?
• Bonus: Should I use a master/slave conﬁguration?


Should I create JMS clients from scratch?

3


Should I create JMS clients from scratch?

• Question:
– Would you create a HTTP client from scratch?
– Would you create a SMTP client from scratch?

• Answer:
– Sometimes, but mostly no

• Solution:
– Use Spring JMS

4


Spring JMS

5


Spring JMS

• JMS Template
– Send and receive messages
synchronously

• Message Listener Container
– Receive messages asynchronously
– Message-Driven POJOs (MDPs)

6


Spring JMS

• JMS Template
– Send and receive messages
synchronously

• Message Listener Container
– Receive messages asynchronously
– Message-Driven POJOs (MDPs)

7


How do I manage connections efﬁciently?

8


How do I manage connections efﬁciently?

• JMS connections are expensive to constantly create and
destroy
• Create a group that never closes, i.e., pooling

• Solutions:
– ActiveMQ PooledConnectionFactory
– Spring CachingConnectionFactory

9


ActiveMQ PooledConnectionFactory

• Based on Apache Commons Pool
– Generic object pooling framework from the ASF
• Highly conﬁgurable
– Instantiate your own custom GenericObjectPool
• Could be improved
– Upon hitting pool limit, grow the pool instead of blocking
– Throw exception when the pool is exhausted
• Caches JMS Sessions and MessageProducers

10


Spring CachingConnectionFactory

• Based on Spring SingleConnectionFactory
– Ignores calls to Connection.close()
• Caches JMS Sessions and MessageProducers
• By default only one session is cached
– Increase the sessionCacheSize property!
• Consumers are not closed until Session is closed
– NOTE: Cache strategy uses the JMS selector as a key

11


Quick Tip: Monitor the Broker

• Q: How can I monitor the message broker while I am
developing?
• A: Manually via JMX
• A: Use the web console (backed by JMX)
• A: Use the advisory messages
– Especially powerful
– http://activemq.apache.org/advisory-message.html

12


How do I consume only certain messages?

13


How do I consume only certain messages?

• ActiveMQ is for sending and receiving events
• ActiveMQ is NOT a message store

• Solutions:
– Use message selectors
– Correct application design

14


JMS Selectors

• Allows a client to filter messages from a destination
• Filters message headers only, not payload
• Conditional expressions using a subset of SQL
• Provide boolean evaluation of message headers

Booleans TRUE/FALSE; numbers such
as 5, -10, +34; numbers with decimal
Literals
or scientific notation such as 43.3E7,
+10.5239
Identifiers A header field

AND, OR, LIKE, BETWEEN, =, <>, <, >,
Operators
<=, =>, +, =, *, /, IS NULL, IS NOT NULL

15


JMS Selector Examples
// Select messages with a header named symbol whose value is APPL
String selector = "symbol = ‘APPL’";

// Create a consumer that only receives messages about Apple Computer stock
MessageConsumer consumer =
session.createConsumer(someDestination, selector);

// Select messages with a header named symbol whose value is APPL
// and with a header named price that is greater than the previous price
String selector = "symbol = ‘APPL’ AND price > " + getPreviousPrice();

// Create a consumer that only receives messages about Apple Computer stock
// that has increased in price
MessageConsumer consumer =
session.createConsumer(someDestination, selector);

16


JMS Selectors

• Very powerful, but like a sharp knife
• Applied to every message on a destination
– Can cause unnecessary overhead

17


Correct Application Design

• ActiveMQ is for sending and receiving events
• ActiveMQ is NOT a message store

• Phase one, consume the messages
– Lightweight processing
• Phase two, conduct further processing
– Heavyweight processing

• I.e., a proper service-oriented architecture

18


Quick Tip: Clearing the DLQ

• Q: Is there a way to automatically clear messages in the DLQ?
• A: Use the DiscardingDLQ plugin or create a custom consumer

<broker brokerName="myBroker” ...>
<plugins>
<discardingDLQBrokerPlugin dropAll="true" dropTemporaryTopics="true"
dropTemporaryQueues="true" />
</plugins>
</broker>

<plugins>
<discardingDLQBrokerPlugin
dropOnly="TEST.FOO.[1-9] EXAMPLE.TOPIC"
reportInverval="5000" />
</plugins>
</broker>
19


Why is ActiveMQ is locking up or freezing?

20


Why is ActiveMQ is locking up or freezing?

• JVM memory
• Broker memory
• Prefetch limit
• Producer ﬂow control
• Message cursors

21


JVM Memory

• ActiveMQ start script
– In 5.4.x, JVM is given 256mb of memory (min and max)

• You may need to increase this!

22


Broker Memory

• ActiveMQ controls how much memory it can use
• Will not automatically use all the JVM memory
• Conﬁgurable but commented out by default

23


Broker Memory Example
...
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="64 mb” />
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb" />
</storeUsage>
<tempUsage>
<tempUsage limit="10 gb" />
</tempUsage>
</systemUsage>
</systemUsage>
...
</broker>

24


Prefetch Limit
• Prevents a consumer from being ﬂooded with messages
• Applied on a per client basis

• Incorrect prefetch limit + slow consumer =
messages remain in a queue unconsumed

• Results in some consumers being starved of messages

• NOTE: Be careful with connection pools

25


Prefetch Limit Example

...
<bean id="connectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory"
p:brokerURL="tcp://localhost:61616"
p:prefetchPolicy-ref="prefetchPolicy"/>

<bean id="prefetchPolicy"
class="org.apache.activemq.ActiveMQPrefetchPolicy"
p:queuePrefetch="1" />
...

26


Producer Flow Control

• Prevents producer from ﬂooding broker
• If memory exceeds limit, a producer will be paused

• NOTE: This setting is enabled by default

27


Broker Memory Example
...
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true"
memoryLimit="10mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true"
memoryLimit="10mb">
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
...
</broker>

28


Message Cursors

• Only so many messages can be held in memory
• Message cursors provide a conﬁgurable message paging

• Two types of cursors
– VM cursors
• Holds only message reference in memory
– File cursors
• Flushes both message and reference to disk

• http://activemq.apache.org/how-do-i-conﬁgure-activemq-to-
hold-100s-of-millions-of-queue-messages-.html

29


Message Cursor Example
...
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true"
memoryLimit="10mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true"
memoryLimit="10mb">
<pendingQueuePolicy>
<ﬁleQueueCursor />
</pendingQueuePolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
...
</broker>
30


Quick Tip: Rebalancing Clients

• Q: After restarting one of my message brokers, how can I force
clients to connect to that broker?
• A: Enforce a deterministic lifetime on a connection by setting
the expiryTimeout on the pool of connections
class="org.apache.activemq.pool.PooledConnectionFactory"
<property name="brokerURL” value="tcp://localhost:61616" />
<property name="expiryTimeout” value="10000" />
<property name="connectionFactory">
class="org.apache.activemq.ActiveMQConnectionFactory"
p:brokerURL="tcp://localhost:61616" />
</property>
</bean>

31


Thank You!

Q&A


Bonus Material!

#apachecorn
33


Do I need a network of brokers?

34


Do I need a network of brokers?

• What is a network of brokers?
– Clustered ActiveMQ instances
• How are they clustered?
– They pass messages between broker instances
– Send a message to one broker, consume the message from a different
broker
• Where might this be useful?
– Situations where a centralized broker is not suitable
• How does this work?
– Using store and forward

35


Store and Forward

36


Topology Example

37


Topology Example

38


Topology Example

39


Topology Example

40


Topology Example

41


Should I use a master/slave conﬁg?

42



• What is a master/slave conﬁguration?
– It helps to provide high availability for ActiveMQ
• What does that mean?
– ActiveMQ brokers are conﬁgured for warm failover
– If one broker fails or becomes unreachable, another one takes over
• Where might this be useful?
– In situations that need highly available message brokers
• How does this work?
– Depends on the type of master/slave

43


Types of Master/Slave

• Shared nothing master/slave
• Shared storage master/slave
– Shared database
– Shared ﬁle system

44


Shared Nothing Master/Slave

• Sometimes called pure master/slave
• Uses a fully replicated data store
– Does not depend on database or ﬁle system
• Slave broker consumes all message states from the Master
broker (messages, acks, tx states)
• Slave does not start any networking or transport connectors
• Master broker will only respond to client when a message
exchange has been successfully passed to the slave broker

45


Shared Nothing Master/Slave

• If the master fails, the slave optionally has two modes of
operation:
1. Start up all it’s network and transport connectors
• All clients connected to failed Master resume on Slave

2. Close down completely
• Slave is simply used to duplicate state from Master

• Clients should use failover transport:

failover://(tcp://masterhost:61616, tcp://slavehost:61616)?randomize=false

46


Shared Database Master/Slave

• Uses tables in a relational database to store data
• No restriction on the number of brokers
• Simple conﬁguration (JDBC URL)
• Clustered database mitigates single point of failure
• One master selected at random



47


Shared File System Master/Slave

• Utilizes a directory on a shared file system to store data
• No restriction on number of brokers
• Simple configuration (point to the data dir)
• Shared file system mitigates single point of failure
• One master selected at random



48



• Are you trying to provide high availability?
– Then, yes
• Which one should I use?
– It depends on your situation

49


ActiveMQ In Action - ApacheCon 2011

More Related Content

What's hot (20)

Similar to ActiveMQ In Action - ApacheCon 2011 (20)

More from Bruce Snyder (8)

Recently uploaded (20)

ActiveMQ In Action - ApacheCon 2011