Apache ActiveMQ and Apache Camel
7 likes4,896 views
ActiveMQ is an open source message broker that implements the Java Message Service (JMS) API. It allows applications written in different languages to communicate asynchronously. Apache Camel is an open source integration framework that can be used to build messaging routes between different transports and APIs using a simple domain-specific language. It provides mediation capabilities and supports common integration patterns. Both ActiveMQ and Camel aim to simplify integration between disparate systems through message-based communication.
Apache ActiveMQ and Apache Camel
- 2. Contents 1. Introduction 2. About JMS 3. Messaging Domains 4. Why ActiveMQ? 5. Benefits
- 3. Introduction ● ActiveMQ is used in enterprise service bus implementations such as Apache ServiceMix and Mule. ● Apache ActiveMQ is a message broker which fully implements the Java Messaging Service API. It can be used by programs written Java,C/C++,.NET,PHP etc.
- 4. JMS (Java Messaging Service) The Java Message Service (JMS) API is a Java Message Oriented Middleware (MOM) API for sending messages between two or more clients. It is a messaging standard that allows application components based on the Java Enterprise Edition (Java EE) to create, send, receive, and read messages. An application can communicate with any number of applications using JMS.This communication is loosely coupled. Instead, those applications are communicated by connecting to a common destination. An application can send messages to the destination and can take messages from the same destination.The concept is shown below as a schematic.This technology is too simple and it can be easily integrated with existing applications so that those applications can communicate each other.
- 5. Destination Type Description Topic In JMS a Topic implements publish and subscribe semantics. When you publish a message it goes to all the subscribers who are interested - so zero to many subscribers will receive a copy of the message. Only subscribers who had an active subscription at the time the broker receives the message will get a copy of the message. Queue A JMS Queue implements load balancer semantics. A single message will be received by exactly one consumer. If there are no consumers available at the time the message is sent it will be kept until a consumer is available that can process the message. If a consumer receives a message and does not acknowledge it before closing then the message will be redelivered to another consumer. A queue can have many consumers with messages load balanced across the available consumers.
- 6. JMS Sessions A session is a single-threaded context for producing and consuming messages. We use sessions to create the following: ● Message producers ● Message consumers ● Messages ● Destination When a client creates a JMS session it must specify the mode in which the Session will acknowledge the messages that it receives and dispatches. The modes supported are summarized in the table below.
- 7. Acknowledge Mode Description AUTO_ACKNOWL EDGE With this acknowledgement mode, the session automatically acknowledges a client's receipt of a message either when the session has successfully returned from a call to receive or when the message listener the session has called to process the message successfully returns. CLIENT_ACKNOW LEDGE With this acknowledgement mode, the client acknowledges a consumed message by calling the message's acknowledge method. Acknowledging a consumed message acknowledges all messages that the session has consumed. When client acknowledgement mode is used, a client may build up a large number of unacknowledged messages while attempting to process them. A JMS provider should provide administrators with a way to limit client overrun so that clients are not driven to resource exhaustion and ensuing failure when some resource they are using is temporarily blocked. DUPS_OK_ACKN OWLEDGE This acknowledgement mode instructs the session to lazily acknowledge the delivery of messages. This is likely to result in the delivery of some duplicate messages if the JMS provider fails, so it should only be used by consumers that can tolerate duplicate messages. Use of this mode can reduce session overhead by minimizing the work the session does to prevent duplicates. SESSION_TRANS ACTED Session is Transacted and the acknowledge of messages is handled internally. INDIVIDUAL_ACK NOWLEDGE Acknowledges are applied to a single message only. Unlike CLIENT_ACKNOWLEDGE where the acknowledgement applies to all messages received up to that point for the entire session, this mode applied only to a single message allowing the client to be more selective about which messages are acknowledged.
- 8. Messaging Domains JMS API supports Point-to-Point and Publish-Subscribe approaches. Point-to-Point Approach : The point to point approach consists of a sender , a receiver and a queue.The sender addresses a message to the queue, and the receiving clients extract messages from the queues established to hold their messages. The queue keeps the message till a receiving client receives it or till the message expires. While any number of producers can send messages to the queue, each message is guaranteed to be delivered, and consumed by one consumer.
- 9. Publish/Subscribe Approach : The publish/subscribe model supports publishing messages to a particular message topic. Subscribers may register interest in receiving messages on a particular message topic. In this model, neither the publisher nor the subscriber knows about each other. A good analogy for this is an anonymous bulletin board. ● Zero or more consumers will receive the message. ● There is a timing dependency between publishers and subscribers. The publisher has to create a message topic for clients to subscribe. The subscriber has to remain continuously active to receive messages, unless it has established a durable subscription. In that case, messages published while the subscriber is not connected will be redistributed whenever it reconnects.
- 10. A JMS message has 3 parts a)header :- The header contains meta-information about the message— who sent it, where it’s going, and so on. b)body :- As the name suggests it is the body of messages. JMS API allows five types of message bodies. 1. TextMessage :- Body contains String data 2. ByteMessage :- Body contains byte data 3. MapMessage :- Body contains data in key/value pair 4. StreamMessage :-Body contains a stream of primitive values 5. ObjectMessage : – Body contains an object 6. Message :- Nothing in body. Only header and properties. c)properties :- Additional properties other than header.
- 11. Why Messaging as Communication Style? ● File Transfer and Shared Database enable applications to share their data but not their functionality. ● Remote Procedure Invocation enables applications to share functionality, but it tightly couples them as well. Often the challenge of integration is about making collaboration between separate systems as timely as possible, without coupling systems together in such a way that they become unreliable either in terms of application execution or application development. ● The data transfer should be asynchronous so that the sender does not need to wait on the receiver, especially when retry is necessary. ● Use Messaging to transfer packets of data frequently, immediately, reliably, and asynchronously, using customizable formats.
- 12. Why ActiveMQ? Imagine you need to communicate with two or more systems. A common approach these days will be web services which is fine if you need an answers right away. However: web services can be down and not available - what do you do then? Putting your message into a message queue (which has a component on your machine/server, too) typically will work in this scenario - your message just doesn't get delivered and thus processed right now - but it will later on, when the other side of the service comes back online. The Message Queue receives the message, places it in the proper queue, and waits for the application to retrieve the message when ready.
- 13. Connect an application to a messaging channel using a Message Endpoint, a client of the messaging system that the application can then use to send or receive Messages.We need a message endpoint to connect the system explicitly to the integration solution. The endpoint can be a special piece of code or a channel adapter provided by an integration software vendor. For example, one data format may store the customer name in two fields, called FIRST_NAME and LAST_NAME, while the other system may use a single field called Customer_Name. Likewise, one system may support multiple customer addresses while the other system only supports a single address. Because the internal data format of an application can often not be changed the middleware needs to provide some mechanism to convert one application’s data format in the other’s. We call this step translation.
- 14. So far we can send data from one system to another and accommodate differences in data formats. What happens if we integrate more than two systems? Where does the data have to be moved? We could expect each application to specify the target system(s) for the data it is sending over the channel. For example, if the customer address changes in the customer care system we could make that system responsible for sending the data to all other systems that store copies of the customer address. As the number of systems increases this becomes very tedious and requires the sending system to have knowledge about all other systems. Every time a new system is added, the customer care system would have to be adjusted to the new environment. Things would be a lot easier of the middleware could take care of sending messages to the correct places. This is the role of a routing component such as a message broker. Integration solutions can quickly become complex because they deal with multiple applications, data formats, channels, routing and transformation. All these elements may be spread across multiple operating platforms and geographic locations. In order to have any idea what is going on inside the system we need a systems management function. This subsystem monitors the flow of data, makes sure that all applications and components are available and reports error conditions to a central location.
- 15. When a standard message is sent to a queue from a producer, it is sent to the broker, which persists it in its message store. By default this is in KahaDB, but it can configured to be stored in memory, which buys performance at the cost of reliability. Once the broker has confirmation that the message has been persisted in the journal (the terms journal and message store are often used interchangeably), it responds with an acknowledgement back to the producer. The thread sending the message from the producer is blocked at this time.
- 16. On the consumption side, when a message listener is registered or a call to receive() is made, the broker creates a subscription to that queue. Messages are fetched from the message store and passed to the consumer; it’s usually done in batches, and the fetching is a lot more complex than simply read from disk, but that’s the general idea. With the default behaviour, assuming the Session.AUTO_ACKNOWLEDGE is being used, the consumer will acknowledge that the message has been received before processing it. On receiving the acknowledgement, the broker updates the message store marking that message as consumed, or just deletes it (this depends on the persistence mechanism).
- 18. This figure also illustrates two important messaging concepts: 1. Send and forget —In step 2, the sending application sends the message to the message channel. Once that send is complete, the sender can go on to other work while the messaging system transmits the message in the background. The sender can be confident that the receiver will eventually receive the message and does not have to wait until that happens. 2. Store and forward —In step 2, when the sending application sends the message to the message channel, the messaging system stores the message on the sender’s computer, either in memory or on disk. In step 3, the messaging system delivers the message by forwarding it from the sender’s computer to the receiver’s computer, and then stores the message once again on the receiver’s computer. This store-and- forward process may be repeated many times as the message is moved from one computer to another until it reaches the receiver’s computer.
- 19. Benefits : Using messaging as an integration or communication style leads to many benefits such as: ● Allowing applications built with different languages and on different operating systems to integrate with each other ● Location transparency – client applications don’t need to know where the service applications are located ● Reliable communication – the producers/consumers of messages don’t have to be available at the same time, or certain segments along the route of the message can go down and come back up without impacting the message getting to the service/consumer ● Scaling – can scale horizontally by adding more services that can handle the messages if too many messages are arriving ● Asynchronous communication – a client can fire a message and continue other processing instead of blocking until the service has sent a response; it can handle the response message only when the message is ready ● Reduced coupling – the assumptions made by the clients and services are greatly reduced as a result of the previous 5 benefits. A service can change details about itself, including its location, protocol, and availability, without affecting or disrupting the client.
- 21. Contents 1) Introduction 2) About Camel 3) Messaging Models 4) Example 5) Architecture
- 23. Introduction ● Apache Camel “a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration”. ● Camel is a mediation and routing engine that lets you implement EIP’s. It uses a simple URI notation to identify the different transport and messaging models. ● Camel is open-source integration framework which allows you to build routes for integration scenario's quickly and efficiently. You can deploy these routes directly on ServiceMix by deploying the plain Spring XML route or by packaging the route in an OSGi bundle. ● The best thing about Camel is that it can be embedded into your current java application to implement integrations. On a larger scale Camel is part of Apache ServiceMix ESB. ● The Camel project was started in early 2007.
- 24. About Camel ● Building complex systems from scratch is a very costly endeavor, and one that’s almost never successful. Apache Camel is such an integration framework that provides simple, manageable abstractions for the complex systems you’re integrating and the “glue” for plugging them together seamlessly. ● Camel’s most important features: message routing. At the core of the Camel framework is a routing engine, or more precisely a routing engine builder. It allows you to define your own routing rules, decide from which sources to accept messages, and determine how to process and send those messages to other destinations. For example: JMS -> JSON, HTTP -> JMS or funneling FTP -> JMS, HTTP -> JMS, JSON -> JMS
- 25. ● Camel has two main ways of defining routing rules: the Java-based domain specific language (DSL) and the Spring XML configuration format. ● One of the fundamental principles of Camel is that it makes no assumptions about the type of data you need to process. This is an important point, because it gives you, the developer, an opportunity to integrate any kind of system, without the need to convert your data to a canonical format. ● Camel isn’t an enterprise service bus(ESB), although some call Camel a lightweight ESB because of its support for routing, transformation, monitoring, orchestration, and so forth. Camel doesn’t have a container or a reliable message bus, but it can be deployed in one, such as Open-ESB or the ServiceMix. For that reason, we prefer to call Camel an integration framework rather than an ESB. Camel was designed not to be a server or ESB but instead to be embedded in whatever platform you choose.
- 26. Camel’s message model In Camel, there are two abstractions for modeling messages, ■ org.apache.camel.Message —The fundamental entity containing the data being carried and routed in Camel. —The Message interface provides an abstraction for a single message, such as a request, reply or exception message. In Camel terminology, the request, reply and exception messages are called in, out and fault messages. ■ org.apache.camel.Exchange —The Camel abstraction for an exchange of messages. This exchange of messages has an “in” message and as a reply, an “out” message. —The Exchange interface provides an abstraction for an exchange of messages, that is, a request message and its corresponding reply or exception message.
- 27. Example on Routing Files: Suppose you need to read files from one directory (data/inbox), process them in some way, and write the result to another directory (data/outbox). For simplicity, you’ll skip the processing, so your output will be merely a copy of the original file. Below figure illustrates this process. It looks pretty simple, right? Here’s a possible solution using pure Java (with no Camel) but it still results in 34 lines of code. You have to use low-level file APIs and ensure that resources get closed properly, a task that can easily go wrong. But if an integration framework like Apache Camel is used, we can create a file-polling route in just one line of Java code
- 28. Architecture
- 29. CamelContext A CamelContext object represents the Camel runtime system. CamelContext is the heart of Camel its where all the routes, endpoints, components, etc. is registered. You typically have one CamelContext object in an application. A typical application executes the following steps. ● Create a CamelContext object. ● Add endpoints – and possibly Components, to the CamelContext object. ● Add routes to the CamelContext object to connect the endpoints. ● Invoke the start() operation on the CamelContext object. This starts Camel-internal threads that are used to process the sending, receiving and processing of messages in the endpoints. ● Eventually invoke the stop() operation on the CamelContext object. Doing this gracefully stops all the endpoints and Camel-internal threads.
- 30. Components Component is confusing terminology; EndpointFactory would have been more appropriate because a Component is a factory for creating Endpoint instances. Processor The Processor interface represents a class that processes a message. Notice that the parameter to the process() method is an Exchange rather than a Message. RouteBuilders A route is the step-by-step movement of a Message from an input queue, through arbitrary types of decision making (such as filters and routers) to a destination queue (if any). A DSL wires Endpoints and Processors together to form routes. Camel provides two ways for an application developer to specify routes. One way is to specify route information in an XML file. A discussion of that approach is outside the scope of this document. The other way is through what Camel calls a Java DSL (domain-specific language).
- 31. Here are some examples of the DSL using different languages and staying functionally equivalent: Java DSL from("file:data/inbox").to("jms:queue:order"); Spring DSL <route> <from uri="file:data/inbox"/> <to uri="jms:queue:order"/> </route> Scala DSL from "file:data/inbox" -> "jms:queue:order" These examples are real code, and they show how easily you can route files from a folder to a JMS queue.
- 32. Content Based Router The Content Based Router from the EIP patterns allows you to route messages to the correct destination based on the contents of the message exchanges. The Content-Based Router examines the message content and routes the message onto a different channel based on data contained in the message.
- 33. Message Filter The Message Filter from the EIP patterns allows you to filter messages. The Message Filter has only a single output channel. If the message content matches the criteria specified by the Message Filter, the message is routed to the output channel. If the message content does not match the criteria, the message is discarded.
- 35. Thank You