Deploying FuseMQ with Fuse Fabric

Deploying FuseMQ in enterprise using Fuse
Fabric

Dejan Bosanac
FuseSource

1 © 2012 FuseSource Corp. All rights reserved. FuseSource Confidential

Presenter: Dejan Bosanac

 Senior Software Engineer at FuseSource -
http://fusesource.com
 Apache ActiveMQ committer and PMC member
 Co-author of ActiveMQ in Action
 Blog:
• http://www.nighttale.net/

 Twitter:
• http://twitter.com/dejanb


Agenda

 Problems of large enterprise deployments
 Fuse Fabric in nutshell
 FuseMQ and Fuse Fabric
• Creating brokers
• Connecting
• Topologies
 Fuse Management Console


Problems of large deployments

Integrate Everything

Problems – Deploying and maintenance

 Main problems
• Installing brokers on multiple hosts
o ssh, untar, set directories and environment
• Setting configuration manually for every broker
o copying xml config, tweaking, testing
• Updating configuration across cluster
• Upgrading brokers

It’s very tedious and error-prone process


Problems – Traditional best-practice tips

 Keep XML as a template and configure node-specific
details through properties
 Keep configuration in SVC system (git, svn, ...)
 Keep configuration separate from installation for easier
upgrades

Deployment with Fuse Fabric moves it to the next level


Problems - Clients

 Topology is very “static”
 Clients need to be aware of topology
 Clients need to know broker locations
 Changes are not easy as clients need to be updated
 Adding new resources (brokers) requires client updates
 Not suitable for “cloud” deployments

Fuse Fabric makes deployments more “elastic”


Fuse Fabric in a nutshell



 How Fabric can help?

• It provides centralized distributed broker configuration
• It provides centralized distributed broker registry
• Uses OSGi and Apache Karaf for easy spawning new broker
instances
• It provides additional tools for centralized configuration and
monitoring (Fuse Management Console)



 Installation
• Features and bundle versions centrally stored and managed
• Easy installation and upgrade
 Configuration
• Stored in one place
• Versioned
 Discovery
• All brokers registered in central registry
• Allows clients to connect without knowing broker locations
• Allows easy creation of advanced topologies


Fuse Fabric Architecture

 Zookeeper

• Replicated in-memory tree
• Similar to file system
• Highly-available
• Distributed
• Support network split
• Proven track record

Ideal for distributed configuration and locking



 Containers
• Apache Karaf instances provisioned through central registry (Zookeeper)



 Profiles:

• Zookeeper nodes with conventional names
• OSGi configuration for the node (so we know what features and
bundles should be used)
• Other configuration (centralized broker configuration)
• Versioned


Fuse Fabric - Profile

FuseFabric:karaf@root> profile-display default
Profile id: default
Version : 1.0
Parents :
Associated Containers :

Container settings
----------------------------
Repositories :
mvn:org.fusesource.fabric/fuse-fabric/7.0-SNAPSHOT/xml/features

Features :
fabric-agent
karaf
fabric-jaas
fabric-core


Fuse Fabric - Profile

Agent Properties :
org.ops4j.pax.url.mvn.repositories =
http://repo1.maven.org/maven2,
http://repo.fusesource.com/nexus/content/repositories/releases,
http://repo.fusesource.com/nexus/content/groups/ea,
http://repository.springsource.com/maven/bundles/release,
http://repository.springsource.com/maven/bundles/external,
http://scala-tools.org/repo-releases
org.ops4j.pax.url.mvn.defaultRepositories =
file:${karaf.home}/${karaf.default.repository}@snapshots,
file:${karaf.home}/local-repo@snapshots

Configuration details
----------------------------
PID: org.fusesource.fabric.zookeeper
zookeeper.url ${zk:root/ip}:2181


FuseMQ and Fuse Fabric


FuseMQ features

 mq-base profile
• Defines OSGi features and bundles to be installed
• Defines basic broker settings
 mq-create command
• Helper command for creating brokers
• Creates an new profile based on mq-base
• Optionally creates new containers
• Assigns the profile to containers (essentially starts the broker)


MQ – Creating broker

FuseFabric:karaf@root> mq-create --create-container broker1 fusebroker
MQ profile fusebroker ready
Successfully created container broker1


MQ Profile

FuseFabric:karaf@root> profile-display fusebroker
Profile id: fusebroker
Version : 1.0
Parents : mq-base
Associated Containers : broker1

Configuration details
----------------------------
PID: org.fusesource.mq.fabric.server-fusebroker
standby.pool default
connectors openwire
broker-name fusebroker
data data/fusebroker
config zk:/fabric/configs/versions/1.0/profiles/mq-base/broker.xml
group default


MQ – Assigning profile

FuseFabric:karaf@root> container-create-ssh --host 192.168.1.106
--user dejanb --password xxx broker1

FuseFabric:karaf@root> mq-create --assign-container broker1 fusebroker
MQ profile fusebroker ready
Profile successfully assigned to broker1


MQ - Benefits

 What did we achieve with this?

• We can easily create new brokers with the same profiles
• We can create new profile version with updated broker version
and/or changed configuration
• We can easily update all (or some) brokers by applying the new
profile


MQ Profile - Management

 Create a new profile version
• with upgraded bundles
• and configuration changes
 Try it out on a non-production container
 Deploy to one or a few production containers
 Roll the full upgrade
 Easy rollback if anything goes wrong


Broker Registry


Broker Registry

 Brokers are organized in groups (clusters)
• Cluster can have any number of brokers (with different names)
• Put in “default” group if not specified


Connecting to the Broker

 Clients need to have ZooKeeper URL
 There is a new discovery protocol (called fabric)
 Connecting is as easy as defining the group


Connecting - Factory

ActiveMQConnectionFactory factory =
new ActiveMQConnectionFactory(“discovery:(fabric:default)”);


Connecting - Reconnecting

 Clients don’t need to know brokers location
 Works like a failover transport
 Supports options for tuning reconnecting options

discovery:(fabric:default)?reconnectDelay=1000&useExponentialBackOff=false


Connecting - Camel

<camelContext xmlns="http://camel.apache.org/schema/spring">
<!– Do your magic here -->
</camelContext>

<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value=”discovery:(fabric:discovery)"/>
</bean>


Topologies


Master/Slave

 Create master slave configuration by starting multiple
brokers with the same name (in the same group)
• First one stared becomes a master
• Everyone else is a slave
• Locked on Zookeeper node
• When master dies, a first slave to get a lock becomes next master


Master/Slave




Master/Slave

 No more relying on shared storage locking
 You’ll still need shared storage for preserving the state
among brokers
 Easy creating non-persistent master slave configurations
 Clients again don’t need to know topology as fabric
discovery will do that work


Master/Slave

 Multiple master slave over the same containers
• Resource utilization
mq-create --create-container broker1,broker2,broker3 hq-broker
mq-create --assign-container broker1,broker2,broker3 web-broker


Networks

 Controlled through profile
 Uses fabric discovery, just as clients
mq-create --group us-east --networks us-west --create-container us-east1,us-east2 us-east

mq-create --group us-west --networks us-east --create-container us-west1,us-west2 us-west


Elastic clusters

 Request-reply pattern over JMS
 Load Balance Traffic
 Non-persistent, not-connected brokers
 Elastic cluster
• Allow adding new brokers, without updating clients
• Allow rebalancing of clients


Elastic clusters

mq-create --create-container broker1 broker1


Tooling


Fuse Management Console

 Centralized Unified Console
 Web UI for managing and monitoring infrastructure
 Uses Fabric to discover resources
 Features
• Container Management
• Profile Management
• Centralized Security
• Centralized Monitoring


FMC – containers


FMC – Container


FMC – broker view


FMC - Profiles


FMC - Profile


Future

 More things for developers
• Make it even easier to write applications for Fuse Enterprise

 More things for operations
• Visualization of clusters
• Centralized logging (collect and search all logs centrally)


Conclusion

 Helps with complex and large deployments
 Use central registry for distributed configuration and
locking
 Make clients location agnostic of brokers (needed for
cloud deployments)
 Easy upgrades and updates
 Support for incremental patching
 Tools


Questions


Deploying FuseMQ with Fuse Fabric

More Related Content

What's hot (20)

Similar to Deploying FuseMQ with Fuse Fabric (20)

More from dejanb (6)

Recently uploaded (20)

Deploying FuseMQ with Fuse Fabric