Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)
2 likes989 views
The document is a presentation by James Titcomb on using RabbitMQ for message queuing, covering its installation, basic operations, and various exchange types like fanout, direct, and topic exchanges. It also discusses real-world applications, the importance of clustering and high availability solutions in RabbitMQ, and provides PHP code examples for implementing messaging patterns. Overall, it aims to explain how RabbitMQ can enhance application performance through effective message handling and processing.
![fanout/consumer.php
$q = $channel->queue_declare(
'', // Lets RabbitMQ pick a name for queue
false, false, false,
true // Delete this queue
);
$queue_name = $q[0];
$channel->exchange_declare(
'test_exchange', 'fanout', false, false, false);
$channel->queue_bind($queue_name, 'test_exchange');](https://image.slidesharecdn.com/141202-adding1-141203071811-conversion-gate01/85/Adding-1-21-Gigawatts-to-Applications-with-RabbitMQ-PHPNW-Dec-2014-Meetup-25-320.jpg)
![direct/consumer.php
$q = $channel->queue_declare('', false, false, false, true);
$queue_name = $q[0];
$channel->exchange_declare(
'test_direct', 'direct', false, false, false);
// Bind for each routing key we want (BINDING KEY)
$channel->queue_bind($queue_name, 'test_direct', 'apple');
$channel->queue_bind($queue_name, 'test_direct', 'orange');
$channel->queue_bind($queue_name, 'test_direct', 'banana');](https://image.slidesharecdn.com/141202-adding1-141203071811-conversion-gate01/85/Adding-1-21-Gigawatts-to-Applications-with-RabbitMQ-PHPNW-Dec-2014-Meetup-34-320.jpg)
![Creating a cluster
node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}]
...done.
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}]
...done.
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}]
...done.](https://image.slidesharecdn.com/141202-adding1-141203071811-conversion-gate01/85/Adding-1-21-Gigawatts-to-Applications-with-RabbitMQ-PHPNW-Dec-2014-Meetup-55-320.jpg)
![Creating a cluster
node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}]
...done.](https://image.slidesharecdn.com/141202-adding1-141203071811-conversion-gate01/85/Adding-1-21-Gigawatts-to-Applications-with-RabbitMQ-PHPNW-Dec-2014-Meetup-56-320.jpg)
![Removing Nodes
node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqctl start_app
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node3]}]
...done.](https://image.slidesharecdn.com/141202-adding1-141203071811-conversion-gate01/85/Adding-1-21-Gigawatts-to-Applications-with-RabbitMQ-PHPNW-Dec-2014-Meetup-58-320.jpg)
Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)
- 1. Adding 1.21 Gigawatts to Applications with RabbitMQ James Titcumb PHPNW December 2014
- 2. Who is this guy? James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim
- 3. What is message queueing?
- 5. Scaling with Rabbit Application RabbitMQ Background processing
- 6. Scaling with Rabbit Application RabbitMQ Background processing Background processing
- 7. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing
- 8. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing Background processing
- 9. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing Background processing Background processing
- 10. Real world uses?
- 11. Why RabbitMQ?
- 12. Installing RabbitMQ (on precise64, other OSs may vary)
- 13. Using Apt ● add apt repo ○ deb http://www.rabbitmq.com/debian/ testing main ● add signing key ○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc ● apt-get update ● apt-get install rabbitmq-server ● rabbitmq-plugins enable rabbitmq_management ● sudo service rabbitmq-server restart
- 16. Objective: Basic Queuing test_queue 1 2 3 4 5 Producer Consumer
- 17. composer.json { "require": { "videlalvaro/php-amqplib": "2.*" } } then composer install
- 18. Please wait, connecting... use PhpAmqpLibConnectionAMQPConnection; $connection = new AMQPConnection( 'localhost', 5672, 'guest', 'guest', '/' ); $channel = $connection->channel();
- 19. basic/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->queue_declare( 'test_queue', false, true, false, false); $message = new AMQPMessage('my test message'); $channel->basic_publish($message, '', 'test_queue');
- 20. basic/consumer.php $channel->basic_consume( 'test_queue', // Queue to consume '', // Consumer identifier false, true, // No-ack means messages are "auto acknowledged" false, // Exclusive - no other consumers can use the queue false, function(AMQPMessage $message) { echo $message->body . "n"; } ); while (count($channel->callbacks)) { $channel->wait(); }
- 23. Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
- 24. fanout/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $message = new AMQPMessage('my test message #' . $id); $channel->basic_publish($message, 'test_exchange');
- 25. fanout/consumer.php $q = $channel->queue_declare( '', // Lets RabbitMQ pick a name for queue false, false, false, true // Delete this queue ); $queue_name = $q[0]; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $channel->queue_bind($queue_name, 'test_exchange');
- 27. A word on Temporary Queues Producer test_exchange Messages go nowhere
- 29. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer BK = orange, banana, apple Consumer
- 30. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
- 31. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
- 32. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
- 33. direct/producer.php $channel->exchange_declare( 'test_direct', 'fanout', false, false, false); $messageContent = 'my test message, key=' . $routingKey; $message = new AMQPMessage($messageContent); $channel->basic_publish($message, 'test_direct', $routingKey);
- 34. direct/consumer.php $q = $channel->queue_declare('', false, false, false, true); $queue_name = $q[0]; $channel->exchange_declare( 'test_direct', 'direct', false, false, false); // Bind for each routing key we want (BINDING KEY) $channel->queue_bind($queue_name, 'test_direct', 'apple'); $channel->queue_bind($queue_name, 'test_direct', 'orange'); $channel->queue_bind($queue_name, 'test_direct', 'banana');
- 36. Exchanges: Topic
- 37. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
- 38. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
- 39. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
- 40. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
- 43. Fetch message Logging Sequence Browser Application Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
- 44. Flexibility!
- 45. RPC
- 46. TTL
- 47. DLX
- 48. Infrastructure
- 49. Problem: SPOF
- 51. Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 6 RabbitMQ Node 5 Load Balance / Floating IP / Low TTL DNS etc.
- 52. Everything Replicates (except queues…)
- 53. RAM / Disk
- 54. Configuration...
- 55. Creating a cluster node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}] ...done. node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}] ...done.
- 56. Creating a cluster node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done.
- 58. Removing Nodes node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done.
- 59. Solution 2: HA
- 60. HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
- 61. Have a go yourself! https://github.com/asgrim/rmq-slides
- 62. Questions?
- 63. Thanks for watching! James Titcumb @asgrim