Get queued
Download as PPTX, PDF0 likes358 views
This document discusses using queues to improve asynchronous processing in web applications. It introduces queues and their components like messages, queues, and service workers. It then demonstrates how to implement a queue using Beanstalkd, including installing it, adding and getting messages from the queue, and executing tasks. Sample code is provided for a messaging system that queues email sending. The document encourages queueing long-running tasks like notifications, image processing, and API requests to improve performance.
![MORE USEFUL
EXAMPLE
Chad Windnagle - @drmmr763
$message = [
‘messageType => ‘sendEmail’,
‘email’ => [
‘to’ => ‘speakers@jdayflorida.com’,
‘from’ => ‘chad@chadwindnagle.com’,
‘subject => ‘Hello speakers!’,
‘body’ => ‘Please come to my session!’
]
];](https://image.slidesharecdn.com/getqueued-170225142331/85/Get-queued-28-320.jpg)
![MORE USEFUL
EXAMPLE
Chad Windnagle - @drmmr763
// get the message from the queued job
$jobData = json_decode($job->getData());
$messageType = $jobData[‘messageType’];
if ($messageType == ‘sendEmail’ {
// “to”, “from”, “subject”, & “body”
$this->sendEmail($jobData[‘email’]);
}
Service worker executes based on message type](https://image.slidesharecdn.com/getqueued-170225142331/85/Get-queued-30-320.jpg)
Get queued
- 2. DON’T MAKE ME WAIT Let’s get queued up Chad Windnagle - @drmmr763
- 3. INTRODUCTION • Recovering Joomla Addict (Clean 2yrs) • Joomla GSoC Admin (2012-2015) • Joomla Resource Team Editor • Joomla!Day, JWC, & JAB Speaker • JWC 2014 Keynote Speaker Chad Windnagle - @drmmr763
- 5. Chad Windnagle - @drmmr763 JOOMLA INSPIRED ME
- 6. AUDIENCE INTRODUCTION Chad Windnagle - @drmmr763
- 8. Chad Windnagle - @drmmr763 SYNCHRONOUS FLOW User Sends Request Server Sends Response Response Rendered
- 9. Chad Windnagle - @drmmr763 A-SYNCHRONOUS FLOW User Sends Request Server Sends Response Response Rendered Background Task Initialized Background Task Executed Notify User Task Complete
- 10. Chad Windnagle - @drmmr763 WHAT SHOULD WE QUEUE?•Notifications (Email / SMS) •Statistical Calculations •Data Matching Algorithms •Regularly Scheduled Tasks •Image / Video Processing •Data Indexing •Traditional Page Requests •User File Uploads •Login / Logout Functions •File Downloads •Add to Cart Functions •REST API Endpoints Use Queues: Don’t Use Queues:
- 11. COMPONENTS OF A QUEUE SYSTEM Chad Windnagle - @drmmr763
- 12. THE MESSAGE A message is string-based content which informs the application which task to perform, and encapsulates any data needed to perform that task. Chad Windnagle - @drmmr763
- 13. { "messageType" : "sendEmail", "email" : { "to" : "speakers@jdayflorida.com", "from" : "chad@chadwindnagle.com", "subject": "Hello speakers!", "body" : "Can't wait to see you all." } } MESSAGE EXAMPLE Chad Windnagle - @drmmr763
- 14. THE QUEUE / TUBE The message queue, or tube, contains a collection of messages that need to be processed by the queue system. Chad Windnagle - @drmmr763 Newest Oldest Typical Processing Order
- 15. SERVICE WORKER The service worker is code that is responsible for executing the tasks on the queue. Chad Windnagle - @drmmr763 Newest Oldest Typical Processing Order
- 16. EXECUTION CYCLE Chad Windnagle - @drmmr763 User Requests Queueable Task Message with Data Generated Message Added To Queue Server Boots Service Worker Service Worker Performs Task Message Removed From Queue
- 18. VARIOUS VENDORS Beanstalkd Laravel Redis Gearman RabbitMQ AmazonSQS IronMQ Symfony Chad Windnagle - @drmmr763 Beanstalkd
- 19. BEANSTALKD BENEFITS Chad Windnagle - @drmmr763 RAM Based Data Storage - High Performance! HTTP Protocol - Allows Multi-Server Set Up PHP Composer Library Available - Fast Development Community Resources (Blogs, Documentation, Etc.)
- 20. Chad Windnagle - @drmmr763 STACK OVERVIEW PHP Application (Joomla!) PHP Library (Pheanstalk) Beanstalkd Service Queue Jobs Queue / Execute Job
- 21. Chad Windnagle - @drmmr763 apt-get install beanstalkd INSTALLING BEANSTALKD
- 22. Chad Windnagle - @drmmr763 php composer.phar require pda/pheanstalk INSTALL PHEANSTALK
- 23. RUNNING BEANSTALKD Chad Windnagle - @drmmr763 ./beanstalkd -l 127.0.0.1 -p 11300 Options: -l Specify the address to listen on -p specify the port to listen on
- 24. CURRENT STATUS? Chad Windnagle - @drmmr763 Beanstalkd is running: - It can add jobs to a queue - Returns job details to service worker - Jobs can be removed from the queue
- 25. ADD MESSAGE TO QUEUE Chad Windnagle - @drmmr763 <?php use PheanstalkPheanstalk; // connect to Beanstalkd Service $pheanstalk = new Pheanstalk(‘127.0.0.1'); // add the job to the queue $pheanstalk->put(‘a special message’);
- 26. GET MESSAGE FROM QUEUE Chad Windnagle - @drmmr763 <?php use PheanstalkPheanstalk; // connect to Beanstalkd Service $pheanstalk = new Pheanstalk(‘127.0.0.1'); // get next available job from queue $job = $pheanstalk ->watch(‘default’) // specific tube ->reserve(); // job in progress
- 27. EXECUTE A TASK Chad Windnagle - @drmmr763 <?php // get the message from the queued job $jobData = $job->getData(); // prints “a special message” print $jobData; // remove job from queue $job->delete();
- 28. MORE USEFUL EXAMPLE Chad Windnagle - @drmmr763 $message = [ ‘messageType => ‘sendEmail’, ‘email’ => [ ‘to’ => ‘speakers@jdayflorida.com’, ‘from’ => ‘chad@chadwindnagle.com’, ‘subject => ‘Hello speakers!’, ‘body’ => ‘Please come to my session!’ ] ];
- 29. MORE USEFUL EXAMPLE Chad Windnagle - @drmmr763 <?php // add the job to the queue $pheanstalk->put(json_encode($message)); Application code would add tasks to the queue
- 30. MORE USEFUL EXAMPLE Chad Windnagle - @drmmr763 // get the message from the queued job $jobData = json_decode($job->getData()); $messageType = $jobData[‘messageType’]; if ($messageType == ‘sendEmail’ { // “to”, “from”, “subject”, & “body” $this->sendEmail($jobData[‘email’]); } Service worker executes based on message type
- 31. TRIGGERING SERVICE WORKER Chad Windnagle - @drmmr763 php ./service-worker.php
- 32. Chad Windnagle - @drmmr763 github.com/drmmr763/jdayflorida PHP Project Composer Based Uses Pheanstalk Uses Beanstalkd Command Line Utilities PHP Class Message Types EXAMPLE GITHUB CODE
- 33. ANY QUESTIONS? Chad Windnagle - @drmmr763
- 34. ICON CREDITS Kirby Wu Schmidt Sergey IconSpher e Carl Holdnerss ProSymbol s Above icons licensed via Creative Commons, sourced from TheNounProject.com Gregor Cresnar Olivia Stoian Rudolf Horaczek ProSymbol s icon 54 ChangHoo n Baek Daniel Baker
- 35. Chad Windnagle - @drmmr763 THANK YOU!Code: github.com/drmmr763/jdayflorida Follow Me: @drmmr763 Slides available on slideshare
Editor's Notes
- #2: Mention the sponsor - say something nice.
- #6: Joomla inspired me excited to be back share what I’ve learned go see if the grass is greener on the other side smell the roses
- #7: the curious coder the devops engineer (what version of linux is running in the background eh?) The newbie (totally new!) - I’ve got something for all of you (I Hope)
- #8: Before we start talking about how to use a queue I want to explain what a queue is and why we might use one mention examples such as submitting a contact form, placing an order, processing a credit card, etc.. why would we want to change this flow? - Doesn’t allow for certain scalability needs - Can cause other processes to become slow or unstable - we can improve the user experience by not making them wait
- #9: Mention telephone call example Mention example about submitting an email form.
- #14: As I said before jobs are STRING based content. We can luckily use JSON to encode strings of complex data so the string won’t really limit us at all.
- #16: your service worker as a ghost user.
- #17: our application code will be responsible for generating the message and adding it to the queue.
- #18: There are a number of different vendors on the market for queue systems
- #21: Here’s an overview of the structure of our system once we have it all set up. We’re going to install two main pieces of software: Beanstalkd. The messaging queue service Pheanstalk - A PHP library which was created for interacting with Beanstalkd. This just makes it a little easier to write PHP code that our Joomla site might use to perform tasks. Beanstalkd is language agnostic, though. You could use any language to do this.
- #26: Application code can add jobs to the queue.
- #29: Application code can add jobs to the queue.
- #30: Instead of a simple message we can actually put a complex JSON object into the message queue.
- #32: An important question you might be asking is how does the service worker know when to go to work or get triggered by the server? Our
- #33: An important question you might be asking is how does the service worker know when to go to work or get triggered by the server? Our
- #34: An important question you might be asking is how does the service worker know when to go to work or get triggered by the server? Our