Do you queue (updated)
Download as PPTX, PDF0 likes665 views
This document discusses using queues to improve performance and scalability in PHP applications. It notes that queues allow asynchronous and distributed processing of tasks rather than handling everything synchronously. It recommends loosening coupling between application components and using queues to process background jobs like data processing, analysis and caching. The document discusses using Zend Server's job queue to schedule and run jobs in the background. It provides examples of class structures and execution flows for implementing a job queue in a PHP application using Zend Server.
Do you queue (updated)
- 1. Do you queue?By Kevin SchroederTechnology EvangelistZend Technologies
- 2. About Kevin Past: Programming/Sys Admin Current: Technology Evangelist/Author/Composer @kpschrade
- 3. Could your PHP apps benefit from being able to process data or execute asynchronously?Twtpoll results
- 4. PerformanceExecute logic apart from the main request (asynchronicity)ScalabilityThe ability to handle non-immediate logic as resources are availableWhy would you want to queue?
- 5. People often say that performance and scalability are separate things.This is inaccurate (consider Google)Performance is the speed by which a request is executedScalability is the ability of that request to maintain its performance as load/infrastructure increasesPerformance & Scalability
- 6. DON’T!!You are not FacebookYou probably won’t beDon’t overcomplicate your problems by trying to beSo how do you scale to Facebook size?
- 7. Typical anatomy of a PHP ApplicationBad for scala-bility!PresentationApplication ControlDatabase AccessBusiness LogicPresentationApplication ControlBusiness LogicPresentation| 7
- 8. Good for ScalabilityPresentationSomething.phtmlSomething.phtmlSomething.phtmlSomething.phtmlSomething.phtmlSomething.phtmlSomething.phtmlSomething.phtmlApplication ControlSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpBusiness LogicSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpDatabase AccessSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.phpSomething.php| 8 | 6-Jun-11
- 9. Defined tasksLoose couplingResource discoveryWhat helps make software scalable?
- 10. The Golden Rule of Scalability“It can probably wait”
- 11. Pre-caching dataData analysisPre-analysis (predicting where your users will go)Data processingPre-calculating (preparing data for the next request)Data is “out of date” once it leaves the web serverImmediacy is seldom necessaryAsynchronous execution uses
- 12. A sledgehammer can hit a machineScalability and High Availability are yin and yangA site that can’t keep running is not scalableA site that can’t scale will fail (if it gets really popular)Machines can be added and removed at willNot “cloudy” necessarilyNo single point of failureData exists in at least two, preferably at least three, placesCharacteristics
- 13. Waste disk spaceControl usage (don’t let users do anything they want)Pre-calculate as much as possibleCalculate and cache/storeDon’t scan large amounts of dataKeep data processing off the front end serversDon’t just cacheDon’t let it substitute for thoughtCache hit rates can be meaningless if you have hundreds of cache hits for a requestConsiderations
- 14. Build a deployment mechanism with NO hardcoded values like directory or resource locations
- 15. Make as much as possible configurable/discoverableDecouple/PartitionDon’t tie everything (relationships and such) into the databaseUse queues/messagingStomp interfaces are really good for PHP – Can also use Java BridgeZend_Queue has several interfacesTry to use stateless interfacespolling is more scalable than idle connections; introduces lagConsiderations
- 16. Use Cron /w PHP CLI (probably shouldn’t do this)Use GearmanUse home-grown (don’t do this)Use pcntl_fork() (NEVER do this)Use Zend Server Job QueueOptions
- 17. Your only real optionsVery cloud friendly* I am not an expert on Gearman. Corrections will be taken in the spirit that they are given.For obvious reasons, I will focus on Zend Server
- 18. Schedule jobs in the futureSet recurring jobsExecute immediately, as resources are available (my fav)Utilize ZendJobQueue()Using the Zend Server Job Queue
- 19. Job Queue Architecture – Elastic BackendWeb Server /w JQWeb ServerUsers!Web Server /w JQWeb ServerLoad BalancerWeb Server /w JQWeb ServerProsScale the backend as necessaryDefault (easy) mechanismConsGetting the job status requires using a DB
- 20. Job Queue Architecture – Elastic FrontendWeb ServerWeb Server /w JQUsers!Web ServerWeb Server /w JQLoad BalancerWeb ServerWeb Server /w JQProsEasy to communicate with the Job Queue server handling the jobConsRequires you to build your own interface
- 21. Create a task-handling controllerCreate an abstract task classUnderstands the Job QueueSelf containedIf Elastic Backend: connects to localhostIf Elastic Frontend: connects to load balancer (my preferred), load balanced JQ servers manage themselvesExecute the task, have it serialize itself and send it to send to the task handlerKevin’s favorite way to implement it
- 22. com\zend\jobqueue\ManagerHandles connecting to the queue and passing results back and forthcom\zend\jobqueue\JobAbstractAbstract class that a job would be based off ofcom\zend\jobqueue\ResponseThe response from the manager when a job is queued. Contains the server name and job numberorg\eschrade\jobs\ScandirThe actual job that scans the directoryorg\eschrade\jobs\ScandirResultAn object that represents the data foundClasses involved in the demo
- 23. Create job and set dataExecute jobJob passes itself to the queue managerManager serializes jobManager uses HTTP call through a load balancer to queue the jobThe queue on the other end returns the job id and server nameJob ID and server name is passed to the clientClient polls the manager to get a completed jobWhen the job is returned pass the serialized version of the executed jobExecution Flow
- 24. Let’s write some code(no I’m not copping out with slides. We’re all told to show our work in grade skool)
- 26. Get this information and all the examples at eschrade.com…